Three-Toed Sloth

Advanced Data Analysis from an Elementary Point of View: Self-Evaluation and Lessons Learned

Accidentally left in my drafts folder for two months. I still haven't looked at my student evaluations.

Now that most of the final exams are graded, but before I've gotten to see my student evaluations, it seems like a good time to reflect on the class. Also, I have had enough May wine, with woodruff from my garden, that the prospect of teaching it again next year can be greeted with equanimity.

First, and conditioning everything else, this was by far the largest class I've taught (70 students), and to the extent it went well it's entirely due to my teaching assistants, Gaia Bellone, Shuhei Okumura and Zachary Kurtz. I'd say I couldn't thank them enough, but clearly I'll have to do 30--40% better than that next year, when there will be between 90 and 100 students. (Memo to self: does the university allow me to pay bonuses to TAs in whiskey?)

• Getting anonymous feedback mid-semester seems to have gone over pretty well. Unfortunately from my point of view, the feedback was all over the map, making it hard to know what to change. I do however need to be clearer about how, exactly, what I am lecturing on feeds into the data analyses I am asking them to do each week.
• I am pretty happy with the subject matter and its arrangement. Just as an undergraduate "introduction to modern physics" class aims to bring the student up to about 1926, my hope was to bring them, methodologically, up to about 1990. (Statistics is after all a younger field than physics.) Judging by the final, most of them got most of it, and were able to use it with only minimal prompts.
• I should have done more to emphasize the identification/estimation distinction throughout the class, rather than just at the end in causal inference. This could I think be done fairly straightforwardly.
• There were three topics I seriously regret cutting: relative distribution methods and smooth tests of goodness of fit; time series and longitudinal data analysis; and hierarchical regression models. I am not sure what I would remove or shorten from the current curriculum to make room for them.
• The lecture notes come to about 400 pages, more than half of it new text. From a purely selfish point of view, I should have written maybe 40 pages, if that, and trimmed my content to some existing textbook. On the bright side, chunks of it could be re-cycled for STACS.
• I was surprised at how many students had no real programming knowledge (operationally: didn't know how to write R functions). With any luck, the new statistical computing class I'll teach in the fall will help keep that from repeating in the spring. Similarly, it would have saved a lot of time for both me and the students if they'd all had copies of something like The R Cookbook.
• This was the first class I've had at CMU where (attempted!) cheating was at all an issue. I suspect this was due to a combination of the size and the fact that it's required for several majors and programs. This raises the question of whether I need to come up with all-new assignments next year. I am inclined not to, and just flunk anyone who copies the old solutions, but am open to suggestions.
• I offered too much easy extra credit on several assignments. The ranking by grades sometimes noticeably distorted the obvious ranking by actual knowledge. I do not think the final grades suffered from this, but points must be revised for the future.
• I didn't give enough opportunities to practice writing, and get feedback. Perhaps, next year, all of the data analysis assignments will require at least a page of prose? (I am not sure what I will cut to make time for that on the students' part.)
• Having "Intro to Nietzsche" in the lecture room right before our class probably did the students no favors, because it kept reminding me of things like the passage from The Gay Science about how science ought to become "the great dispenser of pain".

Advanced Data Analysis from an Elementary Point of View