Three-Toed Sloth

Statistics 36-350: Data Mining (Fall 2008)

Since class begins Monday, this is a good time for the public website to make its appearance. As before, lecture notes will also be posted here; you can use the RSS feed for this entry to keep track of them.

0. Introduction to the course (25 August)
1. Information retrieval and similarity searching (25 August)
2. Multidimensional scaling and a first glance and classification (27 August)
3. A little about page-rank (29 August)
   Homework #1, due 8 September: assignment, R, newsgroups.tgz data file
   Solutions
4. Image search, abstraction and invariance; the accompanying slides (8 September)
5. Finding informative features (10 September)
   Additional reading: David Feldman, "Introduction to Information Theory", chapter 1
6. Information and interaction among features (12 September)
   Additional reading: Aleks Jakulin and Ivan Bratko, "Quantifying and Visualizing Attribute Interactions", arxiv:cs.AI/0308002
   Homework #2, due 22 September: assignment
   solutions, solutions code
   Note: Information theory, axiomatic foundations, connections to statistics — elaboration on some points raised in lecture (12 September)
7. Categorization: types of categorization, basic classifiers and finding simple clusters in data (15 September)
8. Hierarchical clustering; how many clusters? (17 September)
9. Yet more clustering (19 September; slides)
10. Making better features: transformations, principal components (22 September)
11. Mathematics of principal components analysis; interpretations and limitations of PCA (24 September)
12. Yet more on linear dimensionality reduction: PCA + information retrieval = Latent semantic indexing. Factor analysis: motivations, historical roots, preliminaries to estimation (26 September)
    Optional reading: Deerwester et al., "Indexing by Latent Semantic Analysis" [PDF]
    Optional reading: Landauer and Dumais, "A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge" [PDF]
    Optional reading: Thurstone, "The Vectors of Mind"
    Home #3, due 3 October: assignment
13. More on factor analysis: estimation and the rotation problem (29 September)
14. Principal Components versus Factor Analysis: worked examples, basic goodness-of-fit testing for factor analysis; R code for lecture (1 October)
15. The truth about principal components and factor analysis: strengths, limitations, factor models as graphical models, factor models and mixture models, Thomson's sampling model; R code for Thomson's model (3 October)
    Homework #4, due Friday, 10 October: assignment, nci.kmeans, nci.pca2.kmeans
16. Regression: predicting quantiative features: point prediction; expectations and mean-square optimality; regression functions; regression as smoothing; linear regression as linear smoothing; other kinds of linear smoothers; nearest-neighbor regression; kernel regression. R code for figures, data for running example (6 October)
17. The truth about linear regression: optimal linear prediction; shifting distributions and omitted variables; rights and obligations of probabilistic assumptions; abuses of linear regression; how to hurt angels (8 October)
18. Extending linear regression: weighted least-squares, heteroskedasticity, local linear regression. R code for figures, data for running example (10 October)
19. Mid-term review (13 October; no hand-out)
20. Mid-term: exam, solutions (15 October)
21. Evaluating preditive models: in-sample and generalization error; over-fitting and under-fitting; model selection, capacity control, cross-validation. R for figures. (20 October)
22. Using cross-validation: mechanics and examples (22 October; notes forthcoming)
23. Using non-parametric smoothing: adaptive smoothing, testing parametric forms (24 October; notes forthcoming)
    Homework #5, due Friday, 31 October: assignment; solutions
24. Prediction trees 1: mostly regression trees, plus a "classification tree we can believe in" (27 October)
25. Prediction trees 2: classification trees (29 October and 3 November)
26. Bootstrapping, Bagging, and Random Forests (5 November)
27. Combining Predictive Models and the Power of Diversity (7 November)
28. Linear Classifiers and the Perceptron Algorithm (10 November)
29. Logistic Regression and Newton's Method (12 November)
    Homework #7, due Friday, 21 November: assignment; solutions
30. Neural Networks: The Mathematical Reality (14 November)
31. Neural Networks: The Biological Myth (17 November)
32. Support Vector Machines (19 November)
33. Support vector machines continued (21 November; same handout as previous)
    Homework #8, due Monday, 1 December: assignment; solutions
34. The Lecture Full of Fail: The wrong data, lying data, covariate shift, low base-rates and overwhelming false positives, response Waste, fraud and abuse (24 November)
    Homework #9, due 15 December: assignment; solutions

Corrupting the Young; Enigmas of Chance