Three-Toed Sloth

Course Announcement: "Conceptual Foundations of Statistical Learning" (36-465/665, Spring 2021)

Attention conservation notice: Self-promoting notice of a class at a university you don't attend, on an arcane subject you're not that interested in, presuming background you don't have.

Coming terrifyingly soon:

Conceptual Foundations of Statistical Learning (36-465/665), Spring 2021

Description: This course is an introduction to the core ideas and theories of statistical learning, and their uses in designing and analyzing machine-learning systems. Statistical learning theory studies how to fit predictive models to training data, usually by solving an optimization problem, in such a way that the model will predict well, on average, on new data. The course will focus on the key concepts and theoretical tools, at a mathematical level accessible to students who have taken 36-401, "modern regression" (or equivalent) and its pre-requisites. The course will also illustrate those concepts and tools by applying them to carefully selected kinds of machine learning systems (such as kernel machines).

Time and place: Tuesdays and Thursdays 2:20--3:40 pm, Pittsburgh Time, via Zoom

Pre-requisites: Undergraduates taking the course as 36-465 must have a C or better in 36-401. Graduate students taking it as 36-665 are expected to have similar background in the theory and practice of linear regression models, linear algebra, mathematical statistics, probability, and calculus in multiple variables.

Topics in brief (subject to revision): Prediction as a decision problem; elements of statistical decision theory; "risk"; "probably approximately correct"; optimizing on training data; the origins of over-fitting; deviation inequalities; uniform convergence and concentration inequalities; measures of model complexity (Rademacher complexity, VC dimension, etc.); "algorithmic stability" arguments; optimizing noisy functions; regularization and its effects on model complexity; model selection; kernel machines; random-feature machines; mixture models; and some combination of stochastic-process prediction, sequential decision-making/reinforcement learning, and low-regret ("on-line") learning.

This course vs. alternatives: Students wanting exposure to a broad range of learning algorithms and their applications would be better served by other courses, especially 36-462/662 ("data mining", "methods of statistical learning"), 10-301/601 ("introduction to machine learning") or 10-701 ("introduction to machine learning" for Ph.D. students). This class is for those who want a deeper understanding of the underlying principles. It will mean a lot more math than coding, and it won't help you move up a leader-board, but it will help you understand the statistical reasons why learning machines work (when they do).

I have until classes begin on 1 February to figure out how I am actually going to make this happen.

Corrupting the Young; Enigmas of Chance