Course Announcement: "Conceptual Foundations of Statistical Learning" (36-465/665, Spring 2021)
conservation notice: Self-promoting notice of a class at a university
you don't attend, on an arcane subject you're not that interested in, presuming
background you don't have.
Coming terrifyingly soon:
- Conceptual Foundations of Statistical Learning (36-465/665), Spring 2021
- Description: This course is an introduction to the core ideas and
theories of statistical learning, and their uses in designing and analyzing
machine-learning systems. Statistical learning theory studies how to fit
predictive models to training data, usually by solving an optimization problem,
in such a way that the model will predict well, on average, on new data. The
course will focus on the key concepts and theoretical tools, at a mathematical
level accessible to students who have
taken 36-401, "modern
regression" (or equivalent) and its pre-requisites. The course will also
illustrate those concepts and tools by applying them to carefully selected
kinds of machine learning systems (such as kernel machines).
- Time and place: Tuesdays and Thursdays 2:20--3:40 pm, Pittsburgh Time, via Zoom
- Pre-requisites: Undergraduates taking the course as 36-465 must have a C or better in 36-401. Graduate students taking it as 36-665 are expected to have similar background in the theory and practice of linear regression models, linear algebra, mathematical statistics, probability, and calculus in multiple variables.
- Topics in brief (subject to revision): Prediction as a decision problem; elements of statistical decision theory; "risk"; "probably approximately correct"; optimizing on training data; the origins of over-fitting; deviation inequalities; uniform convergence and concentration inequalities; measures of model complexity (Rademacher complexity, VC dimension, etc.); "algorithmic stability" arguments; optimizing noisy functions; regularization and its effects on model complexity; model selection; kernel machines; random-feature machines; mixture models; and some combination of stochastic-process prediction, sequential decision-making/reinforcement learning, and low-regret ("on-line") learning.
- This course vs. alternatives: Students wanting exposure to a broad
range of learning algorithms and their applications would be better served by
especially 36-462/662 ("data
mining", "methods of statistical learning"), 10-301/601 ("introduction to
machine learning") or 10-701 ("introduction to machine learning" for
Ph.D. students). This class is for those who want a deeper understanding of
the underlying principles. It will mean a lot more math than coding, and it
won't help you move up a leader-board, but it will help you understand the
statistical reasons why learning machines work (when they do).
I have until classes begin on 1 February to figure out how I am actually
going to make this happen.
Corrupting the Young;
Enigmas of Chance
Posted at November 24, 2020 18:25 | permanent link