## Hilbert Space Methods for Statistics and Probability

*15 May 2019 13:45*

For representing possible regression functions (very natural); for representing possible probability distributions in density estimation, independence tests, and two-sample tests (less natural but important).

See also: Basis Selection in Function Decomposition; Math I Ought to Learn

- Recommended, big picture:
- Bharath K. Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard
Schölkopf, Gert R.G. Lanckriet, "Hilbert Space Embeddings and Metrics on
Probability
Measures", Journal
of Machine Learning Research
**11**(2010): 1517--1561 - Grace Wahba, Spline Models for Observational Data

- Recommended, close ups:
- Eduardo Corona, Terran Lane, Curtis Storlie, Joshua Neil, "Using Laplacian Methods, RKHS Smoothing Splines and Bayesian Estimation as a framework for Regression on Graph and Graph Related Domains" [Technical report, University of New Mexico Computer Science, 2008-06, PDF]
- Kenji Fukumizu, Le Song, Arthur Gretton, "Kernel Bayes' rule", arxiv:1009.5736

- To read:
- Nancy Heckman, "The theory and application of penalized methods or Reproducing Kernel Hilbert Spaces made easy", arxiv:1111.1915
- Alexander Jung, Sebastian Schmutzhard, Franz Hlawatsch, "The RKHS Approach to Minimum Variance Estimation Revisited: Variance Bounds, Sufficient Statistics, and Exponential Families", arxiv:1210.6516
- Vern I. Paulsen and Mrinal Raghupathi, An Introduction to the Theory of Reproducing Kernel Hilbert Spaces `
- James Robins and Aad van der Vaart, "Adaptive
nonparametric confidence sets", Annals of Statistics
**34**(2006): 229--253, arxiv:math/0605473 ["We construct honest confidence regions for a Hilbert space-valued parameter in various statistical models. The confidence sets can be centered at arbitrary adaptive estimators, and have diameter which adapts optimally to a given selection of models."] - Christopher G. Small and D. L. McLeish, Hilbert Space Methods in Probability and Statistical Inference
- Bharath K. Sriperumbudur, Kenji Fukumizu, Gert R. G. Lanckriet, "Universality, Characteristic Kernels and RKHS Embedding of Measures", arxiv:1003.0887