Notebooks

## Independence Tests, Conditional Independence Tests, Measures of Dependence and Conditional Dependence

08 Dec 2018 11:57

My interest in graphical models and Markov models make me very interested in the following related problems:

• Given samples of two (possible high-dimensional) random variables $X$ and $Y$, are they independent?
• Given samples of three random variables $X$, $Y$ and $Z$, are $X$ and $Y$ conditionally independent given $Z$?
• How can we reliably calculate a measure of dependence from sample values?
• Ditto for a measure of conditional dependence.

See also: Density Estimation; Entropy Estimation (mutual information being a fine measure of dependence); Statistics; Two Sample Tests (since if $X|Y=y_1$ has a different distribution than $X | Y=y_2$, $X$ and $Y$ are dependent)

Recommended:
• Thomas B. Berrett, Richard J. Samworth, "Nonparametric independence testing via mutual information", arxiv:1711.06642
• Jochen Brocker, "A Lower Bound on Arbitrary $f$-Divergences in Terms of the Total Variation" arxiv:0903.1765
• Steve Fienberg, The Analysis of Cross-Classified Categorical Data
• Peter Hall, Jeff Racine and Qi Li, "Cross-Validation and the Estimation of Conditional Probability Densities", Journal of the American Statistical Association 99 (2004): 1015--1026 [PDF]
• Jeffrey D. Hart, Nonparametric Smoothing and Lack-of-Fit Tests [comments]
• Solomon W. Kullback, Information Theory and Statistics
• David Lopez-Paz, Philipp Hennig, Bernhard Schölkopf, "The Randomized Dependence Coefficient", arxiv:1304.7717
• Kun Zhang, Jonas Peters, Dominik Janzing, Bernhard Schölkopf, "Kernel-based Conditional Independence Test and Application in Causal Discovery", arxiv:1202.3775
Modesty forbids me to recommend:
• Daniel J. McDonald, CRS and Mark Schervish