Notebooks

## Data Mining

22 Sep 2015 19:41

I've taught a course on this, so I ought to be able to describe it, oughtn't I? Data mining, more stuffily "knowledge discovery in databases", is the art of finding and extracting useful patterns in very large collections of data. It's not quite the same as machine learning, because, while it certainly uses ML techniques, the aim is to directly guide action (praxis!), rather than to develop a technology and theory of induction. In some ways, in fact, it's closer to what statistics calls "exploratory data analysis", though with certain advantages and limitations that come from having really big data to explore.

Kernel methods probably deserve their own notebook.

Recommended, close-ups:
• Gavin Brown, Adam Pocock, Ming-Jie Zhao, Mikel Luján, "Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection", Journal of Machine Learning Research 13 (2012): 27--66
• Sharad Goel, Jake M. Hofman, Sébastien Lahaie, David M. Pennock, and Duncan J. Watts, "Predicting consumer behavior with Web search", Proceedings of the National Academy of Sciences (USA) 107 (2010): 17486--17490 [A case study in using data mining, while recognizing limitations]
• Aleks Jakulin and Ivan Bratko, "Quantifying and Visualizing Attribute Interactions", cs.AI/0308002
• Jon Kleinberg, Christos Papadimitriou and Prabhakar Raghavan, "A Microeconomic View of Data Mining", Data Mining and Knowledge Discovery 2 (1998) [PDF]
• Ariel Kleiner, Ameet Talwalkar, Purnamrita Sarkar, Michael I. Jordan, "A Scalable Bootstrap for Massive Data", arxiv:1112.5016
• Kling, Scherson and Allen, "Parallel Computing and Information Capitalism," in Metropolis and Rota (eds.), A New Era in Computation (1992) [A batch of UC Irvine comp. sci. professors who write like sociologists. " `Information capitalism' refers to forms of organization in which data-intensive techniques and computerization are key strategic resources for corporate production."]
• Erik Larson, The Naked Consumer: How Our Private Lives Become Public Commodities
• R. Dean Malmgren, Jake M. Hofman, Luis A. N. Amaral, Duncan J. Watts, "Characterizing Individual Communication Patterns", arxiv:0905.0106
• John Shawe-Taylor and Nello Cristianini, Kernel Methods for Pattern Analysis
• Ryan J. Tibshirani, "Degrees of Freedom and Model Search", arxiv:1402.1920
• Yong Wang, Ilze Ziedins, Mark Holmes, Neal Challands, "Tree Models for Difference and Change Detection in a Complex Environment", Annals of Applied Statistics 6 (2012): 1162--1184, arxiv:1202.1561 [In an ordinary classification tree, we are interested in the distribution of the class labels $Y$ given the predictors $X$, i.e., $\Pr(Y|X)$, and make splits on $X$ so that (in essence) the conditional entropy $H[Y|X]$ becomes small. This is of course equivalent to making splits so that the divergence of $Pr(Y|X)$ from $Pr(Y)$ is maximized. What they are interested in is not classification but describing how the different classes are distinct, so the relevant distribution is $Pr(X|Y)$, and they want a big divergence between $Pr(X)$ and $Pr(X|Y)$.]
• Jianming Ye, "On Measuring and Correcting the Effects of Data Mining and Model Selection", Journal of the American Statistical Association 93 (1998): 120--131