Three-Toed Sloth

"Optimal Nonparametric Prediction and Automated Pattern Recognition in Dynamical Space-Time Systems" (or, Our New Filtering Techniques are Unstoppable!, Part II: The Rise of the Austrian Machines)

My student Georg M. Goerg, who I co-advise with Larry Wasserman, has just defended his thesis proposal:

"Optimal Nonparametric Prediction and Automated Pattern Recognition in Dynamical Space-Time Systems"

Abstract: Many methods in statistics, machine learning, and signal processing, such as speech analysis or pattern recognition in images and videos, try to extract informative structures from a dynamic system and remove noisy uninformative parts. Although such methods and algorithms work well in practice, they often do so because they have been specifically tuned to work in a very particular setting, and thus may break down when conditions and properties of the data do not hold anymore.

It would be very useful to have an automated pattern recognition method for dynamic system, which does not rely on any particular model or data structure, but gives informative patterns for any kind of system. Shalizi (2003) showed for discrete fields that an automated pattern discovery can be constructed by a characterization and classification of local conditional predictive distributions. The underlying idea is that statistically optimal predictors not only predict well but — for this very reason — also describe the data well, and therefore reveal informative structure inherent in the system.

In this thesis I extend previous work from Shalizi, Klinkner, and Haslinger (2004) to obtain a fully automated pattern recognition for continuous-valued space-time systems — such as videos — by means of optimal local prediction of the space-time field. Applications to simulated one-dimensional spatial dynamics and a real-world image pattern recognition demonstrate the usefulness and generality of the presented methods.

slides, full proposal (both somewhat large PDFs)

Very constant readers may recall having seen this line of research at various points down the years, most recently in "Our New Filtering Techniques Are Unstoppable!". Georg's goal is to make those methods work for continuous-valued fields, which was not needed for studying cellular automata but will be very handy for data analysis, and where already has some preliminary results. Beyond that, the goal is to develop the statistical theory which would go along with it and let us get things like confidence intervals on statistical complexity.

I can say without any shame that I was quite pleased with Georg's presentation, because I really had no part in making it; all the credit goes to him in the first place, and to provided by Larry, Chris Genovese, Cris Moore and Chad Schafer. Based on this experience, and Georg's publication record, I imagine he will have all the problems polished off by the NIPS deadline, with a monograph or two to follow by the end of the summer.

I will, however, try not to read any omens into my first Austrian student commencing a dissertation on automatic pattern discovery on the day Skynet declares war on humanity.

Kith and Kin; Enigmas of Chance; Complexity