Notebooks
http://bactra.org/notebooks
Cosma's NotebooksenState-Space Reconstruction
http://bactra.org/notebooks/2023/09/03#state-space-reconstruction
<P>An aspect of <a href="time-series.html">time series analysis</a>: given that
the time series came from a <a href="chaos.html">dynamical system</a>, figure
out the state space of that system from observation <em>alone</em>.
<P>Here's the basic set-up. Suppose we have a deterministic dynamical system
with state \( z(t) \) on a smooth manifold of dimension \( m \), evolving
according to a nice system of differential equations, \( \dot{z}(t) = f(z(t))
\). What we observe is not the state \( z(t) \) but rather a smooth,
instantaneous function of the state, \( x(t) = g(z(t)) \). Now, it should be
obvious that in this set-up \( z \) is only going to
be <a href="partial-identification.html">identified</a> up to a smooth change
of coordinates --- basically because we can use any coordinate system we like
on the hidden manifold, without changing anything at all. What is surprising
is that the system <em>can</em>, in fact, be identified up to a smooth,
invertible change of coordinates (i.e., a diffeomorphism).
<P>Fix a finite length of time \( \tau \) and a whole number \( k \), and set
\[
s(t) = \left(x(t), x(t-\tau), x(t-2\tau), \ldots x(t-(k-1)\tau)\right)
\]
<P>For generic choices of \( f, g \) and \( \tau \), if \( k \geq 2m+1 \) ,
then \( z(t) = \phi(s(t)) \). This \( \phi \) is smooth and invertible (a
diffeomorphism), and commutes with time-evolution, \( \frac{d}{dt}\phi(s(t)) = f(\phi(s(t))) \). Indeed, <a href="regression.html">regressing</a> \( \dot{s}(t) \) on \( s(t) \) will give
\( \phi^{-1} \circ f \).
<P>The first publication this subject was that by Packard et al. The
first <em>proof</em> that this can work was that of Takens, which remains the
standard reference. Note 8 in Packard et al. leads me to believe that the
idea may actually have originated with David Ruelle.
<P>I am especially interested in <a href="prediction-process.html">ways of
making this idea work for stochastic systems</a>.
<P>See also:
<a href="manifold-learning.html">Manifold Learning</a>;
<a href="equations-of-motion-from-a-time-series.html">Equations of Motion from a Time Series</a>;
<a href="prediction-process.html">Prediction Processes; Markovian (and Conceivably Causal) Representations of Stochastic Processes</a>
<ul>Recommended (big picture):
<li>Holger Kantz and Thomas Schreiber, <cite>Nonlinear Time Series
Analysis</cite> [An excellent presentation of the nonlinear dynamical systems
approach, which comes out of physics]
<li>Norman H. Packard, James P. Crutchfield, J. Doyne Farmer and Robert
S. Shaw, "Geomtry from a Time Series," <cite>Physical Review
Letters</cite> <strong>45</strong> (1980): 712--716
<li>David Ruelle, <cite>Chaotic Evolution and Strange Attractors: The
Statistical Analysis of Deterministic Nonlinear Systems</cite> [From notes
prepared by Stefano Isola]
<li>Floris Takens, "Detecting Strange Attractors in Fluid Turbulence",
pp. 366--381 in D. A. Rand and L. S. Young (eds.), <cite>Symposium on Dynamical
Systems and Turbulence</cite> (Springer Lecture Notes in Mathematics vol. 898;
1981)
</ul>
<ul>Recommended (close-ups):
<li>Markus Abel, Karsten Ahnert, Jürgen Kurths and Simon Mandelj,
"Additive nonparametric reconstruction of dynamical systems from time
series", <a href="http://dx.doi.org/10.1103/PhysRevE.71.015203"><cite>Physical
Review E</cite> <strong>71</strong> (2005): 015203</a> [Thanks to Prof.
Kürths for a reprint]
<li>Andrew M. Fraser and Harry L. Swinney, "Independent coordinates for strange attractors from mutual information", <a href="https://doi.org/10.1103/PhysRevA.33.1134"><cite>Physical Review A</cite> <strong>33</strong> (1986): 1134</a>
<li>Gershenfeld and Weigend (eds.), <cite>Time Series Prediction:
Forecasting the Future and Understanding the Past</cite>
<li>Kevin Judd, "Chaotic-time-series reconstruction by the Bayesian
paradigm: Right results by wrong methods,"
<citE>Physical Review E</cite> <strong>67</strong> (2003): 026212
<li>G. Langer and U. Parlitz, "Modeling parameter dependence from time
series", <a href="http://dx.doi.org/10.1103/PhysRevE.70.056217"><cite>Physical
Review E</cite> <strong>70</strong> (2004): 056217</a>
<li>Tim Sauer, James A. Yorke and Martin Casdagli, "Embedology",
<a href="https://doi.org/10.1007/BF01053745"><cite>Journal of Statistical Physics</cite> <strong>65</strong> (1991): 579--616</a>, <a href="https://www.santafe.edu/research/results/working-papers/embedology">SFI Working Paper 91-01-008</a>
<li>J. Stark, D. S. Broomhead, M. E. Davies and J. Huke, "Takens
embedding theorems for forced and stochastic systems",
<a href="http://dx.doi.org/10.1016/S0362-546X(96)00149-6"><cite>Nonlinear
Analysis</cite> <strong>30</strong> (1997): 5303--5314</a> [Unfortunately,
the stochastic case
is handled by treating it as forcing by a shift map on sequence space, which is
an infinite-dimensional space... Thanks to Martin
Nilsson Jacobi for telling me about this.]
<li>J. Timmer, H. Rust, W. Horbelt and H. U. Voss, "Parametric,
nonparametric and parametric modelling of a chaotic circuit time series,"
<a href="http://arxiv.org/abs/nlin/CD/0009040">nlin.cd/0009040</a>
</ul>
<ul>To read:
<li>Frank Boettcher, Joachim Peinke, David Kleinhans, Rudolf Friedrich,
Pedro G. Lind, and Maria Haase, "On the proper reconstruction of complex
dynamical systems spoilt by strong measurement
noise", <a href="http://arxiv.org/abs/nlin.CD/0607002">nlin.CD/0607002</a>
<li>Abraham Boyarsky and Pawel Gora, "Chaotic maps derived from
trajectory data", <cite>Chaos</cite> <strong>12</strong> (2002): 42--48
<li>Joseph L. Breeden and Alfred Hübler, "Reconstructing
Equations of Motion from Experimental Data with Unobserved Variables,"
<cite>Physical Review E</cite> <strong>42</strong> (1990): 5817--5826
<li>Cees Diks, <cite>Nonlinear Time Series Analysis: Methods and
Applications</cite>
<li>Sara P. Garcia and Jonas S. Almedia, "Multivariate phase space
reconstruction by nearest neighbor embedding with different time
delays", <citE>Physical Review E</cite> <strong>72</strong> (2006): 027205, <a href="http://arxiv.org/abs/nlin.CD/0609029">nlin.CD/0609029</a>
<li>Joachim Holzfuss, "Prediction of long-term dynamics from
transients", <a
href="http://dx.doi.org/10.1103/PhysRevE.71.016214"><cite>Physical Review
E</cite> <strong>71</strong> (2005): 016214</a> [State-space reconstruction by
experimentation, rather than just observation. Sounds very cool.]
<li>S. Ishii and M.-A. Sato, "Reconstruction of chaotic dynamics by
on-line EM algorithm," <cite>Neural Networks</cite> <strong>14</strong>
(2001): 1239--1256
<li>Kevin Judd and Tomomichi Nakamura, "Degeneracy of time series
models: The best model is not always the correct model", <a
href="http://dx.doi.org/10.1063/1.2213957"><cite>Chaos</cite>
<strong>16</strong> (2006): 033105</a>
<li>Claudia Lainscsek and Terrence J. Sejnowski, "Delay Differential Analysis of Time Series", <a href="http://dx.doi.org/10.1162/NECO_a_00706"><cite>Neural Computation</cite> <strong>27</strong> (2015): 594--614</a>
<li>A. P. Nawroth and J. Peinke, "Multiscale reconstruction of time
series", <a href="http://arxiv.org/abs/physics/0608069">physics/0608069</a>
<li>Louis M. Pecora, Linda Moniz, Jonathan Nichols, Thomas L. Carroll, "A Unified Approach to Attractor Reconstruction", <a href="http://arxiv.org/abs/0602048">arxiv:0602048</a>
<li>James C. Robinson, "A topological delay embedding theorem for
infinite-dimensional dynamical systems", <a
href="http://dx.doi.org/10.1088/0951-7715/18/5/013"><cite>Nonlinearity</cite> <strong>18</strong>
(2005): 2135--2143</a> ["A time delay reconstruction theorem inspired by that
of Takens ... is shown to hold for finite-dimensional subsets of
infinite-dimensional spaces, thereby generalizing previous results which were
valid only for subsets of finite-dimensional spaces."]
<li>Michael Small
<ul>
<li><cite>Applied Nonlinear Time Series Analysis:
Applications in Physics, Physiology and Finance</cite>
<li>"Optimal time delay embedding for nonlinear time
series modeling", <a
href="http://arxiv.org/abs/nlin.CD/0312011">nlin.CD/0312011</a>
</ul>
<li>Michael Small and C. K. Tse, "Optimal embedding parameters: A
modeling paradigm", <a
href="http://arxiv.org/abs/physics/0308114">physics/0308114</a>
<li>Ronen Talmon and Ronald R. Coifman, "Empirical intrinsic geometry for nonlinear modeling and time series filtering", <a href="http://dx.doi.org/10.1073/pnas.1307298110"><cite>Proceedings of the National Academy of Sciences</cite> (USA) <strong>110</strong> (2013): 12535--12540</a>
</ul>