Notebooks
http://bactra.org/notebooks
Cosma's NotebooksenStochastic Differential Equations
http://bactra.org/notebooks/2021/04/16#stoch-diff-eqs
<P>Non-stochastic differential equations are models
of <a href="chaos.html">dynamical systems</a> where the state evolves
continuously in time. If they are autonomous, then the state's future values
depend only on the present state; if they are non-autonomous, it is allowed to
depend on an exogeneous "driving" term as well. (This may not be the standard
way of putting it, but I think it's both correct and more illuminating than the
more analytical viewpoints, and anyway is the line taken by V. I. Arnol'd in
his excellent book on differential equations.) Stochastic differential
equations (SDEs) are, conceptually, ones where the the exogeneous driving term
is a <a href="stochastic-processes.html">stochatic process</a>. --- While
"differential equation", unmodified, covers both ordinary differential
equations, containing only time derivatives, and partial differential
equations, containing both time and space derivatives, "stochastic differential
equation", unmodified, refers only to the ordinary case. Stochastic partial
differential equations are just what you'd think.
<P>The solution of an SDE is, itself, a stochastic process. To understand how
this comes about, it helps to start by thinking through what an ordinary
differential equation is, what constitutes a solution, and how to find a
solution. The canonical sort of autonomous ordinary differential equation
looks like
\[
\frac{dx}{dt} = f(x)
\]
or
\[
\frac{dx}{dt} = f(x,t)
\]
if it's non-autonomous. (I won't keep noting all the modifications for
non-autonomous equations when they're clear.) Here $f$ is the "vector field"
which gives the rate of change of the system's variables, either as a function
of their current value (autonomous), or their current value and the time (non-autonomous). A solution to the equation is a function
$x(t)$ whose time derivative matches the field:
\[
\frac{dx}{dt}(t) = f(x(t))
\]
Now, you'll recall from calculus that $x(t)$ and $x(t)+c$ have the same time
derivative, for any constant $c$. This means that the solution to an ODE isn't
unique. We typically add on an extra requirement, such as an initial condition,
\[
x(0) = x_0
\]
to make the solution unique (and to represent the initial state of the system).
<P>Finding a solution to the ODE means finding such a function, which in
principle we can do by integrating. This is easiest to see if the equation
isn't just non-autonomous, but doesn't depend on $x$ at all,
\[
\frac{dx}{dt} = f(t)
\]
Then we integrate both sides over time, from $0$ to our time of interest $t$;
I'll write the variable of integration as $s$ to keep it distinct:
\[
\begin{eqnarray*}
\int_{0}^{t}{\frac{dx}{dt} ds} & = & \int_{0}^{t}{f(s) ds}\\
x(t) - x(0) & = & \int_{0}^{t}{f(s) ds}\\
x(t) & = & x_0 + \int_{0}^{t}{f(s) ds}
\end{eqnarray*}
\]
using the fundamental theorem of calculus, and the initial condition. Even if the equation isn't completely externally driven, we can still in
principle do the same thing:
\[
x(s) = x_0 + \int_{0}^{t}{f(x(s), s) ds}
\]
<P>We can actually <em>calculate</em> such a thing by using many different
numerical schemes. One which is particularly helpful to going forward
is Euler's method. Pick a small increment of time $h$. We
start with $x(0) = x_0$. Then we say
\[
x(t+h) = x(t) + h f(x(t))
\]
for the points $t=0$, $t=h$, $t=2h$, etc. In between those points we
interpolate linearly. This gives us a function which doesn't quite obey the
different equation, but one can show that it comes closer and closer to doing
so as $h \rightarrow 0$.
<P>Now let's thing about making all this stochastic. The easiest thing to
do is to add some sort of stochastic noise on the right-hand side of the
differential equation:
\[
\frac{dX}{dt}(t) = f(X(t)) + Z(t)
\]
A solution will, once again, be a function $X$ which satisfies this equation.
Since $Z$ is a random function of time, $X$ will also be random. But if we
could somehow fix $Z$, we'd just be back to the kind of equation we know how
to solve:
\[
X(t) = x_0 + \int_{0}^{t}{f(X(s)) ds} + \int_{0}^{t}{Z(t) dt}
\]
<P>To make sense of such an expression <em>in general</em>, we need to know how
to integrate stochastic processes. In particular, we need to understand when
one process, say $\zeta(t)$, is the time-integral of another, say $Z(t)$, and
vice versa. If we know that, then we can actually use Euler's method
to find solutions. We would, once again, start with $X(t) = x_0$,
set
\[
X(t+h) = X(t) + h f(X(t)) + \zeta(t+h) - \zeta(t)
\]
and linearly interpolate between the points $t=0$, $t=h$, $t=2h$, etc. We
then let $h\rightarrow 0$ to recover an exact solution.
<P>In fact, so long as expressions like this last one make sense, we could use
them to <em>define</em> what it means for one stochastic process, say $Z(t)$,
to be the time-derivative of another, $\zeta(t)$. This may seem crazy or, more
politely, of merely mathematical interest, but there are lots of situations
where $\zeta(t)$ is a much better behaved process than is $Z(t)$. The premier
example is when $\zeta(t)$ is the Gaussian process called the "Wiener process"
or (slightly inaccurately) "Brownian motion", defined by $W(0) = 0$, $W(t) \sim
\mathcal{N}(0, t)$ (i.e., a Gaussian or "Normal" distribution with mean 0 and
variance $t$), and the "increment" $W(t) - W(s)$ being statistically
independent of the increment over any other (non-overlapping) interval of time.
If we try to compute the time-derivative of $W(t)$, we find that it is
almost-surely ill-defined <a href="#no-deriv-for-wiener"
name="deriv-for-wiener">[1]</a>. You might say "Well, then maybe we shouldn't
use the Wiener process as something giving us noise in a differential
equation", but look at what it means in the Euler scheme: over the time
interval from $t$ to $t+h$, the trajectory moves from $X(t)$ to $X(t) +
hf(X(t)) + W(t+h) - W(t)$. That is to say, the the process follows the
deterministic differential equation, <em>plus</em> a little Gaussian random
kick --- that seems very natural! It's even natural that the variance of the
Gaussian perturbation should scale with $h$, since that's what we'd see in,
say, a random walk...
<P>So there should be <em>some</em> way of making sense of seeing a Wiener
process as an integral. There is, in fact, a whole theory of stochastic
integrals, developed in the 1940s, by M. Loeve, K. Ito, and R. Stratonovich
(all building on earlier work by, among
others, <a href="wiener.html">N. Wiener</a>). The theory of SDEs more strictly
is largely owed to Ito and Stratonovich (in two slightly different forms,
corresponding to subtle differences between Euler methods).
<P>Most of what one encounters, in applications, as the theory of SDEs assumes
that the driving noise is in fact white noise, i.e., Gaussian and uncorrelated
over time. On the one hand, this is less of a restriction than it might seem,
because many other natural sorts of noise process can be represented as
stochastic integrals of white noise. On the other hand, the same mathematical
structure can be used directly to define stochastic integrals and stochastic
DEs driven by a far broader class of stochastic processes; on this topic
Kallenberg is a very good introduction.
<P><font size="-1"><a name="no-derive-for-wiener">[1]</a>: The argument is so pretty that I can't resist repeating it.
The time-derivative is of course \( \lim_{h\rightarrow 0}{\frac{W(t+h) - W(t)}{h}} \). By the defining properties of the Wiener process, the numerator inside the
limit is a Gaussian random variable with expectation 0 and variance $h$. The
ratio in the limit is therefore a Gaussian random variable with expectation 0
and variance $1/h$, so the variance will blow up to infinity as $h$ shrinks.
Moreover, for any other time $s$, no matter how close $s$ might be to $t$,
eventually the intervals $(s, s+h)$ and $(t, t+h)$ will not over-lap, so
the increments will be statistically independent. <a href="#deriv-for-wiener">^</a></font>
<P>See also:
<a href="sde-inference.html">Inference for Stochastic Differential Equations</a>;
<a href="inference-markov.html">Inference for Markov and Hidden Markov Models</a>;
<a href="markov.html">Markov Models</a>;
<a href="classical-path-integrals.html">Path Integrals and Feynman Diagrams for Classical Stochastic Processes</a>;
<ul>Recommended, more introductory:
<li>Geoffrey Grimmett and David Stirzaker, <cite>Probability and Random
Processes</cite> [The last chapter of the 3rd edition has a good, if rather
heuristic, first-glimpse look at SDEs. I can't recall if it appeared in
earlier editions or not.]
<li>Josef Honerkamp, <cite>Stochastic Dynamical Systems</cite>
<li>Joel Keizer, <cite>Statistical Thermodynamics of Nonequilibrium
Processes</cite> [Describes SDEs, in the Ito framework, from a heuristic
viewpoint, motivated by, precisely, the need to model non-equilibrium
thermodynamic processes; goes on to use them in many interesting physical
applications.]
<li>Andrzej Lasota and Michael C. Mackey, <cite>Chaos, Fractals and
Noise: Stochastic Aspects of Dynamics</cite> [The later chapters give
a solid introduction to SDEs, starting from the Euler-Bernstein approach, but to
my mind somewhat slighting the quite real advantages of the Ito calculus
for more advanced problems]
<li><a href="http://www.eng.tau.ac.il/~liptser/">Robert
S. Liptser</a>, <cite><a
href="http://www.eng.tau.ac.il/~liptser/list.html">Lectures on Stochastic
Processes</a></cite> [See especially <a
href="http://www.eng.tau.ac.il/~liptser/lectures/lect_new10.pdf">lecture
10</a>, on white noise,
and <a href="http://www.eng.tau.ac.il/~liptser/lectures/lect_new12.pdf">lecture
12</a>, on Ito integrals.]
<li>Bernt Oksendal, <cite>Stochastic Differential Equations</cite>
[First-rate introductory textbook, loaded with examples and
intuition-building.]
</ul>
<ul>Recommended, more advanced:
<li>David R. Brillinger, "The 2005 Neyman Lecture: Dynamic Indeterminism in Science", <a href="http://dx.doi.org/10.1214/07-STS246"><cite>Statistical Science</citE>
<strong>23</strong> (2008): 48--64</a>, <a href="http://arxiv.org/abs/0808.0620">arxiv:0808.0620</a> [With discussions and response]
<li>David R. Brillinger, Brent S. Stewart, Charles L. Littnan, "Three
months journeying of a Hawaiian monk
seal", <a href="http://dx.doi.org/10.1214/193940307000000473">pp. 246--264 of
Deborah Nolan and Terry Speed (eds.), <cite>Probability and Statistics: Essays
in Honor of David A. Freedman</citE> (2008)</a>,
<a href="http://arxiv.org/abs/0805.3019">arxiv:0805.3019</a> [A pretty
application]
<li>I. I. Gikhman and A. V. Skorokhod, <cite>Introduction to the Theory
of Random Processes</cite> [Excellent long chapter on stochastic integrals and
SDEs; these authors went on to publish several more books on SDEs, but I
confess I have not read them.]
<li>Olav Kallenberg, <cite>Foundations of Modern Probability</cite>
[Builds up the theory of stochastic integrals and stochastic differential
equations from scratch, ending with a very general framework which makes it
clear just which parts of the original approach, tied to the Wiener process,
were necessary and which were accidental. However, Kallenberg's book is
intended as a comprehensive textbook on <a href="probability.html">probability theory</a>, from measure theory
through <a href="large-deviations.html">large deviations</a>. This means that
it is both mathematically demanding, and that he takes a "spiral" approach,
revisitng this topic, like many others, repeatedly through the text. There
are, however, abundant cross-references.]
<li>Robert S. Liptser and Albert N. Shiryaev, <cite>Statistics of
Random Processes</cite> [Vol. I gives a very detailed account of the classical,
Wiener-process theory and its uses in <a href="filtering.html">optimal
filtering</a>; vol. II considers numerous applications in statistics and
signal-processing, as well as some generalizations and extensions.]
<li>Michel Loeve, <cite>Probability Theory</cite> [Gives
a very elegant account of Loeve's contributions to the theory of
stochastic integrals]
<li>L. C. G. Rogers and D. Williams, <cite>Diffusions, Markov
Processes, and Martingales</cite> [See especially Vol. II, <cite>Ito
Calculus</cite>.]
</ul>
<ul>Modesty forbids me to recommend:
<li>CRS with
A. Kontorovich, <cite><a
href="http://www.stat.cmu.edu/~cshalizi/almost-none">Almost None of the Theory
of Stochastic Processes</a></cite> [A fairly large chunk of this manuscript is
devoted to stochastic integrals and SDEs.]
</ul>
<ul>To read:
<li>Lakhdar Aggoun and Robert Elliott, <cite>Measure Theory and
Filtering: Introduction with Applications</cite>
<li>David Applebaum, <cite><a href="http://cambridge.org/9780521832632">Lévy Processes and Stochastic
Calculus</a></cite>
<li>Ari Arapostathis, Vivek S. Borkar, Mrinal K. Ghosh,
<cite><a href="http://cambridge.org/9780521768405">Ergodic Control of Diffusion Processes</a></cite>
<li>Yuri Bakhtin and Jonathan C. Mattingly, "Stationary Solutions of
Stochastic Differential Equation with Memory and Stochastic Partial
Differential Equations", <a
href="http://arxiv.org/abs/math.PR/0509166">math.PR/0509166</a>
<li>Viorel Barbu, Philippe Blanchard, Giuseppe Da Prato, Michael
Röckner, "Self-organized criticality via stochastic partial differential
equations",
<a href="http://arxiv.org/abs/0811.2093">arxiv:0811.2093</a>
<li>Ole E. Barndorff-Nielsen and Albert Shiryaev, <cite><a href="http://www.worldscientific.com/worldscibooks/10.1142/9609">Change of Time and Change of Measure</a></cite>
<li>François Bolley, Ivan Gentil, Arnaud Guillin, "Convergence
to equilibrium in Wasserstein distance for Fokker-Planck equations",
<a href="http://dx.doi.org/10.1016/j.bbr.2011.03.031"><cite>Journal of Functional Analysis</cite> <strong>263</strong> (2012): 2430--2457</a>, <a href="http://arxiv.org/abs/1110.3606">arxiv:1110.3606</a>
<li>Nicolas Bouleau and Dominique Lépngle, <cite>Numerical
Methods for Stochastic Process</cite>
<li>A. A. Budini and M.O. Caceres, "Functional characterization of
generalized Langevin
equations", <a href="http://arxiv.org/abs/cond-mat/0402311">cond-mat/0402311</a>
["exact functional formalism to deal with linear Langevin equations with
arbitrary memory kernels and driven by any noise structure characterized
through its characteristic functional..."]
<li>Alberto Chiarini, Markus Fischer, "On large deviations for small noise Ito processes", <a href="http://dx.doi.org/10.1239/aap/1418396246"><cite>Advances in Applied Probability</cite> <strong>46</strong> (2014): 1126--147</a>, <a href="http://arxiv.org/abs/1212.3223">arxiv:1212.3223</a>
<li>Carson C. Chow, Michael A. Buice, "Path Integral Methods for Stochastic Differential Equations", <a href="http://arxiv.org/abs/1009.5966">arxiv:1009.5966</a>
<li>Emmanuelle Clément, Arturo Kohatsu-Higa, Damien Lamberton,
"A duality approach for the weak approximation of stochastic differential
equations", <a href="http://arxiv.org/abs/math.PR/0610178">math.PR/0610178</a>
= <cite>Annals of Applied Probability</cite> <strong>16</strong> (2006):
1124--1154 ["a new methodology to prove weak approximation results for general
stochastic differential equations. Instead of using a partial differential
equation approach as is usually done for diffusions, the approach considered
here uses the properties of the linear equation satisfied by the error
process"]
<li>Jacky Cresson and Sébastien Darses, "Stochastic embedding of
dynamical
systems", <a href="http://arxiv.org/abs/math.PR/0509713">math.PR/0509713</a>
<li>A. M. Davie, "Uniqueness of solutions of stochastic differential equations", <a href="http://arxiv.org/abs/0709.4147">arxiv:0709.4147</a>
<li>Freddy Delbaen, Jinniao Qiu, Shanjian Tang, "Forward-Backward Stochastic Differential Systems Associated to Navier-Stokes Equations in the Whole Space", <a href="http://arxiv.org/abs/1303.5329">arxiv:1303.5329</a>
<li>Peter Friz and Nicolas B. Victoir, <cite><a href="http://cambridge.org/9780521876070">Multidimensional Stochastic Processes as Rough Paths:
Theory and Applications</a></cite>
<li>Jorge Garcia, "A Large Deviation Principle for Stochastic
Integrals",
<a href="http://dx.doi.org/10.1007/s10959-007-0136-4"><cite>Journal of
Theoretical Probability</cite>
<strong>21</strong> (2008): 476--501</a>
<li>Leszek Gawarecki and Vidyadhar Mandrekar, <cite><a href="http://www.springer.com/book/978-3-642-16193-3">Stochastic Differential Equations in Infinite Dimensions, with Applications to Stochastic Partial Differential Equations</a></cite>
<li>Martin Hairer, "Exponential Mixing Properties of Stochastic PDEs
Through Asymptotic Coupling," <a
href="http://arXiv.org/abs/math/0109115">math.PR/0109115</a>
<li>David Hochberg, Carmen Molina-Paris, Juan Pérez-Mercader and
Matt Visser, "Effective Action for Stochastic Partial Differential
Equations," <a
href="http://arxiv.org/abs/cond-mat/9904215">cond-mat/9904215</a>
<li>Helge Holden, <cite>Stochastic Partial Differential
Equations: A Modeling, White Noise Functional Approach</cite>
<li>Xiangping Hu, Daniel Simpson, Finn Lindgren, Havard Rue, "Multivariate Gaussian Random Fields Using Systems of Stochastic Partial Differential Equations", <a href="http://arxiv.org/abs/1307.1379">arxiv:1307.1379</a>
<li>Yoshifusa Ito and Izumi Kubo, "Calculus on Gaussian and Poisson
White Noises", <a href="https://projecteuclid.org/euclid.nmj/1118781051"><cite>Nagoya Mathematical Journal</cite>
<strong>111</strong> (1988): 41--84</a>
<li>Gopinath Kallianpur and Jie Xiong, <cite><a href="http://projecteuclid.org/euclid.lnms/1215451864">Stochastic
Differential Equations in Infinite Dimensional Spaces</a></cite>
<li>Hye-Won Kang, Thomas G. Kurtz, Lea Popovic, "Central limit theorems and diffusion approximations for multiscale Markov chain models", <a href="http://arxiv.org/abs/1208.3783">arxiv:1208.3783</a>
<li>Ioannis Kontoyiannis, Sean P. Meyn, "Approximating a Diffusion by a Hidden Markov Model", <a href="http://arxiv.org/abs/0906.0259">arxiv:0906.0259</a>
<li>Peter Kotelenez, <cite><a href="http://www.springer.com/math/probability/book/978-0-387-74316-5">Stochastic Ordinary and Stochastic Partial Differential Equations: Transition from Microscopic to Macroscopic Equations</a></cite>
<li>Peter M. Kotelenez and Thomas G. Kurtz, "Macroscopic limits for stochastic partial differential equations of McKean-Vlasov type", <a href="http://dx.doi.org/10.1007/s00440-008-0188-0"><cite>Probability Theory and Related Fields</cite> <strong>146</strong>
(2010): 189--222</a>
<li>H. Kunita, <cite><a href="http://cambridge.org/0521599253">Stochastic Flows and Stochastic Differential Equations</a></cite>
<li>Kai Liu, <cite><a href="http://cambridge.org/9781108705172">Stochastic Stability of Differential Equations in Abstract Spaces</a></cite>
<li>S. V. Lototsky and B. L. Rozovskii
<ul>
<li>"Wiener Chaos Solutions of Linear Stochastic Evolution
Equations", <a href="http://arxiv.org/abs/math.PR/0504558">math.PR/0504558</a>
<li>"Stochastic Differential Equations: A Wiener Chaos
Approach", <a href="http://arxiv.org/abs/math.PR/0504559">math.PR/0504559</a>
</ul>
<li>Yutao Ma, Ran Wang, Liming Wu, "Moderate Deviation Principle for dynamical systems with small random perturbation", <a href="http://arxiv.org/abs/1107.3432">arxiv:1107.3432</a>
<li>Jonathan C. Mattingly, Andrew M. Stuart, M.V. Tretyakov, "Convergence of Numerical Time-Averaging and Stationary Measures via Poisson Equations", <a href="http://arxiv.org/abs/0908.4450">arxiv:0908.4450</a>
<li>Anatolii V. Mokshin, Renat M. Yulmetyev, and Peter Hänggi,
"Simple Measure of Memory for Dynamical Processes Described by a Generalized
Langevin Equation", <a
href="http://dx.doi.org/10.1103/PhysRevLett.95.200601"><cite>Physical Review
Letters</cite> <strong>95</strong> (2005): 200601</a>
<li>Esteban Moro and Henri Schurz, "Non-negativity preserving numerical
algorithms for stochastic differential
equations", <a href="http://arxiv.org/abs/math.NA/0509724">math.NA/0509724</a>
<li>Cyril Odasso, "Exponential mixing for stochastic PDEs: the
non-additive
case", <a href="http://dx.doi.org/10.1007/s00440-007-0057-2"><cite>Probability
Theory and Related Fields</cite> <strong>140</strong> (2008): 41--82</a>
<li>Fabien Panloup, "Recursive computation of the invariant measure of
a stochastic differential equation driven by a L\'{e}vy
process", <a href="http://arxiv.org/abs/math.PR/0509712">math.PR/0509712</a>
<li>S. Peszat and J. Zabczyk, <cite><a href="http://cambridge.org/9780521879897">Stochastic Partial Differential Equations with Lévy Noise: An evolution Equation Approach</a></cite>
<li>Philip Protter, <cite>Stochastic Integration and Differential
Equations</cite>
<LI>A. J. Roberts, "Normal form transforms separate slow and fast modes in stochastic dynamical systems", <a href="http://arxiv.org/abs/math.DS/0701623">math.DS/0701623</a>
<li>Simo Särkkä and Arno Solin, <cite><a href="http://cambridge.org/9781316649466">Applied Stochastic Differential Equations</a></cite>
<li>Alexander Sokol, Niels Richard Hansen, "Causal interpretation of stochastic differential equations", <a href="http://dx.doi.org/10.1214/EJP.v19-2891"><cite>Electronic Journal of Probability</cite> <strong>19</strong> (2014): 100</a>, <a href="http://arxiv.org/abs/1304.0217">arxiv:1304.0217</a>
<li>Arne Traulsen, Jens Christian Claussen, Christoph Hauert,
"Stochastic differential equations for evolutionary dynamics with demographic
noise and mutations", <a href="http://arxiv.org/abs/1203.3367">arxiv:1203.3367</a>
<li>Ramon van Handel, "Almost Global Stochastic Stability", <a
href="http://arxiv.org/abs/math.PR/0411311">math.PR/0411311</a> ["We develop a
method to prove almost global stability of stochastic differential equations in
the sense that almost every intial point ... is asymptotically attracted to the
origin with unit probability."]
<li>Wei Wang and Jinqiao Duan, "Invariant manifold reduction and
bifurcation for stochastic partial differential
equations", <a href="http://arxiv.org/abs/math.DS/0607050">math.DS/0607050</a>
<li>Wei Wang, A. J. Roberts and Jinqiao Duan, "Large deviations for
slow-fast stochastic partial differential
equations", <a href="http://arxiv.org/abs/1001.4826">arxiv:1001.4826</a> ["the
rate function is exactly that of the averaged equation plus the fluctuating
deviation which is a stochastic partial differential equation with small
Gaussian perturbation"]
</ul>