Three-Toed Sloth

Books to Read While the Algae Grow in Your Fur, November 2022

Attention conservation notice: I have no taste, and no qualifications to expound on optimal control theory.

(Left almost-finished in 2022, because I got interrupted, and posted in 2026, because I wanted to procrastinate about half-a-dozen research projects.)

Martin L. Weitzman, Income, Wealth, and the Maximum Principle [doi:10.2307/j.ctv1pncrxj]

Enid R. Pinch, Optimal Control and the Calculus of Variations [doi:10.1093/oso/9780198532170.001.0001]

Daniel Liberzon, Calculus of Variations and Optimal Control Theory: A Concise Introduction [doi:10.2307/j.ctvcm4g0s]

I read three books on optimal control in a month for two reasons:

1. For the last two years, I've included big segments of explicit optimization theory in my spring undergraduate courses, rather than trusting the math pre-reqs to handle it, and I wanted to see what I could assign that would fit my audience;
2. A key calculation in an on-going research project is an optimal control problem, and I'm frankly rusty because I haven't used it in two decades.

(I retain enough of the mental habits of a physicist that I initially dealt with (2) by assuming that the functions involved were simultaneously piecewise-constant and continuously differentiable, and that it's always fine to Taylor-expand the argument to an exponential and truncate the series at first order. But now that I know the answer, conscience and the threat of referees alike prompt me to justify it.)

Suppose we are faced with a control problem, where we want to maximize (or minimize) some function of the whole trajectory \( x: [0,T] \mapsto \mathbb{R}^d \), which can be expressed as an integral over time, e.g., we want to pick a control process \( u: [0,T] \mapsto \mathbb{R}^{q} \) so as to maximize \( J[x,u] = \int_{0}^{T}{f(t, x(t), u(t)) dt} \) where \( \dot{x}(t) = g(x(t), u(t)) \) and \( x(0) = x_0 \). This is however subject to some constraints on the control signals. We are thus trying to maximize a functional of the whole trajectory of the state process. To oversimplify, the "maximum principle" of Pontryagin says that the optimal trajectory is one where the control \( u(t) \) is picked so as to maximize a certain instantaneous of the state \( x(t) \) and its derivatives, and gives a prescription for constructing this function out of the integrand \( f \), the response function \( g \), and the constraints. Maximizing this function, say \( H \), becomes (in principle) just a calculus exercise, and we can write down differential equations for the evolution of both the state and the control in terms of derivatives of \( H \). (If you know enough to have some analysis quibbles at this point, please accept that I am aware of all the ways in which that is a "lie told to children".)

In his little book, aimed at economics students, Weitzman introduces the reader to a slightly-simplified form of the maximum principle which is suitable for solving problems about maximizing the discounted net-present-value of a stream of future benefits (and costs). In this setting, the control variables become prices which not only enforce the constraints, but also ensure an optimal trade-off over time. The instantaneous function to be maximized \( H \) becomes something like "net income right now, plus the present value of future income made possible by net investments". Providing a more exact economic characterization of the maximand and saying how it relates to concerns about sustainability, is the burden of the last few chapters, and I don't feel like trying to summarize more than I have.

Weitzman assumes a primary audience which is familiar with solving static optimization problems in microeconomics, including using Lagrange multipliers to enforce constraints and recognizing the values of the multipliers as shadow prices. He also assumes a general familiarity with microeconomic jargon. (Something like Varian's textbook would be more than enough.) There are however many asides which are intended for the primary audience's teachers, and/or for those who understand Hamiltonian mechanics. (Weitzman does not say that a momentum variable is the shadow price of investment in the conjugate spatial coordinate, but I think he should have.) One notable part of these remarks, to me, is how readily Weitzman passes back and forth between asking himself "What would I do if I were an omniscient social planner?" and "What would be the equilibrium of a perfectly competitive market?", regarding these two perspectives as perfectly in harmony. This is in line with a long-standing tradition within economic theory, and indeed some of Weitzman's own work in the 1970s.

Since, for pedagogical purposes, I am looking for something to introduce control theory and optimization-over-time to larval statisticians, this isn't quite what I need, but I enjoyed reading it, and hope to mine it for examples.

Pinch's and Liberzon's books are more straightforwardly good math textbooks. Pinch, in particular, is a model of clear exposition and well-employed geometry. (I say this as someone who is often frustrated, even confused, by mathematicians' attempts to provide geometric intuition.) Liberzon is, however, broader in the range of problems considered, and a bit more rigorous.

Books to Read While the Algae Grow in Your Fur; Mathematics; The Dismal Science