Communicated by Notices Associate Editor Daniela De Silva
1. Introduction
The story of the Hamilton-Jacobi equation begins in classical mechanics as developed in the 19th century. A basic problem within this theory is to describe the motion of a particle with mass $m$ subject to a force $F$ depending on its position. If this particle happens to be moving along the $x$-axis, Newton’s second law takes the form
Here $s$ denotes time, $x=\gamma (s)$ is the position of the particle at time $s$, and $\cdot$ is differentiation with respect to time. Observe that in order to determine the trajectory of the particle, we are left to solve a differential equation for given initial conditions $\gamma (0)$ and $\dot{\gamma }(0)$.
Figure 1.
The position $x=\gamma (s)$ of a point mass $m$ versus time on the interval $s\in [0,t]$. This particle moves along the $x$-axis and is subject to a force $F$. We can find $\gamma$ by solving the differential equation 1.
An interesting thing happens when we select a potential energy function $V$ for which
$$F=-V'.$$
This choice allows us to derive the differential equation 1 through a least action principle. Indeed we may consider the action integral
for a given path $\zeta :[0,t]\rightarrow \mathbb{R}$; here and throughout, all paths are assumed to be absolutely continuous. If there is a path $\gamma$ such that
Therefore, action-minimizing paths are natural candidates to describe the motion of a particle with mass $m$ subject to a force $-V'$.
Figure 2.
Various trajectories $\zeta : [0,t]\rightarrow \mathbb{R}$ with the same right endpoint as $\gamma$ indicated with the gray dot. All of these trajectories are candidates to minimize the action $\mathcal{S}_t$ subject to this right endpoint constraint.
The action integral $\mathcal{S}_t$ is also known to satisfy a partial differential equation (PDE). Assume that for each $x\in \mathbb{R}$ and $t>0$,
$$\gamma ^{x,t}:[0,t]\rightarrow \mathbb{R}$$
is a path which minimizes $\mathcal{S}_t(\zeta )$ among all paths $\zeta$ which fulfill the right endpoint constraint $\zeta (t)=x$. Further suppose that when the action integral is evaluated along $\gamma ^{x,t}$,
$$u(x,t)=\mathcal{S}_t(\gamma ^{x,t}),$$
it is a smooth function. Then direct computation (as explained in lecture 19 of Jac84) shows that $u$ solves what is now known as a Hamilton-Jacobi equation
Here $\partial _tu$ and $\partial _xu$ denote partial differentiation with respect to $t$ and $x$, respectively.
In this article, we will examine some of the mathematical developments concerning Hamilton-Jacobi equations since the mid-20th century. This includes the advent of viscosity solutions, which gives us a way to interpret how functions like $u$ defined above generally solve Hamilton-Jacobi equations. Another key idea we will discuss is the dynamic programming principle from control theory. In addition, we will present research directions involving interacting systems of particles and noncooperative differential games which aim to further expand what we know about Hamilton-Jacobi type equations.
2. Modeling Applications
Before studying the particular equation 3, we will discuss various optimization problems in which Hamilton-Jacobi equations arise. We will not analyze each of the resulting equations separately nor present a theory which encompasses them all. However, the ideas that we will subsequently introduce for equation 3 can be adapted to all of these equations. The purpose of this section is to give some perspective on the applicability of these concepts. We also note that some of the PDEs in the examples below are typically called Hamilton-Jacobi-Bellman equations, as they can be derived with Bellman’s dynamic programming method which we will discuss later in this article.
2.1. Escape of a light ray
Let us first consider a two-dimensional light ray passing through an inhomogeneous medium. We will represent the medium by a bounded open subset $U$ of the $xy$-plane and the light ray by a path
We will also assume that at each point $(x,y)$ in $U$, the inhomogeneity of the medium constrains the speed of light to be no more than $c(x,y)$ for a given positive function $c$.
Figure 3.
A planar domain $U$ which represents an inhomogeneous medium. The dashed curves display possible light ray paths $\zeta : [0,\tau _\zeta ]\rightarrow U$ emanating from a common point $\zeta (0)=(x,y)$ in $U$. Fermat’s principle tells us that light ray paths $\gamma$ are among those which minimize their exit time $\tau _\gamma$ from $U$.
For any path $\zeta : [0,\infty )\rightarrow \mathbb{R}^2$ with $\zeta (0)$ in $U$, we define
as the first time that $\zeta$ exits $U$ and $\tau _\zeta =\infty$ if no such time exists. According to Fermat’s principle, a ray of light assumes a path which takes the least amount of time to exit this region. As a result we will try to find a path $\zeta$ which satisfies the speed constraint
This PDE may be the first Hamilton-Jacobi equation ever written down Ham28. An important example occurs when
$$c(x,y)=1\;\text{ for all $(x,y)$ in $U$}.$$
This corresponds to a homogeneous medium. Here $u(x,y)$ is simply the distance from $(x,y)$ to the boundary of $U$. In particular, the distance to the boundary function is a solution of the PDE
$$\sqrt {(\partial _xu)^2+(\partial _yu)^2}=1.$$
We recommend BCD97 and Tra21 for more on the eikonal and other Hamilton-Jacobi equations.
2.2. Optimal production of a commodity
Figure 4.
A graph $x=z(s)$ of the amount of a commodity produced by a hypothetical factory. The boxes represent a changing inventory over time.
A basic problem in economics is to minimize the costs associated with the production of a given commodity on fixed time horizon $[0,T]$. Let us represent $z(s)$ as the amount of commodity stored in inventory at time $s$, which is driven by a variable production rate $\xi \ge 0$; for simplicity, we assume there is a constant demand rate $d$. That is,
modeling the sum of specific types of holding and operational costs. Here $w^+\coloneq \max \{ w,0\}$, and we’ll also use $w^- \coloneq \max \{ -w,0\}$ below.
Considering this optimization problem for a given level of commodity $x\in \mathbb{R}$ at time $t\in [0,T]$, we are led to the function
Here the minimum is taken over paths $z:[t,T]\rightarrow \mathbb{R}$ satisfying 4 for a nonnegative production rate $\xi$. It turns out that $u$ can be interpreted as a solution of the PDE
It would be interesting to know if we can find $u$ explicitly and if it will tell us something about optimal production rates $\xi$. Finally, we note that this optimization problem is a particular case of Example 2.1 in section I.2 of FS06.
2.3. Eradicating an infectious disease
There are many optimization problems of interest in epidemiology. We encounter one when considering the following differential equations in a compartmental model:
$(t>0)$. Here $S(t)$ and $I(t)$ represent the respective susceptible and infected components of a population at time $t$ which is subject to an infectious disease, $\beta$ is the transmission rate, $\gamma$ is the recovery rate, and $r(t)$ represents a vaccination rate at time $t$. As we have seen in the present COVID-19 epidemic, there may be logistical constraints in administering vaccines. As a result, we will suppose for simplicity that
$$0\le r(t)\le 1$$
for all $t\ge 0$.
Figure 5.
Solution of the differential equations 5 with $S(0)=2$,$I(0)=1$,$\beta =2$, and $\gamma =2$. The graph of $S$ is shown in blue, and the graph of $I$ is shown in red. Here the vaccination rate is $r(t)=0$ for $t\in [0,2.5]$ and $r(t)=1$ for $t\in (2.5,\infty )$.
Let us fix $\mu \in (0,I(0))$. Our goal is to choose a vaccination rate $r$ so that the time
$$\tau ^r=\min \{t\ge 0: I^r(t)=\mu \}$$
it takes for the infected population $I^r$ to drop down to the given threshold $\mu$ is as small as possible. Here and below, we’ll write $S^r$ and $I^r$ for the solution of 5 with vaccination rate $r$. The time $\tau ^r$ corresponding to a minimizing $r$ is called an eradication time.
This problem was considered by a group of math biologists BBSG17, who showed that a minimizing vaccination rate is always of the form
for a switching time $T\ge 0$. While it seems intuitive that $T=0$ for an optimal vaccination rate, there are initial conditions $S(0), I(0)$ which correspond to positive switching times (Figure 2a of BBSG17). However, it is not well understood how the quadrant of initial conditions $(S(0),I(0))\in (0,\infty )\times (\mu ,\infty )$ is divided between points associated with immediate or delayed switching.
for $(x,y)\in (0,\infty )\times (\mu ,\infty )$HIP20. It also turns out that $u$ solves another PDE which allowed us to give a new interpretation of optimal vaccination rates.
models the front for times $t\ge 0$. Here $v=v(x,y)$ and $w=w(x,y)$ represent the respective $x$ and $y$ coordinates of the “turbulent” velocity field $V$ of the flame; the parameter $s>0$ is the “laminar” flame speed.
Figure 6.
Schematic of the flame front $\Gamma _t$ determined by the zero level set $u(x,y,t)=0$; note that $n$ is the outward unit normal to $\Gamma _t$. This level set is time-dependent and will evolve according to the $G$-equation6.
The primary assumption used in the derivation of the $G$-equation is that the level set $\Gamma _t$ moves in direction $n$ with speed
$$V\cdot n+s.$$
Here $n$ is the outward unit normal to $\Gamma _t$, which points in the negative direction of the spatial gradient of $u$ provided the burnt region corresponds to points for which $u$ is positive. A central goal of this mathematical model is to describe
$$\lim _{t\rightarrow \infty }u(x,y,t)/t,$$
which is interpreted as the large time flame front speed. To this end, it has been exploited that the $G$-equation is a Hamilton-Jacobi equation NXY09.
In particular, we may represent a solution $u$ of 6 as
The “controls” $\alpha ,\beta$ are required to observe the constraint
$$\alpha (\tau )^2+\beta (\tau )^2\le s^2$$
for $\tau \in [0,t]$. Consequently, the $G$-equation is indeed a PDE which can be identified as a Hamilton-Jacobi equation.
3. Dynamic Programming
Let us now return to minimizing the action associated with Newton’s second law that we considered in the introduction. As before, $F=-V'$, where $V$ is the potential energy. We will consider paths whose right endpoint is fixed and also assume that $m=1$ for convenience. As a result, our problem is to minimize
Notice that we’ve included a function $g: \mathbb{R}\rightarrow \mathbb{R}$ into our minimization problem. This is done with a bit of foresight, and we hope the reader will come to appreciate this addition. We also note that this definition of $u(x,t)$ should be made with an infimum rather than a minimum, since we have not verified the existence of a minimizing path. We won’t lose much by this omission as it is fairly routine to find conditions on $g$ and $V$ so that the minimization problem associated with $u(x,t)$ has a solution.
3.1. Deriving the Hamilton-Jacobi equation
A key insight made by Bellman is that the values of $u(\cdot ,s)$ determine $u(x,t)$ for $s< t$Bel54. Specifically, there is a relationship between the prior optimal values $u(\cdot ,s)$ and the current one $u(x, t)$. In general, this is called the dynamic programming principle. For our particular problem, it takes the form
As a result, dynamic programming heuristically implies the Hamilton-Jacobi equation 3.
3.2. Action-minimizing paths
It is also not hard to check that if $\zeta$ is an action-minimizing path in 7, $\zeta |_{[s,t]}$ is also a minimizer in 8 for any $0\le s<t$. Making use of this fact, we can apply the technique used to derive the least action principle to 8 and show
for any action-minimizing path $\gamma$. This conclusion assumes differentiability of the value function along minimizing trajectories. We also note that the necessary condition 10 is a part of a more general set of conditions due to Pontryagin (as detailed in Chapter 1 of FS06).
In view of 7, we see that $u(x,0)=g(x)$. Therefore, $u$ is a candidate for a solution of the initial value problem
Alternatively, if $w$ is a continuously differentiable solution of this initial value problem and $\zeta :[0,t]\rightarrow \mathbb{R}$ is a path with $\zeta (t)=x$, then