Calculus of Variations

Perturbing a curve to test stationarity, the Euler-Lagrange residual along it, and a brachistochrone race against the cycloid.

Calculus of variations optimizes over functions instead of numbers. A curve $y(x)$ is plugged into a functional

$$ J[y] = \int_a^b L(x,y,y')\,dx, $$

and the question is whether small perturbations of the whole curve make $J$ go up, down, or stay flat to first order. The widgets below put three views next to each other: the global perturbation test $\delta J=0$, the local Euler-Lagrange residual, and a physical race where the optimum is a cycloid.

One sentence: a stationary curve is not necessarily the lowest, shortest, or steepest descent; it is one whose first-order change in the functional vanishes for every allowed perturbation.

1. First variation microscope

The base curve is a sine-series path with fixed endpoints. The perturbation $\eta(x)$ also vanishes at the endpoints, so $y_\epsilon(x)=y(x)+\epsilon\eta(x)$ respects the boundary conditions. The functional is a loaded string energy,

$$ J[y] = \int_0^1 \left(\frac12 y'(x)^2 - 4y(x)\right)dx. $$

The stationary curve satisfies $y''=-4$, hence $y^\star(x)=2x(1-x)$. The right panel plots $J[y+\epsilon\eta]$ against $\epsilon$. If the tangent at zero is not flat, the current curve still has a first-order way to improve.

What this functional is. $J[y] = \int_0^1 \bigl(\tfrac12 y'^2 - 4y\bigr)dx$ is the small-deflection elastic energy of a string pinned at both ends under a uniform downward load of intensity $4$. The $\tfrac12 y'^2$ term is elastic stretch energy, and $-4y$ is the work done by gravity as the string sags into shape $y(x)$. The Euler-Lagrange equation $y''=-4$ is exactly the static balance: curvature opposing load. So a stationary curve here is a string in equilibrium, and "dropping" toward the optimum (Figure 1's Relax button) is what the physical string would do under damping.

A second perspective: stationary means flat in every direction at once. The readout lists $dJ/d\epsilon|_0$ for all four perturbation modes simultaneously. A non-stationary curve will show at least one mode with a non-zero slope; the stationary curve flattens all of them together.

Drag a point on the left panel to deform the curve at that $x$, or drag horizontally on the right panel to scrub $\epsilon$.

Figure 1 · First variation: J[y + epsilon eta]

curve coefficient c1 0.25

curve coefficient c2 0.42

curve coefficient c3 -0.25

perturbation epsilon 0.45

perturbation mode 2

2. Euler-Lagrange residual heatmap

For the same functional, the Euler-Lagrange residual is

$$ R(x) = \frac{\partial L}{\partial y} - \frac{d}{dx}\frac{\partial L}{\partial y'} = -4 - y''(x). $$

A stationary curve has $R(x)=0$ at every interior point. This is the local diagnostic version of the first-variation test above: red means that piece of the curve is still violating the Euler-Lagrange equation. Drag a point on the curve to deform it at that $x$ and watch the residual color shift in response.

Figure 2 · Local Euler-Lagrange residual along the curve

large residual near zero residual Euler-Lagrange solution

3. Brachistochrone race

The brachistochrone asks for the fastest frictionless track between two points under gravity. The shortest path is not fastest: an initially steep curve lets the bead build speed early. The time functional is

$$ T[y] = \int_{x_A}^{x_B} \sqrt{\frac{1+y'(x)^2}{2g\,y(x)}}\,dx, $$

where $y$ is the vertical drop below the start. Among all admissible tracks, the cycloid is the curve on which $T$ is stationary. Drag the two control points. The bead races your cubic Bezier track against the cycloid with the same endpoints, and the histogram records every curve you try.

Figure 3 · Race your curve against the cycloid

user curve cycloid optimum tried curve times

What next

Variational ideas reappear in probability as optimization over distributions.

Bayesian inference

Free Energy & Variational Inference

See the ELBO identity that turns posterior approximation into a variational optimization problem.

Mean-field VI

Mean-Field VI in Bayesian Regression

Watch factorized variational Bayes underestimate uncertainty in Bayesian linear regression.

Information

KL Divergence

Study the directed mismatch measure that drives variational Bayes.