Today I’d like to discuss (in the Tricks Wiki format) a fundamental trick in “soft” analysis, sometimes known as the “limiting argument” or “epsilon regularisation argument”.

Title: Give yourself an epsilon of room.

Quick description: You want to prove some statement $S_0$ about some object $x_0$ (which could be a number, a point, a function, a set, etc.).  To do so, pick a small $\varepsilon > 0$, and first prove a weaker statement $S_\varepsilon$ (which allows for “losses” which go to zero as $\varepsilon \to 0$) about some perturbed object $x_\varepsilon$.  Then, take limits $\varepsilon \to 0$.  Provided that the dependency and continuity of the weaker conclusion $S_\varepsilon$ on $\varepsilon$ are sufficiently controlled, and $x_\varepsilon$ is converging to $x_0$ in an appropriately strong sense, you will recover the original statement.

One can of course play a similar game when proving a statement $S_\infty$ about some object $X_\infty$, by first proving a weaker statement $S_N$ on some approximation $X_N$ to $X_\infty$ for some large parameter N, and then send $N \to \infty$ at the end.

General discussion: Here are some typical examples of a target statement $S_0$, and the approximating statements $S_\varepsilon$ that would converge to $S$:

 $S_0$ $S_\varepsilon$ $f(x_0) = g(x_0)$ $f(x_\varepsilon) = g(x_\varepsilon) + o(1)$ $f(x_0) \leq g(x_0)$ $f(x_\varepsilon) \leq g(x_\varepsilon) + o(1)$ $f(x_0) > 0$ $f(x_\varepsilon) \geq c - o(1)$ for some $c>0$ independent of $\varepsilon$ $f(x_0)$ is finite $f(x_\varepsilon)$ is bounded uniformly in $\varepsilon$ $f(x_0) \geq f(x)$ for all $x \in X$ (i.e. $x_0$ maximises f) $f(x_\varepsilon) \geq f(x)-o(1)$ for all $x \in X$ (i.e. $x_\varepsilon$ nearly maximises f) $f_n(x_0)$ converges as $n \to \infty$ $f_n(x_\varepsilon)$ fluctuates by at most o(1) for sufficiently large n $f_0$ is a measurable function $f_\varepsilon$ is a measurable function converging pointwise to $f_0$ $f_0$ is a continuous function $f_\varepsilon$ is an equicontinuous family of functions converging pointwise to $f_0$ OR $f_\varepsilon$ is continuous and converges (locally) uniformly to $f_0$ The event $E_0$ holds almost surely The event $E_\varepsilon$ holds with probability 1-o(1) The statement $P_0(x)$ holds for almost every x The statement $P_\varepsilon(x)$ holds for x outside of a set of measure o(1)

Of course, to justify the convergence of $S_\varepsilon$ to $S_0$, it is necessary that $x_\varepsilon$ converge to $x_0$ (or $f_\varepsilon$ converge to $f_0$, etc.) in a suitably strong sense. (But for the purposes of proving just upper bounds, such as $f(x_0) \leq M$, one can often get by with quite weak forms of convergence, thanks to tools such as Fatou’s lemma or the weak closure of the unit ball.)  Similarly, we need some continuity (or at least semi-continuity) hypotheses on the functions f, g appearing above.

It is also necessary in many cases that the control $S_\varepsilon$ on the approximating object $x_\varepsilon$ is somehow “uniform in $\varepsilon$“, although for “$\sigma$-closed” conclusions, such as measurability, this is not required. [It is important to note that it is only the final conclusion $S_\varepsilon$ on $x_\varepsilon$ that needs to have this uniformity in $\varepsilon$; one is permitted to have some intermediate stages in the derivation of $S_\varepsilon$ that depend on $\varepsilon$ in a non-uniform manner, so long as these non-uniformities cancel out or otherwise disappear at the end of the argument.]

By giving oneself an epsilon of room, one can evade a lot of familiar issues in soft analysis.  For instance, by replacing “rough”, “infinite-complexity”, “continuous”,  “global”, or otherwise “infinitary” objects $x_0$ with “smooth”, “finite-complexity”, “discrete”, “local”, or otherwise “finitary” approximants $x_\varepsilon$, one can finesse most issues regarding the justification of various formal operations (e.g. exchanging limits, sums, derivatives, and integrals).  [It is important to be aware, though, that any quantitative measure on how smooth, discrete, finite, etc. $x_\varepsilon$ should be expected to degrade in the limit $\varepsilon \to 0$, and so one should take extreme caution in using such quantitative measures to derive estimates that are uniform in $\varepsilon$.]  Similarly, issues such as whether the supremum $M := \sup \{ f(x): x \in X \}$ of a function on a set is actually attained by some maximiser $x_0$ become moot if one is willing to settle instead for an almost-maximiser $x_\varepsilon$, e.g. one which comes within an epsilon of that supremum M (or which is larger than $1/\varepsilon$, if M turns out to be infinite).  Last, but not least, one can use the epsilon room to avoid degenerate solutions, for instance by perturbing a non-negative function to be strictly positive, perturbing a non-strictly monotone function to be strictly monotone, and so forth.

To summarise: one can view the epsilon regularisation argument as a “loan” in which one borrows an epsilon here and there in order to be able to ignore soft analysis difficulties, and can temporarily be able to utilise estimates which are non-uniform in epsilon, but at the end of the day one needs to “pay back” the loan by establishing a final “hard analysis” estimate which is uniform in epsilon (or whose error terms decay to zero as epsilon goes to zero).

A variant: It may seem that the epsilon regularisation trick is useless if one is already in “hard analysis” situations when all objects are already “finitary”, and all formal computations easily justified.  However, there is an important variant of this trick which applies in this case: namely, instead of sending the epsilon parameter to zero, choose epsilon to be a sufficiently small (but not infinitesimally small) quantity, depending on other parameters in the problem, so that one can eventually neglect various error terms and to obtain a useful bound at the end of the day.  (For instance, any result proven using the Szemerédi regularity lemma is likely to be of this type.)  Since one is not sending epsilon to zero, not every term in the final bound needs to be uniform in epsilon, though for quantitative applications one still would like the dependencies on such parameters to be as favourable as possible.

Prerequisites: Graduate real analysis.  (Actually, this isn’t so much a prerequisite as it is a corequisite: the limiting argument plays a central role in many fundamental results in real analysis.)  Some examples also require some exposure to PDE.

Example 0. The “soft analysis” components of any real analysis textbook will contain a large number of examples of this trick in action.  In particular, any argument which exploits Littlewood’s three principles of real analysis is likely to utilise this trick.  Of course, this trick will also occur repeatedly in my 245B lecture notes$\diamond$

Example 1. (Riemann-Lebesgue lemma)  Given any absolutely integrable function $f \in L^1({\Bbb R})$, the Fourier transform $\hat f: {\Bbb R} \to {\Bbb C}$ is defined by the formula

$\displaystyle \hat f(\xi) := \int_{\Bbb R} f(x) e^{-2\pi i x \xi}\ dx$.

The Riemann-Lebesgue lemma asserts that $\hat f(\xi) \to 0$ as $\xi \to \infty$.  It is difficult to prove this estimate for f directly, because this function is too “rough”: it is absolutely integrable (which is enough to ensure that $\hat f$ exists and is bounded), but need not be continuous, differentiable, compactly supported, bounded, or otherwise “nice”.  But suppose we give ourselves an epsilon of room.  Then, as the space $C^\infty_0$ of test functions is dense in $L^1({\Bbb R})$ (a fact I will prove later in this course), we can approximate $f$ to any desired accuracy $\varepsilon > 0$ in the $L^1$ norm by a smooth, compactly supported function $f_\varepsilon: {\Bbb R} \to {\Bbb C}$, thus

$\displaystyle \int_{\Bbb R} |f(x)-f_\varepsilon(x)|\ dx \leq \varepsilon$. (1)

The point is that $f_\varepsilon$ is much better behaved than f, and it is not difficult to show the analogue of the Riemann-Lebesgue lemma for $f_\varepsilon$.  Indeed, being smooth and compactly supported, we can now justifiably integrate by parts to obtain

$\displaystyle \hat f_\varepsilon(\xi) = \frac{1}{2\pi i \xi} \int_{\Bbb R} f'_\varepsilon(x) e^{-2\pi i x \xi}\ dx$

for any non-zero $\xi$, and it is now clear (since $f'$ is bounded and compactly supported) that $\hat f_\varepsilon(\xi) \to 0$ as $\xi \to \infty$.

Now we need to take limits as $\varepsilon \to 0$.  It will be enough to have $\hat f_\varepsilon$ converge uniformly to $\hat f$.  But from (1) and the basic estimate

$\displaystyle \sup_\xi |\hat g(\xi)| \leq \int_{\Bbb R} |g(x)|\ dx$ (2)

(which is the single “hard analysis” ingredient in the proof of the lemma) applied to $g := f - f_\varepsilon$, we see (by the linearity of the Fourier transform) that

$\displaystyle \sup_\xi |\hat f(\xi) - \hat f_\varepsilon(\xi)| \leq \varepsilon$

and we obtain the desired uniform convergence.  $\diamond$

Remark 1. The same argument also shows that $\hat f$ is continuous; we leave this as an exercise to the reader.  $\diamond$

Remark 2. Example 1 is a model case of a much more general instance of the limiting argument: in order to prove a convergence or continuity theorem for all “rough” functions in a function space, it suffices to first prove convergence or continuity for a dense subclass of “smooth” functions, and combine that with some quantitative estimate in the function space (in this case, (2)) in order to justify the limiting argument. $\diamond$

Example 2. The limiting argument in Example 1 relied on the linearity of the Fourier transform $f \mapsto \hat f$.  But, with more effort, it is also possible to extend this type of argument to nonlinear settings.  We will sketch (omitting several technical details, which can be found for instance in my PDE book) a very typical instance.  Consider a nonlinear PDE, e.g. the nonlinear wave equation

$\displaystyle - u_{tt} + u_{xx} = u^3$ (3)

where $u: {\Bbb R} \times {\Bbb R} \to {\Bbb R}$ is some scalar field, and the t and x subscripts denote differentiation of the field $u(t,x)$.  Formally – if u is sufficiently smooth, and sufficiently decaying at spatial infinity, one can show that the energy

$\displaystyle E(u)(t) := \int_{\Bbb R} \frac{1}{2} |u_t(t,x)|^2 + \frac{1}{2} |u_x(t,x)|^2 + \frac{1}{4} |u(t,x)|^4\ dx$ (4)

is conserved, thus $E(u)(t) = E(u)(0)$ for all t.  This can be formally justified by computing the derivative $\partial_t E(u)(t)$ by differentiating under the integral sign, integrating by parts, and then applying the PDE (3); we leave this as an exercise for the reader.  (There are also more fancy ways to see why the energy is conserved, using Hamiltonian or Lagrangian mechanics or by the more general theory of stress-energy tensors, but we will not discuss these here.)  However, these justifications do require a fair amount of regularity on the solution u; for instance, requiring u to be three-times continuously differentiable in space and time, and compactly supported in space on each bounded time interval, would be sufficient to make the computations rigorous by applying “off the shelf” theorems about differentiation under the integration sign, etc.

But suppose one only has a much rougher solution, for instance an energy class solution which has finite energy (4), but for which higher derivatives of u need not exist in the classical sense. (There is a non-trivial issue regarding how to make sense of the PDE (3) when u is only in the energy class, since the terms $u_{tt}$ and $u_{xx}$ do not then make sense classically, but there are standard ways to deal with this, e.g. using weak derivatives, which we will not discuss further here.)  Then it is difficult to justify the energy conservation law directly.  However, it is still possible to obtain energy conservation by the limiting argument.  Namely, one takes the energy class solution u at some initial time (e.g. t=0) and approximates that initial data (the initial position $u(0)$ and initial data $u_t(0)$) by a much smoother (and compactly supported) choice $(u^{(\varepsilon)}(0), u^{(\varepsilon)}_t(0))$ of initial data, which converges back to $(u(0), u_t(0))$ in a suitable “energy topology” related to (4) which we will not define here.  It then turns out (from the existence theory of the PDE (3)) that one can extend the smooth initial data $(u^{(\varepsilon)}(0), u^{(\varepsilon)}_t(0))$ to other times t, providing a smooth solution $u^{(\varepsilon)}$ to that data.  For this solution, the energy conservation law $E( u^{(\varepsilon)} )(t) = E( u^{(\varepsilon)} )(0)$ can be justified.

Now we take limits as $\varepsilon \to 0$ (keeping t fixed).  Since $(u^{(\varepsilon)}(0), u^{(\varepsilon)}_t(0))$ converges in the energy topology to $(u(0), u_t(0))$, and the energy functional E is continuous in this topology, $E( u^{(\varepsilon)} )(0)$ converges to $E( u )(0)$.  To conclude the argument, we will also need $E( u^{(\varepsilon)} )(t)$ to converge to $E( u )(t)$, which will be possible if $(u^{(\varepsilon)}(t), u^{(\varepsilon)}_t(t))$ converges in the energy topology to $(u(t), u_t(t))$.  Thus in turn follows from a fundamental fact (which requires a certain amount of effort to prove) about the PDE to (4), namely that it is well-posed in the energy class.  This means that not only do solutions exist and are unique for initial data in the energy class, but they depend continuously on the initial data in the energy topology; small perturbations in the data lead to small perturbations in the solution, or more formally that the map $(u(0),u_t(0)) \to (u(t),u_t(t))$ from data to solution (say, at some fixed time t) is continuous in the energy topology.  This final fact concludes the limiting argument and gives us the desired conservation law $E(u(t)) = E(u(0))$. $\diamond$

Remark 3. It is important that one have a suitable well-posedness theory in order to make the limiting argument work for rough solutions to a PDE; without such a well-posedness theory, it is possible for quantities which are formally conserved to cease being conserved when the solutions become too rough or otherwise “weak”; energy, for instance, could disappear into a singularity and not come back.   $\diamond$

Example 3. (Maximum principle)  The maximum principle is a fundamental tool in elliptic and parabolic PDE (for example, it is used heavily in the proof of the Poincaré conjecture, see e.g. my lecture notes on this topic).  Here is a model example of this principle:

Proposition 1. Let $u: \overline{\Bbb D} \to {\Bbb R}$ be a smooth harmonic function on the closed unit disk $\overline{\Bbb D} := \{ (x,y): x^2+y^2 \leq 1\}$.  If M is a bound such that $u(x,y) \leq M$ on the boundary $\partial{\Bbb D} := \{ (x,y): x^2+y^2 = 1 \}$.  Then $u(x,y) \leq M$ on the interior as well.

A naive attempt to prove Proposition 1 comes very close to working, and goes like this:  suppose for contradiction that the proposition failed, thus u exceeds M somewhere in the interior of the disk.  Since u is continuous, and the disk is compact, there must then be a point $(x_0,y_0)$ in the interior of the disk where the maximum is attained.  Undergraduate calculus then tells us that $u_{xx}(x_0,y_0)$ and $u_{yy}(x_0,y_0)$ are non-positive, which almost contradicts the harmonicity hypothesis $u_{xx} + u_{yy} = 0$.  However, it is still possible that $u_{xx}$ and $u_{yy}$ both vanish at $(x_0,y_0)$, so we don’t yet get a contradiction.

But we can finish the proof by giving ourselves an epsilon of room.  The trick is to work not with the function u directly, but with the modified function $u^{(\varepsilon)}(x,y) := u(x,y) + \varepsilon (x^2+y^2)$, to boost the harmonicity into subharmonicity.  Indeed, we have $u^{(\varepsilon)}_{xx} + u^{(\varepsilon)}_{yy} = 4\varepsilon > 0$.  The preceding argument now shows that $u^{(\varepsilon)}$ cannot attain its maximum in the interior of the disk; since it is bounded by $M+\varepsilon$ on the boundary of the disk, we we conclude that $u^{(\varepsilon)}$ is bounded by $M + \varepsilon$ on the interior of the disk as well.  Sending $\varepsilon \to 0$ we obtain the claim.  $\diamond$

Remark 4. Of course, Proposition 1 can also be proven by much more direct means, for instance via the Green’s function for the disk.  However, the argument given is extremely robust and applies to a large class of both linear and nonlinear elliptic and parabolic equations, including those with rough variable coefficients. $\diamond$

Exercise 1.  Use the maximum modulus principle to prove the Phragmén-Lindelöf principle: if f is complex analytic on the strip $\{ z: 0 \leq \hbox{Re}(z) \leq 1\}$, is bounded in magnitude by 1 on the boundary of this strip, and obeys a growth condition $|f(z)| \leq C e^{|z|^C}$ on the interior of the strip, then show that f is bounded in magnitude by 1 throughout the strip.   (Hint: multiply f by $e^{-\varepsilon z^m}$ for some even integer m.) $\diamond$

Example 4. (Manipulating generalised functions) In PDE one is primarily interested in smooth (classical) solutions; but for a variety of reasons it is useful to also consider rougher solutions.  Sometimes, these solutions are so rough that they are no longer functions, but are measures, distributions, or some other concept of “generalised function” or “generalised solution“.  For instance, the fundamental solution to a PDE is typically just a distribution or measure, rather than a classical function.  A typical example: a (sufficiently smooth) solution to the three-dimensional wave equation $-u_{tt} + \Delta u = 0$ with initial position $u(0,x)=0$ and initial velocity $u_t(0,x) = g(x)$ is given by the classical formula

$\displaystyle u(t) = t g * \sigma_t$

where $\sigma_t$ is the unique rotation-invariant probability measure on the sphere $S_t := \{ (x,y,z) \in {\Bbb R}^3: x^2+y^2+z^2 = t^2 \}$ of radius t, or equivalently, the area element dS on that sphere divided by the surface area $4\pi t^2$ of that sphere.  (The convolution $f*\mu$ of a smooth function $f$ and a (compactly supported) finite measure $\mu$ is defined by $f*\mu(x) := \int f(x-y)\ d\mu(y)$.)

For this and many other reasons, it is important to manipulate measures and distributions in various ways.  For instance, in addition to convolving functions with measures, it is also useful to convolve measures with measures; the convolution $\mu * \nu$ of two finite measures on ${\Bbb R}^n$ is defined as the measure which assigns to each measurable set E in ${\Bbb R}^n$, the measure

$\displaystyle \mu * \nu(E) := \int \int 1_E(x + y)\ d\mu(x) d\nu(y)$. (5)

For sake of concreteness, let’s focus on a specific question, namely to compute (or at least estimate) the measure $\sigma * \sigma$, where $\sigma$ is the normalised rotation-invariant measure on the unit circle $\{ x \in {\Bbb R}^2: |x|=1 \}$.  It turns out that while $\sigma$ is not absolutely continuous with respect to Lebesgue measure $m$, the convolution is: $d(\sigma*\sigma) = f d m$ for some absolutely integrable function f on ${\Bbb R}^2$.  But what is this function f?  It certainly is possible to compute it from the definition (5), or by other methods (e.g. the Fourier transform), but I would like to give one approach to computing these sorts of expressions involving measures (or other generalised functions) based on epsilon regularisation, which requires a certain amount of geometric computation but which I find to be rather visual and conceptual, compared to more algebraic approaches (e.g. based on Fourier transforms).  The idea is to approximate a singular object, such as the singular measure $\sigma$, by a smoother object $\sigma_\varepsilon$, such as an absolutely continuous measure.  For instance, one can approximate $\sigma$ by

$\displaystyle d\sigma_\varepsilon := \frac{1}{m(A_\varepsilon)} 1_{A_\varepsilon}\ dm$

where $A_\varepsilon := \{ x \in {\Bbb R}^2: 1-\varepsilon \leq |x| \leq 1+\varepsilon \}$.  It is clear that $\sigma_\varepsilon$ converges to $\sigma$ in the vague topology, which implies that $\sigma_\varepsilon * \sigma_\varepsilon$ converges to $\sigma*\sigma$ in the vague topology also.  Since

$\displaystyle \sigma_\varepsilon * \sigma_\varepsilon = \frac{1}{m(A_\varepsilon)^2} 1_{A_\varepsilon} * 1_{A_\varepsilon}\ dm$,

we will be able to understand the limit f by first considering the function

$\displaystyle f_\varepsilon(x) := \frac{1}{m(A_\varepsilon)^2} 1_{A_\varepsilon} * 1_{A_\varepsilon}(x) = \frac{m( A_\varepsilon \cap (x - A_\varepsilon))}{m(A_\varepsilon)^2}$

and then taking (weak) limits as $\varepsilon \to 0$ to recover f.

Up to constants, one can compute from elementary geometry that $m(A_\varepsilon)$ is comparable to $\varepsilon$, and $m( A_\varepsilon \cap (x - A_\varepsilon))$ vanishes for $|x| \geq 2 + 2 \varepsilon$, and is comparable to $\varepsilon^2 (2-|x|)^{-1/2}$ for $1 \leq |x| \leq 2 - 2 \varepsilon$ (and of size $O(\varepsilon^{3/2})$ in the transition region $|x| = 2 + O(\varepsilon)$) and is comparable to $\varepsilon^2 |x|^{-1}$ for $\varepsilon \leq |x| \leq 1$ (and of size about $O(\varepsilon)$ when $|x| \leq \varepsilon$.  (This is a good exercise for anyone who wants practice in quickly computing the orders of magnitude of geometric quantities such as areas; for such order of magnitude calculations, quick and dirty geometric methods tend to work better here than the more algebraic calculus methods you would have learned as an undergraduate.) The bounds here are strong enough to allow one to take limits and conclude what f looks like: it is comparable to $|x|^{-1} (2-|x|)^{-1/2} 1_{|x| \leq 2}$.  And by being more careful with the computations of area, one can compute the exact formula for f(x), though I will not do so here. $\diamond$

Remark 5. Epsilon regularisation also sheds light on why certain operations on measures or distributions are not permissible.  For instance, squaring the Dirac delta function $\delta$ will not give a measure or distribution, because if one looks at the squares $\delta_\varepsilon^2$ of some smoothed out approximations $\delta_\varepsilon$ to the Dirac function (i.e. approximations to the identity), one sees that their masses go to infinity in the limit $\varepsilon \to 0$, and so cannot be integrated against test functions uniformly in $\varepsilon$.  On the other hand, derivatives of the delta function, while no longer measures (the total variation of derivatives of $\delta_\varepsilon$ become unbounded), are at least still distributions (the integrals of derivatives of $\delta_\varepsilon$ against test functions remain convergent).  $\diamond$