You are currently browsing the category archive for the ‘expository’ category.

Suppose we have an ${n \times n}$ matrix ${M}$ that is expressed in block-matrix form as

$\displaystyle M = \begin{pmatrix} A & B \\ C & D \end{pmatrix}$

where ${A}$ is an ${(n-k) \times (n-k)}$ matrix, ${B}$ is an ${(n-k) \times k}$ matrix, ${C}$ is an ${k \times (n-k)}$ matrix, and ${D}$ is a ${k \times k}$ matrix for some ${1 < k < n}$. If ${A}$ is invertible, we can use the technique of Schur complementation to express the inverse of ${M}$ (if it exists) in terms of the inverse of ${A}$, and the other components ${B,C,D}$ of course. Indeed, to solve the equation

$\displaystyle M \begin{pmatrix} x & y \end{pmatrix} = \begin{pmatrix} a & b \end{pmatrix},$

where ${x, a}$ are ${(n-k) \times 1}$ column vectors and ${y,b}$ are ${k \times 1}$ column vectors, we can expand this out as a system

$\displaystyle Ax + By = a$

$\displaystyle Cx + Dy = b.$

Using the invertibility of ${A}$, we can write the first equation as

$\displaystyle x = A^{-1} a - A^{-1} B y \ \ \ \ \ (1)$

and substituting this into the second equation yields

$\displaystyle (D - C A^{-1} B) y = b - C A^{-1} a$

and thus (assuming that ${D - CA^{-1} B}$ is invertible)

$\displaystyle y = - (D - CA^{-1} B)^{-1} CA^{-1} a + (D - CA^{-1} B)^{-1} b$

and then inserting this back into (1) gives

$\displaystyle x = (A^{-1} + A^{-1} B (D - CA^{-1} B)^{-1} C A^{-1}) a - A^{-1} B (D - CA^{-1} B)^{-1} b.$

Comparing this with

$\displaystyle \begin{pmatrix} x & y \end{pmatrix} = M^{-1} \begin{pmatrix} a & b \end{pmatrix},$

we have managed to express the inverse of ${M}$ as

$\displaystyle M^{-1} =$

$\displaystyle \begin{pmatrix} A^{-1} + A^{-1} B (D - CA^{-1} B)^{-1} C A^{-1} & - A^{-1} B (D - CA^{-1} B)^{-1} \\ - (D - CA^{-1} B)^{-1} CA^{-1} & (D - CA^{-1} B)^{-1} \end{pmatrix}. \ \ \ \ \ (2)$

One can consider the inverse problem: given the inverse ${M^{-1}}$ of ${M}$, does one have a nice formula for the inverse ${A^{-1}}$ of the minor ${A}$? Trying to recover this directly from (2) looks somewhat messy. However, one can proceed as follows. Let ${U}$ denote the ${n \times k}$ matrix

$\displaystyle U := \begin{pmatrix} 0 \\ I_k \end{pmatrix}$

(with ${I_k}$ the ${k \times k}$ identity matrix), and let ${V}$ be its transpose:

$\displaystyle V := \begin{pmatrix} 0 & I_k \end{pmatrix}.$

Then for any scalar ${t}$ (which we identify with ${t}$ times the identity matrix), one has

$\displaystyle M + UtV = \begin{pmatrix} A & B \\ C & D+t \end{pmatrix},$

and hence by (2)

$\displaystyle (M+UtV)^{-1} =$

$\displaystyle \begin{pmatrix} A^{-1} + A^{-1} B (D + t - CA^{-1} B)^{-1} C A^{-1} & - A^{-1} B (D + t- CA^{-1} B)^{-1} \\ - (D + t - CA^{-1} B)^{-1} CA^{-1} & (D + t - CA^{-1} B)^{-1} \end{pmatrix}.$

noting that the inverses here will exist for ${t}$ large enough. Taking limits as ${t \rightarrow \infty}$, we conclude that

$\displaystyle \lim_{t \rightarrow \infty} (M+UtV)^{-1} = \begin{pmatrix} A^{-1} & 0 \\ 0 & 0 \end{pmatrix}.$

On the other hand, by the Woodbury matrix identity (discussed in this previous blog post), we have

$\displaystyle (M+UtV)^{-1} = M^{-1} - M^{-1} U (t^{-1} + V M^{-1} U)^{-1} V M^{-1}$

and hence on taking limits and comparing with the preceding identity, one has

$\displaystyle \begin{pmatrix} A^{-1} & 0 \\ 0 & 0 \end{pmatrix} = M^{-1} - M^{-1} U (V M^{-1} U)^{-1} V M^{-1}.$

This achieves the aim of expressing the inverse ${A^{-1}}$ of the minor in terms of the inverse of the full matrix. Taking traces and rearranging, we conclude in particular that

$\displaystyle \mathrm{tr} A^{-1} = \mathrm{tr} M^{-1} - \mathrm{tr} (V M^{-2} U) (V M^{-1} U)^{-1}. \ \ \ \ \ (3)$

In the ${k=1}$ case, this can be simplified to

$\displaystyle \mathrm{tr} A^{-1} = \mathrm{tr} M^{-1} - \frac{e_n^T M^{-2} e_n}{e_n^T M^{-1} e_n} \ \ \ \ \ (4)$

where ${e_n}$ is the ${n^{th}}$ basis column vector.

We can apply this identity to understand how the spectrum of an ${n \times n}$ random matrix ${M}$ relates to that of its top left ${n-1 \times n-1}$ minor ${A}$. Subtracting any complex multiple ${z}$ of the identity from ${M}$ (and hence from ${A}$), we can relate the Stieltjes transform ${s_M(z) := \frac{1}{n} \mathrm{tr}(M-z)^{-1}}$ of ${M}$ with the Stieltjes transform ${s_A(z) := \frac{1}{n-1} \mathrm{tr}(A-z)^{-1}}$ of ${A}$:

$\displaystyle s_A(z) = \frac{n}{n-1} s_M(z) - \frac{1}{n-1} \frac{e_n^T (M-z)^{-2} e_n}{e_n^T (M-z)^{-1} e_n} \ \ \ \ \ (5)$

At this point we begin to proceed informally. Assume for sake of argument that the random matrix ${M}$ is Hermitian, with distribution that is invariant under conjugation by the unitary group ${U(n)}$; for instance, ${M}$ could be drawn from the Gaussian Unitary Ensemble (GUE), or alternatively ${M}$ could be of the form ${M = U D U^*}$ for some real diagonal matrix ${D}$ and ${U}$ a unitary matrix drawn randomly from ${U(n)}$ using Haar measure. To fix normalisations we will assume that the eigenvalues of ${M}$ are typically of size ${O(1)}$. Then ${A}$ is also Hermitian and ${U(n)}$-invariant. Furthermore, the law of ${e_n^T (M-z)^{-1} e_n}$ will be the same as the law of ${u^* (M-z)^{-1} u}$, where ${u}$ is now drawn uniformly from the unit sphere (independently of ${M}$). Diagonalising ${M}$ into eigenvalues ${\lambda_j}$ and eigenvectors ${v_j}$, we have

$\displaystyle u^* (M-z)^{-1} u = \sum_{j=1}^n \frac{|u^* v_j|^2}{\lambda_j - z}.$

One can think of ${u}$ as a random (complex) Gaussian vector, divided by the magnitude of that vector (which, by the Chernoff inequality, will concentrate to ${\sqrt{n}}$). Thus the coefficients ${u^* v_j}$ with respect to the orthonormal basis ${v_1,\dots,v_j}$ can be thought of as independent (complex) Gaussian vectors, divided by that magnitude. Using this and the Chernoff inequality again, we see (for ${z}$ distance ${\sim 1}$ away from the real axis at least) that one has the concentration of measure

$\displaystyle u^* (M-z)^{-1} u \approx \frac{1}{n} \sum_{j=1}^n \frac{1}{\lambda_j - z}$

and thus

$\displaystyle e_n^T (M-z)^{-1} e_n \approx \frac{1}{n} \mathrm{tr} (M-z)^{-1} = s_M(z)$

(that is to say, the diagonal entries of ${(M-z)^{-1}}$ are roughly constant). Similarly we have

$\displaystyle e_n^T (M-z)^{-2} e_n \approx \frac{1}{n} \mathrm{tr} (M-z)^{-2} = \frac{d}{dz} s_M(z).$

Inserting this into (5) and discarding terms of size ${O(1/n^2)}$, we thus conclude the approximate relationship

$\displaystyle s_A(z) \approx s_M(z) + \frac{1}{n} ( s_M(z) - s_M(z)^{-1} \frac{d}{dz} s_M(z) ).$

This can be viewed as a difference equation for the Stieltjes transform of top left minors of ${M}$. Iterating this equation, and formally replacing the difference equation by a differential equation in the large ${n}$ limit, we see that when ${n}$ is large and ${k \approx e^{-t} n}$ for some ${t \geq 0}$, one expects the top left ${k \times k}$ minor ${A_k}$ of ${M}$ to have Stieltjes transform

$\displaystyle s_{A_k}(z) \approx s( t, z ) \ \ \ \ \ (6)$

where ${s(t,z)}$ solves the Burgers-type equation

$\displaystyle \partial_t s(t,z) = s(t,z) - s(t,z)^{-1} \frac{d}{dz} s(t,z) \ \ \ \ \ (7)$

with initial data ${s(0,z) = s_M(z)}$.

Example 1 If ${M}$ is a constant multiple ${M = cI_n}$ of the identity, then ${s_M(z) = \frac{1}{c-z}}$. One checks that ${s(t,z) = \frac{1}{c-z}}$ is a steady state solution to (7), which is unsurprising given that all minors of ${M}$ are also ${c}$ times the identity.

Example 2 If ${M}$ is GUE normalised so that each entry has variance ${\sigma^2/n}$, then by the semi-circular law (see previous notes) one has ${s_M(z) \approx \frac{-z + \sqrt{z^2-4\sigma^2}}{2\sigma^2} = -\frac{2}{z + \sqrt{z^2-4\sigma^2}}}$ (using an appropriate branch of the square root). One can then verify the self-similar solution

$\displaystyle s(t,z) = \frac{-z + \sqrt{z^2 - 4\sigma^2 e^{-t}}}{2\sigma^2 e^{-t}} = -\frac{2}{z + \sqrt{z^2 - 4\sigma^2 e^{-t}}}$

to (7), which is consistent with the fact that a top ${k \times k}$ minor of ${M}$ also has the law of GUE, with each entry having variance ${\sigma^2 / n \approx \sigma^2 e^{-t} / k}$ when ${k \approx e^{-t} n}$.

One can justify the approximation (6) given a sufficiently good well-posedness theory for the equation (7). We will not do so here, but will note that (as with the classical inviscid Burgers equation) the equation can be solved exactly (formally, at least) by the method of characteristics. For any initial position ${z_0}$, we consider the characteristic flow ${t \mapsto z(t)}$ formed by solving the ODE

$\displaystyle \frac{d}{dt} z(t) = s(t,z(t))^{-1} \ \ \ \ \ (8)$

with initial data ${z(0) = z_0}$, ignoring for this discussion the problems of existence and uniqueness. Then from the chain rule, the equation (7) implies that

$\displaystyle \frac{d}{dt} s( t, z(t) ) = s(t,z(t))$

and thus ${s(t,z(t)) = e^t s(0,z_0)}$. Inserting this back into (8) we see that

$\displaystyle z(t) = z_0 + s(0,z_0)^{-1} (1-e^{-t})$

and thus (7) may be solved implicitly via the equation

$\displaystyle s(t, z_0 + s(0,z_0)^{-1} (1-e^{-t}) ) = e^t s(0, z_0) \ \ \ \ \ (9)$

for all ${t}$ and ${z_0}$.

Remark 3 In practice, the equation (9) may stop working when ${z_0 + s(0,z_0)^{-1} (1-e^{-t})}$ crosses the real axis, as (7) does not necessarily hold in this region. It is a cute exercise (ultimately coming from the Cauchy-Schwarz inequality) to show that this crossing always happens, for instance if ${z_0}$ has positive imaginary part then ${z_0 + s(0,z_0)^{-1}}$ necessarily has negative or zero imaginary part.

Example 4 Suppose we have ${s(0,z) = \frac{1}{c-z}}$ as in Example 1. Then (9) becomes

$\displaystyle s( t, z_0 + (c-z_0) (1-e^{-t}) ) = \frac{e^t}{c-z_0}$

for any ${t,z_0}$, which after making the change of variables ${z = z_0 + (c-z_0) (1-e^{-t}) = c - e^{-t} (c - z_0)}$ becomes

$\displaystyle s(t, z ) = \frac{1}{c-z}$

as in Example 1.

Example 5 Suppose we have

$\displaystyle s(0,z) = \frac{-z + \sqrt{z^2-4\sigma^2}}{2\sigma^2} = -\frac{2}{z + \sqrt{z^2-4\sigma^2}}.$

as in Example 2. Then (9) becomes

$\displaystyle s(t, z_0 - \frac{z_0 + \sqrt{z_0^2-4\sigma^2}}{2} (1-e^{-t}) ) = e^t \frac{-z_0 + \sqrt{z_0^2-4\sigma^2}}{2\sigma^2}.$

If we write

$\displaystyle z := z_0 - \frac{z_0 + \sqrt{z_0^2-4\sigma^2}}{2} (1-e^{-t})$

$\displaystyle = \frac{(1+e^{-t}) z_0 - (1-e^{-t}) \sqrt{z_0^2-4\sigma^2}}{2}$

one can calculate that

$\displaystyle z^2 - 4 \sigma^2 e^{-t} = (\frac{(1-e^{-t}) z_0 - (1+e^{-t}) \sqrt{z_0^2-4\sigma^2}}{2})^2$

and hence

$\displaystyle \frac{-z + \sqrt{z^2 - 4\sigma^2 e^{-t}}}{2\sigma^2 e^{-t}} = e^t \frac{-z_0 + \sqrt{z_0^2-4\sigma^2}}{2\sigma^2}$

which gives

$\displaystyle s(t,z) = \frac{-z + \sqrt{z^2 - 4\sigma^2 e^{-t}}}{2\sigma^2 e^{-t}}. \ \ \ \ \ (10)$

One can recover the spectral measure ${\mu}$ from the Stieltjes transform ${s(z)}$ as the weak limit of ${x \mapsto \frac{1}{\pi} \mathrm{Im} s(x+i\varepsilon)}$ as ${\varepsilon \rightarrow 0}$; we write this informally as

$\displaystyle d\mu(x) = \frac{1}{\pi} \mathrm{Im} s(x+i0^+)\ dx.$

In this informal notation, we have for instance that

$\displaystyle \delta_c(x) = \frac{1}{\pi} \mathrm{Im} \frac{1}{c-x-i0^+}\ dx$

which can be interpreted as the fact that the Cauchy distributions ${\frac{1}{\pi} \frac{\varepsilon}{(c-x)^2+\varepsilon^2}}$ converge weakly to the Dirac mass at ${c}$ as ${\varepsilon \rightarrow 0}$. Similarly, the spectral measure associated to (10) is the semicircular measure ${\frac{1}{2\pi \sigma^2 e^{-t}} (4 \sigma^2 e^{-t}-x^2)_+^{1/2}}$.

If we let ${\mu_t}$ be the spectral measure associated to ${s(t,\cdot)}$, then the curve ${e^{-t} \mapsto \mu_t}$ from ${(0,1]}$ to the space of measures is the high-dimensional limit ${n \rightarrow \infty}$ of a Gelfand-Tsetlin pattern (discussed in this previous post), if the pattern is randomly generated amongst all matrices ${M}$ with spectrum asymptotic to ${\mu_0}$ as ${n \rightarrow \infty}$. For instance, if ${\mu_0 = \delta_c}$, then the curve is ${\alpha \mapsto \delta_c}$, corresponding to a pattern that is entirely filled with ${c}$‘s. If instead ${\mu_0 = \frac{1}{2\pi \sigma^2} (4\sigma^2-x^2)_+^{1/2}}$ is a semicircular distribution, then the pattern is

$\displaystyle \alpha \mapsto \frac{1}{2\pi \sigma^2 \alpha} (4\sigma^2 \alpha -x^2)_+^{1/2},$

thus at height ${\alpha}$ from the top, the pattern is semicircular on the interval ${[-2\sigma \sqrt{\alpha}, 2\sigma \sqrt{\alpha}]}$. The interlacing property of Gelfand-Tsetlin patterns translates to the claim that ${\alpha \mu_\alpha(-\infty,\lambda)}$ (resp. ${\alpha \mu_\alpha(\lambda,\infty)}$) is non-decreasing (resp. non-increasing) in ${\alpha}$ for any fixed ${\lambda}$. In principle one should be able to establish these monotonicity claims directly from the PDE (7) or from the implicit solution (9), but it was not clear to me how to do so.

An interesting example of such a limiting Gelfand-Tsetlin pattern occurs when ${\mu_0 = \frac{1}{2} \delta_{-1} + \frac{1}{2} \delta_1}$, which corresponds to ${M}$ being ${2P-I}$, where ${P}$ is an orthogonal projection to a random ${n/2}$-dimensional subspace of ${{\bf C}^n}$. Here we have

$\displaystyle s(0,z) = \frac{1}{2} \frac{1}{-1-z} + \frac{1}{2} \frac{1}{1-z} = \frac{z}{1-z^2}$

and so (9) in this case becomes

$\displaystyle s(t, z_0 + \frac{1-z_0^2}{z_0} (1-e^{-t}) ) = \frac{e^t z_0}{1-z_0^2}$

A tedious calculation then gives the solution

$\displaystyle s(t,z) = \frac{(2e^{-t}-1)z + \sqrt{z^2 - 4e^{-t}(1-e^{-t})}}{2e^{-t}(1-z^2)}. \ \ \ \ \ (11)$

For ${\alpha = e^{-t} > 1/2}$, there are simple poles at ${z=-1,+1}$, and the associated measure is

$\displaystyle \mu_\alpha = \frac{2\alpha-1}{2\alpha} \delta_{-1} + \frac{2\alpha-1}{2\alpha} \delta_1 + \frac{1}{2\pi \alpha(1-x^2)} (4\alpha(1-\alpha)-x^2)_+^{1/2}\ dx.$

This reflects the interlacing property, which forces ${\frac{2\alpha-1}{2\alpha} \alpha n}$ of the ${\alpha n}$ eigenvalues of the ${\alpha n \times \alpha n}$ minor to be equal to ${-1}$ (resp. ${+1}$). For ${\alpha = e^{-t} \leq 1/2}$, the poles disappear and one just has

$\displaystyle \mu_\alpha = \frac{1}{2\pi \alpha(1-x^2)} (4\alpha(1-\alpha)-x^2)_+^{1/2}\ dx.$

For ${\alpha=1/2}$, one has an inverse semicircle distribution

$\displaystyle \mu_{1/2} = \frac{1}{\pi} (1-x^2)_+^{-1/2}.$

There is presumably a direct geometric explanation of this fact (basically describing the singular values of the product of two random orthogonal projections to half-dimensional subspaces of ${{\bf C}^n}$), but I do not know of one off-hand.

The evolution of ${s(t,z)}$ can also be understood using the ${R}$-transform and ${S}$-transform from free probability. Formally, letlet ${z(t,s)}$ be the inverse of ${s(t,z)}$, thus

$\displaystyle s(t,z(t,s)) = s$

for all ${t,s}$, and then define the ${R}$-transform

$\displaystyle R(t,s) := z(t,-s) - \frac{1}{s}.$

The equation (9) may be rewritten as

$\displaystyle z( t, e^t s ) = z(0,s) + s^{-1} (1-e^{-t})$

and hence

$\displaystyle R(t, -e^t s) = R(0, -s)$

or equivalently

$\displaystyle R(t,s) = R(0, e^{-t} s). \ \ \ \ \ (12)$

See these previous notes for a discussion of free probability topics such as the ${R}$-transform.

Example 6 If ${s(t,z) = \frac{1}{c-z}}$ then the ${R}$ transform is ${R(t,s) = c}$.

Example 7 If ${s(t,z)}$ is given by (10), then the ${R}$ transform is

$\displaystyle R(t,s) = \sigma^2 e^{-t} s.$

Example 8 If ${s(t,z)}$ is given by (11), then the ${R}$ transform is

$\displaystyle R(t,s) = \frac{-1 + \sqrt{1 + 4 s^2 e^{-2t}}}{2 s e^{-t}}.$

This simple relationship (12) is essentially due to Nica and Speicher (thanks to Dima Shylakhtenko for this reference). It has the remarkable consequence that when ${\alpha = 1/m}$ is the reciprocal of a natural number ${m}$, then ${\mu_{1/m}}$ is the free arithmetic mean of ${m}$ copies of ${\mu}$, that is to say ${\mu_{1/m}}$ is the free convolution ${\mu \boxplus \dots \boxplus \mu}$ of ${m}$ copies of ${\mu}$, pushed forward by the map ${\lambda \rightarrow \lambda/m}$. In terms of random matrices, this is asserting that the top ${n/m \times n/m}$ minor of a random matrix ${M}$ has spectral measure approximately equal to that of an arithmetic mean ${\frac{1}{m} (M_1 + \dots + M_m)}$ of ${m}$ independent copies of ${M}$, so that the process of taking top left minors is in some sense a continuous analogue of the process of taking freely independent arithmetic means. There ought to be a geometric proof of this assertion, but I do not know of one. In the limit ${m \rightarrow \infty}$ (or ${\alpha \rightarrow 0}$), the ${R}$-transform becomes linear and the spectral measure becomes semicircular, which is of course consistent with the free central limit theorem.

In a similar vein, if one defines the function

$\displaystyle \omega(t,z) := \alpha \int_{\bf R} \frac{zx}{1-zx}\ d\mu_\alpha(x) = e^{-t} (- 1 - z^{-1} s(t, z^{-1}))$

and inverts it to obtain a function ${z(t,\omega)}$ with

$\displaystyle \omega(t, z(t,\omega)) = \omega$

for all ${t, \omega}$, then the ${S}$-transform ${S(t,\omega)}$ is defined by

$\displaystyle S(t,\omega) := \frac{1+\omega}{\omega} z(t,\omega).$

Writing

$\displaystyle s(t,z) = - z^{-1} ( 1 + e^t \omega(t, z^{-1}) )$

for any ${t}$, ${z}$, we have

$\displaystyle z_0 + s(0,z_0)^{-1} (1-e^{-t}) = z_0 \frac{\omega(0,z_0^{-1})+e^{-t}}{\omega(0,z_0^{-1})+1}$

and so (9) becomes

$\displaystyle - z_0^{-1} \frac{\omega(0,z_0^{-1})+1}{\omega(0,z_0^{-1})+e^{-t}} (1 + e^{t} \omega(t, z_0^{-1} \frac{\omega(0,z_0^{-1})+1}{\omega(0,z_0^{-1})+e^{-t}}))$

$\displaystyle = - e^t z_0^{-1} (1 + \omega(0, z_0^{-1}))$

which simplifies to

$\displaystyle \omega(t, z_0^{-1} \frac{\omega(0,z_0^{-1})+1}{\omega(0,z_0^{-1})+e^{-t}})) = \omega(0, z_0^{-1});$

replacing ${z_0}$ by ${z(0,\omega)^{-1}}$ we obtain

$\displaystyle \omega(t, z(0,\omega) \frac{\omega+1}{\omega+e^{-t}}) = \omega$

and thus

$\displaystyle z(0,\omega)\frac{\omega+1}{\omega+e^{-t}} = z(t, \omega)$

and hence

$\displaystyle S(0, \omega) = \frac{\omega+e^{-t}}{\omega+1} S(t, \omega).$

One can compute ${\frac{\omega+e^{-t}}{\omega+1}}$ to be the ${S}$-transform of the measure ${(1-\alpha) \delta_0 + \alpha \delta_1}$; from the link between ${S}$-transforms and free products (see e.g. these notes of Guionnet), we conclude that ${(1-\alpha)\delta_0 + \alpha \mu_\alpha}$ is the free product of ${\mu_1}$ and ${(1-\alpha) \delta_0 + \alpha \delta_1}$. This is consistent with the random matrix theory interpretation, since ${(1-\alpha)\delta_0 + \alpha \mu_\alpha}$ is also the spectral measure of ${PMP}$, where ${P}$ is the orthogonal projection to the span of the first ${\alpha n}$ basis elements, so in particular ${P}$ has spectral measure ${(1-\alpha) \delta_0 + \alpha \delta_1}$. If ${M}$ is unitarily invariant then (by a fundamental result of Voiculescu) it is asymptotically freely independent of ${P}$, so the spectral measure of ${PMP = P^{1/2} M P^{1/2}}$ is asymptotically the free product of that of ${M}$ and of ${P}$.

Szemerédi’s theorem asserts that all subsets of the natural numbers of positive density contain arbitrarily long arithmetic progressions.  Roth’s theorem is the special case when one considers arithmetic progressions of length three.  Both theorems have many important proofs using tools from additive combinatorics, (higher order) Fourier analysis, (hyper) graph regularity theory, and ergodic theory.  However, the original proof by Endre Szemerédi, while extremely intricate, was purely combinatorial (and in particular “elementary”) and almost entirely self-contained, except for an invocation of the van der Waerden theorem.  It is also notable for introducing a prototype of what is now known as the Szemerédi regularity lemma.

Back in 2005, I rewrote Szemerédi’s original proof in order to understand it better, however my rewrite ended up being about the same length as the original argument and was probably only usable to myself.  In 2012, after Szemerédi was awarded the Abel prize, I revisited this argument with the intention to try to write up a more readable version of the proof, but ended up just presenting some ingredients of the argument in a blog post, rather than try to rewrite the whole thing.  In that post, I suspected that the cleanest way to write up the argument would be through the language of nonstandard analysis (perhaps in an iterated hyperextension that could handle various hierarchies of infinitesimals), but was unable to actually achieve any substantial simplifications by passing to the nonstandard world.

A few weeks ago, I participated in a week-long workshop at the American Institute of Mathematics on “Nonstandard methods in combinatorial number theory”, and spent some time in a working group with Shabnam Akhtari, Irfam Alam, Renling Jin, Steven Leth, Karl Mahlburg, Paul Potgieter, and Henry Towsner to try to obtain a manageable nonstandard version of Szemerédi’s original proof.  We didn’t end up being able to do so – in fact there are now signs that perhaps nonstandard analysis is not the optimal framework in which to place this argument – but we did at least clarify the existing standard argument, to the point that I was able to go back to my original rewrite of the proof and present it in a more civilised form, which I am now uploading here as an unpublished preprint.   There are now a number of simplifications to the proof.  Firstly, one no longer needs the full strength of the regularity lemma; only the simpler “weak” regularity lemma of Frieze and Kannan is required.  Secondly, the proof has been “factored” into a number of stand-alone propositions of independent interest, in particular involving just (families of) one-dimensional arithmetic progressions rather than the complicated-looking multidimensional arithmetic progressions that occur so frequently in the original argument of Szemerédi.  Finally, the delicate manipulations of densities and epsilons via double counting arguments in Szemerédi’s original paper have been abstracted into a certain key property of families of arithmetic progressions that I call the “double counting property”.

The factoring mentioned above is particularly simple in the case of proving Roth’s theorem, which is now presented separately in the above writeup.  Roth’s theorem seeks to locate a length three progression ${(P(1),P(2),P(3)) = (a, a+r, a+2r)}$ in which all three elements lie in a single set.  This will be deduced from an easier variant of the theorem in which one locates (a family of) length three progressions in which just the first two elements ${P(1), P(2)}$ of the progression lie in a good set (and some other properties of the family are also required).  This is in turn derived from an even easier variant in which now just the first element of the progression is required to be in the good set.

More specifically, Roth’s theorem is now deduced from

Theorem 1.5.  Let ${L}$ be a natural number, and let ${S}$ be a set of integers of upper density at least ${1-1/10L}$.  Then, whenever ${S}$ is partitioned into finitely many colour classes, there exists a colour class ${A}$ and a family ${(P_l(1),P_l(2),P_l(3))_{l=1}^L}$ of 3-term arithmetic progressions with the following properties:

1. For each ${l}$, ${P_l(1)}$ and ${P_l(2)}$ lie in ${A}$.
2. For each ${l}$, ${P_l(3)}$ lie in ${S}$.
3. The ${P_l(3)}$ for ${l=1,\dots,L}$ are in arithmetic progression.

The situation in this theorem is depicted by the following diagram, in which elements of $A$ are in blue and elements of $S$ are in grey:

Theorem 1.5 is deduced in turn from the following easier variant:

Theorem 1.6.  Let ${L}$ be a natural number, and let ${S}$ be a set of integers of upper density at least ${1-1/10L}$.  Then, whenever ${S}$ is partitioned into finitely many colour classes, there exists a colour class ${A}$ and a family ${(P_l(1),P_l(2),P_l(3))_{l=1}^L}$ of 3-term arithmetic progressions with the following properties:

1. For each ${l}$, ${P_l(1)}$ lie in ${A}$.
2. For each ${l}$, ${P_l(2)}$ and ${P_l(3)}$ lie in ${S}$.
3. The ${P_l(2)}$ for ${l=1,\dots,L}$ are in arithmetic progression.

The situation here is described by the figure below.

Theorem 1.6 is easy to prove.  To derive Theorem 1.5 from Theorem 1.6, or to derive Roth’s theorem from Theorem 1.5, one uses double counting arguments, van der Waerden’s theorem, and the weak regularity lemma, largely as described in this previous blog post; see the writeup for the full details.  (I would be interested in seeing a shorter proof of Theorem 1.5 though that did not go through these arguments, and did not use the more powerful theorems of  Roth or Szemerédi.)

Fix a non-negative integer ${k}$. Define an (weak) integer partition of length ${k}$ to be a tuple ${\lambda = (\lambda_1,\dots,\lambda_k)}$ of non-increasing non-negative integers ${\lambda_1 \geq \dots \geq \lambda_k \geq 0}$. (Here our partitions are “weak” in the sense that we allow some parts of the partition to be zero. Henceforth we will omit the modifier “weak”, as we will not need to consider the more usual notion of “strong” partitions.) To each such partition ${\lambda}$, one can associate a Young diagram consisting of ${k}$ left-justified rows of boxes, with the ${i^{th}}$ row containing ${\lambda_i}$ boxes. A semi-standard Young tableau (or Young tableau for short) ${T}$ of shape ${\lambda}$ is a filling of these boxes by integers in ${\{1,\dots,k\}}$ that is weakly increasing along rows (moving rightwards) and strictly increasing along columns (moving downwards). The collection of such tableaux will be denoted ${{\mathcal T}_\lambda}$. The weight ${|T|}$ of a tableau ${T}$ is the tuple ${(n_1,\dots,n_k)}$, where ${n_i}$ is the number of occurrences of the integer ${i}$ in the tableau. For instance, if ${k=3}$ and ${\lambda = (6,4,2)}$, an example of a Young tableau of shape ${\lambda}$ would be

$\displaystyle \begin{tabular}{|c|c|c|c|c|c|} \hline 1 & 1 & 1 & 2 & 3 & 3 \\ \cline{1-6} 2 & 2 & 2 &3\\ \cline{1-4} 3 & 3\\ \cline{1-2} \end{tabular}$

The weight here would be ${|T| = (3,4,5)}$.

To each partition ${\lambda}$ one can associate the Schur polynomial ${s_\lambda(u_1,\dots,u_k)}$ on ${k}$ variables ${u = (u_1,\dots,u_k)}$, which we will define as

$\displaystyle s_\lambda(u) := \sum_{T \in {\mathcal T}_\lambda} u^{|T|}$

using the multinomial convention

$\displaystyle (u_1,\dots,u_k)^{(n_1,\dots,n_k)} := u_1^{n_1} \dots u_k^{n_k}.$

Thus for instance the Young tableau ${T}$ given above would contribute a term ${u_1^3 u_2^4 u_3^5}$ to the Schur polynomial ${s_{(6,4,2)}(u_1,u_2,u_3)}$. In the case of partitions of the form ${(n,0,\dots,0)}$, the Schur polynomial ${s_{(n,0,\dots,0)}}$ is just the complete homogeneous symmetric polynomial ${h_n}$ of degree ${n}$ on ${k}$ variables:

$\displaystyle s_{(n,0,\dots,0)}(u_1,\dots,u_k) := \sum_{n_1,\dots,n_k \geq 0: n_1+\dots+n_k = n} u_1^{n_1} \dots u_k^{n_k},$

thus for instance

$\displaystyle s_{(3,0)}(u_1,u_2) = u_1^3 + u_1^2 u_2 + u_1 u_2^2 + u_2^3.$

Schur polyomials are ubiquitous in the algebraic combinatorics of “type ${A}$ objects” such as the symmetric group ${S_k}$, the general linear group ${GL_k}$, or the unitary group ${U_k}$. For instance, one can view ${s_\lambda}$ as the character of an irreducible polynomial representation of ${GL_k({\bf C})}$ associated with the partition ${\lambda}$. However, we will not focus on these interpretations of Schur polynomials in this post.

This definition of Schur polynomials allows for a way to describe the polynomials recursively. If ${k > 1}$ and ${T}$ is a Young tableau of shape ${\lambda = (\lambda_1,\dots,\lambda_k)}$, taking values in ${\{1,\dots,k\}}$, one can form a sub-tableau ${T'}$ of some shape ${\lambda' = (\lambda'_1,\dots,\lambda'_{k-1})}$ by removing all the appearances of ${k}$ (which, among other things, necessarily deletes the ${k^{th}}$ row). For instance, with ${T}$ as in the previous example, the sub-tableau ${T'}$ would be

$\displaystyle \begin{tabular}{|c|c|c|c|} \hline 1 & 1 & 1 & 2 \\ \cline{1-4} 2 & 2 & 2 \\ \cline{1-3} \end{tabular}$

and the reduced partition ${\lambda'}$ in this case is ${(4,3)}$. As Young tableaux are required to be strictly increasing down columns, we can see that the reduced partition ${\lambda'}$ must intersperse the original partition ${\lambda}$ in the sense that

$\displaystyle \lambda_{i+1} \leq \lambda'_i \leq \lambda_i \ \ \ \ \ (1)$

for all ${1 \leq i \leq k-1}$; we denote this interspersion relation as ${\lambda' \prec \lambda}$ (though we caution that this is not intended to be a partial ordering). In the converse direction, if ${\lambda' \prec \lambda}$ and ${T'}$ is a Young tableau with shape ${\lambda'}$ with entries in ${\{1,\dots,k-1\}}$, one can form a Young tableau ${T}$ with shape ${\lambda}$ and entries in ${\{1,\dots,k\}}$ by appending to ${T'}$ an entry of ${k}$ in all the boxes that appear in the ${\lambda}$ shape but not the ${\lambda'}$ shape. This one-to-one correspondence leads to the recursion

$\displaystyle s_\lambda(u) = \sum_{\lambda' \prec \lambda} s_{\lambda'}(u') u_k^{|\lambda| - |\lambda'|} \ \ \ \ \ (2)$

where ${u = (u_1,\dots,u_k)}$, ${u' = (u_1,\dots,u_{k-1})}$, and the size ${|\lambda|}$ of a partition ${\lambda = (\lambda_1,\dots,\lambda_k)}$ is defined as ${|\lambda| := \lambda_1 + \dots + \lambda_k}$.

One can use this recursion (2) to prove some further standard identities for Schur polynomials, such as the determinant identity

$\displaystyle s_\lambda(u) V(u) = \det( u_i^{\lambda_j+k-j} )_{1 \leq i,j \leq k} \ \ \ \ \ (3)$

for ${u=(u_1,\dots,u_k)}$, where ${V(u)}$ denotes the Vandermonde determinant

$\displaystyle V(u) := \prod_{1 \leq i < j \leq k} (u_i - u_j), \ \ \ \ \ (4)$

or the Jacobi-Trudi identity

$\displaystyle s_\lambda(u) = \det( h_{\lambda_j - j + i}(u) )_{1 \leq i,j \leq k}, \ \ \ \ \ (5)$

with the convention that ${h_d(u) = 0}$ if ${d}$ is negative. Thus for instance

$\displaystyle s_{(1,1,0,\dots,0)}(u) = h_1^2(u) - h_0(u) h_2(u) = \sum_{1 \leq i < j \leq k} u_i u_j.$

We review the (standard) derivation of these identities via (2) below the fold. Among other things, these identities show that the Schur polynomials are symmetric, which is not immediately obvious from their definition.

One can also iterate (2) to write

$\displaystyle s_\lambda(u) = \sum_{() = \lambda^0 \prec \lambda^1 \prec \dots \prec \lambda^k = \lambda} \prod_{j=1}^k u_j^{|\lambda^j| - |\lambda^{j-1}|} \ \ \ \ \ (6)$

where the sum is over all tuples ${\lambda^1,\dots,\lambda^k}$, where each ${\lambda^j}$ is a partition of length ${j}$ that intersperses the next partition ${\lambda^{j+1}}$, with ${\lambda^k}$ set equal to ${\lambda}$. We will call such a tuple an integral Gelfand-Tsetlin pattern based at ${\lambda}$.

One can generalise (6) by introducing the skew Schur functions

$\displaystyle s_{\lambda/\mu}(u) := \sum_{\mu = \lambda^i \prec \dots \prec \lambda^k = \lambda} \prod_{j=i+1}^k u_j^{|\lambda^j| - |\lambda^{j-1}|} \ \ \ \ \ (7)$

for ${u = (u_{i+1},\dots,u_k)}$, whenever ${\lambda}$ is a partition of length ${k}$ and ${\mu}$ a partition of length ${i}$ for some ${0 \leq i \leq k}$, thus the Schur polynomial ${s_\lambda}$ is also the skew Schur polynomial ${s_{\lambda /()}}$ with ${i=0}$. (One could relabel the variables here to be something like ${(u_1,\dots,u_{k-i})}$ instead, but this labeling seems slightly more natural, particularly in view of identities such as (8) below.)

By construction, we have the decomposition

$\displaystyle s_{\lambda/\nu}(u_{i+1},\dots,u_k) = \sum_\mu s_{\mu/\nu}(u_{i+1},\dots,u_j) s_{\lambda/\mu}(u_{j+1},\dots,u_k) \ \ \ \ \ (8)$

whenever ${0 \leq i \leq j \leq k}$, and ${\nu, \mu, \lambda}$ are partitions of lengths ${i,j,k}$ respectively. This gives another recursive way to understand Schur polynomials and skew Schur polynomials. For instance, one can use it to establish the generalised Jacobi-Trudi identity

$\displaystyle s_{\lambda/\mu}(u) = \det( h_{\lambda_j - j - \mu_i + i}(u) )_{1 \leq i,j \leq k}, \ \ \ \ \ (9)$

with the convention that ${\mu_i = 0}$ for ${i}$ larger than the length of ${\mu}$; we do this below the fold.

The Schur polynomials (and skew Schur polynomials) are “discretised” (or “quantised”) in the sense that their parameters ${\lambda, \mu}$ are required to be integer-valued, and their definition similarly involves summation over a discrete set. It turns out that there are “continuous” (or “classical”) analogues of these functions, in which the parameters ${\lambda,\mu}$ now take real values rather than integers, and are defined via integration rather than summation. One can view these continuous analogues as a “semiclassical limit” of their discrete counterparts, in a manner that can be made precise using the machinery of geometric quantisation, but we will not do so here.

The continuous analogues can be defined as follows. Define a real partition of length ${k}$ to be a tuple ${\lambda = (\lambda_1,\dots,\lambda_k)}$ where ${\lambda_1 \geq \dots \geq \lambda_k \geq 0}$ are now real numbers. We can define the relation ${\lambda' \prec \lambda}$ of interspersion between a length ${k-1}$ real partition ${\lambda' = (\lambda'_1,\dots,\lambda'_{k-1})}$ and a length ${k}$ real partition ${\lambda = (\lambda_1,\dots,\lambda_{k})}$ precisely as before, by requiring that the inequalities (1) hold for all ${1 \leq i \leq k-1}$. We can then define the continuous Schur functions ${S_\lambda(x)}$ for ${x = (x_1,\dots,x_k) \in {\bf R}^k}$ recursively by defining

$\displaystyle S_{()}() = 1$

and

$\displaystyle S_\lambda(x) = \int_{\lambda' \prec \lambda} S_{\lambda'}(x') \exp( (|\lambda| - |\lambda'|) x_k ) \ \ \ \ \ (10)$

for ${k \geq 1}$ and ${\lambda}$ of length ${k}$, where ${x' := (x_1,\dots,x_{k-1})}$ and the integral is with respect to ${k-1}$-dimensional Lebesgue measure, and ${|\lambda| = \lambda_1 + \dots + \lambda_k}$ as before. Thus for instance

$\displaystyle S_{(\lambda_1)}(x_1) = \exp( \lambda_1 x_1 )$

and

$\displaystyle S_{(\lambda_1,\lambda_2)}(x_1,x_2) = \int_{\lambda_2}^{\lambda_1} \exp( \lambda'_1 x_1 + (\lambda_1+\lambda_2-\lambda'_1) x_2 )\ d\lambda'_1.$

More generally, we can define the continuous skew Schur functions ${S_{\lambda/\mu}(x)}$ for ${\lambda}$ of length ${k}$, ${\mu}$ of length ${j \leq k}$, and ${x = (x_{j+1},\dots,x_k) \in {\bf R}^{k-j}}$ recursively by defining

$\displaystyle S_{\mu/\mu}() = 1$

and

$\displaystyle S_{\lambda/\mu}(x) = \int_{\lambda' \prec \lambda} S_{\lambda'/\mu}(x') \exp( (|\lambda| - |\lambda'|) x_k )$

for ${k > j}$. Thus for instance

$\displaystyle S_{(\lambda_1,\lambda_2,\lambda_3)/(\mu_1,\mu_2)}(x_3) = 1_{\lambda_3 \leq \mu_2 \leq \lambda_2 \leq \mu_1 \leq \lambda_1} \exp( x_3 (\lambda_1+\lambda_2+\lambda_3 - \mu_1 - \mu_2 ))$

and

$\displaystyle S_{(\lambda_1,\lambda_2,\lambda_3)/(\mu_1)}(x_2, x_3) = \int_{\lambda_2 \leq \lambda'_2 \leq \lambda_2, \mu_1} \int_{\mu_1, \lambda_2 \leq \lambda'_1 \leq \lambda_1}$

$\displaystyle \exp( x_2 (\lambda'_1+\lambda'_2 - \mu_1) + x_3 (\lambda_1+\lambda_2+\lambda_3 - \lambda'_1 - \lambda'_2))\ d\lambda'_1 d\lambda'_2.$

By expanding out the recursion, one obtains the analogue

$\displaystyle S_\lambda(x) = \int_{\lambda^1 \prec \dots \prec \lambda^k = \lambda} \exp( \sum_{j=1}^k x_j (|\lambda^j| - |\lambda^{j-1}|))\ d\lambda^1 \dots d\lambda^{k-1},$

of (6), and more generally one has

$\displaystyle S_{\lambda/\mu}(x) = \int_{\mu = \lambda^i \prec \dots \prec \lambda^k = \lambda} \exp( \sum_{j=i+1}^k x_j (|\lambda^j| - |\lambda^{j-1}|))\ d\lambda^{i+1} \dots d\lambda^{k-1}.$

We will call the tuples ${(\lambda^1,\dots,\lambda^k)}$ in the first integral real Gelfand-Tsetlin patterns based at ${\lambda}$. The analogue of (8) is then

$\displaystyle S_{\lambda/\nu}(x_{i+1},\dots,x_k) = \int S_{\mu/\nu}(x_{i+1},\dots,x_j) S_{\lambda/\mu}(x_{j+1},\dots,x_k)\ d\mu$

where the integral is over all real partitions ${\mu}$ of length ${j}$, with Lebesgue measure.

By approximating various integrals by their Riemann sums, one can relate the continuous Schur functions to their discrete counterparts by the limiting formula

$\displaystyle N^{-k(k-1)/2} s_{\lfloor N \lambda \rfloor}( \exp[ x/N ] ) \rightarrow S_\lambda(x) \ \ \ \ \ (11)$

as ${N \rightarrow \infty}$ for any length ${k}$ real partition ${\lambda = (\lambda_1,\dots,\lambda_k)}$ and any ${x = (x_1,\dots,x_k) \in {\bf R}^k}$, where

$\displaystyle \lfloor N \lambda \rfloor := ( \lfloor N \lambda_1 \rfloor, \dots, \lfloor N \lambda_k \rfloor )$

and

$\displaystyle \exp[x/N] := (\exp(x_1/N), \dots, \exp(x_k/N)).$

More generally, one has

$\displaystyle N^{j(j-1)/2-k(k-1)/2} s_{\lfloor N \lambda \rfloor / \lfloor N \mu \rfloor}( \exp[ x/N ] ) \rightarrow S_{\lambda/\mu}(x)$

as ${N \rightarrow \infty}$ for any length ${k}$ real partition ${\lambda}$, any length ${j}$ real partition ${\mu}$ with ${0 \leq j \leq k}$, and any ${x = (x_{j+1},\dots,x_k) \in {\bf R}^{k-j}}$.

As a consequence of these limiting formulae, one expects all of the discrete identities above to have continuous counterparts. This is indeed the case; below the fold we shall prove the discrete and continuous identities in parallel. These are not new results by any means, but I was not able to locate a good place in the literature where they are explicitly written down, so I thought I would try to do so here (primarily for my own internal reference, but perhaps the calculations will be worthwhile to some others also).

The determinant ${\det_n(A)}$ of an ${n \times n}$ matrix (with coefficients in an arbitrary field) obey many useful identities, starting of course with the fundamental multiplicativity ${\det_n(AB) = \det_n(A) \det_n(B)}$ for ${n \times n}$ matrices ${A,B}$. This multiplicativity can in turn be used to establish many further identities; in particular, as shown in this previous post, it implies the Schur determinant identity

$\displaystyle \det_{n+k}\begin{pmatrix} A & B \\ C & D \end{pmatrix} = \det_n(A) \det_k( D - C A^{-1} B ) \ \ \ \ \ (1)$

whenever ${A}$ is an invertible ${n \times n}$ matrix, ${B}$ is an ${n \times k}$ matrix, ${C}$ is a ${k \times n}$ matrix, and ${D}$ is a ${k \times k}$ matrix. The matrix ${D - CA^{-1} B}$ is known as the Schur complement of the block ${A}$.

I only recently discovered that this identity in turn immediately implies what I always found to be a somewhat curious identity, namely the Dodgson condensation identity (also known as the Desnanot-Jacobi identity)

$\displaystyle \det_n(M) \det_{n-2}(M^{1,n}_{1,n}) = \det_{n-1}( M^1_1 ) \det_{n-1}(M^n_n)$

$\displaystyle - \det_{n-1}(M^1_n) \det_{n-1}(M^n_1)$

for any ${n \geq 3}$ and ${n \times n}$ matrix ${M}$, where ${M^i_j}$ denotes the ${n-1 \times n-1}$ matrix formed from ${M}$ by removing the ${i^{th}}$ row and ${j^{th}}$ column, and similarly ${M^{i,i'}_{j,j'}}$ denotes the ${n-2 \times n-2}$ matrix formed from ${M}$ by removing the ${i^{th}}$ and ${(i')^{th}}$ rows and ${j^{th}}$ and ${(j')^{th}}$ columns. Thus for instance when ${n=3}$ we obtain

$\displaystyle \det_3 \begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix} \cdot e$

$\displaystyle = \det_2 \begin{pmatrix} e & f \\ h & i \end{pmatrix} \cdot \det_2 \begin{pmatrix} a & b \\ d & e \end{pmatrix}$

$\displaystyle - \det_2 \begin{pmatrix} b & c \\ e & f \end{pmatrix} \cdot \det_2 \begin{pmatrix} d & e \\ g & h \end{pmatrix}$

for any scalars ${a,b,c,d,e,f,g,h,i}$. (Charles Dodgson, better known by his pen name Lewis Caroll, is of course also known for writing “Alice in Wonderland” and “Through the Looking Glass“.)

The derivation is not new; it is for instance noted explicitly in this paper of Brualdi and Schneider, though I do not know if this is the earliest place in the literature where it can be found. (EDIT: Apoorva Khare has pointed out to me that the original arguments of Dodgson can be interpreted as implicitly following this derivation.) I thought it is worth presenting the short derivation here, though.

Firstly, by swapping the first and ${(n-1)^{th}}$ rows, and similarly for the columns, it is easy to see that the Dodgson condensation identity is equivalent to the variant

$\displaystyle \det_n(M) \det_{n-2}(M^{n-1,n}_{n-1,n}) = \det_{n-1}( M^{n-1}_{n-1} ) \det_{n-1}(M^n_n) \ \ \ \ \ (2)$

$\displaystyle - \det_{n-1}(M^{n-1}_n) \det_{n-1}(M^n_{n-1}).$

Now write

$\displaystyle M = \begin{pmatrix} A & B_1 & B_2 \\ C_1 & d_{11} & d_{12} \\ C_2 & d_{21} & d_{22} \end{pmatrix}$

where ${A}$ is an ${n-2 \times n-2}$ matrix, ${B_1, B_2}$ are ${n-2 \times 1}$ column vectors, ${C_1, C_2}$ are ${1 \times n-2}$ row vectors, and ${d_{11}, d_{12}, d_{21}, d_{22}}$ are scalars. If ${A}$ is invertible, we may apply the Schur determinant identity repeatedly to conclude that

$\displaystyle \det_n(M) = \det_{n-2}(A) \det_2 \begin{pmatrix} d_{11} - C_1 A^{-1} B_1 & d_{12} - C_1 A^{-1} B_2 \\ d_{21} - C_2 A^{-1} B_1 & d_{22} - C_2 A^{-1} B_2 \end{pmatrix}$

$\displaystyle \det_{n-2} (M^{n-1,n}_{n-1,n}) = \det_{n-2}(A)$

$\displaystyle \det_{n-1}( M^{n-1}_{n-1} ) = \det_{n-2}(A) (d_{22} - C_2 A^{-1} B_2 )$

$\displaystyle \det_{n-1}( M^{n-1}_{n} ) = \det_{n-2}(A) (d_{21} - C_2 A^{-1} B_1 )$

$\displaystyle \det_{n-1}( M^{n}_{n-1} ) = \det_{n-2}(A) (d_{12} - C_1 A^{-1} B_2 )$

$\displaystyle \det_{n-1}( M^{n}_{n} ) = \det_{n-2}(A) (d_{11} - C_1 A^{-1} B_1 )$

and the claim (2) then follows by a brief calculation (and the explicit form ${\det_2 \begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad-bc}$ of the ${2 \times 2}$ determinant). To remove the requirement that ${A}$ be invertible, one can use a limiting argument, noting that one can work without loss of generality in an algebraically closed field, and in such a field, the set of invertible matrices is dense in the Zariski topology. (In the case when the scalars are reals or complexes, one can just use density in the ordinary topology instead if desired.)

The same argument gives the more general determinant identity of Sylvester

$\displaystyle \det_n(M) \det_{n-k}(M^S_S)^{k-1} = \det_k \left( \det_{n-k+1}(M^{S \backslash \{i\}}_{S \backslash \{j\}}) \right)_{i,j \in S}$

whenever ${n > k \geq 1}$, ${S}$ is a ${k}$-element subset of ${\{1,\dots,n\}}$, and ${M^S_{S'}}$ denotes the matrix formed from ${M}$ by removing the rows associated to ${S}$ and the columns associated to ${S'}$. (The Dodgson condensation identity is basically the ${k=2}$ case of this identity.)

A closely related proof of (2) proceeds by elementary row and column operations. Observe that if one adds some multiple of one of the first ${n-2}$ rows of ${M}$ to one of the last two rows of ${M}$, then the left and right sides of (2) do not change. If the minor ${A}$ is invertible, this allows one to reduce to the case where the components ${C_1,C_2}$ of the matrix vanish. Similarly, using elementary column operations instead of row operations we may assume that ${B_1,B_2}$ vanish. All matrices involved are now block-diagonal and the identity follows from a routine computation.

The latter approach can also prove the cute identity

$\displaystyle \det_2 \begin{pmatrix} \det_n( X_1, Y_1, A ) & \det_n( X_1, Y_2, A ) \\ \det_n(X_2, Y_1, A) & \det_n(X_2,Y_2, A) \end{pmatrix} = \det_n( X_1,X_2,A) \det_n(Y_1,Y_2,A)$

for any ${n \geq 2}$, any ${n \times 1}$ column vectors ${X_1,X_2,Y_1,Y_2}$, and any ${n \times n-2}$ matrix ${A}$, which can for instance be found in page 7 of this text of Karlin. Observe that both sides of this identity are unchanged if one adds some multiple of any column of ${A}$ to one of ${X_1,X_2,Y_1,Y_2}$; for generic ${A}$, this allows one to reduce ${X_1,X_2,Y_1,Y_2}$ to have only the first two entries allowed to be non-zero, at which point the determinants split into ${2 \times 2}$ and ${n -2 \times n-2}$ determinants and we can reduce to the ${n=2}$ case (eliminating the role of ${A}$). One can now either proceed by a direct computation, or by observing that the left-hand side is quartilinear in ${X_1,X_2,Y_1,Y_2}$ and antisymmetric in ${X_1,X_2}$ and ${Y_1,Y_2}$ which forces it to be a scalar multiple of ${\det_2(X_1,X_2) \det_2(Y_1,Y_2)}$, at which point one can test the identity at a single point (e.g. ${X_1=Y_1 = e_1}$ and ${X_2=Y_2=e_2}$ for the standard basis ${e_1,e_2}$) to conclude the argument. (One can also derive this identity from the Sylvester determinant identity but I think the calculations are a little messier if one goes by that route. Conversely, one can recover the Dodgson condensation identity from Karlin’s identity by setting ${X_1=e_1}$, ${X_2=e_2}$ (for instance) and then permuting some rows and columns.)

In one of the earliest posts on this blog, I talked about the ability to “arbitrage” a disparity of symmetry in an inequality, and in particular to “amplify” such an inequality into a stronger one. (The principle can apply to other mathematical statements than inequalities, with the “hypothesis” and “conclusion” of that statement generally playing the role of the “right-hand side” and “left-hand side” of an inequality, but for sake of discussion I will restrict attention here to inequalities.) One can formalise this principle as follows. Many inequalities in analysis can be expressed in the form

$\displaystyle A(f) \leq B(f) \ \ \ \ \ (1)$

for all ${f}$ in some space ${X}$ (in many cases ${X}$ will be a function space, and ${f}$ a function in that space), where ${A(f)}$ and ${B(f)}$ are some functionals of ${f}$ (that is to say, real-valued functions of ${f}$). For instance, ${B(f)}$ might be some function space norm of ${f}$ (e.g. an ${L^p}$ norm), and ${A(f)}$ might be some function space norm of some transform of ${f}$. In addition, we assume we have some group ${G}$ of symmetries ${T: X \rightarrow X}$ acting on the underlying space. For instance, if ${X}$ is a space of functions on some spatial domain, the group might consist of translations (e.g. ${Tf(x) = f(x-h)}$ for some shift ${h}$), or perhaps dilations with some normalisation (e.g. ${Tf(x) = \frac{1}{\lambda^\alpha} f(\frac{x}{\lambda})}$ for some dilation factor ${\lambda > 0}$ and some normalisation exponent ${\alpha \in {\bf R}}$, which can be thought of as the dimensionality of length one is assigning to ${f}$). If we have

$\displaystyle A(Tf) = A(f)$

for all symmetries ${T \in G}$ and all ${f \in X}$, we say that ${A}$ is invariant with respect to the symmetries in ${G}$; otherwise, it is not.

Suppose we know that the inequality (1) holds for all ${f \in X}$, but that there is an imbalance of symmetry: either ${A}$ is ${G}$-invariant and ${B}$ is not, or vice versa. Suppose first that ${A}$ is ${G}$-invariant and ${B}$ is not. Substituting ${f}$ by ${Tf}$ in (1) and taking infima, we can then amplify (1) to the stronger inequality

$\displaystyle A(f) \leq \inf_{T \in G} B(Tf).$

In particular, it is often the case that there is a way to send ${T}$ off to infinity in such a way that the functional ${B(Tf)}$ has a limit ${B_\infty(f)}$, in which case we obtain the amplification

$\displaystyle A(f) \leq B_\infty(f) \ \ \ \ \ (2)$

of (1). Note that these amplified inequalities will now be ${G}$-invariant on both sides (assuming that the way in which we take limits as ${T \rightarrow \infty}$ is itself ${G}$-invariant, which it often is in practice). Similarly, if ${B}$ is ${G}$-invariant but ${A}$ is not, we may instead amplify (1) to

$\displaystyle \sup_{T \in G} A(Tf) \leq B(f)$

and in particular (if ${A(Tf)}$ has a limit ${A_\infty(f)}$ as ${T \rightarrow \infty}$)

$\displaystyle A_\infty(f) \leq B(f). \ \ \ \ \ (3)$

If neither ${A(f)}$ nor ${B(f)}$ has a ${G}$-symmetry, one can still use the ${G}$-symmetry by replacing ${f}$ by ${Tf}$ and taking a limit to conclude that

$\displaystyle A_\infty(f) \leq B_\infty(f),$

though now this inequality is not obviously stronger than the original inequality (1) (for instance it could well be trivial). In some cases one can also average over ${G}$ instead of taking a limit as ${T \rightarrow \infty}$, thus averaging a non-invariant inequality into an invariant one.

As discussed in the previous post, this use of amplification gives rise to a general principle about inequalities: the most efficient inequalities are those in which the left-hand side and right-hand side enjoy the same symmetries. It is certainly possible to have true inequalities that have an imbalance of symmetry, but as shown above, such inequalities can always be amplified to more efficient and more symmetric inequalities. In the case when limits such as ${A_\infty}$ and ${B_\infty}$ exist, the limiting functionals ${A_\infty(f)}$ and ${B_\infty(f)}$ are often simpler in form, or more tractable analytically, than their non-limiting counterparts ${A(f)}$ and ${B(f)}$ (this is one of the main reasons why we take limits at infinity in the first place!), and so in many applications there is really no reason to use the weaker and more complicated inequality (1), when stronger, simpler, and more symmetric inequalities such as (2), (3) are available. Among other things, this explains why many of the most useful and natural inequalities one sees in analysis are dimensionally consistent.

One often tries to prove inequalities (1) by directly chaining together simpler inequalities. For instance, one might attempt to prove (1) by by first bounding ${A(f)}$ by some auxiliary quantity ${C(f)}$, and then bounding ${C(f)}$ by ${B(f)}$, thus obtaining (1) by chaining together two inequalities

$\displaystyle A(f) \leq C(f) \leq B(f). \ \ \ \ \ (4)$

A variant of the above principle then asserts that when proving inequalities by such direct methods, one should, whenever possible, try to maintain the symmetries that are present in both sides of the inequality. Why? Well, suppose that we ignored this principle and tried to prove (1) by establishing (4) for some ${C}$ that is not ${G}$-invariant. Assuming for sake of argument that (4) were actually true, we could amplify the first half ${A(f) \leq C(f)}$ of this inequality to conclude that

$\displaystyle A(f) \leq \inf_{T \in G} C(Tf)$

and also amplify the second half ${C(f) \leq B(f)}$ of the inequality to conclude that

$\displaystyle \sup_{T \in G} C(Tf) \leq B(f)$

and hence (4) amplifies to

$\displaystyle A(f) \leq \inf_{T \in G} C(Tf) \leq \sup_{T \in G} C(Tf) \leq B(f). \ \ \ \ \ (5)$

Let’s say for sake of argument that all the quantities involved here are positive numbers (which is often the case in analysis). Then we see in particular that

$\displaystyle \frac{\sup_{T \in G} C(Tf)}{\inf_{T \in G} C(Tf)} \leq \frac{B(f)}{A(f)}. \ \ \ \ \ (6)$

Informally, (6) asserts that in order for the strategy (4) used to prove (1) to work, the extent to which ${C}$ fails to be ${G}$-invariant cannot exceed the amount of “room” present in (1). In particular, when dealing with those “extremal” ${f}$ for which the left and right-hand sides of (1) are comparable to each other, one can only have a bounded amount of non-${G}$-invariance in the functional ${C}$. If ${C}$ fails so badly to be ${G}$-invariant that one does not expect the left-hand side of (6) to be at all bounded in such extremal situations, then the strategy of proving (1) using the intermediate quantity ${C}$ is doomed to failure – even if one has already produced some clever proof of one of the two inequalities ${A(f) \leq C(f)}$ or ${C(f) \leq B(f)}$ needed to make this strategy work. And even if it did work, one could amplify (4) to a simpler inequality

$\displaystyle A(f) \leq C_\infty(f) \leq B(f) \ \ \ \ \ (7)$

(assuming that the appropriate limit ${C_\infty(f) = \lim_{T \rightarrow \infty} C(Tf)}$ existed) which would likely also be easier to prove (one can take whatever proofs one had in mind of the inequalities in (4), conjugate them by ${T}$, and take a limit as ${T \rightarrow \infty}$ to extract a proof of (7)).

Here are some simple (but somewhat contrived) examples to illustrate these points. Suppose one wishes to prove the inequality

$\displaystyle xy \leq x^2 + y^2 \ \ \ \ \ (8)$

for all ${x,y>0}$. Both sides of this inequality are invariant with respect to interchanging ${x}$ with ${y}$, so the principle suggests that when proving this inequality directly, one should only use sub-inequalities that are also invariant with respect to this interchange. However, in this particular case there is enough “room” in the inequality that it is possible (though somewhat unnatural) to violate this principle. For instance, one could decide (for whatever reason) to start with the inequality

$\displaystyle 0 \leq (x - y/2)^2 = x^2 - xy + y^2/4$

to conclude that

$\displaystyle xy \leq x^2 + y^2/4$

and then use the obvious inequality ${x^2 + y^2/4 \leq x^2+y^2}$ to conclude the proof. Here, the intermediate quantity ${x^2 + y^2/4}$ is not invariant with respect to interchange of ${x}$ and ${y}$, but the failure is fairly mild (changing ${x}$ and ${y}$ only modifies the quantity ${x^2 + y^2/4}$ by a multiplicative factor of ${4}$ at most), and disappears completely in the most extremal case ${x=y}$, which helps explain why one could get away with using this quantity in the proof here. But it would be significantly harder (though still not impossible) to use non-symmetric intermediaries to prove the sharp version

$\displaystyle xy \leq \frac{x^2 + y^2}{2}$

of (8) (that is to say, the arithmetic mean-geometric mean inequality). Try it!

Similarly, consider the task of proving the triangle inequality

$\displaystyle |z+w| \leq |z| + |w| \ \ \ \ \ (9)$

for complex numbers ${z, w}$. One could try to leverage the triangle inequality ${|x+y| \leq |x| + |y|}$ for real numbers by using the crude estimate

$\displaystyle |z+w| \leq |\hbox{Re}(z+w)| + |\hbox{Im}(z+w)|$

and then use the real triangle inequality to obtain

$\displaystyle |\hbox{Re}(z+w)| \leq |\hbox{Re}(z)| + |\hbox{Re}(w)|$

and

$\displaystyle |\hbox{Im}(z+w)| \leq |\hbox{Im}(z)| + |\hbox{Im}(w)|$

and then finally use the inequalities

$\displaystyle |\hbox{Re}(z)|, |\hbox{Im}(z)| \leq |z| \ \ \ \ \ (10)$

and

$\displaystyle |\hbox{Re}(w)|, |\hbox{Im}(w)| \leq |w| \ \ \ \ \ (11)$

but when one puts this all together at the end of the day, one loses a factor of two:

$\displaystyle |z+w| \leq 2(|z| + |w|).$

One can “blame” this loss on the fact that while the original inequality (9) was invariant with respect to phase rotation ${(z,w) \mapsto (e^{i\theta} z, e^{i\theta} w)}$, the intermediate expressions we tried to use when proving it were not, leading to inefficient estimates. One can try to be smarter than this by using Pythagoras’ theorem ${|z|^2 = |\hbox{Re}(z)|^2 + |\hbox{Im}(z)|^2}$; this reduces the loss from ${2}$ to ${\sqrt{2}}$ but does not eliminate it completely, which is to be expected as one is still using non-invariant estimates in the proof. But one can remove the loss completely by using amplification; see the previous blog post for details (we also give a reformulation of this amplification below).

Here is a slight variant of the above example. Suppose that you had just learned in class to prove the triangle inequality

$\displaystyle (\sum_{n=1}^\infty |a_n+b_n|^2)^{1/2} \leq (\sum_{n=1}^\infty |a_n|^2)^{1/2} + (\sum_{n=1}^\infty |b_n|^2)^{1/2} \ \ \ \ \ (12)$

for (say) real square-summable sequences ${(a_n)_{n=1}^\infty}$, ${(b_n)_{n=1}^\infty}$, and was tasked to conclude the corresponding inequality

$\displaystyle (\sum_{n \in {\bf Z}} |a_n+b_n|^2)^{1/2} \leq (\sum_{n \in {\bf Z}} |a_n|^2)^{1/2} + (\sum_{n \in {\bf Z}} |b_n|^2)^{1/2} \ \ \ \ \ (13)$

for doubly infinite square-summable sequences ${(a_n)_{n \in {\bf Z}}, (b_n)_{n \in {\bf Z}}}$. The quickest way to do this is of course to exploit a bijection between the natural numbers ${1,2,\dots}$ and the integers, but let us say for sake of argument that one was unaware of such a bijection. One could then proceed instead by splitting the integers into the positive integers and the non-positive integers, and use (12) on each component separately; this is very similar to the strategy of proving (9) by splitting a complex number into real and imaginary parts, and will similarly lose a factor of ${2}$ or ${\sqrt{2}}$. In this case, one can “blame” this loss on the abandonment of translation invariance: both sides of the inequality (13) are invariant with respect to shifting the sequences ${(a_n)_{n \in {\bf Z}}}$, ${(b_n)_{n \in {\bf Z}}}$ by some shift ${h}$ to arrive at ${(a_{n-h})_{n \in {\bf Z}}, (b_{n-h})_{n \in {\bf Z}}}$, but the intermediate quantities caused by splitting the integers into two subsets are not invariant. Another way of thinking about this is that the splitting of the integers gives a privileged role to the origin ${n=0}$, whereas the inequality (13) treats all values of ${n}$ equally thanks to the translation invariance, and so using such a splitting is unnatural and not likely to lead to optimal estimates. On the other hand, one can deduce (13) from (12) by sending this symmetry to infinity; indeed, after applying a shift to (12) we see that

$\displaystyle (\sum_{n=-N}^\infty |a_n+b_n|^2)^{1/2} \leq (\sum_{n=-N}^\infty |a_n|^2)^{1/2} + (\sum_{n=-N}^\infty |b_n|^2)^{1/2}$

for any ${N}$, and on sending ${N \rightarrow \infty}$ we obtain (13) (one could invoke the monotone convergence theorem here to justify the limit, though in this case it is simple enough that one can just use first principles).

Note that the principle of preserving symmetry only applies to direct approaches to proving inequalities such as (1). There is a complementary approach, discussed for instance in this previous post, which is to spend the symmetry to place the variable ${f}$ “without loss of generality” in a “normal form”, “convenient coordinate system”, or a “good gauge”. Abstractly: suppose that there is some subset ${Y}$ of ${X}$ with the property that every ${f \in X}$ can be expressed in the form ${f = Tg}$ for some ${T \in G}$ and ${g \in Y}$ (that is to say, ${X = GY}$). Then, if one wishes to prove an inequality (1) for all ${f \in X}$, and one knows that both sides ${A(f), B(f)}$ of this inequality are ${G}$-invariant, then it suffices to check (1) just for those ${f}$ in ${Y}$, as this together with the ${G}$-invariance will imply the same inequality (1) for all ${f}$ in ${GY=X}$. By restricting to those ${f}$ in ${Y}$, one has given up (or spent) the ${G}$-invariance, as the set ${Y}$ will in typical not be preserved by the group action ${G}$. But by the same token, by eliminating the invariance, one also eliminates the prohibition on using non-invariant proof techniques, and one is now free to use a wider range of inequalities in order to try to establish (1). Of course, such inequalities should make crucial use of the restriction ${f \in Y}$, for if they did not, then the arguments would work in the more general setting ${f \in X}$, and then the previous principle would again kick in and warn us that the use of non-invariant inequalities would be inefficient. Thus one should “spend” the symmetry wisely to “buy” a restriction ${f \in Y}$ that will be of maximal utility in calculations (for instance by setting as many annoying factors and terms in one’s analysis to be ${0}$ or ${1}$ as possible).

As a simple example of this, let us revisit the complex triangle inequality (9). As already noted, both sides of this inequality are invariant with respect to the phase rotation symmetry ${(z,w) \mapsto (e^{i\theta} z, e^{i\theta} w)}$. This seems to limit one to using phase-rotation-invariant techniques to establish the inequality, in particular ruling out the use of real and imaginary parts as discussed previously. However, we can instead spend the phase rotation symmetry to restrict to a special class of ${z}$ and ${w}$. It turns out that the most efficient way to spend the symmetry is to achieve the normalisation of ${z+w}$ being a nonnegative real; this is of course possible since any complex number ${z+w}$ can be turned into a nonnegative real by multiplying by an appropriate phase ${e^{i\theta}}$. Once ${z+w}$ is a nonnegative real, the imaginary part disappears and we have

$\displaystyle |z+w| = \hbox{Re}(z+w) = \hbox{Re}(z) + \hbox{Re}(w),$

and the triangle inequality (9) is now an immediate consequence of (10), (11). (But note that if one had unwisely spent the symmetry to normalise, say, ${z}$ to be a non-negative real, then one is no closer to establishing (9) than before one had spent the symmetry.)

The complete homogeneous symmetric polynomial ${h_d(x_1,\dots,x_n)}$ of ${n}$ variables ${x_1,\dots,x_n}$ and degree ${d}$ can be defined as

$\displaystyle h_d(x_1,\dots,x_n) := \sum_{1 \leq i_1 \leq \dots \leq i_d \leq n} x_{i_1} \dots x_{i_d},$

thus for instance

$\displaystyle h_0(x_1,\dots,x_n) = 0,$

$\displaystyle h_1(x_1,\dots,x_n) = x_1 + \dots + x_n,$

and

$\displaystyle h_2(x_1,\dots,x_n) = x_1^2 + \dots + x_n^2 + \sum_{1 \leq i < j \leq n} x_i x_j.$

One can also define all the complete homogeneous symmetric polynomials of ${n}$ variables simultaneously by means of the generating function

$\displaystyle \sum_{d=0}^\infty h_d(x_1,\dots,x_n) t^d = \frac{1}{(1-t x_1) \dots (1-tx_n)}. \ \ \ \ \ (1)$

We will think of the variables ${x_1,\dots,x_n}$ as taking values in the real numbers. When one does so, one might observe that the degree two polynomial ${h_2}$ is a positive definite quadratic form, since it has the sum of squares representation

$\displaystyle h_2(x_1,\dots,x_n) = \frac{1}{2} \sum_{i=1}^n x_i^2 + \frac{1}{2} (x_1+\dots+x_n)^2.$

In particular, ${h_2(x_1,\dots,x_n) > 0}$ unless ${x_1=\dots=x_n=0}$. This can be compared against the superficially similar quadratic form

$\displaystyle x_1^2 + \dots + x_n^2 + \sum_{1 \leq i < j \leq n} \epsilon_{ij} x_i x_j$

where ${\epsilon_{ij} = \pm 1}$ are independent randomly chosen signs. The Wigner semicircle law says that for large ${n}$, the eigenvalues of this form will be mostly distributed in the interval ${[-\sqrt{n}, \sqrt{n}]}$ using the semicircle distribution, so in particular the form is quite far from being positive definite despite the presence of the first ${n}$ positive terms. Thus the positive definiteness is coming from the finer algebraic structure of ${h_2}$, and not just from the magnitudes of its coefficients.

One could ask whether the same positivity holds for other degrees ${d}$ than two. For odd degrees, the answer is clearly no, since ${h_d(-x_1,\dots,-x_n) = -h_d(x_1,\dots,x_n)}$ in that case. But one could hope for instance that

$\displaystyle h_4(x_1,\dots,x_n) = \sum_{1 \leq i \leq j \leq k \leq l \leq n} x_i x_j x_k x_l$

also has a sum of squares representation that demonstrates positive definiteness. This turns out to be true, but is remarkably tedious to establish directly. Nevertheless, we have a nice result of Hunter that gives positive definiteness for all even degrees ${d}$. In fact, a modification of his argument gives a little bit more:

Theorem 1 Let ${n \geq 1}$, let ${d \geq 0}$ be even, and let ${x_1,\dots,x_n}$ be reals.

• (i) (Positive definiteness) One has ${h_d(x_1,\dots,x_n) \geq 0}$, with strict inequality unless ${x_1=\dots=x_n=0}$.
• (ii) (Schur convexity) One has ${h_d(x_1,\dots,x_n) \geq h_d(y_1,\dots,y_n)}$ whenever ${(x_1,\dots,x_n)}$ majorises ${(y_1,\dots,y_n)}$, with equality if and only if ${(y_1,\dots,y_n)}$ is a permutation of ${(x_1,\dots,x_n)}$.
• (iii) (Schur-Ostrowski criterion for Schur convexity) For any ${1 \leq i < j \leq n}$, one has ${(x_i - x_j) (\frac{\partial}{\partial x_i} - \frac{\partial}{\partial x_j}) h_d(x_1,\dots,x_n) \geq 0}$, with strict inequality unless ${x_i=x_j}$.

Proof: We induct on ${d}$ (allowing ${n}$ to be arbitrary). The claim is trivially true for ${d=0}$, and easily verified for ${d=2}$, so suppose that ${d \geq 4}$ and the claims (i), (ii), (iii) have already been proven for ${d-2}$ (and for arbitrary ${n}$).

If we apply the differential operator ${(x_i - x_j) (\frac{\partial}{\partial x_i} - \frac{\partial}{\partial x_j})}$ to ${\frac{1}{(1-t x_1) \dots (1-tx_n)}}$ using the product rule, one obtains after a brief calculation

$\displaystyle \frac{(x_i-x_j)^2 t^2}{(1-t x_1) \dots (1-tx_n) (1-t x_i) (1-t x_j)}.$

Using (1) and extracting the ${t^d}$ coefficient, we obtain the identity

$\displaystyle (x_i - x_j) (\frac{\partial}{\partial x_i} - \frac{\partial}{\partial x_j}) h_d(x_1,\dots,x_n)$

$\displaystyle = (x_i-x_j)^2 h_{d-2}( x_1,\dots,x_n,x_i,x_j). \ \ \ \ \ (2)$

The claim (iii) then follows from (i) and the induction hypothesis.

To obtain (ii), we use the more general statement (known as the Schur-Ostrowski criterion) that (ii) is implied from (iii) if we replace ${h_d}$ by an arbitrary symmetric, continuously differentiable function. To establish this criterion, we induct on ${n}$ (this argument can be made independently of the existing induction on ${d}$). If ${(y_1,\dots,y_n)}$ is majorised by ${(x_1,\dots,x_n)}$, it lies in the permutahedron of ${(x_1,\dots,x_n)}$. If ${(y_1,\dots,y_n)}$ lies on a face of this permutahedron, then after permuting both the ${x_i}$ and ${y_j}$ we may assume that ${(y_1,\dots,y_m)}$ is majorised by ${(x_1,\dots,x_m)}$, and ${(y_{m+1},\dots,y_n)}$ is majorised by ${(x_{m+1},\dots,x_n)}$ for some ${1 \leq m < n}$, and the claim then follows from two applications of the induction hypothesis. If instead ${(y_1,\dots,y_n)}$ lies in the interior of the permutahedron, one can follow it to the boundary by using one of the vector fields ${(x_i - x_j) (\frac{\partial}{\partial x_i} - \frac{\partial}{\partial x_j})}$, and the claim follows from the boundary case.

Finally, to obtain (i), we observe that ${(x_1,\dots,x_n)}$ majorises ${(x,\dots,x)}$, where ${x}$ is the arithmetic mean of ${x_1,\dots,x_n}$. But ${h_d(x,\dots,x)}$ is clearly a positive multiple of ${x^d}$, and the claim now follows from (ii). $\Box$

If the variables ${x_1,\dots,x_n}$ are restricted to be nonnegative, the same argument gives Schur convexity for odd degrees also.

The proof in Hunter of positive definiteness is arranged a little differently than the one above, but still relies ultimately on the identity (2). I wonder if there is a genuinely different way to establish positive definiteness that does not go through this identity.

A few days ago, I was talking with Ed Dunne, who is currently the Executive Editor of Mathematical Reviews (and in particular with its online incarnation at MathSciNet).  At the time, I was mentioning how laborious it was for me to create a BibTeX file for dozens of references by using MathSciNet to locate each reference separately, and to export each one to BibTeX format.  He then informed me that underneath to every MathSciNet reference there was a little link to add the reference to a Clipboard, and then one could export the entire Clipboard at once to whatever format one wished.  In retrospect, this was a functionality of the site that had always been visible, but I had never bothered to explore it, and now I can populate a BibTeX file much more quickly.

This made me realise that perhaps there are many other useful features of popular mathematical tools out there that only a few users actually know about, so I wanted to create a blog post to encourage readers to post their own favorite tools, or features of tools, that are out there, often in plain sight, but not always widely known.  Here are a few that I was able to recall from my own workflow (though for some of them it took quite a while to consciously remember, since I have been so used to them for so long!):

1. TeX for Gmail.  A Chrome plugin that lets one write TeX symbols in emails sent through Gmail (by writing the LaTeX code and pressing a hotkey, usually F8).
2. Boomerang for Gmail.  Another Chrome plugin for Gmail, which does two main things.  Firstly, it can “boomerang” away an email from your inbox to return at some specified later date (e.g. one week from today).  I found this useful to declutter my inbox regarding mail that I needed to act on in the future, but was unable to deal with at present due to travel, or because I was waiting for some other piece of data to arrive first.   Secondly, it can send out email with some specified delay (e.g. by tomorrow morning), giving one time to cancel the email if necessary.  (Thanks to Julia Wolf for telling me about Boomerang!)
3. Which just reminds me, the Undo Send feature on Gmail has saved me from embarrassment a few times (but one has to set it up first; it delays one’s emails by a short period, such as 30 seconds, during which time it is possible to undo the email).
4. LaTeX rendering in Inkscape.  I used to use plain text to write mathematical formulae in my images, which always looked terrible.  It took me years to realise that Inkscape had the functionality to compile LaTeX within it.
5. Bookmarks in TeXnicCenter.  I probably only use a tiny fraction of the functionality that TeXnicCenter offers, but one little feature I quite like is the ability to bookmark a portion of the TeX file (e.g. the bibliography at the end, or the place one is currently editing) with one hot-key (Ctrl-F2) and then one can cycle quickly between one bookmarked location and another with some further hot-keys (F2 and shift-F2).
6. Actually, there are a number of Windows keyboard shortcuts that are worth experimenting with (and similarly for Mac or Linux systems of course).
7. Detexify has been the quickest way for me to locate the TeX code for a symbol that I couldn’t quite remember (or when hunting for a new symbol that would roughly be shaped like something I had in mind).
8. For writing LaTeX on my blog, I use Luca Trevisan’s LaTeX to WordPress Python script (together with a little batch file I wrote to actually run the python script).
9. Using the camera on my phone to record a blackboard computation or a slide (or the wifi password at a conference centre, or any other piece of information that is written or displayed really).  If the phone is set up properly this can be far quicker than writing it down with pen and paper.  (I guess this particular trick is now quite widely used, but I still see people surprised when someone else uses a phone instead of a pen to record things.)
10. Using my online calendar not only to record scheduled future appointments, but also to block out time to do specific tasks (e.g. reserve 2-3pm at Tuesday to read paper X, or do errand Y).  I have found I am able to get a much larger fraction of my “to do” list done on days in which I had previously blocked out such specific chunks of time, as opposed to days in which I had left several hours unscheduled (though sometimes those hours were also very useful for finding surprising new things to do that I had not anticipated).  (I learned of this little trick online somewhere, but I have long since lost the original reference.)

Anyway, I would very much like to hear what other little tools or features other readers have found useful in their work.

Suppose ${F: X \rightarrow Y}$ is a continuous (but nonlinear) map from one normed vector space ${X}$ to another ${Y}$. The continuity means, roughly speaking, that if ${x_0, x \in X}$ are such that ${\|x-x_0\|_X}$ is small, then ${\|F(x)-F(x_0)\|_Y}$ is also small (though the precise notion of “smallness” may depend on ${x}$ or ${x_0}$, particularly if ${F}$ is not known to be uniformly continuous). If ${F}$ is known to be differentiable (in, say, the Fréchet sense), then we in fact have a linear bound of the form

$\displaystyle \|F(x)-F(x_0)\|_Y \leq C(x_0) \|x-x_0\|_X$

for some ${C(x_0)}$ depending on ${x_0}$, if ${\|x-x_0\|_X}$ is small enough; one can of course make ${C(x_0)}$ independent of ${x_0}$ (and drop the smallness condition) if ${F}$ is known instead to be Lipschitz continuous.

In many applications in analysis, one would like more explicit and quantitative bounds that estimate quantities like ${\|F(x)-F(x_0)\|_Y}$ in terms of quantities like ${\|x-x_0\|_X}$. There are a number of ways to do this. First of all, there is of course the trivial estimate arising from the triangle inequality:

$\displaystyle \|F(x)-F(x_0)\|_Y \leq \|F(x)\|_Y + \|F(x_0)\|_Y. \ \ \ \ \ (1)$

This estimate is usually not very good when ${x}$ and ${x_0}$ are close together. However, when ${x}$ and ${x_0}$ are far apart, this estimate can be more or less sharp. For instance, if the magnitude of ${F}$ varies so much from ${x_0}$ to ${x}$ that ${\|F(x)\|_Y}$ is more than (say) twice that of ${\|F(x_0)\|_Y}$, or vice versa, then (1) is sharp up to a multiplicative constant. Also, if ${F}$ is oscillatory in nature, and the distance between ${x}$ and ${x_0}$ exceeds the “wavelength” of the oscillation of ${F}$ at ${x_0}$ (or at ${x}$), then one also typically expects (1) to be close to sharp. Conversely, if ${F}$ does not vary much in magnitude from ${x_0}$ to ${x}$, and the distance between ${x}$ and ${x_0}$ is less than the wavelength of any oscillation present in ${F}$, one expects to be able to improve upon (1).

When ${F}$ is relatively simple in form, one can sometimes proceed simply by substituting ${x = x_0 + h}$. For instance, if ${F: R \rightarrow R}$ is the squaring function ${F(x) = x^2}$ in a commutative ring ${R}$, one has

$\displaystyle F(x_0+h) = (x_0+h)^2 = x_0^2 + 2x_0 h+ h^2$

and thus

$\displaystyle F(x_0+h) - F(x_0) = 2x_0 h + h^2$

or in terms of the original variables ${x, x_0}$ one has

$\displaystyle F(x) - F(x_0) = 2 x_0 (x-x_0) + (x-x_0)^2.$

If the ring ${R}$ is not commutative, one has to modify this to

$\displaystyle F(x) - F(x_0) = x_0 (x-x_0) + (x-x_0) x_0 + (x-x_0)^2.$

Thus, for instance, if ${A, B}$ are ${n \times n}$ matrices and ${\| \|_{op}}$ denotes the operator norm, one sees from the triangle inequality and the sub-multiplicativity ${\| AB\|_{op} \leq \| A \|_{op} \|B\|_{op}}$ of operator norm that

$\displaystyle \| A^2 - B^2 \|_{op} \leq \| A - B \|_{op} ( 2 \|B\|_{op} + \|A - B \|_{op} ). \ \ \ \ \ (2)$

If ${F(x)}$ involves ${x}$ (or various components of ${x}$) in several places, one can sometimes get a good estimate by “swapping” ${x}$ with ${x_0}$ at each of the places in turn, using a telescoping series. For instance, if we again use the squaring function ${F(x) = x^2 = x x}$ in a non-commutative ring, we have

$\displaystyle F(x) - F(x_0) = x x - x_0 x_0$

$\displaystyle = (x x - x_0 x) + (x_0 x - x_0 x_0)$

$\displaystyle = (x-x_0) x + x_0 (x-x_0)$

which for instance leads to a slight improvement of (2):

$\displaystyle \| A^2 - B^2 \|_{op} \leq \| A - B \|_{op} ( \| A\|_{op} + \|B\|_{op} ).$

More generally, for any natural number ${n}$, one has the identity

$\displaystyle x^n - x_0^n = (x-x_0) (x^{n-1} + x^{n-2} x_0 + \dots + x x_0^{n-2} + x_0^{n-1}) \ \ \ \ \ (3)$

in a commutative ring, while in a non-commutative ring one must modify this to

$\displaystyle x^n - x_0^n = \sum_{i=0}^{n-1} x_0^i (x-x_0) x^{n-1-i},$

and for matrices one has

$\displaystyle \| A^n - B^n \|_{op} \leq \| A-B\|_{op} ( \|A\|_{op}^{n-1} + \| A\|_{op}^{n-2} \| B\|_{op} + \dots + \|B\|_{op}^{n-1} ).$

Exercise 1 If ${U}$ and ${V}$ are unitary ${n \times n}$ matrices, show that the commutator ${[U,V] := U V U^{-1} V^{-1}}$ obeys the inequality

$\displaystyle \| [U,V] - I \|_{op} \leq 2 \| U - I \|_{op} \| V - I \|_{op}.$

(Hint: first control ${\| UV - VU \|_{op}}$.)

Now suppose (for simplicity) that ${F: {\bf R}^d \rightarrow {\bf R}^{d'}}$ is a map between Euclidean spaces. If ${F}$ is continuously differentiable, then one can use the fundamental theorem of calculus to write

$\displaystyle F(x) - F(x_0) = \int_0^1 \frac{d}{dt} F( \gamma(t) )\ dt$

where ${\gamma: [0,1] \rightarrow Y}$ is any continuously differentiable path from ${x_0}$ to ${x}$. For instance, if one uses the straight line path ${\gamma(t) := (1-t) x_0 + tx}$, one has

$\displaystyle F(x) - F(x_0) = \int_0^1 ((x-x_0) \cdot \nabla F)( (1-t) x_0 + t x )\ dt.$

In the one-dimensional case ${d=1}$, this simplifies to

$\displaystyle F(x) - F(x_0) = (x-x_0) \int_0^1 F'( (1-t) x_0 + t x )\ dt. \ \ \ \ \ (4)$

Among other things, this immediately implies the factor theorem for ${C^k}$ functions: if ${F}$ is a ${C^k({\bf R})}$ function for some ${k \geq 1}$ that vanishes at some point ${x_0}$, then ${F(x)}$ factors as the product of ${x-x_0}$ and some ${C^{k-1}}$ function ${G}$. Another basic consequence is that if ${\nabla F}$ is uniformly bounded in magnitude by some constant ${C}$, then ${F}$ is Lipschitz continuous with the same constant ${C}$.

Applying (4) to the power function ${x \mapsto x^n}$, we obtain the identity

$\displaystyle x^n - x_0^n = n (x-x_0) \int_0^1 ((1-t) x_0 + t x)^{n-1}\ dt \ \ \ \ \ (5)$

which can be compared with (3). Indeed, for ${x_0}$ and ${x}$ close to ${1}$, one can use logarithms and Taylor expansion to arrive at the approximation ${((1-t) x_0 + t x)^{n-1} \approx x_0^{(1-t) (n-1)} x^{t(n-1)}}$, so (3) behaves a little like a Riemann sum approximation to (5).

Exercise 2 For each ${i=1,\dots,n}$, let ${X^{(1)}_i}$ and ${X^{(0)}_i}$ be random variables taking values in a measurable space ${R_i}$, and let ${F: R_1 \times \dots \times R_n \rightarrow {\bf R}^m}$ be a bounded measurable function.

• (i) (Lindeberg exchange identity) Show that

$\displaystyle \mathop{\bf E} F(X^{(1)}_1,\dots,X^{(1)}_n) - \mathop{\bf E} F(X^{(0)}_1,\dots,X^{(0)}_n)$

$\displaystyle = \sum_{i=1}^n \mathop{\bf E} F( X^{(1)}_1,\dots, X^{(1)}_{i-1}, X^{(1)}_i, X^{(0)}_{i+1}, \dots, X^{(0)}_n)$

$\displaystyle - \mathop{\bf E} F( X^{(1)}_1,\dots, X^{(1)}_{i-1}, X^{(0)}_i, X^{(0)}_{i+1}, \dots, X^{(0)}_n).$

• (ii) (Knowles-Yin exchange identity) Show that

$\displaystyle \mathop{\bf E} F(X^{(1)}_1,\dots,X^{(1)}_n) - \mathop{\bf E} F(X^{(0)}_1,\dots,X^{(0)}_n)$

$\displaystyle = \int_0^1 \sum_{i=1}^n \mathop{\bf E} F( X^{(t)}_1,\dots, X^{(t)}_{i-1}, X^{(1)}_i, X^{(t)}_{i+1}, \dots, X^{(t)}_n)$

$\displaystyle - \mathop{\bf E} F( X^{(t)}_1,\dots, X^{(t)}_{i-1}, X^{(0)}_i, X^{(t)}_{i+1}, \dots, X^{(t)}_n)\ dt,$

where ${X^{(t)}_i = 1_{I_i \leq t} X^{(0)}_i + 1_{I_i > t} X^{(1)}_i}$ is a mixture of ${X^{(0)}_i}$ and ${X^{(1)}_i}$, with ${I_1,\dots,I_n}$ uniformly drawn from ${[0,1]}$ independently of each other and of the ${X^{(0)}_1,\dots,X^{(0)}_n, X^{(1)}_0,\dots,X^{(1)}_n}$.

• (iii) Discuss the relationship between the identities in parts (i), (ii) with the identities (3), (5).

(The identity in (i) is the starting point for the Lindeberg exchange method in probability theory, discussed for instance in this previous post. The identity in (ii) can also be used in the Lindeberg exchange method; the terms in the right-hand side are slightly more symmetric in the indices ${1,\dots,n}$, which can be a technical advantage in some applications; see this paper of Knowles and Yin for an instance of this.)

Exercise 3 If ${F: {\bf R}^d \rightarrow {\bf R}^{d'}}$ is continuously ${k}$ times differentiable, establish Taylor’s theorem with remainder

$\displaystyle F(x) = \sum_{j=0}^{k-1} \frac{1}{j!} (((x-x_0) \cdot \nabla)^j F)( x_0 )$

$\displaystyle + \int_0^1 \frac{(1-t)^{k-1}}{(k-1)!} (((x-x_0) \cdot \nabla)^k F)((1-t) x_0 + t x)\ dt.$

If ${\nabla^k F}$ is bounded, conclude that

$\displaystyle |F(x) - \sum_{j=0}^{k-1} \frac{1}{j!} (((x-x_0) \cdot \nabla)^j F)( x_0 )|$

$\displaystyle \leq \frac{|x-x_0|^k}{k!} \sup_{y \in {\bf R}^d} |\nabla^k F(y)|.$

For real scalar functions ${F: {\bf R}^d \rightarrow {\bf R}}$, the average value of the continuous real-valued function ${(x - x_0) \cdot \nabla F((1-t) x_0 + t x)}$ must be attained at some point ${t}$ in the interval ${[0,1]}$. We thus conclude the mean-value theorem

$\displaystyle F(x) - F(x_0) = ((x - x_0) \cdot \nabla F)((1-t) x_0 + t x)$

for some ${t \in [0,1]}$ (that can depend on ${x}$, ${x_0}$, and ${F}$). This can for instance give a second proof of fact that continuously differentiable functions ${F}$ with bounded derivative are Lipschitz continuous. However it is worth stressing that the mean-value theorem is only available for real scalar functions; it is false for instance for complex scalar functions. A basic counterexample is given by the function ${e(x) := e^{2\pi i x}}$; there is no ${t \in [0,1]}$ for which ${e(1) - e(0) = e'(t)}$. On the other hand, as ${e'}$ has magnitude ${2\pi}$, we still know from (4) that ${e}$ is Lipschitz of constant ${2\pi}$, and when combined with (1) we obtain the basic bounds

$\displaystyle |e(x) - e(y)| \leq \min( 2, 2\pi |x-y| )$

which are already very useful for many applications.

Exercise 4 Let ${H_0, V}$ be ${n \times n}$ matrices, and let ${t}$ be a non-negative real.

• (i) Establish the Duhamel formula

$\displaystyle e^{t(H_0+V)} = e^{tH_0} + \int_0^t e^{(t-s) H_0} V e^{s (H_0+V)}\ ds$

$\displaystyle = e^{tH_0} + \int_0^t e^{(t-s) (H_0+V)} V e^{s H_0}\ ds$

where ${e^A}$ denotes the matrix exponential of ${A}$. (Hint: Differentiate ${e^{(t-s) H_0} e^{s (H_0+V)}}$ or ${e^{(t-s) (H_0+V)} e^{s H_0}}$ in ${s}$.)

• (ii) Establish the iterated Duhamel formula

$\displaystyle e^{t(H_0+V)} = e^{tH_0} + \sum_{j=1}^k \int_{0 \leq t_1 \leq \dots \leq t_j \leq t}$

$\displaystyle e^{(t-t_j) H_0} V e^{(t_j-t_{j-1}) H_0} V \dots e^{(t_2-t_1) H_0} V e^{t_1 H_0}\ dt_1 \dots dt_j$

$\displaystyle + \int_{0 \leq t_1 \leq \dots \leq t_{k+1} \leq t}$

$\displaystyle e^{(t-t_{k+1}) (H_0+V)} V e^{(t_{k+1}-t_k) H_0} V \dots e^{(t_2-t_1) H_0} V e^{t_1 H_0}\ dt_1 \dots dt_{k+1}$

for any ${k \geq 0}$.

• (iii) Establish the infinitely iterated Duhamel formula

$\displaystyle e^{t(H_0+V)} = e^{tH_0} + \sum_{j=1}^\infty \int_{0 \leq t_1 \leq \dots \leq t_j \leq t}$

$\displaystyle e^{(t-t_j) H_0} V e^{(t_j-t_{j-1}) H_0} V \dots e^{(t_2-t_1) H_0} V e^{t_1 H_0}\ dt_1 \dots dt_j.$

• (iv) If ${H(t)}$ is an ${n \times n}$ matrix depending in a continuously differentiable fashion on ${t}$, establish the variation formula

$\displaystyle \frac{d}{dt} e^{H(t)} = (F(\mathrm{ad}(H(t))) H'(t)) e^{H(t)}$

where ${\mathrm{ad}(H)}$ is the adjoint representation ${\mathrm{ad}(H)(V) = HV - VH}$ applied to ${H}$, and ${F}$ is the function

$\displaystyle F(z) := \int_0^1 e^{sz}\ ds$

(thus ${F(z) = \frac{e^z-1}{z}}$ for non-zero ${z}$), with ${F(\mathrm{ad}(H(t)))}$ defined using functional calculus.

We remark that further manipulation of (iv) of the above exercise using the fundamental theorem of calculus eventually leads to the Baker-Campbell-Hausdorff-Dynkin formula, as discussed in this previous blog post.

Exercise 5 Let ${A, B}$ be positive definite ${n \times n}$ matrices, and let ${Y}$ be an ${n \times n}$ matrix. Show that there is a unique solution ${X}$ to the Sylvester equation

$\displaystyle AX + X B = Y$

which is given by the formula

$\displaystyle X = \int_0^\infty e^{-tA} Y e^{-tB}\ dt.$

In the above examples we had applied the fundamental theorem of calculus along linear curves ${\gamma(t) = (1-t) x_0 + t x}$. However, it is sometimes better to use other curves. For instance, the circular arc ${\gamma(t) = \cos(\pi t/2) x_0 + \sin(\pi t/2) x}$ can be useful, particularly if ${x_0}$ and ${x}$ are “orthogonal” or “independent” in some sense; a good example of this is the proof by Maurey and Pisier of the gaussian concentration inequality, given in Theorem 8 of this previous blog post. In a similar vein, if one wishes to compare a scalar random variable ${X}$ of mean zero and variance one with a Gaussian random variable ${G}$ of mean zero and variance one, it can be useful to introduce the intermediate random variables ${\gamma(t) := (1-t)^{1/2} X + t^{1/2} G}$ (where ${X}$ and ${G}$ are independent); note that these variables have mean zero and variance one, and after coupling them together appropriately they evolve by the Ornstein-Uhlenbeck process, which has many useful properties. For instance, one can use these ideas to establish monotonicity formulae for entropy; see e.g. this paper of Courtade for an example of this and further references. More generally, one can exploit curves ${\gamma}$ that flow according to some geometrically natural ODE or PDE; several examples of this occur famously in Perelman’s proof of the Poincaré conjecture via Ricci flow, discussed for instance in this previous set of lecture notes.

In some cases, it is difficult to compute ${F(x)-F(x_0)}$ or the derivative ${\nabla F}$ directly, but one can instead proceed by implicit differentiation, or some variant thereof. Consider for instance the matrix inversion map ${F(A) := A^{-1}}$ (defined on the open dense subset of ${n \times n}$ matrices consisting of invertible matrices). If one wants to compute ${F(B)-F(A)}$ for ${B}$ close to ${A}$, one can write temporarily write ${F(B) - F(A) = E}$, thus

$\displaystyle B^{-1} - A^{-1} = E.$

Multiplying both sides on the left by ${B}$ to eliminate the ${B^{-1}}$ term, and on the right by ${A}$ to eliminate the ${A^{-1}}$ term, one obtains

$\displaystyle A - B = B E A$

and thus on reversing these steps we arrive at the basic identity

$\displaystyle B^{-1} - A^{-1} = B^{-1} (A - B) A^{-1}. \ \ \ \ \ (6)$

For instance, if ${H_0, V}$ are ${n \times n}$ matrices, and we consider the resolvents

$\displaystyle R_0(z) := (H_0 - z I)^{-1}; \quad R_V(z) := (H_0 + V - zI)^{-1}$

then we have the resolvent identity

$\displaystyle R_V(z) - R_0(z) = - R_V(z) V R_0(z) \ \ \ \ \ (7)$

as long as ${z}$ does not lie in the spectrum of ${H_0}$ or ${H_0+V}$ (for instance, if ${H_0}$, ${V}$ are self-adjoint then one can take ${z}$ to be any strictly complex number). One can iterate this identity to obtain

$\displaystyle R_V(z) = \sum_{j=0}^k (-R_0(z) V)^j R_0(z) + (-R_V(z) V) (-R_0(z) V)^k R_0(z)$

for any natural number ${k}$; in particular, if ${R_0(z) V}$ has operator norm less than one, one has the Neumann series

$\displaystyle R_V(z) = \sum_{j=0}^\infty (-R_0(z) V)^j R_0(z).$

Similarly, if ${A(t)}$ is a family of invertible matrices that depends in a continuously differentiable fashion on a time variable ${t}$, then by implicitly differentiating the identity

$\displaystyle A(t) A(t)^{-1} = I$

in ${t}$ using the product rule, we obtain

$\displaystyle (\frac{d}{dt} A(t)) A(t)^{-1} + A(t) \frac{d}{dt} A(t)^{-1} = 0$

and hence

$\displaystyle \frac{d}{dt} A(t)^{-1} = - A(t)^{-1} (\frac{d}{dt} A(t)) A(t)^{-1}$

(this identity may also be easily derived from (6)). One can then use the fundamental theorem of calculus to obtain variants of (6), for instance by using the curve ${\gamma(t) = (1-t) A + tB}$ we arrive at

$\displaystyle B^{-1} - A^{-1} = \int_0^1 ((1-t)A + tB)^{-1} (A-B) ((1-t)A + tB)^{-1}\ dt$

assuming that the curve stays entirely within the set of invertible matrices. While this identity may seem more complicated than (6), it is more symmetric, which conveys some advantages. For instance, using this identity it is easy to see that if ${A, B}$ are positive definite with ${A>B}$ in the sense of positive definite matrices (that is, ${A-B}$ is positive definite), then ${B^{-1} > A^{-1}}$. (Try to prove this using (6) instead!)

Exercise 6 If ${A}$ is an invertible ${n \times n}$ matrix and ${u, v}$ are ${n \times 1}$ vectors, establish the Sherman-Morrison formula

$\displaystyle (A + t uv^T)^{-1} = A^{-1} - \frac{t}{1 + t v^T A^{-1} u} A^{-1} uv^T A^{-1}$

whenever ${t}$ is a scalar such that ${1 + t v^T A^{-1} u}$ is non-zero. (See also this previous blog post for more discussion of these sorts of identities.)

One can use the Cauchy integral formula to extend these identities to other functions of matrices. For instance, if ${F: {\bf C} \rightarrow {\bf C}}$ is an entire function, and ${\gamma}$ is a counterclockwise contour that goes around the spectrum of both ${H_0}$ and ${H_0+V}$, then we have

$\displaystyle F(H_0+V) = \frac{-1}{2\pi i} \int_\gamma F(z) R_V(z)\ dz$

and similarly

$\displaystyle F(H_0) = \frac{-1}{2\pi i} \int_\gamma F(z) R_0(z)\ dz$

and hence by (7) one has

$\displaystyle F(H_0+V) - F(H_0) = \frac{1}{2\pi i} \int_\gamma F(z) R_V(z) V F_0(z)\ dz;$

similarly, if ${H(t)}$ depends on ${t}$ in a continuously differentiable fashion, then

$\displaystyle \frac{d}{dt} F(H(t)) = \frac{1}{2\pi i} \int_\gamma F(z) (H(t) - zI)^{-1} H'(t) (z) (H(t)-zI)^{-1}\ dz$

as long as ${\gamma}$ goes around the spectrum of ${H(t)}$.

Exercise 7 If ${H(t)}$ is an ${n \times n}$ matrix depending continuously differentiably on ${t}$, and ${F: {\bf C} \rightarrow {\bf C}}$ is an entire function, establish the tracial chain rule

$\displaystyle \frac{d}{dt} \hbox{tr} F(H(t)) = \hbox{tr}(F'(H(t)) H'(t)).$

In a similar vein, given that the logarithm function is the antiderivative of the reciprocal, one can express the matrix logarithm ${\log A}$ of a positive definite matrix by the fundamental theorem of calculus identity

$\displaystyle \log A = \int_0^\infty (I + sI)^{-1} - (A + sI)^{-1}\ ds$

(with the constant term ${(I+tI)^{-1}}$ needed to prevent a logarithmic divergence in the integral). Differentiating, we see that if ${A(t)}$ is a family of positive definite matrices depending continuously on ${t}$, that

$\displaystyle \frac{d}{dt} \log A(t) = \int_0^\infty (A(t) + sI)^{-1} A'(t) (A(t)+sI)^{-1}\ dt.$

This can be used for instance to show that ${\log}$ is a monotone increasing function, in the sense that ${\log A> \log B}$ whenever ${A > B > 0}$ in the sense of positive definite matrices. One can of course integrate this formula to obtain some formulae for the difference ${\log A - \log B}$ of the logarithm of two positive definite matrices ${A,B}$.

To compare the square root ${A^{1/2} - B^{1/2}}$ of two positive definite matrices ${A,B}$ is trickier; there are multiple ways to proceed. One approach is to use contour integration as before (but one has to take some care to avoid branch cuts of the square root). Another to express the square root in terms of exponentials via the formula

$\displaystyle A^{1/2} = \frac{1}{\Gamma(-1/2)} \int_0^\infty (e^{-tA} - I) t^{-1/2} \frac{dt}{t}$

where ${\Gamma}$ is the gamma function; this formula can be verified by first diagonalising ${A}$ to reduce to the scalar case and using the definition of the Gamma function. Then one has

$\displaystyle A^{1/2} - B^{1/2} = \frac{1}{\Gamma(-1/2)} \int_0^\infty (e^{-tA} - e^{-tB}) t^{-1/2} \frac{dt}{t}$

and one can use some of the previous identities to control ${e^{-tA} - e^{-tB}}$. This is pretty messy though. A third way to proceed is via implicit differentiation. If for instance ${A(t)}$ is a family of positive definite matrices depending continuously differentiably on ${t}$, we can differentiate the identity

$\displaystyle A(t)^{1/2} A(t)^{1/2} = A(t)$

to obtain

$\displaystyle A(t)^{1/2} \frac{d}{dt} A(t)^{1/2} + (\frac{d}{dt} A(t)^{1/2}) A(t)^{1/2} = \frac{d}{dt} A(t).$

This can for instance be solved using Exercise 5 to obtain

$\displaystyle \frac{d}{dt} A(t)^{1/2} = \int_0^\infty e^{-sA(t)^{1/2}} A'(t) e^{-sA(t)^{1/2}}\ ds$

and this can in turn be integrated to obtain a formula for ${A^{1/2} - B^{1/2}}$. This is again a rather messy formula, but it does at least demonstrate that the square root is a monotone increasing function on positive definite matrices: ${A > B > 0}$ implies ${A^{1/2} > B^{1/2} > 0}$.

Several of the above identities for matrices can be (carefully) extended to operators on Hilbert spaces provided that they are sufficiently well behaved (in particular, if they have a good functional calculus, and if various spectral hypotheses are obeyed). We will not attempt to do so here, however.

Suppose one has a bounded sequence ${(a_n)_{n=1}^\infty = (a_1, a_2, \dots)}$ of real numbers. What kinds of limits can one form from this sequence?

Of course, we have the usual notion of limit ${\lim_{n \rightarrow \infty} a_n}$, which in this post I will refer to as the classical limit to distinguish from the other limits discussed in this post. The classical limit, if it exists, is the unique real number ${L}$ such that for every ${\varepsilon>0}$, one has ${|a_n-L| \leq \varepsilon}$ for all sufficiently large ${n}$. We say that a sequence is (classically) convergent if its classical limit exists. The classical limit obeys many useful limit laws when applied to classically convergent sequences. Firstly, it is linear: if ${(a_n)_{n=1}^\infty}$ and ${(b_n)_{n=1}^\infty}$ are classically convergent sequences, then ${(a_n+b_n)_{n=1}^\infty}$ is also classically convergent with

$\displaystyle \lim_{n \rightarrow \infty} (a_n + b_n) = (\lim_{n \rightarrow \infty} a_n) + (\lim_{n \rightarrow \infty} b_n) \ \ \ \ \ (1)$

and similarly for any scalar ${c}$, ${(ca_n)_{n=1}^\infty}$ is classically convergent with

$\displaystyle \lim_{n \rightarrow \infty} (ca_n) = c \lim_{n \rightarrow \infty} a_n. \ \ \ \ \ (2)$

It is also an algebra homomorphism: ${(a_n b_n)_{n=1}^\infty}$ is also classically convergent with

$\displaystyle \lim_{n \rightarrow \infty} (a_n b_n) = (\lim_{n \rightarrow \infty} a_n) (\lim_{n \rightarrow \infty} b_n). \ \ \ \ \ (3)$

We also have shift invariance: if ${(a_n)_{n=1}^\infty}$ is classically convergent, then so is ${(a_{n+1})_{n=1}^\infty}$ with

$\displaystyle \lim_{n \rightarrow \infty} a_{n+1} = \lim_{n \rightarrow \infty} a_n \ \ \ \ \ (4)$

and more generally in fact for any injection ${\phi: {\bf N} \rightarrow {\bf N}}$, ${(a_{\phi(n)})_{n=1}^\infty}$ is classically convergent with

$\displaystyle \lim_{n \rightarrow \infty} a_{\phi(n)} = \lim_{n \rightarrow \infty} a_n. \ \ \ \ \ (5)$

The classical limit of a sequence is unchanged if one modifies any finite number of elements of the sequence. Finally, we have boundedness: for any classically convergent sequence ${(a_n)_{n=1}^\infty}$, one has

$\displaystyle \inf_n a_n \leq \lim_{n \rightarrow \infty} a_n \leq \sup_n a_n. \ \ \ \ \ (6)$

One can in fact show without much difficulty that these laws uniquely determine the classical limit functional on convergent sequences.

One would like to extend the classical limit notion to more general bounded sequences; however, when doing so one must give up one or more of the desirable limit laws that were listed above. Consider for instance the sequence ${a_n = (-1)^n}$. On the one hand, one has ${a_n^2 = 1}$ for all ${n}$, so if one wishes to retain the homomorphism property (3), any “limit” of this sequence ${a_n}$ would have to necessarily square to ${1}$, that is to say it must equal ${+1}$ or ${-1}$. On the other hand, if one wished to retain the shift invariance property (4) as well as the homogeneity property (2), any “limit” of this sequence would have to equal its own negation and thus be zero.

Nevertheless there are a number of useful generalisations and variants of the classical limit concept for non-convergent sequences that obey a significant portion of the above limit laws. For instance, we have the limit superior

$\displaystyle \limsup_{n \rightarrow \infty} a_n := \inf_N \sup_{n \geq N} a_n$

$\displaystyle \liminf_{n \rightarrow \infty} a_n := \sup_N \inf_{n \geq N} a_n$

which are well-defined real numbers for any bounded sequence ${(a_n)_{n=1}^\infty}$; they agree with the classical limit when the sequence is convergent, but disagree otherwise. They enjoy the shift-invariance property (4), and the boundedness property (6), but do not in general obey the homomorphism property (3) or the linearity property (1); indeed, we only have the subadditivity property

$\displaystyle \limsup_{n \rightarrow \infty} (a_n + b_n) \leq (\limsup_{n \rightarrow \infty} a_n) + (\limsup_{n \rightarrow \infty} b_n)$

for the limit superior, and the superadditivity property

$\displaystyle \liminf_{n \rightarrow \infty} (a_n + b_n) \geq (\liminf_{n \rightarrow \infty} a_n) + (\liminf_{n \rightarrow \infty} b_n)$

for the limit inferior. The homogeneity property (2) is only obeyed by the limits superior and inferior for non-negative ${c}$; for negative ${c}$, one must have the limit inferior on one side of (2) and the limit superior on the other, thus for instance

$\displaystyle \limsup_{n \rightarrow \infty} (-a_n) = - \liminf_{n \rightarrow \infty} a_n.$

The limit superior and limit inferior are examples of limit points of the sequence, which can for instance be defined as points that are limits of at least one subsequence of the original sequence. Indeed, the limit superior is always the largest limit point of the sequence, and the limit inferior is always the smallest limit point. However, limit points can be highly non-unique (indeed they are unique if and only if the sequence is classically convergent), and so it is difficult to sensibly interpret most of the usual limit laws in this setting, with the exception of the homogeneity property (2) and the boundedness property (6) that are easy to state for limit points.

Another notion of limit are the Césaro limits

$\displaystyle \mathrm{C}\!-\!\lim_{n \rightarrow \infty} a_n := \lim_{N \rightarrow \infty} \frac{1}{N} \sum_{n=1}^N a_n;$

if this limit exists, we say that the sequence is Césaro convergent. If the sequence ${(a_n)_{n=1}^\infty}$ already has a classical limit, then it also has a Césaro limit that agrees with the classical limit; but there are additional sequences that have a Césaro limit but not a classical one. For instance, the non-classically convergent sequence ${a_n= (-1)^n}$ discussed above is Césaro convergent, with a Césaro limit of ${0}$. However, there are still bounded sequences that do not have Césaro limit, such as ${a_n := \sin( \log n )}$ (exercise!). The Césaro limit is linear, bounded, and shift invariant, but not an algebra homomorphism and also does not obey the rearrangement property (5).

Using the Hahn-Banach theorem, one can extend the classical limit functional to generalised limit functionals ${\mathop{\widetilde \lim}_{n \rightarrow \infty} a_n}$, defined to be bounded linear functionals from the space ${\ell^\infty({\bf N})}$ of bounded real sequences to the real numbers ${{\bf R}}$ that extend the classical limit functional (defined on the space ${c_0({\bf N}) + {\bf R}}$ of convergent sequences) without any increase in the operator norm. (In some of my past writings I made the slight error of referring to these generalised limit functionals as Banach limits, though as discussed below, the latter actually refers to a subclass of generalised limit functionals.) It is not difficult to see that such generalised limit functionals will range between the limit inferior and limit superior. In fact, for any specific sequence ${(a_n)_{n=1}^\infty}$ and any number ${L}$ lying in the closed interval ${[\liminf_{n \rightarrow \infty} a_n, \limsup_{n \rightarrow \infty} a_n]}$, there exists at least one generalised limit functional ${\mathop{\widetilde \lim}_{n \rightarrow \infty}}$ that takes the value ${L}$ when applied to ${a_n}$; for instance, for any number ${\theta}$ in ${[-1,1]}$, there exists a generalised limit functional that assigns that number ${\theta}$ as the “limit” of the sequence ${a_n = (-1)^n}$. This claim can be seen by first designing such a limit functional on the vector space spanned by the convergent sequences and by ${(a_n)_{n=1}^\infty}$, and then appealing to the Hahn-Banach theorem to extend to all sequences. This observation also gives a necessary and sufficient criterion for a bounded sequence ${(a_n)_{n=1}^\infty}$ to classically converge to a limit ${L}$, namely that all generalised limits of this sequence must equal ${L}$.

Because of the reliance on the Hahn-Banach theorem, the existence of generalised limits requires the axiom of choice (or some weakened version thereof); there are presumably models of set theory without the axiom of choice in which no generalised limits exist, but I do not know of an explicit reference for this.

Generalised limits can obey the shift-invariance property (4) or the algebra homomorphism property (2), but as the above analysis of the sequence ${a_n = (-1)^n}$ shows, they cannot do both. Generalised limits that obey the shift-invariance property (4) are known as Banach limits; one can for instance construct them by applying the Hahn-Banach theorem to the Césaro limit functional; alternatively, if ${\mathop{\widetilde \lim}}$ is any generalised limit, then the Césaro-type functional ${(a_n)_{n=1}^\infty \mapsto \mathop{\widetilde \lim}_{N \rightarrow \infty} \frac{1}{N} \sum_{n=1}^N a_n}$ will be a Banach limit. The existence of Banach limits can be viewed as a demonstration of the amenability of the natural numbers (or integers); see this previous blog post for further discussion.

Generalised limits that obey the algebra homomorphism property (2) are known as ultrafilter limits. If one is given a generalised limit functional ${p\!-\!\lim_{n \rightarrow \infty}}$ that obeys (2), then for any subset ${A}$ of the natural numbers ${{\bf N}}$, the generalised limit ${p\!-\!\lim_{n \rightarrow \infty} 1_A(n)}$ must equal its own square (since ${1_A(n)^2 = 1_A(n)}$) and is thus either ${0}$ or ${1}$. If one defines ${p \subset 2^{2^{\bf N}}}$ to be the collection of all subsets ${A}$ of ${{\bf N}}$ for which ${p\!-\!\lim_{n \rightarrow \infty} 1_A(n) = 1}$, one can verify that ${p}$ obeys the axioms of a non-principal ultrafilter. Conversely, if ${p}$ is a non-principal ultrafilter, one can define the associated generalised limit ${p\!-\!\lim_{n \rightarrow \infty} a_n}$ of any bounded sequence ${(a_n)_{n=1}^\infty}$ to be the unique real number ${L}$ such that the sets ${\{ n \in {\bf N}: |a_n - L| \leq \varepsilon \}}$ lie in ${p}$ for all ${\varepsilon>0}$; one can check that this does indeed give a well-defined generalised limit that obeys (2). Non-principal ultrafilters can be constructed using Zorn’s lemma. In fact, they do not quite need the full strength of the axiom of choice; see the Wikipedia article on the ultrafilter lemma for examples.

We have previously noted that generalised limits of a sequence can converge to any point between the limit inferior and limit superior. The same is not true if one restricts to Banach limits or ultrafilter limits. For instance, by the arguments already given, the only possible Banach limit for the sequence ${a_n = (-1)^n}$ is zero. Meanwhile, an ultrafilter limit must converge to a limit point of the original sequence, but conversely every limit point can be attained by at least one ultrafilter limit; we leave these assertions as an exercise to the interested reader. In particular, a bounded sequence converges classically to a limit ${L}$ if and only if all ultrafilter limits converge to ${L}$.

There is no generalisation of the classical limit functional to any space that includes non-classically convergent sequences that obeys the subsequence property (5), since any non-classically-convergent sequence will have one subsequence that converges to the limit superior, and another subsequence that converges to the limit inferior, and one of these will have to violate (5) since the limit superior and limit inferior are distinct. So the above limit notions come close to the best generalisations of limit that one can use in practice.

We summarise the above discussion in the following table:

 Limit Always defined Linear Shift-invariant Homomorphism Constructive Classical No Yes Yes Yes Yes Superior Yes No Yes No Yes Inferior Yes No Yes No Yes Césaro No Yes Yes No Yes Generalised Yes Yes Depends Depends No Banach Yes Yes Yes No No Ultrafilter Yes Yes No Yes No

A sequence ${a: {\bf Z} \rightarrow {\bf C}}$ of complex numbers is said to be quasiperiodic if it is of the form

$\displaystyle a(n) = F( \alpha_1 n \hbox{ mod } 1, \dots, \alpha_k n \hbox{ mod } 1)$

for some real numbers ${\alpha_1,\dots,\alpha_k}$ and continuous function ${F: ({\bf R}/{\bf Z})^k \rightarrow {\bf C}}$. For instance, linear phases such as ${n \mapsto e(\alpha n + \beta)}$ (where ${e(\theta) := e^{2\pi i \theta}}$) are examples of quasiperiodic sequences; the top order coefficient ${\alpha}$ (modulo ${1}$) can be viewed as a “frequency” of the integers, and an element of the Pontryagin dual ${\hat {\bf Z} \equiv {\bf R}/{\bf Z}}$ of the integers. Any periodic sequence is also quasiperiodic (taking ${k=1}$ and ${\alpha_1}$ to be the reciprocal of the period). A sequence is said to be almost periodic if it is the uniform limit of quasiperiodic sequences. For instance any Fourier series of the form

$\displaystyle a(n) = \sum_{j=1}^\infty c_j e(\alpha_j n)$

with ${\alpha_1,\alpha_2,\dots}$ real numbers and ${c_1,c_2,\dots}$ an absolutely summable sequence of complex coefficients, will be almost periodic.

These sequences arise in various “complexity one” problems in arithmetic combinatorics and ergodic theory. For instance, if ${(X, \mu, T)}$ is a measure-preserving system – a probability space ${(X,\mu)}$ equipped with a measure-preserving shift, and ${f_1,f_2 \in L^\infty(X)}$ are bounded measurable functions, then the correlation sequence

$\displaystyle a(n) := \int_X f_1(x) f_2(T^n x)\ d\mu(x)$

can be shown to be an almost periodic sequence, plus an error term ${b_n}$ which is “null” in the sense that it has vanishing uniform density:

$\displaystyle \sup_N \frac{1}{M} \sum_{n=N+1}^{N+M} |b_n| \rightarrow 0 \hbox{ as } M \rightarrow \infty.$

This can be established in a number of ways, for instance by writing ${a(n)}$ as the Fourier coefficients of the spectral measure of the shift ${T}$ with respect to the functions ${f_1,f_2}$, and then decomposing that measure into pure point and continuous components.

In the last two decades or so, it has become clear that there are natural higher order versions of these concepts, in which linear polynomials such as ${\alpha n + \beta}$ are replaced with higher degree counterparts. The most obvious candidates for these counterparts would be the polynomials ${\alpha_d n^d + \dots + \alpha_0}$, but this turns out to not be a complete set of higher degree objects needed for the theory. Instead, the higher order versions of quasiperiodic and almost periodic sequences are now known as basic nilsequences and nilsequences respectively, while the higher order version of a linear phase is a nilcharacter; each nilcharacter then has a symbol that is a higher order generalisation of the concept of a frequency (and the collection of all symbols forms a group that can be viewed as a higher order version of the Pontryagin dual of ${{\bf Z}}$). The theory of these objects is spread out in the literature across a number of papers; in particular, the theory of nilcharacters is mostly developed in Appendix E of this 116-page paper of Ben Green, Tamar Ziegler, and myself, and is furthermore written using nonstandard analysis and treating the more general setting of higher dimensional sequences. I therefore decided to rewrite some of that material in this blog post, in the simpler context of the qualitative asymptotic theory of one-dimensional nilsequences and nilcharacters rather than the quantitative single-scale theory that is needed for combinatorial applications (and which necessitated the use of nonstandard analysis in the previous paper).

For technical reasons (having to do with the non-trivial topological structure on nilmanifolds), it will be convenient to work with vector-valued sequences, that take values in a finite-dimensional complex vector space ${{\bf C}^m}$ rather than in ${{\bf C}}$. By doing so, the space of sequences is now, technically, no longer a ring, as the operations of addition and multiplication on vector-valued sequences become ill-defined. However, we can still take complex conjugates of a sequence, and add sequences taking values in the same vector space ${{\bf C}^m}$, and for sequences taking values in different vector spaces ${{\bf C}^m}$, ${{\bf C}^{m'}}$, we may utilise the tensor product ${\otimes: {\bf C}^m \times {\bf C}^{m'} \rightarrow {\bf C}^{mm'}}$, which we will normalise by defining

$\displaystyle (z_1,\dots,z_m) \otimes (w_1,\dots,w_{m'}) = (z_1 w_1, \dots, z_1 w_{m'}, \dots, z_m w_1, \dots, z_m w_{m'} ).$

This product is associative and bilinear, and also commutative up to permutation of the indices. It also interacts well with the Hermitian norm

$\displaystyle \| (z_1,\dots,z_m) \| := \sqrt{|z_1|^2 + \dots + |z_m|^2}$

since we have ${\|z \otimes w\| = \|z\| \|w\|}$.

The traditional definition of a basic nilsequence (as defined for instance by Bergelson, Host, and Kra) is as follows:

Definition 1 (Basic nilsequence, first definition) A nilmanifold of step at most ${d}$ is a quotient ${G/\Gamma}$, where ${G}$ is a connected, simply connected nilpotent Lie group of step at most ${d}$ (thus, all ${d+1}$-fold commutators vanish) and ${\Gamma}$ is a discrete cocompact lattice in ${G}$. A basic nilsequence of degree at most ${d}$ is a sequence of the form ${n \mapsto F(g^n g_0 \Gamma)}$, where ${g_0 \Gamma \in G/\Gamma}$, ${g \in G}$, and ${F: G/\Gamma \rightarrow {\bf C}^m}$ is a continuous function.

For instance, it is not difficult using this definition to show that a sequence is a basic nilsequence of degree at most ${1}$ if and only if it is quasiperiodic. The requirement that ${G}$ be simply connected can be easily removed if desired by passing to a universal cover, but it is technically convenient to assume it (among other things, it allows for a well-defined logarithm map that obeys the Baker-Campbell-Hausdorff formula). When one wishes to perform a more quantitative analysis of nilsequences (particularly when working on a “single scale”. sich as on a single long interval ${\{ N+1, \dots, N+M\}}$), it is common to impose additional regularity conditions on the function ${F}$, such as Lipschitz continuity or smoothness, but ordinary continuity will suffice for the qualitative discussion in this blog post.

Nowadays, and particularly when one needs to understand the “single-scale” equidistribution properties of nilsequences, it is more convenient (as is for instance done in this ICM paper of Green) to use an alternate definition of a nilsequence as follows.

Definition 2 Let ${d \geq 0}$. A filtered group of degree at most ${d}$ is a group ${G}$ together with a sequence ${G_\bullet = (G_0,G_1,G_2,\dots)}$ of subgroups ${G \geq G_0 \geq G_1 \geq \dots}$ with ${G_{d+1}=\{\hbox{id}\}}$ and ${[G_i,G_j] \subset G_{i+j}}$ for ${i,j \geq 0}$. A polynomial sequence ${g: {\bf Z} \rightarrow G}$ into a filtered group is a function such that ${\partial_{h_i} \dots \partial_{h_1} g(n) \in G_i}$ for all ${i \geq 0}$ and ${n,h_1,\dots,h_i \in{\bf Z}}$, where ${\partial_h g(n) := g(n+h) g(n)^{-1}}$ is the difference operator. A filtered nilmanifold of degree at most ${s}$ is a quotient ${G/\Gamma}$, where ${G}$ is a filtered group of degree at most ${s}$ such that ${G}$ and all of the subgroups ${G_i}$ are connected, simply connected nilpotent filtered Lie group, and ${\Gamma}$ is a discrete cocompact subgroup of ${G}$ such that ${\Gamma_i := \Gamma \cap G_i}$ is a discrete cocompact subgroup of ${G_i}$. A basic nilsequence of degree at most ${d}$ is a sequence of the form ${n \mapsto F(g(n))}$, where ${g: {\bf Z} \rightarrow G}$ is a polynomial sequence, ${G/\Gamma}$ is a filtered nilmanifold of degree at most ${d}$, and ${F: G \rightarrow {\bf C}^m}$ is a continuous function which is ${\Gamma}$-automorphic, in the sense that ${F(g \gamma) = F(g)}$ for all ${g \in G}$ and ${\gamma \in \Gamma}$.

One can easily identify a ${\Gamma}$-automorphic function on ${G}$ with a function on ${G/\Gamma}$, but there are some (very minor) advantages to working on the group ${G}$ instead of the quotient ${G/\Gamma}$, as it becomes slightly easier to modify the automorphy group ${\Gamma}$ when needed. (But because the action of ${\Gamma}$ on ${G}$ is free, one can pass from ${\Gamma}$-automorphic functions on ${G}$ to functions on ${G/\Gamma}$ with very little difficulty.) The main reason to work with polynomial sequences ${n \mapsto g(n)}$ rather than geometric progressions ${n \mapsto g^n g_0 \Gamma}$ is that they form a group, a fact essentially established by by Lazard and Leibman; see Corollary B.4 of this paper of Green, Ziegler, and myself for a proof in the filtered group setting.

It is easy to see that any sequence that is a basic nilsequence of degree at most ${d}$ in the sense of the first definition, is also a basic nilsequence of degree at most ${d}$ in the second definition, since a nilmanifold of degree at most ${d}$ can be filtered using the lower central series, and any linear sequence ${n \mapsto g^n g_0}$ will be a polynomial sequence with respect to that filtration. The converse implication is a little trickier, but still not too hard to show: see Appendix C of this paper of Ben Green, Tamar Ziegler, and myself. There are two key examples of basic nilsequences to keep in mind. The first are the polynomially quasiperiodic sequences

$\displaystyle a(n) = F( P_1(n), \dots, P_k(n) ),$

where ${P_1,\dots,P_k: {\bf Z} \rightarrow {\bf R}}$ are polynomials of degree at most ${d}$, and ${F: {\bf R}^k \rightarrow {\bf C}^m}$ is a ${{\bf Z}^k}$-automorphic (i.e., ${{\bf Z}^k}$-periodic) continuous function. The map ${P: {\bf Z} \rightarrow {\bf R}^k}$ defined by ${P(n) := (P_1(n),\dots,P_k(n))}$ is a polynomial map of degree at most ${d}$, if one filters ${{\bf R}^k}$ by defining ${({\bf R}^k)_i}$ to equal ${{\bf R}^k}$ when ${i \leq d}$, and ${\{0\}}$ for ${i > d}$. The torus ${{\bf R}^k/{\bf Z}^k}$ then becomes a filtered nilmanifold of degree at most ${d}$, and ${a(n)}$ is thus a basic nilsequence of degree at most ${d}$ as per the second definition. It is also possible explicitly describe ${a_n}$ as a basic nilsequence of degree at most ${d}$ as per the first definition, for instance (in the ${k=1}$ case) by taking ${G}$ to be the space of upper triangular unipotent ${d+1 \times d+1}$ real matrices, and ${\Gamma}$ the subgroup with integer coefficients; we leave the details to the interested reader.

The other key example is a sequence of the form

$\displaystyle a(n) = F( \alpha n, \{ \alpha n \} \beta n )$

where ${\alpha,\beta}$ are real numbers, ${\{ \alpha n \} = \alpha n - \lfloor \alpha n \rfloor}$ denotes the fractional part of ${\alpha n}$, and and ${F: {\bf R}^2 \rightarrow {\bf C}^m}$ is a ${{\bf Z}^2}$-automorphic continuous function that vanishes in a neighbourhood of ${{\bf Z} \times {\bf R}}$. To describe this as a nilsequence, we use the nilpotent connected, simply connected degree ${2}$, Heisenberg group

$\displaystyle G := \begin{pmatrix} 1 & {\bf R} & {\bf R} \\ 0 & 1 & {\bf R} \\ 0 & 0 & 1 \end{pmatrix}$

with the lower central series filtration ${G_0=G_1=G}$, ${G_2= [G,G] = \begin{pmatrix} 1 &0 & {\bf R} \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}}$, and ${G_i = \{ \mathrm{id} \}}$ for ${i > 2}$, ${\Gamma}$ to be the discrete compact subgroup

$\displaystyle \Gamma := \begin{pmatrix} 1 & {\bf Z} & {\bf Z} \\ 0 & 1 & {\bf Z} \\ 0 & 0 & 1 \end{pmatrix},$

${g: {\bf Z} \rightarrow G}$ to be the polynomial sequence

$\displaystyle g(n) := \begin{pmatrix} 1 & \beta n & \alpha \beta n^2 \\ 0 & 1 & \alpha n \\ 0 & 0 & 1 \end{pmatrix}$

and ${\tilde F: G \rightarrow {\bf C}^m}$ to be the ${\Gamma}$-automorphic function

$\displaystyle \tilde F( \begin{pmatrix} 1 & x & y \\ 0 & 1 & z \\ 0 & 0 & 1 \end{pmatrix} ) = F( \{ z \}, y - \lfloor z \rfloor x );$

one easily verifies that this function is indeed ${\Gamma}$-automorphic, and it is continuous thanks to the vanishing properties of ${F}$. Also we have ${a(n) = \tilde F(g(n))}$, so ${a}$ is a basic nilsequence of degree at most ${2}$. One can concoct similar examples with ${\{ \alpha n \} \beta n}$ replaced by other “bracket polynomials” of ${n}$; for instance

$\displaystyle a(n) = F( \alpha n, \{ \alpha n - \frac{1}{2} \} \beta n )$

will be a basic nilsequence if ${F}$ now vanishes in a neighbourhood of ${(\frac{1}{2}+{\bf Z}) \times {\bf R}}$ rather than ${{\bf Z} \times {\bf R}}$. See this paper of Bergelson and Leibman for more discussion of bracket polynomials (also known as generalised polynomials) and their relationship to nilsequences.

A nilsequence of degree at most ${d}$ is defined to be a sequence that is the uniform limit of basic nilsequences of degree at most ${d}$. Thus for instance a sequence is a nilsequence of degree at most ${1}$ if and only if it is almost periodic, while a sequence is a nilsequence of degree at most ${0}$ if and only if it is constant. Such objects arise in higher order recurrence: for instance, if ${h_0,\dots,h_d}$ are integers, ${(X,\mu,T)}$ is a measure-preserving system, and ${f_0,\dots,f_d \in L^\infty(X)}$, then it was shown by Leibman that the sequence

$\displaystyle n \mapsto \int_X f_0(T^{h_0 n} x) \dots f_d(T^{h_d n} x)\ d\mu(x)$

is equal to a nilsequence of degree at most ${d}$, plus a null sequence. (The special case when the measure-preserving system was ergodic and ${h_i = i}$ for ${i=0,\dots,d}$ was previously established by Bergelson, Host, and Kra.) Nilsequences also arise in the inverse theory of the Gowers uniformity norms, as discussed for instance in this previous post.

It is easy to see that a sequence ${a: {\bf Z} \rightarrow {\bf C}^m}$ is a basic nilsequence of degree at most ${d}$ if and only if each of its ${m}$ components are. The scalar basic nilsequences ${a: {\bf Z} \rightarrow {\bf C}}$ of degree ${d}$ are easily seen to form a ${*}$-algebra (that is to say, they are a complex vector space closed under pointwise multiplication and complex conjugation), which implies similarly that vector-valued basic nilsequences ${a: {\bf Z} \rightarrow {\bf C}^m}$ of degree at most ${d}$ form a complex vector space closed under complex conjugation for each ${m}$, and that the tensor product of any two basic nilsequences of degree at most ${d}$ is another basic nilsequence of degree at most ${d}$. Similarly with “basic nilsequence” replaced by “nilsequence” throughout.

Now we turn to the notion of a nilcharacter, as defined in this paper of Ben Green, Tamar Ziegler, and myself:

Definition 3 (Nilcharacters) Let ${d \geq 1}$. A sub-nilcharacter of degree ${d}$ is a basic nilsequence ${\chi: n \mapsto F(g(n))}$ of degree at most ${d}$, such that ${F}$ obeys the additional modulation property

$\displaystyle F( g_d g ) = e( \xi \cdot g_d ) F(g) \ \ \ \ \ (1)$

for all ${g \in G}$ and ${g_d \in G_d}$, where ${\xi: G_d \rightarrow {\bf R}}$ is a continuous homomorphism ${g_d \mapsto \xi \cdot g_d}$. (Note from (1) and ${\Gamma}$-automorphy that unless ${F}$ vanishes identically, ${\xi}$ must map ${\Gamma_d}$ to ${{\bf Z}}$, thus without loss of generality one can view ${\xi}$ as an element of the Pontryagial dual of the torus ${G_d / \Gamma_d}$.) If in addition one has ${\|F(g)\|=1}$ for all ${g \in G}$, we call ${\chi}$ a nilcharacter of degree ${d \geq 1}$.

In the degree one case ${d=1}$, the only sub-nilcharacters are of the form ${\chi(n) = e(\alpha n)}$ for some vector ${c \in {\bf C}^m}$ and ${\alpha \in {\bf R}}$, and this is a nilcharacter if ${c}$ is a unit vector. Similarly, in higher degree, any sequence of the form ${\chi(n) = c e(P(n))}$, where ${c \in {\bf C}^m}$ is a vector and ${P: {\bf Z} \rightarrow {\bf R}}$ is a polynomial of degree at most ${d}$, is a sub-nilcharacter of degree ${d}$, and a character if ${c}$ is a unit vector. A nilsequence of degree at most ${d-1}$ is automatically a sub-nilcharacter of degree ${d}$, and a nilcharacter if it is of magnitude ${1}$. A further example of a nilcharacter is provided by the two-dimensional sequence ${\chi: {\bf Z} \rightarrow {\bf C}^2}$ defined by

$\displaystyle \chi(n) := ( F_0( \alpha n ) e( \{ \alpha n \} \beta n ), F_{1/2}( \alpha n ) e( \{ \alpha n - \frac{1}{2} \} \beta n ) ) \ \ \ \ \ (2)$

where ${F_0, F_{1/2}: {\bf R} \rightarrow {\bf C}}$ are continuous, ${{\bf Z}}$-automorphic functions that vanish on a neighbourhood of ${{\bf Z}}$ and ${\frac{1}{2}+{\bf Z}}$ respectively, and which form a partition of unity in the sense that

$\displaystyle |F_0(x)|^2 + |F_{1/2}(x)|^2 = 1$

for all ${x \in {\bf R}}$. Note that one needs both ${F_0}$ and ${F_{1/2}}$ to be not identically zero in order for all these conditions to be satisfied; it turns out (for topological reasons) that there is no scalar nilcharacter that is “equivalent” to this nilcharacter in a sense to be defined shortly. In some literature, one works exclusively with sub-nilcharacters rather than nilcharacters, however the former space contains zero-divisors, which is a little annoying technically. Nevertheless, both nilcharacters and sub-nilcharacters generate the same set of “symbols” as we shall see later.

We claim that every degree ${d}$ sub-nilcharacter ${f: {\bf Z} \rightarrow {\bf C}^m}$ can be expressed in the form ${f = c \chi}$, where ${\chi: {\bf Z} \rightarrow {\bf C}^{m'}}$ is a degree ${d}$ nilcharacter, and ${c: {\bf C}^{m'} \rightarrow {\bf C}^m}$ is a linear transformation. Indeed, by scaling we may assume ${f(n) = F(g(n))}$ where ${|F| < 1}$ uniformly. Using partitions of unity, one can find further functions ${F_1,\dots,F_m}$ also obeying (1) for the same character ${\xi}$ such that ${|F_1|^2 + \dots + |F_m|^2}$ is non-zero; by dividing out the ${F_1,\dots,F_m}$ by the square root of this quantity, and then multiplying by ${\sqrt{1-|F|^2}}$, we may assume that

$\displaystyle |F|^2 + |F_1|^2 + \dots + |F_m|^2 = 1,$

and then

$\displaystyle \chi(n) := (F(g(n)), F_1(g(n)), \dots, F_m(g(n)))$

becomes a degree ${d}$ nilcharacter that contains ${f(n)}$ amongst its components, giving the claim.

As we shall show below, nilsequences can be approximated uniformly by linear combinations of nilcharacters, in much the same way that quasiperiodic or almost periodic sequences can be approximated uniformly by linear combinations of linear phases. In particular, nilcharacters can be used as “obstructions to uniformity” in the sense of the inverse theory of the Gowers uniformity norms.

The space of degree ${d}$ nilcharacters forms a semigroup under tensor product, with the constant sequence ${1}$ as the identity. One can upgrade this semigroup to an abelian group by quotienting nilcharacters out by equivalence:

Definition 4 Let ${d \geq 1}$. We say that two degree ${d}$ nilcharacters ${\chi: {\bf Z} \rightarrow {\bf C}^m}$, ${\chi': {\bf Z} \rightarrow {\bf C}^{m'}}$ are equivalent if ${\chi \otimes \overline{\chi'}: {\bf Z} \rightarrow {\bf C}^{mm'}}$ is equal (as a sequence) to a basic nilsequence of degree at most ${d-1}$. (We will later show that this is indeed an equivalence relation.) The equivalence class ${[\chi]_{\mathrm{Symb}^d({\bf Z})}}$ of such a nilcharacter will be called the symbol of that nilcharacter (in analogy to the symbol of a differential or pseudodifferential operator), and the collection of such symbols will be denoted ${\mathrm{Symb}^d({\bf Z})}$.

As we shall see below the fold, ${\mathrm{Symb}^d({\bf Z})}$ has the structure of an abelian group, and enjoys some nice “symbol calculus” properties; also, one can view symbols as precisely describing the obstruction to equidistribution for nilsequences. For ${d=1}$, the group is isomorphic to the Ponytragin dual ${\hat {\bf Z} = {\bf R}/{\bf Z}}$ of the integers, and ${\mathrm{Symb}^d({\bf Z})}$ for ${d > 1}$ should be viewed as higher order generalisations of this Pontryagin dual. In principle, this group can be explicitly described for all ${d}$, but the theory rapidly gets complicated as ${d}$ increases (much as the classification of nilpotent Lie groups or Lie algebras of step ${d}$ rapidly gets complicated even for medium-sized ${d}$ such as ${d=3}$ or ${d=4}$). We will give an explicit description of the ${d=2}$ case here. There is however one nice (and non-trivial) feature of ${\mathrm{Symb}^d({\bf Z})}$ for ${d \geq 2}$ – it is not just an abelian group, but is in fact a vector space over the rationals ${{\bf Q}}$!