You are currently browsing the monthly archive for March 2012.

I recently finished the first draft of the the first of my books, entitled “Hilbert’s fifth problem and related topics“, based on the lecture notes for my graduate course of the same name.    The PDF of this draft is available here.  As always, comments and corrections are welcome.

A few days ago, Endre Szemerédi was awarded the 2012 Abel prize “for his fundamental contributions to discrete mathematics and theoretical computer science, and in recognition of the profound and lasting impact of these contributions on additive number theory and ergodic theory.” The full citation for the prize may be found here, and the written notes for a talk given by Tim Gowers on Endre’s work at the announcement may be found here (and video of the talk can be found here).

As I was on the Abel prize committee this year, I won’t comment further on the prize, but will instead focus on what is arguably Endre’s most well known result, namely Szemerédi’s theorem on arithmetic progressions:

Theorem 1 (Szemerédi’s theorem) Let ${A}$ be a set of integers of positive upper density, thus ${\lim \sup_{N \rightarrow\infty} \frac{|A \cap [-N,N]|}{|[-N,N]|} > 0}$, where ${[-N,N] := \{-N, -N+1,\ldots,N\}}$. Then ${A}$ contains an arithmetic progression of length ${k}$ for any ${k>1}$.

Szemerédi’s original proof of this theorem is a remarkably intricate piece of combinatorial reasoning. Most proofs of theorems in mathematics – even long and difficult ones – generally come with a reasonably compact “high-level” overview, in which the proof is (conceptually, at least) broken down into simpler pieces. There may well be technical difficulties in formulating and then proving each of the component pieces, and then in fitting the pieces together, but usually the “big picture” is reasonably clear. To give just one example, the overall strategy of Perelman’s proof of the Poincaré conjecture can be briefly summarised as follows: to show that a simply connected three-dimensional manifold is homeomorphic to a sphere, place a Riemannian metric on it and perform Ricci flow, excising any singularities that arise by surgery, until the entire manifold becomes extinct. By reversing the flow and analysing the surgeries performed, obtain enough control on the topology of the original manifold to establish that it is a topological sphere.

In contrast, the pieces of Szemerédi’s proof are highly interlocking, particularly with regard to all the epsilon-type parameters involved; it takes quite a bit of notational setup and foundational lemmas before the key steps of the proof can even be stated, let alone proved. Szemerédi’s original paper contains a logical diagram of the proof (reproduced in Gowers’ recent talk) which already gives a fair indication of this interlocking structure. (Many years ago I tried to present the proof, but I was unable to find much of a simplification, and my exposition is probably not that much clearer than the original text.) Even the use of nonstandard analysis, which is often helpful in cleaning up armies of epsilons, turns out to be a bit tricky to apply here. (In typical applications of nonstandard analysis, one can get by with a single nonstandard universe, constructed as an ultrapower of the standard universe; but to correctly model all the epsilons occuring in Szemerédi’s argument, one needs to repeatedly perform the ultrapower construction to obtain a (finite) sequence of increasingly nonstandard (and increasingly saturated) universes, each one containing unbounded quantities that are far larger than any quantity that appears in the preceding universe, as discussed at the end of this previous blog post. This sequence of universes does end up concealing all the epsilons, but it is not so clear that this is a net gain in clarity for the proof; I may return to the nonstandard presentation of Szemeredi’s argument at some future juncture.)

Instead of trying to describe the entire argument here, I thought I would instead show some key components of it, with only the slightest hint as to how to assemble the components together to form the whole proof. In particular, I would like to show how two particular ingredients in the proof – namely van der Waerden’s theorem and the Szemerédi regularity lemma – become useful. For reasons that will hopefully become clearer later, it is convenient not only to work with ordinary progressions ${P_1 = \{ a, a+r_1, a+2r_1, \ldots, a+(k_1-1)r_1\}}$, but also progressions of progressions ${P_2 := \{ P_1, P_1 + r_2, P_1+2r_2, \ldots, P_1+(k_2-1)r_2\}}$, progressions of progressions of progressions, and so forth. (In additive combinatorics, these objects are known as generalised arithmetic progressions of rank one, two, three, etc., and play a central role in the subject, although the way they are used in Szemerédi’s proof is somewhat different from the way that they are normally used in additive combinatorics.) Very roughly speaking, Szemerédi’s proof begins by building an enormous generalised arithmetic progression of high rank containing many elements of the set ${A}$ (arranged in a “near-maximal-density” configuration), and then steadily prunes this progression to improve the combinatorial properties of the configuration, until one ends up with a single rank one progression of length ${k}$ that consists entirely of elements of ${A}$.

To illustrate some of the basic ideas, let us first consider a situation in which we have located a progression ${P, P + r, \ldots, P+(k-1)r}$ of progressions of length ${k}$, with each progression ${P+ir}$, ${i=0,\ldots,k-1}$ being quite long, and containing a near-maximal amount of elements of ${A}$, thus

$\displaystyle |A \cap (P+ir)| \approx \delta |P|$

where ${\delta := \lim \sup_{|P| \rightarrow \infty} \frac{|A \cap P|}{|P|}}$ is the “maximal density” of ${A}$ along arithmetic progressions. (There are a lot of subtleties in the argument about exactly how good the error terms are in various approximations, but we will ignore these issues for the sake of this discussion and just use the imprecise symbols such as ${\approx}$ instead.) By hypothesis, ${\delta}$ is positive. The objective is then to locate a progression ${a, a+r', \ldots,a+(k-1)r'}$ in ${A}$, with each ${a+ir}$ in ${P+ir}$ for ${i=0,\ldots,k-1}$. It may help to view the progression of progressions ${P, P + r, \ldots, P+(k-1)r}$ as a tall thin rectangle ${P \times \{0,\ldots,k-1\}}$.

If we write ${A_i := \{ a \in P: a+ir \in A \}}$ for ${i=0,\ldots,k-1}$, then the problem is equivalent to finding a (possibly degenerate) arithmetic progression ${a_0,a_1,\ldots,a_{k-1}}$, with each ${a_i}$ in ${A_i}$.

By hypothesis, we know already that each set ${A_i}$ has density about ${\delta}$ in ${P}$:

$\displaystyle |A_i \cap P| \approx \delta |P|. \ \ \ \ \ (1)$

Let us now make a “weakly mixing” assumption on the ${A_i}$, which roughly speaking asserts that

$\displaystyle |A_i \cap E| \approx \delta \sigma |P| \ \ \ \ \ (2)$

for “most” subsets ${E}$ of ${P}$ of density ${\approx \sigma}$ of a certain form to be specified shortly. This is a plausible type of assumption if one believes ${A_i}$ to behave like a random set, and if the sets ${E}$ are constructed “independently” of the ${A_i}$ in some sense. Of course, we do not expect such an assumption to be valid all of the time, but we will postpone consideration of this point until later. Let us now see how this sort of weakly mixing hypothesis could help one count progressions ${a_0,\ldots,a_{k-1}}$ of the desired form.

We will inductively consider the following (nonrigorously defined) sequence of claims ${C(i,j)}$ for each ${0 \leq i \leq j < k}$:

• ${C(i,j)}$: For most choices of ${a_j \in P}$, there are ${\sim \delta^i |P|}$ arithmetic progressions ${a_0,\ldots,a_{k-1}}$ in ${P}$ with the specified choice of ${a_j}$, such that ${a_l \in A_l}$ for all ${l=0,\ldots,i-1}$.

(Actually, to avoid boundary issues one should restrict ${a_j}$ to lie in the middle third of ${P}$, rather than near the edges, but let us ignore this minor technical detail.) The quantity ${\delta^i |P|}$ is natural here, given that there are ${\sim |P|}$ arithmetic progressions ${a_0,\ldots,a_{k-1}}$ in ${P}$ that pass through ${a_i}$ in the ${i^{th}}$ position, and that each one ought to have a probability of ${\delta^i}$ or so that the events ${a_0 \in A_0, \ldots, a_{i-1} \in A_{i-1}}$ simultaneously hold.) If one has the claim ${C(k-1,k-1)}$, then by selecting a typical ${a_{k-1}}$ in ${A_{k-1}}$, we obtain a progression ${a_0,\ldots,a_{k-1}}$ with ${a_i \in A_i}$ for all ${i=0,\ldots,k-1}$, as required. (In fact, we obtain about ${\delta^k |P|^2}$ such progressions by this method.)

We can heuristically justify the claims ${C(i,j)}$ by induction on ${i}$. For ${i=0}$, the claims ${C(0,j)}$ are clear just from direct counting of progressions (as long as we keep ${a_j}$ away from the edges of ${P}$). Now suppose that ${i>0}$, and the claims ${C(i-1,j)}$ have already been proven. For any ${i \leq j < k}$ and for most ${a_j \in P}$, we have from hypothesis that there are ${\sim \delta^{i-1} |P|}$ progressions ${a_0,\ldots,a_{k-1}}$ in ${P}$ through ${a_j}$ with ${a_0 \in A_0,\ldots,a_{i-2}\in A_{i-2}}$. Let ${E = E(a_j)}$ be the set of all the values of ${a_{i-1}}$ attained by these progressions, then ${|E| \sim \delta^{i-1} |P|}$. Invoking the weak mixing hypothesis, we (heuristically, at least) conclude that for most choices of ${a_j}$, we have

$\displaystyle |A_{i-1} \cap E| \sim \delta^i |P|$

which then gives the desired claim ${C(i,j)}$.

The observant reader will note that we only needed the claim ${C(i,j)}$ in the case ${j=k-1}$ for the above argument, but for technical reasons, the full proof requires one to work with more general values of ${j}$ (also the claim ${C(i,j)}$ needs to be replaced by a more complicated version of itself, but let’s ignore this for sake of discussion).

We now return to the question of how to justify the weak mixing hypothesis (2). For a single block ${A_i}$ of ${A}$, one can easily concoct a scenario in which this hypothesis fails, by choosing ${E}$ to overlap with ${A_i}$ too strongly, or to be too disjoint from ${A_i}$. However, one can do better if one can select ${A_i}$ from a long progression of blocks. The starting point is the following simple double counting observation that gives the right upper bound:

Proposition 2 (Single upper bound) Let ${P, P+r, \ldots, P+(M-1)r}$ be a progression of progressions ${P}$ for some large ${M}$. Suppose that for each ${i=0,\ldots,M-1}$, the set ${A_i := \{ a \in P: a+ir \in A \}}$ has density ${\approx \delta}$ in ${P}$ (i.e. (1) holds). Let ${E}$ be a subset of ${P}$ of density ${\approx \sigma}$. Then (if ${M}$ is large enough) one can find an ${i = 0,\ldots,M-1}$ such that

$\displaystyle |A_i \cap E| \lessapprox \delta \sigma |P|.$

Proof: The key is the double counting identity

$\displaystyle \sum_{i=0}^{M-1} |A_i \cap E| = \sum_{a \in E} |A \cap \{ a, a+r, \ldots, a+(M-1) r\}|.$

Because ${A}$ has maximal density ${\delta}$ and ${M}$ is large, we have

$\displaystyle |A \cap \{ a, a+r, \ldots, a+(M-1) r\}| \lessapprox \delta M$

for each ${a}$, and thus

$\displaystyle \sum_{i=0}^{M-1} |A_i \cap E| \lessapprox \delta M |E|.$

The claim then follows from the pigeonhole principle. $\Box$

Now suppose we want to obtain weak mixing not just for a single set ${E}$, but for a small number ${E_1,\ldots,E_m}$ of such sets, i.e. we wish to find an ${i}$ for which

$\displaystyle |A_i \cap E_j| \lessapprox \delta \sigma_j |P|. \ \ \ \ \ (3)$

for all ${j=1,\ldots,m}$, where ${\sigma_j}$ is the density of ${E_j}$ in ${P}$. The above proposition gives, for each ${j}$, a choice of ${i}$ for which (3) holds, but it could be a different ${i}$ for each ${j}$, and so it is not immediately obvious how to use Proposition 2 to find an ${i}$ for which (3) holds simultaneously for all ${j}$. However, it turns out that the van der Waerden theorem is the perfect tool for this amplification:

Proposition 3 (Multiple upper bound) Let ${P, P+r, \ldots, P+(M-1)r}$ be a progression of progressions ${P+ir}$ for some large ${M}$. Suppose that for each ${i=0,\ldots,M-1}$, the set ${A_i := \{ a \in P: a+ir \in A \}}$ has density ${\approx \delta}$ in ${P}$ (i.e. (1) holds). For each ${1 \leq j \leq m}$, let ${E_j}$ be a subset of ${P}$ of density ${\approx \sigma_j}$. Then (if ${M}$ is large enough depending on ${j}$) one can find an ${i = 0,\ldots,M-1}$ such that

$\displaystyle |A_i \cap E_j| \lessapprox \delta \sigma_j |P|$

simultaneously for all ${1 \leq j \leq m}$.

Proof: Suppose that the claim failed (for some suitably large ${M}$). Then, for each ${i = 0,\ldots,M-1}$, there exists ${j \in \{1,\ldots,m\}}$ such that

$\displaystyle |A_i \cap E_j| \gg \delta \sigma_j |P|.$

This can be viewed as a colouring of the interval ${\{1,\ldots,M\}}$ by ${m}$ colours. If we take ${M}$ large compared to ${m}$, van der Waerden’s theorem allows us to then find a long subprogression of ${\{1,\ldots,M\}}$ which is monochromatic, so that ${j}$ is constant on this progression. But then this will furnish a counterexample to Proposition 2. $\Box$

Proposition 4 (Multiple mixing) Let ${P, P+r, \ldots, P+(M-1)r}$ be a progression of progressions ${P+ir}$ for some large ${M}$. Suppose that for each ${i=0,\ldots,M-1}$, the set ${A_i := \{ a \in P: a+ir \in A \}}$ has density ${\approx \delta}$ in ${P}$ (i.e. (1) holds). For each ${1 \leq j \leq m}$, let ${E_j}$ be a subset of ${P}$ of density ${\approx \sigma_j}$. Then (if ${M}$ is large enough depending on ${m}$) one can find an ${i = 0,\ldots,M-1}$ such that

$\displaystyle |A_i \cap E_j| \approx \delta \sigma_j |P|$

simultaneously for all ${1 \leq j \leq m}$.

Proof: By applying the previous proposition to the collection of sets ${E_1,\ldots,E_m}$ and their complements ${P\backslash E_1,\ldots,P \backslash E_m}$ (thus replacing ${m}$ with ${2m}$, one can find an ${i}$ for which

$\displaystyle |A_i \cap E_j| \lessapprox \delta \sigma_j |P|$

and

$\displaystyle |A_i \cap (P \backslash E_j)| \lessapprox \delta (1-\sigma_j) |P|$

which gives the claim. $\Box$

However, this improvement of Proposition 2 turns out to not be strong enough for applications. The reason is that the number ${m}$ of sets ${E_1,\ldots,E_m}$ for which mixing is established is too small compared with the length ${M}$ of the progression one has to use in order to obtain that mixing. However, thanks to the magic of the Szemerédi regularity lemma, one can amplify the above proposition even further, to allow for a huge number of ${E_i}$ to be mixed (at the cost of excluding a small fraction of exceptions):

Proposition 5 (Really multiple mixing) Let ${P, P+r, \ldots, P+(M-1)r}$ be a progression of progressions ${P+ir}$ for some large ${M}$. Suppose that for each ${i=0,\ldots,M-1}$, the set ${A_i := \{ a \in P: a+ir \in A \}}$ has density ${\approx \delta}$ in ${P}$ (i.e. (1) holds). For each ${v}$ in some (large) finite set ${V}$, let ${E_v}$ be a subset of ${P}$ of density ${\approx \sigma_v}$. Then (if ${M}$ is large enough, but not dependent on the size of ${V}$) one can find an ${i = 0,\ldots,M-1}$ such that

$\displaystyle |A_i \cap E_v| \approx \delta \sigma_v |P|$

simultaneously for almost all ${v \in V}$.

Proof: We build a bipartite graph ${G = (P, V, E)}$ connecting the progression ${P}$ to the finite set ${V}$ by placing an edge ${(a,v)}$ between an element ${a \in P}$ and an element ${v \in V}$ whenever ${a \in E_v}$. The number ${|E_v| \approx \sigma_v |P|}$ can then be interpreted as the degree of ${v}$ in this graph, while the number ${|A_i \cap E_v|}$ is the number of neighbours of ${v}$ that land in ${A_i}$.

We now apply the regularity lemma to this graph ${G}$. Roughly speaking, what this lemma does is to partition ${P}$ and ${V}$ into almost equally sized cells ${P = P_1 \cup \ldots P_m}$ and ${V = V_1 \cup \ldots V_m}$ such that for most pairs ${P_j, V_k}$ of cells, the graph ${G}$ resembles a random bipartite graph of some density ${d_{jk}}$ between these two cells. The key point is that the number ${m}$ of cells here is bounded uniformly in the size of ${P}$ and ${V}$. As a consequence of this lemma, one can show that for most vertices ${v}$ in a typical cell ${V_k}$, the number ${|E_v|}$ is approximately equal to

$\displaystyle |E_v| \approx \sum_{j=1}^m d_{ij} |P_j|$

and the number ${|A_i \cap E_v|}$ is approximately equal to

$\displaystyle |A_i \cap E_v| \approx \sum_{j=1}^m d_{ij} |A_i \cap P_j|.$

The point here is that the ${|V|}$ different statistics ${|A_i \cap E_v|}$ are now controlled by a mere ${m}$ statistics ${|A_i \cap P_j|}$ (this is not unlike the use of principal component analysis in statistics, incidentally, but that is another story). Now, we invoke Proposition 4 to find an ${i}$ for which

$\displaystyle |A_i \cap P_j| \approx \delta |P_j|$

simultaneously for all ${j=1,\ldots,m}$, and the claim follows. $\Box$

This proposition now suggests a way forward to establish the type of mixing properties (2) needed for the preceding attempt at proving Szemerédi’s theorem to actually work. Whereas in that attempt, we were working with a single progression of progressions ${P, P+r, \ldots, P+(k-1)r}$ of progressions containing a near-maximal density of elements of ${A}$, we will now have to work with a family ${(P_\lambda, P_\lambda+r_\lambda,\ldots,P_\lambda+(k-1)r_\lambda)_{\lambda \in \Lambda}}$ of such progression of progressions, where ${\Lambda}$ ranges over some suitably large parameter set. Furthermore, in order to invoke Proposition 5, this family must be “well-arranged” in some arithmetic sense; in particular, for a given ${i}$, it should be possible to find many reasonably large subfamilies of this family for which the ${i^{th}}$ terms ${P_\lambda + i r_\lambda}$ of the progression of progressions in this subfamily are themselves in arithmetic progression. (Also, for technical reasons having to do with the fact that the sets ${E_v}$ in Proposition 5 are not allowed to depend on ${i}$, one also needs the progressions ${P_\lambda + i' r_\lambda}$ for any given ${0 \leq i' < i}$ to be “similar” in the sense that they intersect ${A}$ in the same fashion (thus the sets ${A \cap (P_\lambda + i' r_\lambda)}$ as ${\lambda}$ varies need to be translates of each other).) If one has this sort of family, then Proposition 5 allows us to “spend” some of the degrees of freedom of the parameter set ${\Lambda}$ in order to gain good mixing properties for at least one of the sets ${P_\lambda +i r_\lambda}$ in the progression of progressions.

Of course, we still have to figure out how to get such large families of well-arranged progressions of progressions. Szemerédi’s solution was to begin by working with generalised progressions of a much larger rank ${d}$ than the rank ${2}$ progressions considered here; roughly speaking, to prove Szemerédi’s theorem for length ${k}$ progressions, one has to consider generalised progressions of rank as high as ${2^k+1}$. It is possible by a reasonably straightforward (though somewhat delicate) “density increment argument” to locate a huge generalised progression of this rank which is “saturated” by ${A}$ in a certain rather technical sense (related to the concept of “near maximal density” used previously). Then, by another reasonably elementary argument, it is possible to locate inside a suitable large generalised progression of some rank ${d}$, a family of large generalised progressions of rank ${d-1}$ which inherit many of the good properties of the original generalised progression, and which have the arithmetic structure needed for Proposition 5 to be applicable, at least for one value of ${i}$. (But getting this sort of property for all values of ${i}$ simultaneously is tricky, and requires many careful iterations of the above scheme; there is also the problem that by obtaining good behaviour for one index ${i}$, one may lose good behaviour at previous indices, leading to a sort of “Tower of Hanoi” situation which may help explain the exponential factor in the rank ${2^k+1}$ that is ultimately needed. It is an extremely delicate argument; all the parameters and definitions have to be set very precisely in order for the argument to work at all, and it is really quite remarkable that Endre was able to see it through to the end.)

This is an addendum to last quarter’s course notes on Hilbert’s fifth problem, which I am in the process of reviewing in order to transcribe them into a book (as was done similarly for several other sets of lecture notes on this blog). When reviewing the zeroth set of notes in particular, I found that I had made a claim (Proposition 11 from those notes) which asserted, roughly speaking, that any sufficiently large nilprogression was an approximate group, and promised to prove it later in the course when we had developed the ability to calculate efficiently in nilpotent groups. As it turned out, I managed finish the course without the need to develop these calculations, and so the proposition remained unproven. In order to rectify this, I will use this post to lay out some of the basic algebra of nilpotent groups, and use it to prove the above proposition, which turns out to be a bit tricky. (In my paper with Breuillard and Green, we avoid the need for this proposition by restricting attention to a special type of nilprogression, which we call a nilprogression in ${C}$-normal form, for which the computations are simpler.)

There are several ways to think about nilpotent groups; for instance one can use the model example of the Heisenberg group

$\displaystyle H(R) :=\begin{pmatrix} 1 & R & R \\ 0 & 1 & R\\ 0 & 0 & 1 \end{pmatrix}$

over an arbitrary ring ${R}$ (which need not be commutative), or more generally any matrix group consisting of unipotent upper triangular matrices, and view a general nilpotent group as being an abstract generalisation of such concrete groups. (In the case of nilpotent Lie groups, at least, this is quite an accurate intuition, thanks to Engel’s theorem.) Or, one can adopt a Lie-theoretic viewpoint and try to think of nilpotent groups as somehow arising from nilpotent Lie algebras; this intuition is rigorous when working with nilpotent Lie groups (at least when the characteristic is large, in order to avoid issues coming from the denominators in the Baker-Campbell-Hausdorff formula), but also retains some conceptual value in the non-Lie setting. In particular, nilpotent groups (particularly finitely generated ones) can be viewed in some sense as “nilpotent Lie groups over ${{\bf Z}}$“, even though Lie theory does not quite work perfectly when the underlying scalars merely form an integral domain instead of a field.

Another point of view, which arises naturally both in analysis and in algebraic geometry, is to view nilpotent groups as modeling “infinitesimal” perturbations of the identity, where the infinitesimals have a certain finite order. For instance, given a (not necessarily commutative) ring ${R}$ without identity (representing all the “small” elements of some larger ring or algebra), we can form the powers ${R^j}$ for ${j=1,2,\ldots}$, defined as the ring generated by ${j}$-fold products ${r_1 \ldots r_j}$ of elements ${r_1,\ldots,r_j}$ in ${R}$; this is an ideal of ${R}$ which represents the elements which are “${j^{th}}$ order” in some sense. If one then formally adjoins an identity ${1}$ onto the ring ${R}$, then for any ${s \geq 1}$, the multiplicative group ${G := 1+R \hbox{ mod } R^{s+1}}$ is a nilpotent group of step at most ${s}$. For instance, if ${R}$ is the ring of strictly upper ${s \times s}$ matrices (over some base ring), then ${R^{s+1}}$ vanishes and ${G}$ becomes the group of unipotent upper triangular matrices over the same ring, thus recovering the previous matrix-based example. In analysis applications, ${R}$ might be a ring of operators which are somehow of “order” ${O(\epsilon)}$ or ${O(\hbar)}$ for some small parameter ${\epsilon}$ or ${\hbar}$, and one wishes to perform Taylor expansions up to order ${O(\epsilon^s)}$ or ${O(\hbar^s)}$, thus discarding (i.e. quotienting out) all errors in ${R^{s+1}}$.

From a dynamical or group-theoretic perspective, one can also view nilpotent groups as towers of central extensions of a trivial group. Finitely generated nilpotent groups can also be profitably viewed as a special type of polycylic group; this is the perspective taken in this previous blog post. Last, but not least, one can view nilpotent groups from a combinatorial group theory perspective, as being words from some set of generators of various “degrees” subject to some commutation relations, with commutators of two low-degree generators being expressed in terms of higher degree objects, and all commutators of a sufficiently high degree vanishing. In particular, generators of a given degree can be moved freely around a word, as long as one is willing to generate commutator errors of higher degree.

With this last perspective, in particular, one can start computing in nilpotent groups by adopting the philosophy that the lowest order terms should be attended to first, without much initial concern for the higher order errors generated in the process of organising the lower order terms. Only after the lower order terms are in place should attention then turn to higher order terms, working successively up the hierarchy of degrees until all terms are dealt with. This turns out to be a relatively straightforward philosophy to implement in many cases (particularly if one is not interested in explicit expressions and constants, being content instead with qualitative expansions of controlled complexity), but the arguments are necessarily recursive in nature and as such can become a bit messy, and require a fair amount of notation to express precisely. So, unfortunately, the arguments here will be somewhat cumbersome and notation-heavy, even if the underlying methods of proof are relatively simple.

I’ve just uploaded to the arXiv my paper The asymptotic distribution of a single eigenvalue gap of a Wigner matrix, submitted to Probability Theory and Related Fields. This paper (like several of my previous papers) is concerned with the asymptotic distribution of the eigenvalues ${\lambda_1(M_n) \leq \ldots \leq \lambda_n(M_n)}$ of a random Wigner matrix ${M_n}$ in the limit ${n \rightarrow \infty}$, with a particular focus on matrices drawn from the Gaussian Unitary Ensemble (GUE). This paper is focused on the bulk of the spectrum, i.e. to eigenvalues ${\lambda_i(M_n)}$ with ${\delta n \leq i \leq (1-\delta) n}$ for some fixed ${\delta>0}$.

The location of an individual eigenvalue ${\lambda_i(M_n)}$ is by now quite well understood. If we normalise the entries of the matrix ${M_n}$ to have mean zero and variance ${1}$, then in the asymptotic limit ${n \rightarrow \infty}$, the Wigner semicircle law tells us that with probability ${1-o(1)}$ one has

$\displaystyle \lambda_i(M_n) =\sqrt{n} u + o(\sqrt{n})$

where the classical location ${u = u_{i/n} \in [-2,2]}$ of the eigenvalue is given by the formula

$\displaystyle \int_{-2}^{u} \rho_{sc}(x)\ dx = \frac{i}{n}$

and the semicircular distribution ${\rho_{sc}(x)\ dx}$ is given by the formula

$\displaystyle \rho_{sc}(x) := \frac{1}{2\pi} (4-x^2)_+^{1/2}.$

Actually, one can improve the error term here from ${o(\sqrt{n})}$ to ${O( \log^{1/2+\epsilon} n)}$ for any ${\epsilon>0}$ (see this previous recent paper of Van and myself for more discussion of these sorts of estimates, sometimes known as eigenvalue rigidity estimates).

From the semicircle law (and the fundamental theorem of calculus), one expects the ${i^{th}}$ eigenvalue spacing ${\lambda_{i+1}(M_n)-\lambda_i(M_n)}$ to have an average size of ${\frac{1}{\sqrt{n} \rho_{sc}(u)}}$. It is thus natural to introduce the normalised eigenvalue spacing

$\displaystyle X_i := \frac{\lambda_{i+1}(M_n) - \lambda_i(M_n)}{1/\sqrt{n} \rho_{sc}(u)}$

and ask what the distribution of ${X_i}$ is.

As mentioned previously, we will focus on the bulk case ${\delta n \leq i\leq (1-\delta)n}$, and begin with the model case when ${M_n}$ is drawn from GUE. (In the edge case when ${i}$ is close to ${1}$ or to ${n}$, the distribution is given by the famous Tracy-Widom law.) Here, the distribution was almost (but as we shall see, not quite) worked out by Gaudin and Mehta. By using the theory of determinantal processes, they were able to compute a quantity closely related to ${X_i}$, namely the probability

$\displaystyle {\bf P}( N_{[\sqrt{n} u + \frac{x}{\sqrt{n} \rho_{sc}(u)}, \sqrt{n} u + \frac{y}{\sqrt{n} \rho_{sc}(u)}]} = 0) \ \ \ \ \ (1)$

that an interval ${[\sqrt{n} u + \frac{x}{\sqrt{n} \rho_{sc}(u)}, \sqrt{n} u + \frac{y}{\sqrt{n} \rho_{sc}(u)}]}$ near ${\sqrt{n} u}$ of length comparable to the expected eigenvalue spacing ${1/\sqrt{n} \rho_{sc}(u)}$ is devoid of eigenvalues. For ${u}$ in the bulk and fixed ${x,y}$, they showed that this probability is equal to

$\displaystyle \det( 1 - 1_{[x,y]} P 1_{[x,y]} ) + o(1),$

where ${P}$ is the Dyson projection

$\displaystyle P f(x) = \int_{\bf R} \frac{\sin(\pi(x-y))}{\pi(x-y)} f(y)\ dy$

to Fourier modes in ${[-1/2,1/2]}$, and ${\det}$ is the Fredholm determinant. As shown by Jimbo, Miwa, Tetsuji, Mori, and Sato, this determinant can also be expressed in terms of a solution to a Painleve V ODE, though we will not need this fact here. In view of this asymptotic and some standard integration by parts manipulations, it becomes plausible to propose that ${X_i}$ will be asymptotically distributed according to the Gaudin-Mehta distribution ${p(x)\ dx}$, where

$\displaystyle p(x) := \frac{d^2}{dx^2} \det( 1 - 1_{[0,x]} P 1_{[0,x]} ).$

A reasonably accurate approximation for ${p}$ is given by the Wigner surmise ${p(x) \approx \frac{1}{2} \pi x e^{-\pi x^2/4}}$, which was presciently proposed by Wigner as early as 1957; it is exact for ${n=2}$ but not in the asymptotic limit ${n \rightarrow \infty}$.

Unfortunately, when one tries to make this argument rigorous, one finds that the asymptotic for (1) does not control a single gap ${X_i}$, but rather an ensemble of gaps ${X_i}$, where ${i}$ is drawn from an interval ${[i_0 - L, i_0 + L]}$ of some moderate size ${L}$ (e.g. ${L = \log n}$); see for instance this paper of Deift, Kriecherbauer, McLaughlin, Venakides, and Zhou for a more precise formalisation of this statement (which is phrased slightly differently, in which one samples all gaps inside a fixed window of spectrum, rather than inside a fixed range of eigenvalue indices ${i}$). (This result is stated for GUE, but can be extended to other Wigner ensembles by the Four Moment Theorem, at least if one assumes a moment matching condition; see this previous paper with Van Vu for details. The moment condition can in fact be removed, as was done in this subsequent paper with Erdos, Ramirez, Schlein, Vu, and Yau.)

The problem is that when one specifies a given window of spectrum such as ${[\sqrt{n} u + \frac{x}{\sqrt{n} \rho_{sc}(u)}, \sqrt{n} u + \frac{y}{\sqrt{n} \rho_{sc}(u)}]}$, one cannot quite pin down in advance which eigenvalues ${\lambda_i(M_n)}$ are going to lie to the left or right of this window; even with the strongest eigenvalue rigidity results available, there is a natural uncertainty of ${\sqrt{\log n}}$ or so in the ${i}$ index (as can be quantified quite precisely by this central limit theorem of Gustavsson).

The main difficulty here is that there could potentially be some strange coupling between the event (1) of an interval being devoid of eigenvalues, and the number ${N_{(-\infty,\sqrt{n} u + \frac{x}{\sqrt{n} \rho_{sc}(u)})}(M_n)}$ of eigenvalues to the left of that interval. For instance, one could conceive of a possible scenario in which the interval in (1) tends to have many eigenvalues when ${N_{(-\infty,\sqrt{n} u + \frac{x}{\sqrt{n} \rho_{sc}(u)})}(M_n)}$ is even, but very few when ${N_{(-\infty,\sqrt{n} u + \frac{x}{\sqrt{n} \rho_{sc}(u)})}(M_n)}$ is odd. In this sort of situation, the gaps ${X_i}$ may have different behaviour for even ${i}$ than for odd ${i}$, and such anomalies would not be picked up in the averaged statistics in which ${i}$ is allowed to range over some moderately large interval.

The main result of the current paper is that these anomalies do not actually occur, and that all of the eigenvalue gaps ${X_i}$ in the bulk are asymptotically governed by the Gaudin-Mehta law without the need for averaging in the ${i}$ parameter. Again, this is shown first for GUE, and then extended to other Wigner matrices obeying a matching moment condition using the Four Moment Theorem. (It is likely that the moment matching condition can be removed here, but I was unable to achieve this, despite all the recent advances in establishing universality of local spectral statistics for Wigner matrices, mainly because the universality results in the literature are more focused on specific energy levels ${u}$ than on specific eigenvalue indices ${i}$. To make matters worse, in some cases universality is currently known only after an additional averaging in the energy parameter.)

The main task in the proof is to show that the random variable ${N_{(-\infty,\sqrt{n} u + \frac{x}{\sqrt{n} \rho_{sc}(u)})}(M_n)}$ is largely decoupled from the event in (1) when ${M_n}$ is drawn from GUE. To do this we use some of the theory of determinantal processes, and in particular the nice fact that when one conditions a determinantal process to the event that a certain spatial region (such as an interval) contains no points of the process, then one obtains a new determinantal process (with a kernel that is closely related to the original kernel). The main task is then to obtain a sufficiently good control on the distance between the new determinantal kernel and the old one, which we do by some functional-analytic considerations involving the manipulation of norms of operators (and specifically, the operator norm, Hilbert-Schmidt norm, and nuclear norm). Amusingly, the Fredholm alternative makes a key appearance, as I end up having to invert a compact perturbation of the identity at one point (specifically, I need to invert ${1 - 1_{[x,y]}P1_{[x,y]}}$, where ${P}$ is the Dyson projection and ${[x,y]}$ is an interval). As such, the bounds in my paper become ineffective, though I am sure that with more work one can invert this particular perturbation of the identity by hand, without the need to invoke the Fredholm alternative.

In this final set of course notes, we discuss how (a generalisation of) the expansion results obtained in the preceding notes can be used for some number-theoretic applications, and in particular to locate almost primes inside orbits of thin groups, following the work of Bourgain, Gamburd, and Sarnak. We will not attempt here to obtain the sharpest or most general results in this direction, but instead focus on the simplest instances of these results which are still illustrative of the ideas involved.

One of the basic general problems in analytic number theory is to locate tuples of primes of a certain form; for instance, the famous (and still unsolved) twin prime conjecture asserts that there are infinitely many pairs ${(n_1,n_2)}$ in the line ${\{ (n_1,n_2) \in {\bf Z}^2: n_2-n_1=2\}}$ in which both entries are prime. In a similar spirit, one of the Landau conjectures (also still unsolved) asserts that there are infinitely many primes in the set ${\{ n^2+1: n \in {\bf Z} \}}$. The Mersenne prime conjecture (also unsolved) asserts that there are infinitely many primes in the set ${\{ 2^n - 1: n \in {\bf Z} \}}$, and so forth.

More generally, given some explicit subset ${V}$ in ${{\bf R}^d}$ (or ${{\bf C}^d}$, if one wishes), such as an algebraic variety, one can ask the question of whether there are infinitely many integer lattice points ${(n_1,\ldots,n_d)}$ in ${V \cap {\bf Z}^d}$ in which all the coefficients ${n_1,\ldots,n_d}$ are simultaneously prime; let us refer to such points as prime points.

At this level of generality, this problem is impossibly difficult. Indeed, even the much simpler problem of deciding whether the set ${V \cap {\bf Z}^d}$ is non-empty (let alone containing prime points) when ${V}$ is a hypersurface ${\{ x \in {\bf R}^d: P(x) = 0 \}}$ cut out by a polynomial ${P}$ is essentially Hilbert’s tenth problem, which is known to be undecidable in general by Matiyasevich’s theorem. So one needs to restrict attention to a more special class of sets ${V}$, in which the question of finding integer points is not so difficult. One model case is to consider orbits ${V = \Gamma b}$, where ${b \in {\bf Z}^d}$ is a fixed lattice vector and ${\Gamma}$ is some discrete group that acts on ${{\bf Z}^d}$ somehow (e.g. ${\Gamma}$ might be embedded as a subgroup of the special linear group ${SL_d({\bf Z})}$, or on the affine group ${SL_d({\bf Z}) \ltimes {\bf Z}^d}$). In such a situation it is then quite easy to show that ${V = V \cap {\bf Z}^d}$ is large; for instance, ${V}$ will be infinite precisely when the stabiliser of ${b}$ in ${\Gamma}$ has infinite index in ${\Gamma}$.

Even in this simpler setting, the question of determining whether an orbit ${V = \Gamma b}$ contains infinitely prime points is still extremely difficult; indeed the three examples given above of the twin prime conjecture, Landau conjecture, and Mersenne prime conjecture are essentially of this form (possibly after some slight modification of the underlying ring ${{\bf Z}}$, see this paper of Bourgain-Gamburd-Sarnak for details), and are all unsolved (and generally considered well out of reach of current technology). Indeed, the list of non-trivial orbits ${V = \Gamma b}$ which are known to contain infinitely many prime points is quite slim; Euclid’s theorem on the infinitude of primes handles the case ${V = {\bf Z}}$, Dirichlet’s theorem handles infinite arithmetic progressions ${V = a{\bf Z} + r}$, and a somewhat complicated result of Green, Tao, and Ziegler handles “non-degenerate” affine lattices in ${{\bf Z}^d}$ of rank two or more (such as the lattice of length ${d}$ arithmetic progressions), but there are few other positive results known that are not based on the above cases (though we will note the remarkable theorem of Friedlander and Iwaniec that there are infinitely many primes of the form ${a^2+b^4}$, and the related result of Heath-Brown that there are infinitely many primes of the form ${a^3+2b^3}$, as being in a kindred spirit to the above results, though they are not explicitly associated to an orbit of a reasonable action as far as I know).

On the other hand, much more is known if one is willing to replace the primes by the larger set of almost primes – integers with a small number of prime factors (counting multiplicity). Specifically, for any ${r \geq 1}$, let us call an ${r}$-almost prime an integer which is the product of at most ${r}$ primes, and possibly by the unit ${-1}$ as well. Many of the above sorts of questions which are open for primes, are known for ${r}$-almost primes for ${r}$ sufficiently large. For instance, with regards to the twin prime conjecture, it is a result of Chen that there are infinitely many pairs ${p,p+2}$ where ${p}$ is a prime and ${p+2}$ is a ${2}$-almost prime; in a similar vein, it is a result of Iwaniec that there are infinitely many ${2}$-almost primes of the form ${n^2+1}$. On the other hand, it is still open for any fixed ${r}$ whether there are infinitely many Mersenne numbers ${2^n-1}$ which are ${r}$-almost primes. (For the superficially similar situation with the numbers ${2^n+1}$, it is in fact believed (but again unproven) that there are only finitely many ${r}$-almost primes for any fixed ${r}$ (the Fermat prime conjecture).)

The main tool that allows one to count almost primes in orbits is sieve theory. The reason for this lies in the simple observation that in order to ensure that an integer ${n}$ of magnitude at most ${x}$ is an ${r}$-almost prime, it suffices to guarantee that ${n}$ is not divisible by any prime less than ${x^{1/(r+1)}}$. Thus, to create ${r}$-almost primes, one can start with the integers up to some large threshold ${x}$ and remove (or “sieve out”) all the integers that are multiples of any prime ${p}$ less than ${x^{1/(r+1)}}$. The difficulty is then to ensure that a sufficiently non-trivial quantity of integers remain after this process, for the purposes of finding points in the given set ${V}$.

The most basic sieve of this form is the sieve of Eratosthenes, which when combined with the inclusion-exclusion principle gives the Legendre sieve (or exact sieve), which gives an exact formula for quantities such as the number ${\pi(x,z)}$ of natural numbers less than or equal to ${x}$ that are not divisible by any prime less than or equal to a given threshold ${z}$. Unfortunately, when one tries to evaluate this formula, one encounters error terms which grow exponentially in ${z}$, rendering this sieve useful only for very small thresholds ${z}$ (of logarithmic size in ${x}$). To improve the sieve level up to a small power of ${x}$ such as ${x^{1/(r+1)}}$, one has to replace the exact sieve by upper bound sieves and lower bound sieves which only seek to obtain upper or lower bounds on quantities such as ${\pi(x,z)}$, but contain a polynomial number of terms rather than an exponential number. There are a variety of such sieves, with the two most common such sieves being combinatorial sieves (such as the beta sieve), based on various combinatorial truncations of the inclusion-exclusion formula, and the Selberg upper bound sieve, based on upper bounds that are the square of a divisor sum. (There is also the large sieve, which is somewhat different in nature and based on ${L^2}$ almost orthogonality considerations, rather than on any actual sieving, to obtain upper bounds.) We will primarily work with a specific sieve in this notes, namely the beta sieve, and we will not attempt to optimise all the parameters of this sieve (which ultimately means that the almost primality parameter ${r}$ in our results will be somewhat large). For a more detailed study of sieve theory, see the classic text of Halberstam and Richert, or the more recent texts of Iwaniec-Kowalski or of Friedlander-Iwaniec.

Very roughly speaking, the end result of sieve theory is that excepting some degenerate and “exponentially thin” settings (such as those associated with the Mersenne primes), all the orbits which are expected to have a large number of primes, can be proven to at least have a large number of ${r}$-almost primes for some finite ${r}$. (Unfortunately, there is a major obstruction, known as the parity problem, which prevents sieve theory from lowering ${r}$ all the way to ${1}$; see this blog post for more discussion.) One formulation of this principle was established by Bourgain, Gamburd, and Sarnak:

Theorem 1 (Bourgain-Gamburd-Sarnak) Let ${\Gamma}$ be a subgroup of ${SL_2({\bf Z})}$ which is not virtually solvable. Let ${f: {\bf Z}^4 \rightarrow {\bf Z}}$ be a polynomial with integer coefficients obeying the following primitivity condition: for any positive integer ${q}$, there exists ${A \in \Gamma \subset {\bf Z}^4}$ such that ${f(A)}$ is coprime to ${q}$. Then there exists an ${r \geq 1}$ such that there are infinitely many ${A \in \Gamma}$ with ${f(A)}$ non-zero and ${r}$-almost prime.

This is not the strongest version of the Bourgain-Gamburd-Sarnak theorem, but it captures the general flavour of their results. Note that the theorem immediately implies an analogous result for orbits ${\Gamma b \subset {\bf Z}^2}$, in which ${f}$ is now a polynomial from ${{\bf Z}^2}$ to ${{\bf Z}}$, and one uses ${f(Ab)}$ instead of ${f(A)}$. It is in fact conjectured that one can set ${r=1}$ here, but this is well beyond current technology. For the purpose of reaching ${r=1}$, it is very natural to impose the primitivity condition, but as long as one is content with larger values of ${r}$, it is possible to relax the primitivity condition somewhat; see the paper of Bourgain, Gamburd, and Sarnak for more discussion.

By specialising to the polynomial ${f: \begin{pmatrix} a & b \\ c & d \end{pmatrix} \rightarrow abcd}$, we conclude as a corollary that as long as ${\Gamma}$ is primitive in the sense that it contains matrices with all coefficients coprime to ${q}$ for any given ${q}$, then ${\Gamma}$ contains infinitely many matrices whose elements are all ${r}$-almost primes for some ${r}$ depending only on ${\Gamma}$. For further applications of these sorts of results, for instance to Appolonian packings, see the paper of Bourgain, Gamburd, and Sarnak.

It turns out that to prove Theorem 1, the Cayley expansion results in ${SL_2(F_p)}$ from the previous set of notes are not quite enough; one needs a more general Cayley expansion result in ${SL_2({\bf Z}/q{\bf Z})}$ where ${q}$ is square-free but not necessarily prime. The proof of this expansion result uses the same basic methods as in the ${SL_2(F_p)}$ case, but is significantly more complicated technically, and we will only discuss it briefly here. As such, we do not give a complete proof of Theorem 1, but hopefully the portion of the argument presented here is still sufficient to give an impression of the ideas involved.

 Alberto Ibañez on The Collatz conjecture, Little… bizarre. on Twisted convolution and the se… Anonymous on The Collatz conjecture, Little… Anonymous on The Collatz conjecture, Little… Anonymous on The Collatz conjecture, Little… Anonymous on The Collatz conjecture, Little… Alberto Ibañez on The Collatz conjecture, Little… Anonymous on The Collatz conjecture, Little… One trick pony i mad… on There’s more to mathematics th… Κωνσταντίνος Κουρουζ… on The Collatz conjecture, Little… Zachary Kyle Knutson on The Collatz conjecture, Little… Dan Asimov on The Collatz conjecture, Little… Zachary Kyle Knutson on Twisted convolution and the se… Zachary Kyle Knutson on Twisted convolution and the se… Zachary Kyle Knutson on Twisted convolution and the se…