for the divisor function , in particular recovering the calculation of Ingham that obtained the asymptotic

when was fixed and non-zero and went to infinity. It is natural to consider the more general correlations

for fixed and non-zero , where

is the order divisor function. The sum (1) then corresponds to the case . For , , and a routine application of the Dirichlet hyperbola method (or Perron’s formula) gives the asymptotic

or more accurately

where is a certain explicit polynomial of degree with leading coefficient ; see e.g. Exercise 31 of this previous post for a discussion of the case (which is already typical). Similarly if . For more general , there is a conjecture of Conrey and Gonek which predicts that

for some polynomial of degree which is explicit but whose form is rather complicated (one has to compute residues of a various complicated products of zeta functions and local factors). This conjecture has been verified when or , by the work of Linnik, Motohashi, Fouvry-Tenenbaum, and others, but all the remaining cases when are currently open.

In principle, the calculations of the previous post should recover the predictions of Conrey and Gonek. In this post I would like to record this for the top order term:

Conjecture 1If and are fixed, thenas , where the product is over all primes , and the local factors are given by the formula

where is the degree polynomial

where

and one adopts the conventions that and for .

For instance, if then

and hence

and the above conjecture recovers the Ingham formula (2). For , we have

and so we predict

where

Similarly, if we have

and so we predict

where

and so forth.

As in the previous blog, the idea is to factorise

where the local factors are given by

(where means that divides precisely times), or in terms of the valuation of at ,

We then have the following exact local asymptotics:

Proposition 2 (Local correlations)Let be a profinite integer chosen uniformly at random, let be a profinite integer, and let . Then

(For profinite integers it is possible that and hence are infinite, but this is a probability zero event and so can be ignored.)

Conjecture 1 can then be heuristically justified from the local calculations (2) by various pseudorandomness heuristics, as discussed in the previous post.

I’ll give a short proof of the above proposition below, basically using the recursive methods of the previous post. This short proof actually took be quite a while to find; I spent several hours and a fair bit of scratch paper working out the cases laboriously by hand (with some assistance and cross-checking from Maple). Here is an unorganised sample of some of this scratch, just to show how the sausage is actually made:

It was only after expending all this effort that I realised that it would be much more efficient to compute the correlations for all values of simultaneously by using generating functions. After performing this computation, it then became apparent that there would be a direct combinatorial proof of (6) that was shorter than even the generating function proof. (I will not supply the full generating function calculations here, but will at least show them for the easier correlation (5).)

I am confident that Conjecture 1 is consistent with the explicit asymptotic in the Conrey-Gonek conjecture, but have not yet rigorously established that the leading order term in the latter is indeed identical to the expression provided above.

We now prove (5). Here we will use the generating function method. From the binomial formula we have the formal expansion

for any . By the geometric series formula, it thus suffices to show that

in the ring of formal power series in .

With probability , is coprime to and thus . In the remaining probability event, for some with the same distribution as the original random variable , and . This leads to the identity

and the claim then follows from a brief amount of high-school algebra.

Now we prove (6). By (3), our task is to show that

The above generating function method will establish this (and, as I said above, this was how I originally discovered the formula), but we give here a combinatorial proof. We first deal with the case when is not divisible by . In this case, at least one of and must equal ; we can phrase this fact algebraically as the identity

By linearity (and translation invariance) of expectation and (5) we thus have

which gives (7) in this case.

Now suppose that for some (profinite) integer . With probability , is coprime to , so that ; in particular (8) holds here. In the remaining probability event, we can write . From (4) and Pascal triangle identities, we have

and similarly

and hence on subtracting from both equations, multiplying, and rearranging we have a variant of (8):

Putting this together, we see that

and hence by (5) as before

The claim is now easily verified from an induction on , as well as many applications of Pascal triangle identities.

Filed under: expository, math.NT, math.PR, Uncategorized Tagged: correlation, divisor function, generating function ]]>

where denotes the estimate as . Much better error estimates are possible here, but we will not focus on the lower order terms in this discussion. For somewhat idiosyncratic reasons I will interpret this estimate (and the other analytic number theory estimates discussed here) through the probabilistic lens. Namely, if is a random number selected uniformly between and , then the above estimate can be written as

that is to say the random variable has mean approximately . (But, somewhat paradoxically, this is not the median or mode behaviour of this random variable, which instead concentrates near , basically thanks to the Hardy-Ramanujan theorem.)

Now we turn to the pair correlations for a fixed positive integer . There is a classical computation of Ingham that shows that

The error term in (2) has been refined by many subsequent authors, as has the uniformity of the estimates in the aspect, as these topics are related to other questions in analytic number theory, such as fourth moment estimates for the Riemann zeta function; but we will not consider these more subtle features of the estimate here. However, we will look at the next term in the asymptotic expansion for (2) below the fold.

Using our probabilistic lens, the estimate (2) can be written as

From (1) (and the asymptotic negligibility of the shift by ) we see that the random variables and both have a mean of , so the additional factor of represents some arithmetic coupling between the two random variables.

Ingham’s formula can be established in a number of ways. Firstly, one can expand out and use the hyperbola method (splitting into the cases and and removing the overlap). If one does so, one soon arrives at the task of having to estimate sums of the form

for various . For much less than this can be achieved using a further application of the hyperbola method, but for comparable to things get a bit more complicated, necessitating the use of non-trivial estimates on Kloosterman sums in order to obtain satisfactory control on error terms. A more modern approach proceeds using automorphic form methods, as discussed in this previous post. A third approach, which unfortunately is only heuristic at the current level of technology, is to apply the Hardy-Littlewood circle method (discussed in this previous post) to express (2) in terms of exponential sums for various frequencies . The contribution of “major arc” can be computed after a moderately lengthy calculation which yields the right-hand side of (2) (as well as the correct lower order terms that are currently being suppressed), but there does not appear to be an easy way to show directly that the “minor arc” contributions are of lower order, although the methods discussed previously do indirectly show that this is ultimately the case.

Each of the methods outlined above requires a fair amount of calculation, and it is not obvious while performing them that the factor will emerge at the end. One can at least explain the as a normalisation constant needed to balance the factor (at a heuristic level, at least). To see this through our probabilistic lens, introduce an independent copy is an independent copy of , then

using symmetry to order (discarding the diagonal case ) and making the change of variables , we see that (4) is heuristically consistent with (3) as long as the asymptotic mean of in is equal to . (This argument is not rigorous because there was an implicit interchange of limits present, but still gives a good heuristic “sanity check” of Ingham’s formula.) Indeed, if denotes the asymptotic mean in , then we have (heuristically at least)

and we obtain the desired consistency after multiplying by .

This still however does not explain the presence of the factor. Intuitively it is reasonable that if has many prime factors, and has a lot of factors, then will have slightly more factors than average, because any common factor to and will automatically be acquired by . But how to quantify this effect?

One heuristic way to proceed is through analysis of local factors. Observe from the fundamental theorem of arithmetic that we can factor

where the product is over all primes , and is the local version of at (which in this case, is just one plus the –valuation of : ). Note that all but finitely many of the terms in this product will equal , so the infinite product is well-defined. In a similar fashion, we can factor

where

(or in terms of valuations, ). Heuristically, the Chinese remainder theorem suggests that the various factors behave like independent random variables, and so the correlation between and should approximately decouple into the product of correlations between the local factors and . And indeed we do have the following local version of Ingham’s asymptotics:

Proposition 1 (Local Ingham asymptotics)For fixed and integer , we haveand

From the Euler formula

we see that

and so one can “explain” the arithmetic factor in Ingham’s asymptotic as the product of the arithmetic factors in the (much easier) local Ingham asymptotics. Unfortunately we have the usual “local-global” problem in that we do not know how to rigorously derive the global asymptotic from the local ones; this problem is essentially the same issue as the problem of controlling the minor arc contributions in the circle method, but phrased in “physical space” language rather than “frequency space”.

Remark 2The relation between the local means and the global mean can also be seen heuristically through the applicationof Mertens’ theorem, where is Pólya’s magic exponent, which serves as a useful heuristic limiting threshold in situations where the product of local factors is divergent.

Let us now prove this proposition. One could brute-force the computations by observing that for any fixed , the valuation is equal to with probability , and with a little more effort one can also compute the joint distribution of and , at which point the proposition reduces to the calculation of various variants of the geometric series. I however find it cleaner to proceed in a more recursive fashion (similar to how one can prove the geometric series formula by induction); this will also make visible the vague intuition mentioned previously about how common factors of and force to have a factor also.

It is first convenient to get rid of error terms by observing that in the limit , the random variable converges vaguely to a uniform random variable on the profinite integers , or more precisely that the pair converges vaguely to . Because of this (and because of the easily verified uniform integrability properties of and their powers), it suffices to establish the exact formulae

in the profinite setting (this setting will make it easier to set up the recursion).

We begin with (5). Observe that is coprime to with probability , in which case is equal to . Conditioning to the complementary probability event that is divisible by , we can factor where is also uniformly distributed over the profinite integers, in which event we have . We arrive at the identity

As and have the same distribution, the quantities and are equal, and (5) follows by a brief amount of high-school algebra.

We use a similar method to treat (6). First treat the case when is coprime to . Then we see that with probability , and are simultaneously coprime to , in which case . Furthermore, with probability , is divisible by and is not; in which case we can write as before, with and . Finally, in the remaining event with probability , is divisible by and is not; we can then write , so that and . Putting all this together, we obtain

and the claim (6) in this case follows from (5) and a brief computation (noting that in this case).

Now suppose that is divisible by , thus for some integer . Then with probability , and are simultaneously coprime to , in which case . In the remaining event, we can write , and then and . Putting all this together we have

which by (5) (and replacing by ) leads to the recursive relation

and (6) then follows by induction on the number of powers of .

The estimate (2) of Ingham was refined by Estermann, who obtained the more accurate expansion

for certain complicated but explicit coefficients . For instance, is given by the formula

where is the Euler-Mascheroni constant,

The formula for is similar but even more complicated. The error term was improved by Heath-Brown to ; it is conjectured (for instance by Conrey and Gonek) that one in fact has square root cancellation here, but this is well out of reach of current methods.

These lower order terms are traditionally computed either from a Dirichlet series approach (using Perron’s formula) or a circle method approach. It turns out that a refinement of the above heuristics can also predict these lower order terms, thus keeping the calculation purely in physical space as opposed to the “multiplicative frequency space” of the Dirichlet series approach, or the “additive frequency space” of the circle method, although the computations are arguably as messy as the latter computations for the purposes of working out the lower order terms. We illustrate this just for the term below the fold.

** — 1. More refined heuristics — **

To begin with we revisit the heuristic justification of (2) and modify it a bit, in the spirit of the “modified Cramér models” from this previous blog post. Our arguments here will be even less rigorous than in the preceding section, for instance we shall be quite cavalier with the use of the symbol without precisely quantifying error terms.

For this section, it will be convenient to set not to be the uniform distribution from to as before, but rather the uniform distribution from to for some fairly small (e.g. will suffice). The reason for this is that we need to start keeping track of the magnitude of to a relative accuracy of at least , and with this new choice of the magnitude is simply (up to this accuracy). By “differentiating” asymptotics such as (7) in we see that

where we ignore terms of lower order than the term and

and so our task is now to heuristically justify the statement

Let us begin by revisiting the derivation of the simpler asymptotic

without passing immediately to local factors, but instead using a modified Cramér type model that will be more amenable to keeping careful track of the magnitude . For any square-free , write

and

so that we have the factorisation . From the Chinese remainder theorem we expect and to behave approximately independently, at least when is fixed and is large.

We can statistics of to deduce statistics of . This is traditionally done using the device of Möbius inversion, but we will use a recursive probabilistic approach here similar to that used to prove Proposition 1. Let’s start with understanding for some prime . With probability about , is coprime to , so that . In the remaining event of probability about , we may write where is uniformly distributed near , and then . This leads to the approximate identity

To work with this equation we make two further heuristic assumptions, which turn out to be reasonable at the level of accuracy required for justifying (10), but not for (9). Firstly, we assume that the distribution of is largely unchanged after conditioning to the event , which is plausible from the Chinese remainder theorem. Secondly, we assume that the distribution of is largely the same as that of , and similarly for . With these assumptions, (11) simplifies to

(note that this is consistent with Proposition 1 and the presumed independence of and ). Iterating this argument gives

for any fixed squarefree . Applying (1) we conclude

Now let be a moderately large number (growing very slowly with , e.g. ), and let be the primorial of . We factor

There is arithmetic coupling between the and factors; indeed from Proposition 1 and the Chinese remainder theorem we have

Meanwhile, from (13) we have

and

In analogy with the modified Cramér model for the primes, we now assume that the random variables , , and behave like independent random variables; this turns out to be sufficiently accurate for predicting (10), but not (9). This leads to the prediction

and if goes to infinity with then we recover (10).

We now repeat the above analysis but expanding to the next order in , which will require keeping better track of the parameter. The Dirichlet hyperbola method actually gives the refinement

and hence

We previously computed the expectation . Now that we want to be more accurate, it turns out that it is better to compute the slightly different *conditional expectation* . Now we compute the mean of a bit more carefully. Again we begin with for prime. From (11) and (16), and by replacing with , we have

Previously we simplified this equation by assuming that the conditioning had negligible impact on , and that had similar distribution to . As it turns out, we can no longer afford to assume these simplifying assumptions at this level of accuracy, and must argue more carefully. With probability about is coprime to , and in the remaining event of probability about , we have and . This gives the approximate identity

If we refine the approximation (12) to give the ansatz

and

for some constants to be worked out momentarily, the above two equations give after some computation (and canceling out all the terms

which can be solved as

Thus we have

It is instructive (albeit lengthy) to deduce this asymptotic rigorously for fixed square-free and large by using Möbius inversion and the hyperbola method; one can also proceed a bit more quickly by using Perron’s formula and inspecting the pole of the relevant Dirichlet series at . (Not surprisingly, similar calculations appear when computing major arc contributions to (2).)

Now we are ready to predict the value of . Our starting point is again the expansion (14), except that we now strip out the factors within that divide , in order to use (19). More precisely, we adopt the factorisation of arbitrary natural numbers by the formulae

and observe that

and similarly

We are now faced with computing the expression

Previously we had assumed that the factors , , and behaved independently. However this is not quite true at the current level of accuracy, because the magnitude of the natural numbers and are influenced by the size of and respectively, and these in turn are certainly correlated with . Because of the non-trivial dependence on on the right-hand side of (19) at this level of accuracy, the size of and influences the expected value of and , causing a coupling between the three factors of (20). However, if we first condition the magnitude of and to be fixed, then we expect the three factors to be *conditionally* independent (at this level of accuracy). Heuristically, with this conditioning, behaves like a random element of size about that is coprime to , and hence by (19)

and similarly

We can thus heuristically estimate (20) by

This can be expanded as

where

and is something complicated that we do not compute here. Using (15) we have

which recovers the Ingham estimate (10). To obtain the finer approximation (9) we need to show that

It will suffice to obtain the asymptotic

together with the analogue of (21) with replaced by . (One in fact has the approximation , which reflects the Laurent approximation near , but we will not need this.)

We will just prove (21), as the analogue of (21) for is proven similarly. As this is a local calculation we may replace by . Splitting

we can write the left-hand side of (21) as a sum over primes,

The and terms decouple, so by Proposition 1 this becomes

Extracting a common factor of , we reduce to showing that

The left-hand side is a sum over primes , so we work on making the right-hand side a sum over primes as well. This is achieved by the following two lemmas:

Lemma 3We have .

*Proof:* While this identity can be proven elementarily, it is fastest to proceed using Dirichlet series. From Taylor expansion and (8) we have

for close to . Inverting, we have

and hence

The left-hand side is , giving the claim.

Lemma 4We have .

*Proof:* Again it is fastest to proceed using Dirichlet series. Setting and , we have from the fundamental theorem of arithmetic that

Differentiating this at we obtain the claim.

We remark that is an example of a derived multiplicative function, discussed in this previous blog post.

In view of these expansions, we can reduce matters to establishing a purely local estimate at a single prime , namely that

(compare with Proposition 1).

We need two preliminary estimates to handle lower order terms:

*Proof:* Once again we use recursion. With probability , is coprime to , so vanishes. Otherwise, , , and . We conclude that

From Proposition 1 we have

and from the identity we thus have

and the claim follows.

Note that one can view Lemma 5 as the (p-adic) limiting case of Lemma 6.

*Proof:* First suppose that is coprime to . Then is equal to whenever is non-zero, so the claim follows from (23) in this case. Now suppose that for some integer .

When is coprime to the left-hand side is equal to when is coprime to and otherwise, giving the claim. Now suppose that for some integer . With probability , is coprime to , so vanishes; otherwise,, , and . We conclude that

Arguing as before we conclude that

and the claim follows by induction on the number of times divides (noting that ).

Now we can prove (22). First suppose is coprime to . In this case, the random variable is only non-zero when is divisible by , which forces to equal . By Lemma 5, the left-hand side of (22) is thus equal to , thus proving (22) in this case.

Now suppose that is divisible by . With probability , is coprime to , so vanishes. Otherwise, we write and observe that

thus the left-hand side of (22) expands as

This expands into eight terms that can be computed using Proposition 1, (23), and Lemmas 5, (6) as

The claim then follows by induction on the number of times divides , together with a tedious computation using the identities

and

Filed under: expository, math.NT, math.PR Tagged: correlation, divisor function, pseudorandomness ]]>

I will assume familiarity with the notation of my paper. In Section 10, some complicated spaces are constructed for each frequency scale , and then a further space is constructed for a given frequency envelope by the formula

where is the Littlewood-Paley projection of to frequency magnitudes . Then, given a spacetime slab , we define the restrictions

where the infimum is taken over all extensions of to the Minkowski spacetime ; similarly one defines

The gap in the paper is as follows: it was implicitly assumed that one could restrict (1) to the slab to obtain the equality

(This equality is implicitly used to establish the bound (36) in the paper.) Unfortunately, (1) only gives the lower bound, not the upper bound, and it is the upper bound which is needed here. The problem is that the extensions of that are optimal for computing are not necessarily the Littlewood-Paley projections of the extensions of that are optimal for computing .

To remedy the problem, one has to prove an upper bound of the form

for all Schwartz (actually we need affinely Schwartz , but one can easily normalise to the Schwartz case). Without loss of generality we may normalise the RHS to be . Thus

for each , and one has to find a single extension of such that

for each . Achieving a that obeys (4) is trivial (just extend by zero), but such extensions do not necessarily obey (5). On the other hand, from (3) we can find extensions of such that

the extension will then obey (5) (here we use Lemma 9 from my paper), but unfortunately is not guaranteed to obey (4) (the norm does control the norm, but a key point about frequency envelopes for the small energy regularity problem is that the coefficients , while bounded, are not necessarily summable).

This can be fixed as follows. For each we introduce a time cutoff supported on that equals on and obeys the usual derivative estimates in between (the time derivative of size for each ). Later we will prove the truncation estimate

Assuming this estimate, then if we set , then using Lemma 9 in my paper and (6), (7) (and the local stability of frequency envelopes) we have the required property (5). (There is a technical issue arising from the fact that is not necessarily Schwartz due to slow decay at temporal infinity, but by considering partial sums in the summation and taking limits we can check that is the strong limit of Schwartz functions, which suffices here; we omit the details for sake of exposition.) So the only issue is to establish (4), that is to say that

for all .

For this is immediate from (2). Now suppose that for some integer (the case when is treated similarly). Then we can split

where

The contribution of the term is acceptable by (6) and estimate (82) from my paper. The term sums to which is acceptable by (2). So it remains to control the norm of . By the triangle inequality and the fundamental theorem of calculus, we can bound

By hypothesis, . Using the first term in (79) of my paper and Bernstein’s inequality followed by (6) we have

and then we are done by summing the geometric series in .

It remains to prove the truncation estimate (7). This estimate is similar in spirit to the algebra estimates already in my paper, but unfortunately does not seem to follow immediately from these estimates as written, and so one has to repeat the somewhat lengthy decompositions and case checkings used to prove these estimates. We do this below the fold.

** — 1. Proof of truncation estimate — **

Firstly, by rescaling (and changing as necessary) we may assume that . By the triangle inequality and time translation invariance, it suffices to show an estimate of the form

where is a smooth time cutoff that equals on and is supported in , and all norms are understood to be on . We may normalise the right-hand side to be , thus is supported in frequencies , and by equation (79) of my paper one has the estimates

for all , and our objective is to show that

The bound (11) easily follows from (8), the Leibniz rule, and using the frequency localisation of to ignore spatial derivatives. Now we turn to (12). From the definition of the norms, we have

for all integers , and we need to show that

Fix . We can use Littlewood-Paley operators to split , where is supported on time frequencies and is supported on time frequencies . For the contribution of one can replace in (15) by (say) and the claim then follows from (14), the Leibniz rule, and Hölder’s inequality (again ignoring spatial derivatives). For the contribution of , we discard and observe that has an norm of (and its time derivative has a norm of ), so this contribution is then acceptable from (8) and Hölder’s inequality.

Finally we need to show (13). Similarly to before, we split . We also split , leaving us with the task of proving the four estimates

We begin with (16). The multiplier is disposable in the sense of the paper, and similarly if one replaces by a slightly larger multiplier; this lets us bound the left-hand side of (16) by

The time cutoff commutes with the spatial Fourier projection and can then be discarded by equation (66) of my paper. This term is thus acceptable thanks to (10).

Now we turn to (17). We can freely insert a factor of in front of . Applying estimate (75) from my paper, it then suffices to show that

From the Fourier support of the expression inside the norm, the left-hand side is bounded by

discarding the cutoff and using (9) we see that this contribution is acceptable.

Next, we show (18). Here we use the energy estimate from equation (27) (and (25)) of the paper. By repeating the proof of (11) (and using Lemma 4 from my paper) we see that

so it suffices to show that

Expanding out the d’Lambertian using the Leibniz rule, we are reduced to showing the estimates

For (20) we note that has an norm of , while from (9) has an norm of , so the claim follows from Hölder’s inequality. For (21) we can similarly observe that has an norm of while from (8) we see that has an norm of , so the claim again follows from Hölder’s inequality. A similar argument gives (22) (with an additional gain of coming from the second derivative on ).

Finally, for (19), we observe from the Fourier separation between and that we may replace by (in fact one could do a much more drastic replacement if desired). The claim now follows from repeating the proof of (18).

Filed under: math.AP, update Tagged: Hao Jia, wave maps ]]>

In the previous blog post, one of us (Terry) implicitly introduced a notion of rank for tensors which is a little different from the usual notion of tensor rank, and which (following BCCGNSU) we will call “slice rank”. This notion of rank could then be used to encode the Croot-Lev-Pach-Ellenberg-Gijswijt argument that uses the polynomial method to control capsets.

Afterwards, several papers have applied the slice rank method to further problems – to control tri-colored sum-free sets in abelian groups (BCCGNSU, KSS) and from there to the triangle removal lemma in vector spaces over finite fields (FL), to control sunflowers (NS), and to bound progression-free sets in -groups (P).

In this post we investigate the notion of slice rank more systematically. In particular, we show how to give lower bounds for the slice rank. In many cases, we can show that the upper bounds on slice rank given in the aforementioned papers are sharp to within a subexponential factor. This still leaves open the possibility of getting a better bound for the original combinatorial problem using the slice rank of some other tensor, but for very long arithmetic progressions (at least eight terms), we show that the slice rank method cannot improve over the trivial bound using any tensor.

It will be convenient to work in a “basis independent” formalism, namely working in the category of abstract finite-dimensional vector spaces over a fixed field . (In the applications to the capset problem one takes to be the finite field of three elements, but most of the discussion here applies to arbitrary fields.) Given such vector spaces , we can form the tensor product , generated by the tensor products with for , subject to the constraint that the tensor product operation is multilinear. For each , we have the smaller tensor products , as well as the tensor product

defined in the obvious fashion. Elements of of the form for some and will be called *rank one functions*, and the *slice rank* (or *rank* for short) of an element of is defined to be the least nonnegative integer such that is a linear combination of rank one functions. If are finite-dimensional, then the rank is always well defined as a non-negative integer (in fact it cannot exceed . It is also clearly subadditive:

For , is when is zero, and otherwise. For , is the usual rank of the -tensor (which can for instance be identified with a linear map from to the dual space ). The usual notion of tensor rank for higher order tensors uses complete tensor products , as the rank one objects, rather than , giving a rank that is greater than or equal to the slice rank studied here.

From basic linear algebra we have the following equivalences:

Lemma 1Let be finite-dimensional vector spaces over a field , let be an element of , and let be a non-negative integer. Then the following are equivalent:

- (i) One has .
- (ii) One has a representation of the form
where are finite sets of total cardinality at most , and for each and , and .

- (iii) One has
where for each , is a subspace of of total dimension at most , and we view as a subspace of in the obvious fashion.

- (iv) (Dual formulation) There exist subspaces of the dual space for , of total dimension at least , such that is orthogonal to , in the sense that one has the vanishing
for all , where is the obvious pairing.

*Proof:* The equivalence of (i) and (ii) is clear from definition. To get from (ii) to (iii) one simply takes to be the span of the , and conversely to get from (iii) to (ii) one takes the to be a basis of the and computes by using a basis for the tensor product consisting entirely of functions of the form for various . To pass from (iii) to (iv) one takes to be the annihilator of , and conversely to pass from (iv) to (iii).

One corollary of the formulation (iv), is that the set of tensors of slice rank at most is Zariski closed (if the field is algebraically closed), and so the slice rank itself is a lower semi-continuous function. This is in contrast to the usual tensor rank, which is not necessarily semicontinuous.

Corollary 2Let be finite-dimensional vector spaces over an algebraically closed field . Let be a nonnegative integer. The set of elements of of slice rank at most is closed in the Zariski topology.

*Proof:* In view of Lemma 1(i and iv), this set is the union over tuples of integers with of the projection from of the set of tuples with orthogonal to , where is the Grassmanian parameterizing -dimensional subspaces of .

One can check directly that the set of tuples with orthogonal to is Zariski closed in using a set of equations of the form locally on . Hence because the Grassmanian is a complete variety, the projection of this set to is also Zariski closed. So the finite union over tuples of these projections is also Zariski closed.

We also have good behaviour with respect to linear transformations:

Lemma 3Let be finite-dimensional vector spaces over a field , let be an element of , and for each , let be a linear transformation, with the tensor product of these maps. Then

Furthermore, if the are all injective, then one has equality in (2).

Thus, for instance, the rank of a tensor is intrinsic in the sense that it is unaffected by any enlargements of the spaces .

*Proof:* The bound (2) is clear from the formulation (ii) of rank in Lemma 1. For equality, apply (2) to the injective , as well as to some arbitrarily chosen left inverses of the .

Computing the rank of a tensor is difficult in general; however, the problem becomes a combinatorial one if one has a suitably sparse representation of that tensor in some basis, where we will measure sparsity by the property of being an antichain.

Proposition 4Let be finite-dimensional vector spaces over a field . For each , let be a linearly independent set in indexed by some finite set . Let be a subset of .

where for each , is a coefficient in . Then one has

where the minimum ranges over all coverings of by sets , and for are the projection maps.

Now suppose that the coefficients are all non-zero, that each of the are equipped with a total ordering , and is the set of maximal elements of , thus there do not exist distinct , such that for all . Then one has

In particular, if is an antichain (i.e. every element is maximal), then equality holds in (4).

*Proof:* By Lemma 3 (or by enlarging the bases ), we may assume without loss of generality that each of the is spanned by the . By relabeling, we can also assume that each is of the form

with the usual ordering, and by Lemma 3 we may take each to be , with the standard basis.

Let denote the rank of . To show (4), it suffices to show the inequality

for any covering of by . By removing repeated elements we may assume that the are disjoint. For each , the tensor

can (after collecting terms) be written as

for some . Summing and using (1), we conclude the inequality (6).

Now assume that the are all non-zero and that is the set of maximal elements of . To conclude the proposition, it suffices to show that the reverse inequality

Â holds for some covering . By Lemma 1(iv), there exist subspaces of whose dimension sums to

Let . Using Gaussian elimination, one can find a basis of whose representation in the standard dual basis of is in row-echelon form. That is to say, there exist natural numbers

such that for all , is a linear combination of the dual vectors , with the coefficient equal to one.

We now claim that is disjoint from . Suppose for contradiction that this were not the case, thus there exists for each such that

As is the set of maximal elements of , this implies that

for any tuple other than . On the other hand, we know that is a linear combination of , with the coefficient one. We conclude that the tensor product is equal to

plus a linear combination of other tensor products with not in . Taking inner products with (3), we conclude that , contradicting the fact that is orthogonal to . Thus we have disjoint from .

For each , let denote the set of tuples in with not of the form . From the previous discussion we see that the cover , and we clearly have , and hence from (8) we have (7) as claimed.

As an instance of this proposition, we recover the computation of diagonal rank from the previous blog post:

Example 5Let be finite-dimensional vector spaces over a field for some . Let be a natural number, and for , let be a linearly independent set in . Let be non-zero coefficients in . Thenhas rank . Indeed, one applies the proposition with all equal to , with the diagonal in ; this is an antichain if we give one of the the standard ordering, and another of the the opposite ordering (and ordering the remaining arbitrarily). In this case, the are all bijective, and so it is clear that the minimum in (4) is simply .

The combinatorial minimisation problem in the above proposition can be solved asymptotically when working with tensor powers, using the notion of the Shannon entropy of a discrete random variable .

Proposition 6Let be finite-dimensional vector spaces over a field . For each , let be a linearly independent set in indexed by some finite set . Let be a non-empty subset of .Let be a tensor of the form (3) for some coefficients . For each natural number , let be the tensor power of copies of , viewed as an element of . Then

and range over the random variables taking values in .

Now suppose that the coefficients are all non-zero and that each of the are equipped with a total ordering . Let be the set of maximal elements of in the product ordering, and let where range over random variables taking values in . Then

as . In particular, if the maximizer in (10) is supported on the maximal elements of (which always holds if is an antichain in the product ordering), then equality holds in (9).

*Proof:*

as , where is the projection map. Then the same thing will apply to and . Then applying Proposition 4, using the lexicographical ordering on and noting that, if are the maximal elements of , then are the maximal elements of , we obtain both (9) and (11).

We first prove the lower bound. By compactness (and the continuity properties of entropy), we can find a random variable taking values in such that

Let be a small positive quantity that goes to zero sufficiently slowly with . Let denote the set of all tuples in that are within of being distributed according to the law of , in the sense that for all , one has

By the asymptotic equipartition property, the cardinality of can be computed to be

if goes to zero slowly enough. Similarly one has

Now let be an arbitrary covering of . By the pigeonhole principle, there exists such that

which by (13) implies that

noting that the factor can be absorbed into the error). This gives the lower bound in (12).

Now we prove the upper bound. We can cover by sets of the form for various choices of random variables taking values in . For each such random variable , we can find such that ; we then place all of in . It is then clear that the cover and that

for all , giving the required upper bound.

It is of interest to compute the quantity in (10). We have the following criterion for when a maximiser occurs:

Proposition 7Let be finite sets, and be non-empty. Let be the quantity in (10). Let be a random variable taking values in , and let denote the essential range of , that is to say the set of tuples such that is non-zero. Then the following are equivalent:

- (i) attains the maximum in (10).
- (ii) There exist weights and a finite quantity , such that whenever , and such that
for all , with equality if . (In particular, must vanish if there exists a with .)

Furthermore, when (i) and (ii) holds, one has

*Proof:* We first show that (i) implies (ii). The function is concave on . As a consequence, if we define to be the set of tuples such that there exists a random variable taking values in with , then is convex. On the other hand, by (10), is disjoint from the orthant . Thus, by the hyperplane separation theorem, we conclude that there exists a half-space

where are reals that are not all zero, and is another real, which contains on its boundary and in its interior, such that avoids the interior of the half-space. Since is also on the boundary of , we see that the are non-negative, and that whenever .

By construction, the quantity

is maximised when . At this point we could use the method of Lagrange multipliers to obtain the required constraints, but because we have some boundary conditions on the (namely, that the probability that they attain a given element of has to be non-negative) we will work things out by hand. Let be an element of , and an element of . For small enough, we can form a random variable taking values in , whose probability distribution is the same as that for except that the probability of attaining is increased by , and the probability of attaining is decreased by . If there is any for which and , then one can check that

for sufficiently small , contradicting the maximality of ; thus we have whenever . Taylor expansion then gives

for small , where

and similarly for . We conclude that for all and , thus there exists a quantity such that for all , and for all . By construction must be nonnegative. Sampling using the distribution of , one has

almost surely; taking expectations we conclude that

The inner sum is , which equals when is non-zero, giving (17).

Now we show conversely that (ii) implies (i). As noted previously, the function is concave on , with derivative . This gives the inequality

for any (note the right-hand side may be infinite when and ). Let be any random variable taking values in , then on applying the above inequality with and , multiplying by , and summing over and gives

By construction, one has

and

so to prove that (which would give (i)), it suffices to show that

or equivalently that the quantity

is maximised when . Since

it suffices to show this claim for the quantity

One can view this quantity as

By (ii), this quantity is bounded by , with equality if is equal to (and is in particular ranging in ), giving the claim.

The second half of the proof of Proposition 7 only uses the marginal distributions and the equation(16), not the actual distribution of , so it can also be used to prove an upper bound on when the exact maximizing distribution is not known, given suitable probability distributions in each variable. The logarithm of the probability distribution here plays the role that the weight functions do in BCCGNSU.

Remark 8Suppose one is in the situation of (i) and (ii) above; assume the nondegeneracy condition that is positive (or equivalently that is positive). We can assign a “degree” to each element by the formula

then every tuple in has total degree at most , and those tuples in have degree exactly . In particular, every tuple in has degree at most , and hence by (17), each such tuple has a -component of degree less than or equal to for some with . On the other hand, we can compute from (19) and the fact that for that . Thus, by asymptotic equipartition, and assuming , the number of “monomials” in of total degree at most is at most ; one can in fact use (19) and (18) to show that this is in fact an equality. This gives a direct way to cover by sets with , which is in the spirit of the Croot-Lev-Pach-Ellenberg-Gijswijt arguments from the previous post.

We can now show that the rank computation for the capset problem is sharp:

Proposition 9Let denote the space of functions from to . Then the function from to , viewed as an element of , has rank as , where is given by the formula

*Proof:* In , we have

Thus, if we let be the space of functions from to (with domain variable denoted respectively), and define the basis functions

of indexed by (with the usual ordering), respectively, and set to be the set

then is a linear combination of the with , and all coefficients non-zero. Then we have . We will show that the quantity of (10) agrees with the quantity of (20), and that the optimizing distribution is supported on , so that by Proposition 6 the rank of is .

To compute the quantity at (10), we use the criterion in Proposition 7. We take to be the random variable taking values in that attains each of the values with a probability of , and each of with a probability of ; then each of the attains the values of with probabilities respectively, so in particular is equal to the quantity in (20). If we now set and

we can verify the condition (16) with equality for all , which from (17) gives as desired.

This statement already follows from the result of Kleinberg-Sawin-Speyer, which gives a “tri-colored sum-free set” in of size , as the slice rank of this tensor is an upper bound for the size of a tri-colored sum-free set. If one were to go over the proofs more carefully to evaluate the subexponential factors, this argument would give a stronger lower bound than KSS, as it does not deal with the substantial loss that comes from Behrend’s construction. However, because it actually constructs a set, the KSS result rules out more possible approaches to give an exponential improvement of the upper bound for capsets. The lower bound on slice rank shows that the bound cannot be improved using only the slice rank of this particular tensor, whereas KSS shows that the bound cannot be improved using any method that does not take advantage of the “single-colored” nature of the problem.

We can also show that the slice rank upper bound in a result of Naslund-Sawin is similarly sharp:

Proposition 10Let denote the space of functions from to . Then the function from , viewed as an element of , has slice rank

*Proof:* Let and be a basis for the space of functions on , itself indexed by . Choose similar bases for and , with and .

Set . Then is a linear combination of the with , and all coefficients non-zero. Order the usual way so that is an antichain. We will show that the quantity of (10) is , so that applying the last statement of Proposition 6, we conclude that the rank of is ,

Let be the random variable taking values in that attains each of the values with a probability of . Then each of the attains the value with probability and with probability , so

Setting and , we can verify the condition (16) with equality for all , which from (17) gives as desired.

We used a slightly different method in each of the last two results. In the first one, we use the most natural bases for all three vector spaces, and distinguish from its set of maximal elements . In the second one we modify one basis element slightly, with instead of the more obvious choice , which allows us to work with instead of . Because is an antichain, we do not need to distinguish and . Both methods in fact work with either problem, and they are both about equally difficult, but we include both as either might turn out to be substantially more convenient in future work.

Proposition 11Let be a natural number and let be a finite abelian group. Let be any field. Let denote the space of functions from to .Let be any -valued function on that is nonzero only when the elements of form a -term arithmetic progression, and is nonzero on every -term constant progression.

Then the slice rank of is .

*Proof:* We apply Proposition 4, using the standard bases of . Let be the support of . Suppose that we have orderings on such that the constant progressions are maximal elements of and thus all constant progressions lie in . Then for any partition of , can contain at most constant progressions, and as all constant progressions must lie in one of the , we must have . By Proposition 4, this implies that the slice rank of is at least . Since is a tensor, the slice rank is at most , hence exactly .

So it is sufficient to find orderings on such that the constant progressions are maximal element of . We make several simplifying reductions: We may as well assume that consists of all the -term arithmetic progressions, because if the constant progressions are maximal among the set of all progressions then they are maximal among its subset . So we are looking for an ordering in which the constant progressions are maximal among all -term arithmetic progressions. We may as well assume that is cyclic, because if for each cyclic group we have an ordering where constant progressions are maximal, on an arbitrary finite abelian group the lexicographic product of these orderings is an ordering for which the constant progressions are maximal. We may assume , as if we have an -tuple of orderings where constant progressions are maximal, we may add arbitrary orderings and the constant progressions will remain maximal.

So it is sufficient to find orderings on the cyclic group such that the constant progressions are maximal elements of the set of -term progressions in in the -fold product ordering. To do that, let the first, second, third, and fifth orderings be the usual order on and let the fourth, sixth, seventh, and eighth orderings be the reverse of the usual order on .

Then let be a constant progression and for contradiction assume that is a progression greater than in this ordering. We may assume that , because otherwise we may reverse the order of the progression, which has the effect of reversing all eight orderings, and then apply the transformation , which again reverses the eight orderings, bringing us back to the original problem but with .

Take a representative of the residue class in the interval . We will abuse notation and call this . Observe that , and are all contained in the interval modulo . Take a representative of the residue class in the interval . Then is in the interval for some . The distance between any distinct pair of intervals of this type is greater than , but the distance between and is at most , so is in the interval . By the same reasoning, is in the interval . Therefore . But then the distance between and is at most , so by the same reasoning is in the interval . Because is between and , it also lies in the interval . Because is in the interval , and by assumption it is congruent mod to a number in the set greater than or equal to , it must be exactly . Then, remembering that and lie in , we have and , so , hence , thus , which contradicts the assumption that .

In fact, given a -term progressions mod and a constant, we can form a -term binary sequence with a for each step of the progression that is greater than the constant and a for each step that is less. Because a rotation map, viewed as a dynamical system, has zero topological entropy, the number of -term binary sequences that appear grows subexponentially in . Hence there must be, for large enough , at least one sequence that does not appear. In this proof we exploit a sequence that does not appear for .

Filed under: expository, math.CO, math.RA Tagged: polynomial method, tensors, Will Sawin ]]>

as , where is the von Mangoldt function and is the twin prime constant

Because is almost entirely supported on the primes, it is not difficult to see that (1) implies the twin prime conjecture.

One can give a heuristic justification of the asymptotic (1) (and hence the twin prime conjecture) via sieve theoretic methods. Recall that the von Mangoldt function can be decomposed as a Dirichlet convolution

where is the MÃ¶bius function. Because of this, we can rewrite the left-hand side of (1) as

To compute this double sum, it is thus natural to consider sums such as

or (to simplify things by removing the logarithm)

The prime number theorem in arithmetic progressions suggests that one has an asymptotic of the form

where is the multiplicative function with for even and

for odd. Summing by parts, one then expects

and so we heuristically have

The Dirichlet series

has an Euler product factorisation

for ; comparing this with the Euler product factorisation

for the Riemann zeta function, and recalling that has a simple pole of residue at , we see that

has a simple zero at with first derivative

From this and standard multiplicative number theory manipulations, one can calculate the asymptotic

which concludes the heuristic justification of (1).

What prevents us from making the above heuristic argument rigorous, and thus proving (1) and the twin prime conjecture? Note that the variable in (2) ranges to be as large as . On the other hand, the prime number theorem in arithmetic progressions (3) is not expected to hold for anywhere that large (for instance, the left-hand side of (3) vanishes as soon as exceeds ). The best unconditional result known of the type (3) is the Siegel-Walfisz theorem, which allows to be as large as . Even the powerful generalised Riemann hypothesis (GRH) only lets one prove an estimate of the form (3) for up to about .

However, because of the averaging effect of the summation in in (2), we don’t need the asymptotic (3) to be true for *all* in a particular range; having it true for *almost all* in that range would suffice. Here the situation is much better; the celebrated Bombieri-Vinogradov theorem (sometimes known as “GRH on the average”) implies, roughly speaking, that the approximation (3) is valid for *almost all* for any fixed . While this is not enough to control (2) or (1), the Bombieri-Vinogradov theorem can at least be used to control variants of (1) such as

for various sieve weights whose associated divisor function is supposed to approximate the von Mangoldt function , although that theorem only lets one do this when the weights are supported on the range . This is still enough to obtain some partial results towards (1); for instance, by selecting weights according to the Selberg sieve, one can use the Bombieri-Vinogradov theorem to establish the upper bound

which is off from (1) by a factor of about . See for instance this blog post for details.

It has been difficult to improve upon the Bombieri-Vinogradov theorem in its full generality, although there are various improvements to certain restricted versions of the Bombieri-Vinogradov theorem, for instance in the famous work of Zhang on bounded gaps between primes. Nevertheless, it is believed that the Elliott-Halberstam conjecture (EH) holds, which roughly speaking would mean that (3) now holds for almost all for any fixed . (Unfortunately, the factor cannot be removed, as investigated in a series of papers by Friedlander, Granville, and also Hildebrand and Maier.) This comes tantalisingly close to having enough distribution to control all of (1). Unfortunately, it still falls short. Using this conjecture in place of the Bombieri-Vinogradov theorem leads to various improvements to sieve theoretic bounds; for instance, the factor of in (4) can now be improved to .

In two papers from the 1970s (which can be found online here and here respectively, the latter starting on page 255 of the pdf), Bombieri developed what is now known as the *Bombieri asymptotic sieve* to clarify the situation more precisely. First, he showed that on the Elliott-Halberstam conjecture, while one still could not establish the asymptotic (1), one could prove the generalised asymptotic

for all natural numbers , where the generalised von Mangoldt functions are defined by the formula

These functions behave like the von Mangoldt function, but are concentrated on -almost primes (numbers with at most prime factors) rather than primes. The right-hand side of (5) corresponds to what one would expect if one ran the same heuristics used to justify (1). Sadly, the case of (5), which is just (1), is just barely excluded from Bombieri’s analysis.

More generally, on the assumption of EH, the Bombieri asymptotic sieve provides the asymptotic

for any fixed and any tuple of natural numbers other than , where

is a further generalisation of the von Mangoldt function (now concentrated on -almost primes). By combining these asymptotics with some elementary identities involving the , together with the Weierstrass approximation theorem, Bombieri was able to control a wide family of sums including (1), except for one undetermined scalar . Namely, he was able to show (again on EH) that for any fixed and any continuous function on the simplex that had suitable vanishing at the boundary, the sum

when was even, where the integral on is with respect to the measure (this is Dirac measure in the case ). In particular, we have

and the twin prime conjecture would be proved if one could show that is bounded away from zero, while (1) is equivalent to the assertion that is equal to . Unfortunately, no additional bound beyond the inequalities provided by the Bombieri asymptotic sieve is known, even if one assumes all other major conjectures in number theory than the prime tuples conjecture and its variants (e.g. GRH, GEH, GUE, abc, Chowla, …).

To put it another way, the Bombieri asymptotic sieve is able (on EH) to compute asymptotics for sums

without needing to know the unknown scalar , when is a function supported on almost primes of the form

for and some fixed , with vanishing elsewhere and for some continuous (symmetric) functions obeying some vanishing at the boundary, so long as the parity condition

is obeyed (informally: gives the same weight to products of an odd number of primes as to products of an even number of primes, or to put it another way, is asymptotically orthogonal to the MÃ¶bius function ). But when violates the parity condition, the asymptotic involves the unknown . This scalar thus embodies the “parity problem” for the twin prime conjecture (discussed in these previous blog posts).

Because the obstruction to the parity problem is only one-dimensional (on EH), one can replace any parity-violating weight (such as ) with any other parity-violating weight and obtain a logically equivalent estimate. For instance, to prove the twin prime conjecture on EH, it would suffice to show that

for some fixed , or equivalently that there are solutions to the equation in primes with and . (In some cases, this sort of reduction can also be made using other sieves than the Bombieri asymptotic sieve, as was observed by Ng.) As another example, the Bombieri asymptotic sieve can be used to show that the asymptotic (1) is equivalent to the asymptotic

where is the set of numbers that are *rough* in the sense that they have no prime factors less than for some fixed (the function clearly correlates with and so must violate the parity condition). One can replace with similar sieve weights (e.g. a Selberg sieve) that concentrate on almost primes if desired.

As it turns out, if one is willing to strengthen the assumption of the Elliott-Halberstam (EH) conjecture to the assumption of the *generalised Elliott-Halberstam (GEH) conjecture* (as formulated for instance in Claim 2.6 of the Polymath8b paper), one can also swap the factor in the above asymptotics with other parity-violating weights and obtain a logically equivalent estimate, as the Bombieri asymptotic sieve also applies to weights such as under the assumption of GEH. For instance, on GEH one can use two such applications of the Bombieri asymptotic sieve to show that the twin prime conjecture would follow if one could show that there are solutions to the equation

in primes with and , for some . Similarly, on GEH the asymptotic (1) is equivalent to the asymptotic

for some fixed , and similarly with replaced by other sieves. This form of the quantitative twin primes conjecture is appealingly similar to the (special case)

of the Chowla conjecture, for which there has been some recent progress (discussed for instance in these recent posts). Informally, the Bombieri asymptotic sieve lets us (on GEH) view the twin prime conjecture as a sort of Chowla conjecture restricted to almost primes. Unfortunately, the recent progress on the Chowla conjecture relies heavily on the multiplicativity of at small primes, which is completely destroyed by inserting a weight such as , so this does not yet yield a viable path towards the twin prime conjecture even assuming GEH. Still, the similarity is striking, and one can hope that further ways to attack the Chowla conjecture may emerge that could impact the twin prime conjecture. (Alternatively, if one assumes a sufficiently optimistic version of the GEH, one could perhaps relax the notion of “almost prime” to the extent that one could start usefully using multiplicativity at smallish primes, though this seems rather wishful at present, particularly since the most optimistic versions of GEH are known to be false.)

The Bombieri asymptotic sieve is already well explained in the original two papers of Bombieri; there is also a slightly different treatment of the sieve by Friedlander and Iwaniec, as well as a simplified version in the book of Friedlander and Iwaniec (in which the distribution hypothesis is strengthened in order to shorten the arguments. I’ve decided though to write up my own notes on the sieve below the fold; this is primarily for my own benefit, but may be useful to some readers also. I largely follow the treatment of Bombieri, with the one idiosyncratic twist of replacing the usual “elementary” Selberg sieve with the “analytic” Selberg sieve used in particular in many of the breakthrough works in small gaps between primes; I prefer working with the latter due to its Fourier-analytic flavour.

** â€” 1. Controlling generalised von Mangoldt sums â€” **

To prove (5), we shall first generalise it, by replacing the sequence by a more general sequence obeying the following axioms:

- (i) (Non-negativity) One has for all .
- (ii) (Crude size bound) One has for all , where is the divisor function.
- (iii) (Size) We have for some constant .
- (iv) (Elliott-Halberstam type conjecture) For any , one has
where is a multiplicative function with for all primes and .

These axioms are a little bit stronger than what is actually needed to make the Bombieri asymptotic sieve work, but we will not attempt to work with the weakest possible axioms here.

We introduce the function

which is analytic for ; in particular it can be evaluated at to yield

There are two model examples of data to keep in mind. The first, discussed in the introduction, is when , then and is as in the introduction; one of course needs EH to justify axiom (iv) in this case. The other is when , in which case and for all . We will later take advantage of the second example to avoid doing some (routine, but messy) main term computations.

The main result of this section is then

Theorem 1Let be as above. Let be a tuple of natural numbers (independent of ) that is not equal to . Then one has the asymptoticas , where .

Note that this recovers (5) (on EH) as a special case.

We now begin the proof of this theorem. Henceforth we allow implied constants in the or notation to depend on and .

It will be convenient to replace the range by a shorter range by the following standard localisation trick. Let be a large quantity depending on to be chosen later, and let denote the interval . We will show the estimate

from which the original claim follows by a routine summation argument. Observe from axiom (iv) and the triangle inequality that

for any .

Write for the logarithm function , thus for any . Without loss of generality we may assume that ; we then factor , where

This function is just when . When the function is more complicated, but we at least have the following crude bound:

*Proof:* We induct on . The case is obvious, so suppose and the claim has already been proven for . Since , we see from induction hypothesis and the triangle inequality that

Since by MÃ¶bius inversion, the claim follows.

We can write

In the region , we have . Thus

for . The contribution of the error term to to (10) is easily seen to be negligible if is large enough, so we may freely replace with with little difficulty.

If we insert this replacement directly into the left-hand side of (10) and rearrange, we get

We can’t quite control this using axiom (iv) because the range of is a bit too big, as explained in the introduction. So let us introduce a truncated function

where is a small quantity to be chosen later, and is a smooth function that equals on and equals on . Suppose one could establish the following two estimates for any fixed :

where is a quantity that depends on but not on . Then on combining the two estimates we would have

One could in principle compute explicitly from the proof of (13), but one can avoid doing so by the following comparison trick. In the special case , standard multiplicative number theory (noting that the Dirichlet series has a pole of order at , with top Laurent coefficient ) gives the asymptotic

which when compared with (14) for (recalling that in this case) gives the formula

Inserting this back into (14) and recalling that can be made arbitrarily small, we obtain (10).

As it turns out, the estimate (13) is easy to establish, but the estimate (12) is not, roughly speaking because the typical number in has too many divisors in the range , each of which gives a contribution to the error term. (In the book of Friedlander and Iwaniec, the estimate (13) is established anyway, but only after assuming a stronger version of (iv), roughly speaking in which is allowed to be as large as .) To resolve this issue, we will insert a preliminary sieve that will remove most of the potential divisors i the range (leaving only about such divisors on the average for typical ), making the analogue of (12) easier to prove (at the cost of making the analogue of (13) more difficult). Namely, if one can find a function for which one has the estimates

for some quantity that depends on but not on , then by repeating the previous arguments we will again be able to establish (10).

The key estimate is (16). As we shall see, when comparing with , the weight will cost us a factor of , but the term in the definitions of and will recover a factor of , which will give the desired bound since we are assuming .

One has some flexibility in how to select the weight : basically any standard sieve that uses divisors of size at most to localise (at least approximately) to numbers that are rough in the sense that they have no (or at least very few) factors less than , will do. We will use the analytic Selberg sieve choice

where is a smooth function supported on that equals on .

It remains to establish the bounds (15), (16), (17). To warm up and introduce the various methods needed, we begin with the standard bound

where denotes the derivative of . Note the loss of that had previously been pointed out. In the arguments that follows I will be a little brief with the details, as they are standard (see e.g. this previous post).

We now prove (19). The left-hand side can be expanded as

where denotes the least common multiple of and . From the support of we see that the summand is only non-vanishing when . We now use axiom (iv) and split the left-hand side into a main term

and an error term that is at most

From axiom (ii) and elementary multiplicative number theory, we have the bound

so from axiom (iv) and Cauchy-Schwarz we see that the error term (20) is acceptable. Thus it will suffice to establish the bound

The summand here is almost, but not quite, multiplicative in . To make it genuinely multiplicative, we perform a (shifted) Fourier expansion

for some rapidly decreasing function (essentially the Fourier transform of ). Thus

and so the left-hand side of (21) can be rearranged using Fubini’s theorem as

We can factorise as an Euler product:

Taking absolute values and using Mertens’ theorem leads to the crude bound

which when combined with the rapid decrease of , allows us to restrict the region of integration in (23) to the square (say) with negligible error. Next, we use the Euler product

for to factorise

where

For with nonnegative real part, one has

and so by the Weierstrass -test, is continuous at . Since

we thus have

Also, since has a pole of order at with residue , we have

and thus

The quantity (23) can thus be written, up to errors of , as

Using the rapid decrease of , we may remove the restriction on , and it will now suffice to prove the identity

But on differentiating and then squaring (22) we have

and the claim follows by integrating in from zero to infinity (noting that vanishes for ).

We have the following variant of (19):

for any . We also have the variant

If in addition has no prime factors less than for some fixed , one has

Roughly speaking, the above estimates assert that is concentrated on those numbers with no prime factors much less than , but factors without such small prime divisors occur with about the same relative density as they do in the integers.

*Proof:* The left-hand side of (24) can be expanded as

If we define

then the previous expression can be written as

while one has

which gives (25) from Axiom (iv). To prove (24), it now suffices to show that

Arguing as before, the left-hand side is

where

From Mertens’ theorem we have

when , so the contribution of the terms where can be absorbed into the error (after increasing that error slightly). For the remaining contributions, we see that

where if does not divide , and

if divides times for some . In the latter case, Taylor expansion gives the bounds

and the claim (28) follows. When and we have

and (27) follows by repeating the previous calculations. Finally, (26) is proven similarly to (24) (using in place of ).

Now we can prove (15), (16), (17). We begin with (15). Using the Leibniz rule applied to the identity and using and MÃ¶bius inversion (and the associativity and commutativity of Dirichlet convolution) we see that

Next, by applying the Leibniz rule to for some and using (29) we see that

and hence we have the recursive identity

In particular, from induction we see that is supported on numbers with at most distinct prime factors, and hence is supported on numbers with at most distinct prime factors. In particular, from (18) we see that on the support of . Thus it will suffice to show that

If and , then has at most distinct prime factors , with . If we factor , where is the contribution of those with , and is the contribution of those with , then at least one of the following two statements hold:

- (a) (and hence ) is divisible by a square number of size at least .
- (b) .

The contribution of case (a) is easily seen to be acceptable by axiom (ii). For case (b), we observe from (30) and induction that

and so it will suffice to show that

where ranges over numbers bounded by with at most distinct prime factors, the smallest of which is at most , and consists of those numbers with no prime factor less than or equal to . Applying (26) (with replaced by ) gives the bound

so by (25) it suffices to show that

subject to the same constraints on as before. The contribution of those with distinct prime factors can be bounded by

applying Mertens’ theorem and summing over , one obtains the claim.

Now we show (16). As discussed previously in this section, we can replace by with negligible error. Comparing this with (16) and (11), we see that it suffices to show that

From the support of , the summand on the left-hand side is only non-zero when , which makes , where we use the crucial hypothesis to gain enough powers of to make the argument here work. Applying Lemma 2, we reduce to showing that

We can make the change of variables to flip the sum

and then swap the sums to reduce to showing that

By Lemma 3, it suffices to show that

To prove this, we use the Rankin trick, bounding the implied weight by . We can then bound the left-hand side by the Euler product

which can be bounded by

and the claim follows from Mertens’ theorem.

Finally, we show (17). By (11), the left-hand side expands as

We let be a small constant to be chosen later. We divide the outer sum into two ranges, depending on whether only has prime factors greater than or not. In the former case, we can apply (27) to write this contribution as

plus a negligible error, where the is implicitly restricted to numbers with all prime factors greater than . The main term is messy, but it is of the required form up to an acceptable error, so there is no need to compute it any further. It remains to consider those that have at least one prime factor less than . Here we use (24) instead of (27) as well as Lemma 3 to dominate this contribution by

up to negligible errors, where is now restricted to have at least one prime factor less than . This makes at least one of the factors to be at most . A routine application of Rankin’s trick shows that

and so the total contribution of this case is . Since can be made arbitrarily small, (17) follows.

** â€” 2. Weierstrass approximation â€” **

Having proved Theorem 1, we now take linear combinations of this theorem, combined with the Weierstrass approximation theorem, to give the asymptotics (7), (8) described in the introduction.

Let , , , be as in that theorem. It will be convenient to normalise the weights by to make their mean value comparable to . From Theorem 1 and summation by parts we have

whenever does not consist entirely of ones.

We now take a closer look at what happens when does consist entirely of ones. Let denote the -tuple . Convolving the case of (30) with copies of for some and using the Leibniz rule, we see that

and hence

Multiplying by and summing over , and using (31) to control the term, one has

If we define (up to an error of ) by the formula

then an induction then shows that

for odd , and

for even . In particular, after adjusting by if necessary, we have since the left-hand sides are non-negative.

If we now define the comparison sequence , standard multiplicative number theory shows that the above estimates also hold when is replaced by ; thus

for both odd and even . The bound (31) also holds for when does not consist entirely of ones, and hence

for any fixed (which may or may not consist entirely of ones).

Next, from induction (on ), the Leibniz rule, and (30), we see that for any and , , the function

is a finite linear combination of functions of the form for tuples that may possibly consist entirely of ones. We thus have

whenever is one of these functions (32). Specialising to the case , we thus have

where . The contribution of those that are powers of primes can be easily seen to be negligible, leading to

where now . The contribution of the case where two of the primes agree can also be seen to be negligible, as can the error when replacing with , and then by symmetry

By linearity, this implies that

for any polynomial that vanishes on the coordinate hyperplanes . The right-hand side can also be evaluated by Mertens’ theorem as

when is odd and

when is even. Using the Weierstrass approximation theorem, we then have

for any continuous function that is compactly supported in the interior of . Computing the right-hand side using Mertens’ theorem as before, we obtain the claimed asymptotics (7), (8).

Remark 4The Bombieri asymptotic sieve has to use the full power of EH (or GEH); there are constructions due to Ford that show that if one only has a distributional hypothesis up to for some fixed constant , then the asymptotics of sums such as (5), or more generally (9), are not determined by a single scalar parameter , but can also vary in other ways as well. Thus the Bombieri asymptotic sieve really is asymptotic; in order to get type error terms one needs the level of distribution to be asymptotically equal to as . Related to this, the quantitative decay of the error terms in the Bombieri asymptotic sieve are extremely poor; in particular, they depend on the dependence of implied constant in axiom (iv) on the parameters , for which there is no consensus on what one should conjecturally expect.

Filed under: expository, math.NT Tagged: Bombieri sieve, parity problem, sieve theory, twin primes ]]>

in three spatial dimensions, where is the velocity vector field and is the pressure field. In vorticity form, and viewing the vorticity as a -form (rather than a vector), we can rewrite this system using the language of differential geometry as

where is the Lie derivative along , is the codifferential (the adjoint of the differential , or equivalently the negative of the divergence operator) that sends -vector fields to -vector fields, is the Hodge Laplacian, and is the identification of -vector fields with -forms induced by the Euclidean metric . The equation can be viewed as the Biot-Savart law recovering velocity from vorticity, expressed in the language of differential geometry.

One can then generalise this system by replacing the operator by a more general operator from -forms to -vector fields, giving rise to what I call the *generalised Euler equations*

For example, the surface quasi-geostrophic (SQG) equations can be written in this form, as discussed in this previous post. One can view (up to Hodge duality) as a vector potential for the velocity , so it is natural to refer to as a vector potential operator.

The generalised Euler equations carry much of the same geometric structure as the true Euler equations. For instance, the transport equation is equivalent to the Kelvin circulation theorem, which in three dimensions also implies the transport of vortex streamlines and the conservation of helicity. If is self-adjoint and positive definite, then the famous Euler-PoincarÃ© interpretation of the true Euler equations as geodesic flow on an infinite dimensional Riemannian manifold of volume preserving diffeomorphisms (as discussed in this previous post) extends to the generalised Euler equations (with the operator determining the new Riemannian metric to place on this manifold). In particular, the generalised Euler equations have a Lagrangian formulation, and so by Noether’s theorem we expect any continuous symmetry of the Lagrangian to lead to conserved quantities. Indeed, we have a conserved Hamiltonian , and any spatial symmetry of leads to a conserved impulse (e.g. translation invariance leads to a conserved momentum, and rotation invariance leads to a conserved angular momentum). If behaves like a pseudodifferential operator of order (as is the case with the true vector potential operator ), then it turns out that one can use energy methods to recover the same sort of classical local existence theory as for the true Euler equations (up to and including the famous Beale-Kato-Majda criterion for blowup).

The true Euler equations are suspected of admitting smooth localised solutions which blow up in finite time; there is now substantial numerical evidence for this blowup, but it has not been proven rigorously. The main purpose of this paper is to show that such finite time blowup can at least be established for certain generalised Euler equations that are somewhat close to the true Euler equations. This is similar in spirit to my previous paper on finite time blowup on averaged Navier-Stokes equations, with the main new feature here being that the modified equation continues to have a Lagrangian structure and a vorticity formulation, which was not the case with the averaged Navier-Stokes equation. On the other hand, the arguments here are not able to handle the presence of viscosity (basically because they rely crucially on the Kelvin circulation theorem, which is not available in the viscous case).

In fact, three different blowup constructions are presented (for three different choices of vector potential operator ). The first is a variant of one discussed previously on this blog, in which a “neck pinch” singularity for a vortex tube is created by using a non-self-adjoint vector potential operator, in which the velocity at the neck of the vortex tube is determined by the circulation of the vorticity somewhat further away from that neck, which when combined with conservation of circulation is enough to guarantee finite time blowup. This is a relatively easy construction of finite time blowup, and has the advantage of being rather stable (any initial data flowing through a narrow tube with a large positive circulation will blow up in finite time). On the other hand, it is not so surprising in the non-self-adjoint case that finite blowup can occur, as there is no conserved energy.

The second blowup construction is based on a connection between the two-dimensional SQG equation and the three-dimensional generalised Euler equations, discussed in this previous post. Namely, any solution to the former can be lifted to a “two and a half-dimensional” solution to the latter, in which the velocity and vorticity are translation-invariant in the vertical direction (but the velocity is still allowed to contain vertical components, so the flow is not completely horizontal). The same embedding also works to lift solutions to generalised SQG equations in two dimensions to solutions to generalised Euler equations in three dimensions. Conveniently, even if the vector potential operator for the generalised SQG equation fails to be self-adjoint, one can ensure that the three-dimensional vector potential operator is self-adjoint. Using this trick, together with a two-dimensional version of the first blowup construction, one can then construct a generalised Euler equation in three dimensions with a vector potential that is both self-adjoint and positive definite, and still admits solutions that blow up in finite time, though now the blowup is now a vortex sheet creasing at on a line, rather than a vortex tube pinching at a point.

This eliminates the main defect of the first blowup construction, but introduces two others. Firstly, the blowup is less stable, as it relies crucially on the initial data being translation-invariant in the vertical direction. Secondly, the solution is not spatially localised in the vertical direction (though it can be viewed as a compactly supported solution on the manifold , rather than ). The third and final blowup construction of the paper addresses the final defect, by replacing vertical translation symmetry with axial rotation symmetry around the vertical axis (basically, replacing Cartesian coordinates with cylindrical coordinates). It turns out that there is a more complicated way to embed two-dimensional generalised SQG equations into three-dimensional generalised Euler equations in which the solutions to the latter are now axially symmetric (but are allowed to “swirl” in the sense that the velocity field can have a non-zero angular component), while still keeping the vector potential operator self-adjoint and positive definite; the blowup is now that of a vortex ring creasing on a circle.

As with the previous papers in this series, these blowup constructions do not *directly* imply finite time blowup for the true Euler equations, but they do at least provide a barrier to establishing global regularity for these latter equations, in that one is forced to use some property of the true Euler equations that are not shared by these generalisations. They also suggest some possible blowup mechanisms for the true Euler equations (although unfortunately these mechanisms do not seem compatible with the addition of viscosity, so they do not seem to suggest a viable Navier-Stokes blowup mechanism).

Filed under: math.AP, paper Tagged: Euler equations, finite time blowup ]]>

I believe that there is now a real-life instance of this situation in the US presidential election, regarding the following

Proposition 1. Â The presumptive nominee of the Republican Party, Donald Trump, is not even remotely qualified to carry out the duties of the presidency of the United States of America.

Proposition 1Â is a statement which I think is approaching the level of mutual knowledge amongst the US population (and probably a large proportion of people following US politics overseas): even many of Trump’s nominal supporters secretly suspect that this proposition is true, even if they are hesitant to say it out loud. Â And there have been many prominent people, from both major parties, that have made the case for Proposition 1: for instance Mitt Romney, the Republican presidential nominee in 2012, did so back in March,Â and just a few days ago Hillary Clinton, the likely Democratic presidential nominee this year, did so in this speech:

I highly recommend watching the entirety of the (35 mins or so) speech, followed by the entirety ofÂ Trump’sÂ rebuttal.

However, even if Proposition 1Â is approaching the status of “mutual knowledge”, it does not yet seem to be close to the status of “common knowledge”: one may secretly believe that Trump cannot be considered as a serious candidate for the US presidency, but must continue to entertain this possibility, because they feel that others around them, or in politics or the media, appear to be doing so. Â To reconcile these views can require taking on some implausible hypotheses that are not otherwise supported by any evidence, such as the hypothesis that Trump’s displays of policy ignorance, pettiness, and other clearly unpresidential behaviour areÂ merely “for show”, and that behind this facade there is actually a competent and qualified presidential candidate; much like the emperor’s new clothes, this allegedÂ competence is supposedly only visible to a select few. Â And so the charade continues.

I feel that it is time for the charade to end: Trump is unfit to be president, and everybody knows it. Â But more people need to say so, openly.

**Important note**: I anticipate there will be any number of “tu quoque” responses, asserting for instance that Hillary Clinton is also unfit to be the US president. Â I personally do not believe that to be the case (and certainly not to the extent that Trump exhibits), but in any event such an assertionÂ has no logical bearing on the qualification of Trump for the presidency. Â **As such, any comments that are purely of this “tu quoque” nature, and which do not directly address the validity or epistemological status of Proposition 1,Â will be deleted as off-topic**. Â However, there is a legitimate case to be made that there is a fundamental weakness in the current mechanics of the US presidential election, particularly with the “first-past-the-post” voting system, in that (once the presidential primaries are concluded) a voter in the presidential election is effectively limited to choosing between just two viable choices, one from each of the two major parties, or else refusing to vote or making a largely symbolic protest vote.Â This weakness is particularly evident when at least one of these two major choices is demonstrably unfit for office, as per Proposition 1. Â I think there is a serious case for debating the possibility of major electoral reform in the US (I am particularly partial to the Instant Runoff Voting system, used for instance in my home country of Australia, which allows for meaningful votes to third parties), and I would consider such a debate to be on-topic for this post. Â But this is very much a longer term issue, as there is absolutely no chance that any such reform would be implemented by the time of the US elections in November (particularly given that any significantÂ reform would almost certainly require, at minimum, a constitutional amendment).

Filed under: non-technical, opinion Tagged: politics ]]>

Suppose one assigns true-false questions on an examination, with the answers randomised so that each question is equally likely to have “true” as the correct answer as “false”, with no correlation between different questions. Suppose that the students taking the examination must answer each question with exactly one of “true” or “false” (they are not allowed to skip any question). Then it is easy to see how to grade the exam: one can simply count how many questions each student answered correctly (i.e. each correct answer scores one point, and each incorrect answer scores zero points), and give that number as the final grade of the examination. More generally, one could assign some score of points to each correct answer and some score (possibly negative) of points to each incorrect answer, giving a total grade of points. As long as , this grade is simply an affine rescaling of the simple grading scheme and would serve just as well for the purpose of evaluating the students, as well as encouraging each student to answer the questions as correctly as possible.

In practice, though, a student will probably not know the answer to each individual question with absolute certainty. One can adopt a probabilistic model, where for a given student and a given question , the student Â may think that the answer to question is true with probability and false with probability , where is some quantity that can be viewed as a measure of confidence has in the answer (with being confident that the answer is true if is close to , and confident that the answer is false if is close to ); for simplicity let us assume that in ‘s probabilistic model, the answers to each question are independent random variables. Given this model, and assuming that the student wishes to maximise his or her expected grade on the exam, it is an easy matter to see that the optimal strategy for to take is to answer question true if and false if . (If , the student can answer arbitrarily.)

[Important note: here we are *not* using the term “confidence” in the technical sense used in statistics, but rather as an informal term for “subjective probability”.]

This is fine as far as it goes, but for the purposes of evaluating how well the student actually knows the material, it provides only a limited amount of information, in particular we do not get to directly see the student’s subjective probabilities for each question. If for instance answered out of questions correctly, was it because he or she actually knew the right answer for seven of the questions, or was it because he or she was making educated guesses for the ten questions that turned out to be slightly better than random chance? There seems to be no way to discern this if the only input the student is allowed to provide for each question is the single binary choice of true/false.

But what if the student were able to give probabilistic answers to any given question? That is to say, instead of being forced to answer just “true” or “false” for a given question , the student was allowed to give answers such as “ confident that the answer is true” (and hence confidence the answer is false). Such answers would give more insight as to how well the student actually knew the material; in particular, we would theoretically be able to actually see the student’s subjective probabilities .

But now it becomes less clear what the right grading scheme to pick is. Suppose for instance we wish to extend the simple grading scheme in which an correct answer given in confidence is awarded one point. How many points should one award a correct answer given in confidence? How about an incorrect answer given in confidence (or equivalently, a correct answer given in confidence)?

Mathematically, one could design a grading scheme by selecting some grading function and then awarding a student points whenever they indicate the correct answer with a confidence of . For instance, if the student was confident that the answer was “true” (and hence confident that the answer was “false”), then this grading scheme would award the student points if the correct answer actually was “true”, and points if the correct answer actually was “false”. One can then ask the question of what functions would be “best” for this scheme?

Intuitively, one would expect that should be monotone increasing – one should be rewarded more for being correct with high confidence, than correct with low confidence. On the other hand, some sort of “partial credit” should still be assigned in the latter case. One obvious proposal is to just use a linear grading function – thus for instance a correct answer given with confidence might be worth points. But is this the “best” option?

To make the problem more mathematically precise, one needs an objective criterion with which to evaluate a given grading scheme. One criterion that one could use here is the avoidance of perverse incentives. If a grading scheme is designed badly, a student may end up overstating or understating his or her confidence in an answer in order to optimise the (expected) grade: the optimal level of confidence for a student to report on a question may differ from that student’s subjective confidence . So one could ask to design a scheme so that is always equal to , so that the incentive is for the student to honestly report his or her confidence level in the answer.

This turns out to give a precise constraint on the grading function . If a student thinks that the answer to a question is true with probability and false with probability , and enters in an answer of “true” with confidence (and thus “false” with confidence ), then student would expect a grade of

on average for this question. To maximise this expected grade (assuming differentiability of , which is a reasonable hypothesis for a partial credit grading scheme), one performs the usual maneuvre of differentiating in the independent variable and setting the result to zero, thus obtaining

In order to avoid perverse incentives, the maximum should occur at , thus we should have

for all . This suggests that the function should be constant. (Strictly speaking, it only gives the weaker constraint that is symmetric around ; but if one generalised the problem to allow for multiple-choice questions with more than two possible answers, with a grading scheme that depended only on the confidence assigned to the correct answer, the same analysis would in fact force to be constant in ; we leave this computation to the interested reader.) In other words, should be of the form for some ; by monotonicity we expect to be positive. If we make the normalisation (so that no points are awarded for a split in confidence between true and false) and , one arrives at the grading scheme

Thus, if a student believes that an answer is “true” with confidence and “false” with confidence , he or she will be awarded points when the correct answer is “true”, and points if the correct answer is “false”. The following table gives some illustrative values for this scheme:

Confidence that answer is “true” | Points awarded if answer is “true” | Points awarded if answer is “false” |

Note the large penalties for being extremely confident of an answer that ultimately turns out to be incorrect; in particular, answers of confidence should be avoided unless one really is absolutely certain as to the correctness of one’s answer.

The total grade given under such a scheme to a student who answers each question to be “true” with confidence , and “false” with confidence , is

This grade can also be written as

where

is the likelihood of the student ‘s subjective probability model, given the outcome of the correct answers. Thus the grade system here has another natural interpretation, as being an affine rescaling of the log-likelihood. The incentive is thus for the student to maximise the likelihood of his or her own subjective model, which aligns well with standard practices in statistics. From the perspective of Bayesian probability, the grade given to a student can then be viewed as a measurement (in logarithmic scale) of how much the posterior probability that the student’s model was correct has improved over the prior probability.

One could propose using the above grading scheme to evaluate predictions to binary events, such as an upcoming election with only two viable candidates, to see in hindsight just how effective each predictor was in calling these events. One difficulty in doing so is that many predictions do not come with explicit probabilities attached to them, and attaching a default confidence level of to any prediction made without any such qualification would result in an automatic grade of if even one of these predictions turned out to be incorrect. But perhaps if a predictor refuses to attach confidence level to his or her predictions, one can assign some default level of confidence to these predictions, and then (using some suitable set of predictions from this predictor as “training data”) find the value of that maximises this predictor’s grade. This level can then be used going forward as the default level of confidence to apply to any future predictions from this predictor.

The above grading scheme extends easily enough to multiple-choice questions. But one question I had trouble with was how to deal with *uncertainty*, in which the student does not know enough about a question to venture even a probability of being true or false. Here, it is natural to allow a student to leave a question blank (i.e. to answer “I don’t know”); a more advanced option would be to allow the student to enter his or her confidence level as an interval range (e.g. “I am between and confident that the answer is “true””). But now I do not have a good proposal for a grading scheme; once there is uncertainty in the student’s subjective model, the problem of that student maximising his or her expected grade becomes ill-posed due to the “unknown unknowns”, and so the previous criterion of avoiding perverse incentives becomes far less useful.

Filed under: math.PR, math.ST Tagged: grading ]]>

Trivially, one has . Using Fourier methods (and the density increment argument of Roth), the bound of was obtained by Meshulam, and improved only as late as 2012 to for some absolute constant by Bateman and Katz. But in a very recent breakthrough, Ellenberg (and independently Gijswijt) obtained the exponentially superior bound , using a version of the polynomial method recently introduced by Croot, Lev, and Pach. (In the converse direction, a construction of Edel gives capsets as large as .) Given the success of the polynomial method in superficially similar problems such as the finite field Kakeya problem (discussed in this previous post), it was natural to wonder that this method could be applicable to the cap set problem (see for instance this MathOverflow comment of mine on this from 2010), but it took a surprisingly long time before Croot, Lev, and Pach were able to identify the precise variant of the polynomial method that would actually work here.

The proof of the capset bound is very short (Ellenberg’s and Gijswijt’s preprints are both 3 pages long, and Croot-Lev-Pach is 6 pages), but I thought I would present a slight reformulation of the argument which treats the three points on a line in symmetrically (as opposed to treating the third point differently from the first two, as is done in the Ellenberg and Gijswijt papers; Croot-Lev-Pach also treat the middle point of a three-term arithmetic progression differently from the two endpoints, although this is a very natural thing to do in their context of ). The basic starting point is this: if is a capset, then one has the identity

for all , where is the Kronecker delta function, which we view as taking values in . Indeed, (1) reflects the fact that the equation has solutions precisely when are either all equal, or form a line, and the latter is ruled out precisely when is a capset.

To exploit (1), we will show that the left-hand side of (1) is “low rank” in some sense, while the right-hand side is “high rank”. Recall that a function taking values in a field is of *rank one* if it is non-zero and of the form for some , and that the rank of a general function is the least number of rank one functions needed to express as a linear combination. More generally, if , we define the *rank* of a function to be the least number of “rank one” functions of the form

for some and some functions , , that are needed to generate as a linear combination. For instance, when , the rank one functions take the form , , , and linear combinations of such rank one functions will give a function of rank at most .

It is a standard fact in linear algebra that the rank of a diagonal matrix is equal to the number of non-zero entries. This phenomenon extends to higher dimensions:

Lemma 1 (Rank of diagonal hypermatrices)Let , let be a finite set, let be a field, and for each , let be a coefficient. Then the rank of the function

*Proof:* We induct on . As mentioned above, the case follows from standard linear algebra, so suppose now that and the claim has already been proven for .

It is clear that the function (2) has rank at most equal to the number of non-zero (since the summands on the right-hand side are rank one functions), so it suffices to establish the lower bound. By deleting from those elements with (which cannot increase the rank), we may assume without loss of generality that all the are non-zero. Now suppose for contradiction that (2) has rank at most , then we obtain a representation

for some sets of cardinalities adding up to at most , and some functions and .

Consider the space of functions that are orthogonal to all the , in the sense that

for all . This space is a vector space whose dimension is at least . A basis of this space generates a coordinate matrix of full rank, which implies that there is at least one non-singular minor. This implies that there exists a function in this space which is nowhere vanishing on some subset of of cardinality at least .

If we multiply (3) by and sum in , we conclude that

where

The right-hand side has rank at most , since the summands are rank one functions. On the other hand, from induction hypothesis the left-hand side has rank at least , giving the required contradiction.

On the other hand, we have the following (symmetrised version of a) beautifully simple observation of Croot, Lev, and Pach:

*Proof:* Using the identity for , we have

The right-hand side is clearly a polynomial of degree in , which is then a linear combination of monomials

with with

In particular, from the pigeonhole principle, at least one of is at most .

Consider the contribution of the monomials for which . We can regroup this contribution as

where ranges over those with , is the monomial

and is some explicitly computable function whose exact form will not be of relevance to our argument. The number of such is equal to , so this contribution has rank at most . The remaining contributions arising from the cases and similarly have rank at most (grouping the monomials so that each monomial is only counted once), so the claim follows.

Upon restricting from to , the rank of is still at most . The two lemmas then combine to give the Ellenberg-Gijswijt bound

All that remains is to compute the asymptotic behaviour of . This can be done using the general tool of Cramer’s theorem, but can also be derived from Stirling’s formula (discussed in this previous post). Indeed, if , , for some summing to , Stirling’s formula gives

where is the entropy function

We then have

where is the maximum entropy subject to the constraints

A routine Lagrange multiplier computation shows that the maximum occurs when

and is approximately , giving rise to the claimed bound of .

Remark 3As noted in the Ellenberg and Gijswijt papers, the above argument extends readily to other fields than to control the maximal size of subset of that has no non-trivial solutions to the equation , where are non-zero constants that sum to zero. Of course one replaces the function in Lemma 2 by in this case.

Remark 4This symmetrised formulation suggests that one possible way to improve slightly on the numerical quantity by finding a more efficient way to decompose into rank one functions, however I was not able to do so (though such improvements are reminiscent of the Strassen type algorithms for fast matrix multiplication).

Remark 5It is tempting to see if this method can get non-trivial upper bounds for sets with no length progressions, in (say) . One can run the above arguments, replacing the functionwith

this leads to the bound where

Unfortunately, is asymptotic to and so this bound is in fact slightly worse than the trivial bound ! However, there is a slim chance that there is a more efficient way to decompose into rank one functions that would give a non-trivial bound on . I experimented with a few possible such decompositions but unfortunately without success.

Remark 6Return now to the capset problem. Since Lemma 1 is valid for any field , one could perhaps hope to get better bounds by viewing the Kronecker delta function as taking values in another field than , such as the complex numbers . However, as soon as one works in a field of characteristic other than , one can adjoin a cube root of unity, and one now has the Fourier decompositionMoving to the Fourier basis, we conclude from Lemma 1 that the function on now has rank exactly , and so one cannot improve upon the trivial bound of by this method using fields of characteristic other than three as the range field. So it seems one has to stick with (or the algebraic completion thereof).

Thanks to Jordan Ellenberg and Ben Green for helpful discussions.

Filed under: expository, math.CO, math.RA Tagged: cap sets, Dion Gijswijt, Ernie Croot, Jordan Ellenberg, Peter Pach, polynomial method, Seva Lev ]]>

as was demonstrated, where was any positive integer and denoted the Liouville function. The proof proceeded using a method I call the “entropy decrement argument”, which ultimately reduced matters to establishing a bound of the form

whenever was a slowly growing function of . This was in turn established in a previous paper of Matomaki, Radziwill, and myself, using the recent breakthrough of Matomaki and Radziwill.

It is natural to see to what extent the arguments can be adapted to attack the higher-point cases of the logarithmically averaged Chowla conjecture (ignoring for this post the more general Elliott conjecture for other bounded multiplicative functions than the Liouville function). That is to say, one would like to prove that

as for any fixed distinct integers . As it turns out (and as is detailed in the current paper), the entropy decrement argument extends to this setting (after using some known facts about linear equations in primes), and allows one to reduce the above estimate to an estimate of the form

for a slowly growing function of and some fixed (in fact we can take for ), where is the (normalised) local Gowers uniformity norm. (In the case , , this becomes the Fourier-uniformity conjecture discussed in this previous post.) If one then applied the (now proven) inverse conjecture for the Gowers norms, this estimate is in turn equivalent to the more complicated looking assertion

where the supremum is over all possible choices of *nilsequences* of controlled step and complexity (see the paper for definitions of these terms).

The main novelty in the paper (elaborating upon a previous comment I had made on this blog) is to observe that this latter estimate in turn follows from the logarithmically averaged form of Sarnak’s conjecture (discussed in this previous post), namely that

whenever is a zero entropy (i.e. deterministic) sequence. Morally speaking, this follows from the well-known fact that nilsequences have zero entropy, but the presence of the supremum in (1) means that we need a little bit more; roughly speaking, we need the *class* of nilsequences of a given step and complexity to have “uniformly zero entropy” in some sense.

On the other hand, it was already known (see previous post) that the Chowla conjecture implied the Sarnak conjecture, and similarly for the logarithmically averaged form of the two conjectures. Putting all these implications together, we obtain the pleasant fact that the logarithmically averaged Sarnak and Chowla conjectures are equivalent, which is the main result of the current paper. There have been a large number of special cases of the Sarnak conjecture worked out (when the deterministic sequence involved came from a special dynamical system), so these results can now also be viewed as partial progress towards the Chowla conjecture also (at least with logarithmic averaging). However, my feeling is that the full resolution of these conjectures will not come from these sorts of special cases; instead, conjectures like the Fourier-uniformity conjecture in this previous post look more promising to attack.

It would also be nice to get rid of the pesky logarithmic averaging, but this seems to be an inherent requirement of the entropy decrement argument method, so one would probably have to find a way to avoid that argument if one were to remove the log averaging.

Filed under: math.NT, paper Tagged: Chowla conjecture, entropy decrement argument, Sarnak conjecture ]]>