You are currently browsing the tag archive for the ‘sieve theory’ tag.

We continue the discussion of sieve theory from Notes 4, but now specialise to the case of the linear sieve in which the sieve dimension ${\kappa}$ is equal to ${1}$, which is one of the best understood sieving situations, and one of the rare cases in which the precise limits of the sieve method are known. A bit more specifically, let ${z, D \geq 1}$ be quantities with ${z = D^{1/s}}$ for some fixed ${s>1}$, and let ${g}$ be a multiplicative function with

$\displaystyle g(p) = \frac{1}{p} + O(\frac{1}{p^2}) \ \ \ \ \ (1)$

and

$\displaystyle 0 \leq g(p) \leq 1-c \ \ \ \ \ (2)$

for all primes ${p}$ and some fixed ${c>0}$ (we allow all constants below to depend on ${c}$). Let ${P(z) := \prod_{p, and for each prime ${p < z}$, let ${E_p}$ be a set of integers, with ${E_d := \bigcap_{p|d} E_p}$ for ${d|P(z)}$. We consider finitely supported sequences ${(a_n)_{n \in {\bf Z}}}$ of non-negative reals for which we have bounds of the form

$\displaystyle \sum_{n \in E_d} a_n = g(d) X + r_d. \ \ \ \ \ (3)$

for all square-free ${d \leq D}$ and some ${X>0}$, and some remainder terms ${r_d}$. One is then interested in upper and lower bounds on the quantity

$\displaystyle \sum_{n\not \in\bigcup_{p

The fundamental lemma of sieve theory (Corollary 19 of Notes 4) gives us the bound

$\displaystyle \sum_{n\not \in\bigcup_{p

where ${V(z)}$ is the quantity

$\displaystyle V(z) := \prod_{p

This bound is strong when ${s}$ is large, but is not as useful for smaller values of ${s}$. We now give a sharp bound in this regime. We introduce the functions ${F, f: (0,+\infty) \rightarrow {\bf R}^+}$ by

$\displaystyle F(s) := 2e^\gamma ( \frac{1_{s>1}}{s} \ \ \ \ \ (6)$

$\displaystyle + \sum_{j \geq 3, \hbox{ odd}} \frac{1}{j!} \int_{[1,+\infty)^{j-1}} 1_{t_1+\dots+t_{j-1}\leq s-1} \frac{dt_1 \dots dt_{j-1}}{t_1 \dots t_j} )$

and

$\displaystyle f(s) := 2e^\gamma \sum_{j \geq 2, \hbox{ even}} \frac{1}{j!} \int_{[1,+\infty)^{j-1}} 1_{t_1+\dots+t_{j-1}\leq s-1} \frac{dt_1 \dots dt_{j-1}}{t_1 \dots t_j} \ \ \ \ \ (7)$

where we adopt the convention ${t_j := s - t_1 - \dots - t_{j-1}}$. Note that for each ${s}$ one has only finitely many non-zero summands in (6), (7). These functions are closely related to the Buchstab function ${\omega}$ from Exercise 28 of Supplement 4; indeed from comparing the definitions one has

$\displaystyle F(s) + f(s) = 2 e^\gamma \omega(s)$

for all ${s>0}$.

Exercise 1 (Alternate definition of ${F, f}$) Show that ${F(s)}$ is continuously differentiable except at ${s=1}$, and ${f(s)}$ is continuously differentiable except at ${s=2}$ where it is continuous, obeying the delay-differential equations

$\displaystyle \frac{d}{ds}( s F(s) ) = f(s-1) \ \ \ \ \ (8)$

for ${s > 1}$ and

$\displaystyle \frac{d}{ds}( s f(s) ) = F(s-1) \ \ \ \ \ (9)$

for ${s>2}$, with the initial conditions

$\displaystyle F(s) = \frac{2e^\gamma}{s} 1_{s>1}$

for ${s \leq 3}$ and

$\displaystyle f(s) = 0$

for ${s \leq 2}$. Show that these properties of ${F, f}$ determine ${F, f}$ completely.

For future reference, we record the following explicit values of ${F, f}$:

$\displaystyle F(s) = \frac{2e^\gamma}{s} \ \ \ \ \ (10)$

for ${1 < s \leq 3}$, and

$\displaystyle f(s) = \frac{2e^\gamma}{s} \log(s-1) \ \ \ \ \ (11)$

for ${2 \leq s \leq 4}$.

We will show

Theorem 2 (Linear sieve) Let the notation and hypotheses be as above, with ${s > 1}$. Then, for any ${\varepsilon > 0}$, one has the upper bound

$\displaystyle \sum_{n\not \in\bigcup_{p

and the lower bound

$\displaystyle \sum_{n\not \in\bigcup_{p

if ${D}$ is sufficiently large depending on ${\varepsilon, s, c}$. Furthermore, this claim is sharp in the sense that the quantity ${F(s)}$ cannot be replaced by any smaller quantity, and similarly ${f(s)}$ cannot be replaced by any larger quantity.

Comparing the linear sieve with the fundamental lemma (and also testing using the sequence ${a_n = 1_{1 \leq n \leq N}}$ for some extremely large ${N}$), we conclude that we necessarily have the asymptotics

$\displaystyle 1 - O(e^{-s}) \leq f(s) \leq 1 \leq F(s) \leq 1 + O( e^{-s} )$

for all ${s \geq 1}$; this can also be proven directly from the definitions of ${F, f}$, or from Exercise 1, but is somewhat challenging to do so; see e.g. Chapter 11 of Friedlander-Iwaniec for details.

Exercise 3 Establish the integral identities

$\displaystyle F(s) = 1 + \frac{1}{s} \int_s^\infty (1 - f(t-1))\ dt$

and

$\displaystyle f(s) = 1 + \frac{1}{s} \int_s^\infty (1 - F(t-1))\ dt$

for ${s \geq 2}$. Argue heuristically that these identities are consistent with the bounds in Theorem 2 and the Buchstab identity (Equation (16) from Notes 4).

Exercise 4 Use the Selberg sieve (Theorem 30 from Notes 4) to obtain a slightly weaker version of (12) in the range ${1 < s < 3}$ in which the error term ${|r_d|}$ is worsened to ${\tau_3(d) |r_d|}$, but the main term is unchanged.

We will prove Theorem 2 below the fold. The optimality of ${F, f}$ is closely related to the parity problem obstruction discussed in Section 5 of Notes 4; a naive application of the parity arguments there only give the weak bounds ${F(s) \geq \frac{2 e^\gamma}{s}}$ and ${f(s)=0}$ for ${s \leq 2}$, but this can be sharpened by a more careful counting of various sums involving the Liouville function ${\lambda}$.

As an application of the linear sieve (specialised to the ranges in (10), (11)), we will establish a famous theorem of Chen, giving (in some sense) the closest approach to the twin prime conjecture that one can hope to achieve by sieve-theoretic methods:

Theorem 5 (Chen’s theorem) There are infinitely many primes ${p}$ such that ${p+2}$ is the product of at most two primes.

The same argument gives the version of Chen’s theorem for the even Goldbach conjecture, namely that for all sufficiently large even ${N}$, there exists a prime ${p}$ between ${2}$ and ${N}$ such that ${N-p}$ is the product of at most two primes.

The discussion in these notes loosely follows that of Friedlander-Iwaniec (who study sieving problems in more general dimension than ${\kappa=1}$).

Many problems in non-multiplicative prime number theory can be recast as sieving problems. Consider for instance the problem of counting the number ${N(x)}$ of pairs of twin primes ${p,p+2}$ contained in ${[x/2,x]}$ for some large ${x}$; note that the claim that ${N(x) > 0}$ for arbitrarily large ${x}$ is equivalent to the twin prime conjecture. One can obtain this count by any of the following variants of the sieve of Eratosthenes:

1. Let ${A}$ be the set of natural numbers in ${[x/2,x-2]}$. For each prime ${p \leq \sqrt{x}}$, let ${E_p}$ be the union of the residue classes ${0\ (p)}$ and ${-2\ (p)}$. Then ${N(x)}$ is the cardinality of the sifted set ${A \backslash \bigcup_{p \leq \sqrt{x}} E_p}$.
2. Let ${A}$ be the set of primes in ${[x/2,x-2]}$. For each prime ${p \leq \sqrt{x}}$, let ${E_p}$ be the residue class ${-2\ (p)}$. Then ${N(x)}$ is the cardinality of the sifted set ${A \backslash \bigcup_{p \leq \sqrt{x}} E_p}$.
3. Let ${A}$ be the set of primes in ${[x/2+2,x]}$. For each prime ${p \leq \sqrt{x}}$, let ${E_p}$ be the residue class ${2\ (p)}$. Then ${N(x)}$ is the cardinality of the sifted set ${A \backslash \bigcup_{p \leq \sqrt{x}} E_p}$.
4. Let ${A}$ be the set ${\{ n(n+2): x/2 \leq n \leq x-2 \}}$. For each prime ${p \leq \sqrt{x}}$, let ${E_p}$ be the residue class ${0\ (p)}$ Then ${N(x)}$ is the cardinality of the sifted set ${A \backslash \bigcup_{p \leq \sqrt{x}} E_p}$.

Exercise 1 Develop similar sifting formulations of the other three Landau problems.

In view of these sieving interpretations of number-theoretic problems, it becomes natural to try to estimate the size of sifted sets ${A \backslash \bigcup_{p | P} E_p}$ for various finite sets ${A}$ of integers, and subsets ${E_p}$ of integers indexed by primes ${p}$ dividing some squarefree natural number ${P}$ (which, in the above examples, would be the product of all primes up to ${\sqrt{x}}$). As we see in the above examples, the sets ${E_p}$ in applications are typically the union of one or more residue classes modulo ${p}$, but we will work at a more abstract level of generality here by treating ${E_p}$ as more or less arbitrary sets of integers, without caring too much about the arithmetic structure of such sets.

It turns out to be conceptually more natural to replace sets by functions, and to consider the more general the task of estimating sifted sums

$\displaystyle \sum_{n \in {\bf Z}} a_n 1_{n \not \in \bigcup_{p | P} E_p} \ \ \ \ \ (1)$

for some finitely supported sequence ${(a_n)_{n \in {\bf Z}}}$ of non-negative numbers; the previous combinatorial sifting problem then corresponds to the indicator function case ${a_n=1_{n \in A}}$. (One could also use other index sets here than the integers ${{\bf Z}}$ if desired; for much of sieve theory the index set and its subsets ${E_p}$ are treated as abstract sets, so the exact arithmetic structure of these sets is not of primary importance.)

Continuing with twin primes as a running example, we thus have the following sample sieving problem:

Problem 2 (Sieving problem for twin primes) Let ${x, z \geq 1}$, and let ${\pi_2(x,z)}$ denote the number of natural numbers ${n \leq x}$ which avoid the residue classes ${0, -2\ (p)}$ for all primes ${p < z}$. In other words, we have

$\displaystyle \pi_2(x,z) := \sum_{n \in {\bf Z}} a_n 1_{n \not \in \bigcup_{p | P(z)} E_p}$

where ${a_n := 1_{n \in [1,x]}}$, ${P(z) := \prod_{p < z} p}$ is the product of all the primes strictly less than ${z}$ (we omit ${z}$ itself for minor technical reasons), and ${E_p}$ is the union of the residue classes ${0, -2\ (p)}$. Obtain upper and lower bounds on ${\pi_2(x,z)}$ which are as strong as possible in the asymptotic regime where ${x}$ goes to infinity and the sifting level ${z}$ grows with ${x}$ (ideally we would like ${z}$ to grow as fast as ${\sqrt{x}}$).

From the preceding discussion we know that the number of twin prime pairs ${p,p+2}$ in ${(x/2,x]}$ is equal to ${\pi_2(x-2,\sqrt{x}) - \pi_2(x/2,\sqrt{x})}$, if ${x}$ is not a perfect square; one also easily sees that the number of twin prime pairs in ${[1,x]}$ is at least ${\pi_2(x-2,\sqrt{x})}$, again if ${x}$ is not a perfect square. Thus we see that a sufficiently good answer to Problem 2 would resolve the twin prime conjecture, particularly if we can get the sifting level ${z}$ to be as large as ${\sqrt{x}}$.

We return now to the general problem of estimating (1). We may expand

$\displaystyle 1_{n \not \in \bigcup_{p | P} E_p} = \prod_{p | P} (1 - 1_{E_p}(n)) \ \ \ \ \ (2)$

$\displaystyle = \sum_{k=0}^\infty (-1)^k \sum_{p_1 \dots p_k|P: p_1 < \dots < p_k} 1_{E_{p_1}} \dots 1_{E_{p_k}}(n)$

$\displaystyle = \sum_{d|P} \mu(d) 1_{E_d}(n)$

where ${E_d := \bigcap_{p|d} E_p}$ (with the convention that ${E_1={\bf Z}}$). We thus arrive at the Legendre sieve identity

$\displaystyle \sum_{n \in {\bf Z}} a_n 1_{n \not \in \bigcup_{p | P} E_p} = \sum_{d|P} \mu(d) \sum_{n \in E_d} a_n. \ \ \ \ \ (3)$

Specialising to the case of an indicator function ${a_n=1_{n \in A}}$, we recover the inclusion-exclusion formula

$\displaystyle |A \backslash \bigcup_{p|P} E_p| = \sum_{d|P} \mu(d) |A \cap E_d|.$

Such exact sieving formulae are already satisfactory for controlling sifted sets or sifted sums when the amount of sieving is relatively small compared to the size of ${A}$. For instance, let us return to the running example in Problem 2 for some ${x,z \geq 1}$. Observe that each ${E_p}$ in this example consists of ${\omega(p)}$ residue classes modulo ${p}$, where ${\omega(p)}$ is defined to equal ${1}$ when ${p=2}$ and ${2}$ when ${p}$ is odd. By the Chinese remainder theorem, this implies that for each ${d|P(z)}$, ${E_d}$ consists of ${\prod_{p|d} \omega(p)}$ residue classes modulo ${d}$. Using the basic bound

$\displaystyle \sum_{n \leq x: n = a\ (q)} 1 = \frac{x}{q} + O(1) \ \ \ \ \ (4)$

for any ${x > 0}$ and any residue class ${a\ (q)}$, we conclude that

$\displaystyle \sum_{n \in E_d} a_n = g(d) x + O( \prod_{p|d} \omega(p) ) \ \ \ \ \ (5)$

for any ${d|P(z)}$, where ${g}$ is the multiplicative function

$\displaystyle g(d) := \prod_{p|d: p|P(z)} \frac{\omega(p)}{p}.$

Since ${\omega(p) \leq 2}$ and there are at most ${\pi(z)}$ primes dividing ${P(z)}$, we may crudely bound ${\prod_{p|d} \omega(p) \leq 2^{\pi(z)}}$, thus

$\displaystyle \sum_{n \in E_d} a_n = g(d) x + O( 2^{\pi(z)} ). \ \ \ \ \ (6)$

Also, the number of divisors of ${P(z)}$ is at most ${2^{\pi(z)}}$. From the Legendre sieve (3), we thus conclude that

$\displaystyle \pi_2(x,z) = (\sum_{d|P(z)} \mu(d) g(d) x) + O( 4^{\pi(z)} ).$

We can factorise the main term to obtain

$\displaystyle \pi_2(x,z) = x \prod_{p < z} (1-\frac{\omega(p)}{p}) + O( 4^{\pi(z)} ).$

This is compatible with the heuristic

$\displaystyle \pi_2(x,z) \approx x \prod_{p < z} (1-\frac{\omega(p)}{p}) \ \ \ \ \ (7)$

coming from the equidistribution of residues principle (Section 3 of Supplement 4), bearing in mind (from the modified Cramér model, see Section 1 of Supplement 4) that we expect this heuristic to become inaccurate when ${z}$ becomes very large. We can simplify the right-hand side of (7) by recalling the twin prime constant

$\displaystyle \Pi_2 := \prod_{p>2} (1 - \frac{1}{(p-1)^2}) = 0.6601618\dots$

(see equation (7) from Supplement 4); note that

$\displaystyle \prod_p (1-\frac{1}{p})^{-2} (1-\frac{\omega(p)}{p}) = 2 \Pi_2$

so from Mertens’ third theorem (Theorem 42 from Notes 1) one has

$\displaystyle \prod_{p < z} (1-\frac{\omega(p)}{p}) = (2\Pi_2+o(1)) \frac{1}{(e^\gamma \log z)^2} \ \ \ \ \ (8)$

as ${z \rightarrow \infty}$. Bounding ${4^{\pi(z)}}$ crudely by ${\exp(o(z))}$, we conclude in particular that

$\displaystyle \pi_2(x,z) = (2\Pi_2 +o(1)) \frac{x}{(e^\gamma \log z)^2}$

when ${x,z \rightarrow \infty}$ with ${z = O(\log x)}$. This is somewhat encouraging for the purposes of getting a sufficiently good answer to Problem 2 to resolve the twin prime conjecture, but note that ${z}$ is currently far too small: one needs to get ${z}$ as large as ${\sqrt{x}}$ before one is counting twin primes, and currently ${z}$ can only get as large as ${\log x}$.

The problem is that the number of terms in the Legendre sieve (3) basically grows exponentially in ${z}$, and so the error terms in (4) accumulate to an unacceptable extent once ${z}$ is significantly larger than ${\log x}$. An alternative way to phrase this problem is that the estimate (4) is only expected to be truly useful in the regime ${q=o(x)}$; on the other hand, the moduli ${d}$ appearing in (3) can be as large as ${P}$, which grows exponentially in ${z}$ by the prime number theorem.

To resolve this problem, it is thus natural to try to truncate the Legendre sieve, in such a way that one only uses information about the sums ${\sum_{n \in E_d} a_n}$ for a relatively small number of divisors ${d}$ of ${P}$, such as those ${d}$ which are below a certain threshold ${D}$. This leads to the following general sieving problem:

Problem 3 (General sieving problem) Let ${P}$ be a squarefree natural number, and let ${{\mathcal D}}$ be a set of divisors of ${P}$. For each prime ${p}$ dividing ${P}$, let ${E_p}$ be a set of integers, and define ${E_d := \bigcap_{p|d} E_p}$ for all ${d|P}$ (with the convention that ${E_1={\bf Z}}$). Suppose that ${(a_n)_{n \in {\bf Z}}}$ is an (unknown) finitely supported sequence of non-negative reals, whose sums

$\displaystyle X_d := \sum_{n \in E_d} a_n \ \ \ \ \ (9)$

are known for all ${d \in {\mathcal D}}$. What are the best upper and lower bounds one can conclude on the quantity (1)?

Here is a simple example of this type of problem (corresponding to the case ${P = 6}$, ${{\mathcal D} = \{1, 2, 3\}}$, ${X_1 = 100}$, ${X_2 = 60}$, and ${X_3 = 10}$):

Exercise 4 Let ${(a_n)_{n \in {\bf Z}}}$ be a finitely supported sequence of non-negative reals such that ${\sum_{n \in {\bf Z}} a_n = 100}$, ${\sum_{n \in {\bf Z}: 2|n} a_n = 60}$, and ${\sum_{n \in {\bf Z}: 3|n} a_n = 10}$. Show that

$\displaystyle 30 \leq \sum_{n \in {\bf Z}: (n,6)=1} a_n \leq 40$

and give counterexamples to show that these bounds cannot be improved in general, even when ${a_n}$ is an indicator function sequence.

Problem 3 is an example of a linear programming problem. By using linear programming duality (as encapsulated by results such as the Hahn-Banach theorem, the separating hyperplane theorem, or the Farkas lemma), we can rephrase the above problem in terms of upper and lower bound sieves:

Theorem 5 (Dual sieve problem) Let ${P, {\mathcal D}, E_p, E_d, X_d}$ be as in Problem 3. We assume that Problem 3 is feasible, in the sense that there exists at least one finitely supported sequence ${(a_n)_{n \in {\bf Z}}}$ of non-negative reals obeying the constraints in that problem. Define an (normalised) upper bound sieve to be a function ${\nu^+: {\bf Z} \rightarrow {\bf R}}$ of the form

$\displaystyle \nu^+ = \sum_{d \in {\mathcal D}} \lambda^+_d 1_{E_d}$

for some coefficients ${\lambda^+_d \in {\bf R}}$, and obeying the pointwise lower bound

$\displaystyle \nu^+(n) \geq 1_{n \not \in\bigcup_{p|P} E_p}(n) \ \ \ \ \ (10)$

for all ${n \in {\bf Z}}$ (in particular ${\nu^+}$ is non-negative). Similarly, define a (normalised) lower bound sieve to be a function ${\nu^-: {\bf Z} \rightarrow {\bf R}}$ of the form

$\displaystyle \nu^-(n) = \sum_{d \in {\mathcal D}} \lambda^-_d 1_{E_d}$

for some coefficients ${\lambda^-_d \in {\bf R}}$, and obeying the pointwise upper bound

$\displaystyle \nu^-(n) \leq 1_{n \not \in\bigcup_{p|P} E_p}(n)$

for all ${n \in {\bf Z}}$. Thus for instance ${1}$ and ${0}$ are (trivially) upper bound sieves and lower bound sieves respectively.

• (i) The supremal value of the quantity (1), subject to the constraints in Problem 3, is equal to the infimal value of the quantity ${\sum_{d \in {\mathcal D}} \lambda^+_d X_d}$, as ${\nu^+ = \sum_{d \in {\mathcal D}} \lambda^+_d 1_{E_d}}$ ranges over all upper bound sieves.
• (ii) The infimal value of the quantity (1), subject to the constraints in Problem 3, is equal to the supremal value of the quantity ${\sum_{d \in {\mathcal D}} \lambda^-_d X_d}$, as ${\nu^- = \sum_{d \in {\mathcal D}} \lambda^-_d 1_{E_d}}$ ranges over all lower bound sieves.

Proof: We prove part (i) only, and leave part (ii) as an exercise. Let ${A}$ be the supremal value of the quantity (1) given the constraints in Problem 3, and let ${B}$ be the infimal value of ${\sum_{d \in {\mathcal D}} \lambda^+_d X_d}$. We need to show that ${A=B}$.

We first establish the easy inequality ${A \leq B}$. If the sequence ${a_n}$ obeys the constraints in Problem 3, and ${\nu^+ = \sum_{d \in {\mathcal D}} \lambda^+_d 1_{E_d}}$ is an upper bound sieve, then

$\displaystyle \sum_n \nu^+(n) a_n = \sum_{d \in {\mathcal D}} \lambda^+_d X_d$

and hence (by the non-negativity of ${\nu^+}$ and ${a_n}$)

$\displaystyle \sum_{n \not \in \bigcup_{p|P} E_p} a_n \leq \sum_{d \in {\mathcal D}} \lambda^+_d X_d;$

taking suprema in ${f}$ and infima in ${\nu^+}$ we conclude that ${A \leq B}$.

Now suppose for contradiction that ${A, thus ${A < C < B}$ for some real number ${C}$. We will argue using the hyperplane separation theorem; one can also proceed using one of the other duality results mentioned above. (See this previous blog post for some discussion of the connections between these various forms of linear duality.) Consider the affine functional

$\displaystyle \rho_0: (a_n)_{n \in{\bf Z}} \mapsto C - \sum_{n \not \in \bigcup_{p|P} E_p} a_n.$

on the vector space of finitely supported sequences ${(a_n)_{n \in {\bf Z}}}$ of reals. On the one hand, since ${C > A}$, this functional is positive for every sequence ${(a_n)_{n \in{\bf Z}}}$ obeying the constraints in Problem 3. Next, let ${K}$ be the space of affine functionals ${\rho}$ of the form

$\displaystyle \rho: (a_n)_{n \in {\bf Z}} \mapsto -\sum_{d \in {\mathcal D}} \lambda^+_d ( \sum_{n \in E_d} a_n - X_d ) + \sum_n a_n \nu(n) + X$

for some real numbers ${\lambda^+_d \in {\bf R}}$, some non-negative function ${\nu: {\bf Z} \rightarrow {\bf R}^+}$ which is a finite linear combination of the ${1_{E_d}}$ for ${d|P}$, and some non-negative ${X}$. This is a closed convex cone in a finite-dimensional vector space ${V}$; note also that ${\rho_0}$ lies in ${V}$. Suppose first that ${\rho_0 \in K}$, thus we have a representation of the form

$\displaystyle C - \sum_{n \not \in \bigcup_{p|P} E_p} a_n = -\sum_{d \in {\mathcal D}} \lambda^+_d ( \sum_{n \in E_d} a_n - X_d ) + \sum_n a_n \nu(n) + X$

for any finitely supported sequence ${(a_n)_{n \in {\bf Z}}}$. Comparing coefficients, we conclude that

$\displaystyle \sum_{d \in {\mathcal D}} \lambda^+_d 1_{E_d}(n) \geq 1_{n \not \in \bigcup_{p|P} E_p}$

for any ${n}$ (i.e., ${\sum_{d \in {\mathcal D}} \lambda^+_d 1_{E_d}}$ is an upper bound sieve), and also

$\displaystyle C \geq \sum_{d \in {\mathcal D}} \lambda^+_d X_d,$

and thus ${C \geq B}$, a contradiction. Thus ${\rho_0}$ lies outside of ${K}$. But then by the hyperplane separation theorem, we can find an affine functional ${\iota: V \rightarrow {\bf R}}$ on ${V}$ that is non-negative on ${K}$ and negative on ${\rho_0}$. By duality, such an affine functional takes the form ${\iota: \rho \mapsto \rho((b_n)_{n \in {\bf Z}}) + c}$ for some finitely supported sequence ${(b_n)_{n \in {\bf Z}}}$ and ${c \in {\bf R}}$ (indeed, ${(b_n)_{n \in {\bf Z}}}$ can be supported on a finite set consisting of a single representative for each atom of the finite ${\sigma}$-algebra generated by the ${E_p}$). Since ${\iota}$ is non-negative on the cone ${K}$, we see (on testing against multiples of the functionals ${(a_n)_{n \in {\bf Z}} \mapsto \sum_{n \in E_d} a_n - X_d}$ or ${(a_n)_{n \in {\bf Z}} \mapsto a_n}$) that the ${b_n}$ and ${c}$ are non-negative, and that ${\sum_{n \in E_d} b_n - X_d = 0}$ for all ${d \in {\mathcal D}}$; thus ${(b_n)_{n \in {\bf Z}}}$ is feasible for Problem 3. Since ${\iota}$ is negative on ${\rho_0}$, we see that

$\displaystyle \sum_{n \not \in \bigcup_{p|P} E_p} b_n \geq C$

and thus ${A \geq C}$, giving the desired contradiction. $\Box$

Exercise 6 Prove part (ii) of the above theorem.

Exercise 7 Show that the infima and suprema in the above theorem are actually attained (so one can replace “infimal” and “supremal” by “minimal” and “maximal” if desired).

Exercise 8 What are the optimal upper and lower bound sieves for Exercise 4?

In the case when ${{\mathcal D}}$ consists of all the divisors of ${P}$, we see that the Legendre sieve ${\sum_{d|P} \mu(d) 1_{E_d}}$ is both the optimal upper bound sieve and the optimal lower bound sieve, regardless of what the quantities ${X_d}$ are. However, in most cases of interest, ${{\mathcal D}}$ will only be some strict subset of the divisors of ${P}$, and there will be a gap between the optimal upper and lower bounds.

Observe that a sequence ${(\lambda^+_d)_{d \in {\mathcal D}}}$ of real numbers will form an upper bound sieve ${\sum_d \lambda^+_d 1_{E_d}}$ if one has the inequalities

$\displaystyle \lambda^+_1 \geq 1$

and

$\displaystyle \sum_{d|n} \lambda^+_d \geq 0$

for all ${n|P}$; we will refer to such sequences as upper bound sieve coefficients. (Conversely, if the sets ${E_p}$ are in “general position” in the sense that every set of the form ${\bigcap_{p|n} E_p \backslash \bigcup_{p|P; p\not | n} E_p}$ for ${n|P}$ is non-empty, we see that every upper bound sieve arises from a sequence of upper bound sieve coefficients.) Similarly, a sequence ${(\lambda^-_d)_{d \in {\mathcal D}}}$ of real numbers will form a lower bound sieve ${\sum_d \lambda^-_d 1_{E_d}}$ if one has the inequalities

$\displaystyle \lambda^-_1 \leq 1$

and

$\displaystyle \sum_{d|n} \lambda^-_d \leq 0$

for all ${n|P}$ with ${n>1}$; we will refer to such sequences as lower bound sieve coefficients.

Exercise 9 (Brun pure sieve) Let ${P}$ be a squarefree number, and ${k}$ a non-negative integer. Show that the sequence ${(\lambda_d)_{d \in P}}$ defined by

$\displaystyle \lambda_d := 1_{\omega(d) \leq k} \mu(d),$

where ${\omega(d)}$ is the number of prime factors of ${d}$, is a sequence of upper bound sieve coefficients for even ${k}$, and a sequence of lower bound sieve coefficients for odd ${k}$. Deduce the Bonferroni inequalities

$\displaystyle \sum_{n \in {\bf Z}} a_n 1_{n \not \in \bigcup_{p | P} E_p} \leq \sum_{d|P: \omega(p) \leq k} \mu(d) X_d \ \ \ \ \ (11)$

when ${k}$ is even, and

$\displaystyle \sum_{n \in {\bf Z}} a_n 1_{n \not \in \bigcup_{p | P} E_p} \geq \sum_{d|P: \omega(p) \leq k} \mu(d) X_d \ \ \ \ \ (12)$

when ${k}$ is odd, whenever one is in the situation of Problem 3 (and ${{\mathcal D}}$ contains all ${d|P}$ with ${\omega(p) \leq k}$). The resulting upper and lower bound sieves are sometimes known as Brun pure sieves. The Legendre sieve can be viewed as the limiting case when ${k \geq \omega(P)}$.

In many applications the sums ${X_d}$ in (9) take the form

$\displaystyle \sum_{n \in E_d} a_n = g(d) X + r_d \ \ \ \ \ (13)$

for some quantity ${X}$ independent of ${d}$, some multiplicative function ${g}$ with ${0 \leq g(p) \leq 1}$, and some remainder term ${r_d}$ whose effect is expected to be negligible on average if ${d}$ is restricted to be small, e.g. less than a threshold ${D}$; note for instance that (5) is of this form if ${D \leq x^{1-\varepsilon}}$ for some fixed ${\varepsilon>0}$ (note from the divisor bound, Lemma 23 of Notes 1, that ${\prod_{p|d} \omega(p) \ll x^{o(1)}}$ if ${d \ll x^{O(1)}}$). We are thus led to the following idealisation of the sieving problem, in which the remainder terms ${r_d}$ are ignored:

Problem 10 (Idealised sieving) Let ${z, D \geq 1}$ (we refer to ${z}$ as the sifting level and ${D}$ as the level of distribution), let ${g}$ be a multiplicative function with ${0 \leq g(p) \leq 1}$, and let ${{\mathcal D} := \{ d|P(z): d \leq D \}}$. How small can one make the quantity

$\displaystyle \sum_{d \in {\mathcal D}} \lambda^+_d g(d) \ \ \ \ \ (14)$

for a sequence ${(\lambda^+_d)_{d \in {\mathcal D}}}$ of upper bound sieve coefficients, and how large can one make the quantity

$\displaystyle \sum_{d \in {\mathcal D}} \lambda^-_d g(d) \ \ \ \ \ (15)$

for a sequence ${(\lambda^-_d)_{d \in {\mathcal D}}}$ of lower bound sieve coefficients?

Thus, for instance, the trivial upper bound sieve ${\lambda^+_d := 1_{d=1}}$ and the trivial lower bound sieve ${\lambda^-_d := 0}$ show that (14) can equal ${1}$ and (15) can equal ${0}$. Of course, one hopes to do better than these trivial bounds in many situations; usually one can improve the upper bound quite substantially, but improving the lower bound is significantly more difficult, particularly when ${z}$ is large compared with ${D}$.

If the remainder terms ${r_d}$ in (13) are indeed negligible on average for ${d \leq D}$, then one expects the upper and lower bounds in Problem 3 to essentially be the optimal bounds in (14) and (15) respectively, multiplied by the normalisation factor ${X}$. Thus Problem 10 serves as a good model problem for Problem 3, in which all the arithmetic content of the original sieving problem has been abstracted into two parameters ${z,D}$ and a multiplicative function ${g}$. In many applications, ${g(p)}$ will be approximately ${\kappa/p}$ on the average for some fixed ${\kappa>0}$, known as the sieve dimension; for instance, in the twin prime sieving problem discussed above, the sieve dimension is ${2}$. The larger one makes the level of distribution ${D}$ compared to ${z}$, the more choices one has for the upper and lower bound sieves; it is thus of interest to obtain equidistribution estimates such as (13) for ${d}$ as large as possible. When the sequence ${a_d}$ is of arithmetic origin (for instance, if it is the von Mangoldt function ${\Lambda}$), then estimates such as the Bombieri-Vinogradov theorem, Theorem 17 from Notes 3, turn out to be particularly useful in this regard; in other contexts, the required equidistribution estimates might come from other sources, such as homogeneous dynamics, or the theory of expander graphs (the latter arises in the recent theory of the affine sieve, discussed in this previous blog post). However, the sieve-theoretic tools developed in this post are not particularly sensitive to how a certain level of distribution is attained, and are generally content to use sieve axioms such as (13) as “black boxes”.

In some applications one needs to modify Problem 10 in various technical ways (e.g. in altering the product ${P(z)}$, the set ${{\mathcal D}}$, or the definition of an upper or lower sieve coefficient sequence), but to simplify the exposition we will focus on the above problem without such alterations.

As the exercise below (or the heuristic (7)) suggests, the “natural” size of (14) and (15) is given by the quantity ${V(z) := \prod_{p < z} (1 - g(p))}$ (so that the natural size for Problem 3 is ${V(z) X}$):

Exercise 11 Let ${z,D,g}$ be as in Problem 10, and set ${V(z) := \prod_{p \leq z} (1 - g(p))}$.

• (i) Show that the quantity (14) is always at least ${V(z)}$ when ${(\lambda^+_d)_{d \in {\mathcal D}}}$ is a sequence of upper bound sieve coefficients. Similarly, show that the quantity (15) is always at most ${V(z)}$ when ${(\lambda^-_d)_{d \in {\mathcal D}}}$ is a sequence of lower bound sieve coefficients. (Hint: compute the expected value of ${\sum_{d|n} \lambda^\pm_d}$ when ${n}$ is a random factor of ${P(z)}$ chosen according to a certain probability distribution depending on ${g}$.)
• (ii) Show that (14) and (15) can both attain the value of ${V(z)}$ when ${D \geq P(z)}$. (Hint: translate the Legendre sieve to this setting.)

The problem of finding good sequences of upper and lower bound sieve coefficients in order to solve problems such as Problem 10 is one of the core objectives of sieve theory, and has been intensively studied. This is more of an optimisation problem rather than a genuinely number theoretic problem; however, the optimisation problem is sufficiently complicated that it has not been solved exactly or even asymptotically, except in a few special cases. (It can be reduced to a optimisation problem involving multilinear integrals of certain unknown functions of several variables, but this problem is rather difficult to analyse further; see these lecture notes of Selberg for further discussion.) But while we do not yet have a definitive solution to this problem in general, we do have a number of good general-purpose upper and lower bound sieve coefficients that give fairly good values for (14), (15), often coming within a constant factor of the idealised value ${V(z)}$, and which work well for sifting levels ${z}$ as large as a small power of the level of distribution ${D}$. Unfortunately, we also know of an important limitation to the sieve, known as the parity problem, that prevents one from taking ${z}$ as large as ${D^{1/2}}$ while still obtaining non-trivial lower bounds; as a consequence, sieve theory is not able, on its own, to sift out primes for such purposes as establishing the twin prime conjecture. However, it is still possible to use these sieves, in conjunction with additional tools, to produce various types of primes or prime patterns in some cases; examples of this include the theorem of Ben Green and myself in which an upper bound sieve is used to demonstrate the existence of primes in arbitrarily long arithmetic progressions, or the more recent theorem of Zhang in which (among other things) used an upper bound sieve was used to demonstrate the existence of infinitely many pairs of primes whose difference was bounded. In such arguments, the upper bound sieve was used not so much to count the primes or prime patterns directly, but to serve instead as a sort of “container” to efficiently envelop such prime patterns; when used in such a manner, the upper bound sieves are sometimes known as enveloping sieves. If the original sequence was supported on primes, then the enveloping sieve can be viewed as a “smoothed out indicator function” that is concentrated on almost primes, which in this context refers to numbers with no small prime factors.

In a somewhat different direction, it can be possible in some cases to break the parity barrier by assuming additional equidistribution axioms on the sequence ${a_n}$ than just (13), in particular controlling certain bilinear sums involving ${a_{nm}}$ rather than just linear sums of the ${a_n}$. This approach was in particular pursued by Friedlander and Iwaniec, leading to their theorem that there are infinitely many primes of the form ${n^2+m^4}$.

The study of sieves is an immense topic; see for instance the recent 527-page text by Friedlander and Iwaniec. We will limit attention to two sieves which give good general-purpose results, if not necessarily the most optimal ones:

• (i) The beta sieve (or Rosser-Iwaniec sieve), which is a modification of the classical combinatorial sieve of Brun. (A collection of sieve coefficients ${\lambda_d^{\pm}}$ is called combinatorial if its coefficients lie in ${\{-1,0,+1\}}$.) The beta sieve is a family of upper and lower bound combinatorial sieves, and are particularly useful for efficiently sieving out all primes up to a parameter ${z = x^{1/u}}$ from a set of integers of size ${x}$, in the regime where ${u}$ is moderately large, leading to what is sometimes known as the fundamental lemma of sieve theory.
• (ii) The Selberg upper bound sieve, which is a general-purpose sieve that can serve both as an upper bound sieve for classical sieving problems, as well as an enveloping sieve for sets such as the primes. (One can also convert the Selberg upper bound sieve into a lower bound sieve in a number of ways, but we will only touch upon this briefly.) A key advantage of the Selberg sieve is that, due to the “quadratic” nature of the sieve, the difficult optimisation problem in Problem 10 is replaced with a much more tractable quadratic optimisation problem, which can often be solved for exactly.

Remark 12 It is possible to compose two sieves together, for instance by using the observation that the product of two upper bound sieves is again an upper bound sieve, or that the product of an upper bound sieve and a lower bound sieve is a lower bound sieve. Such a composition of sieves is useful in some applications, for instance if one wants to apply the fundamental lemma as a “preliminary sieve” to sieve out small primes, but then use a more precise sieve like the Selberg sieve to sieve out medium primes. We will see an example of this in later notes, when we discuss the linear beta-sieve.

We will also briefly present the (arithmetic) large sieve, which gives a rather different approach to Problem 3 in the case that each ${E_p}$ consists of some number (typically a large number) of residue classes modulo ${p}$, and is powered by the (analytic) large sieve inequality of the preceding section. As an application of these methods, we will utilise the Selberg upper bound sieve as an enveloping sieve to establish Zhang’s theorem on bounded gaps between primes. Finally, we give an informal discussion of the parity barrier which gives some heuristic limitations on what sieve theory is able to accomplish with regards to counting prime patters such as twin primes.

These notes are only an introduction to the vast topic of sieve theory; more detailed discussion can be found in the Friedlander-Iwaniec text, in these lecture notes of Selberg, and in many further texts.

Many problems and results in analytic prime number theory can be formulated in the following general form: given a collection of (affine-)linear forms ${L_1(n),\dots,L_k(n)}$, none of which is a multiple of any other, find a number ${n}$ such that a certain property ${P( L_1(n),\dots,L_k(n) )}$ of the linear forms ${L_1(n),\dots,L_k(n)}$ are true. For instance:

• For the twin prime conjecture, one can use the linear forms ${L_1(n) := n}$, ${L_2(n) := n+2}$, and the property ${P( L_1(n), L_2(n) )}$ in question is the assertion that ${L_1(n)}$ and ${L_2(n)}$ are both prime.
• For the even Goldbach conjecture, the claim is similar but one uses the linear forms ${L_1(n) := n}$, ${L_2(n) := N-n}$ for some even integer ${N}$.
• For Chen’s theorem, we use the same linear forms ${L_1(n),L_2(n)}$ as in the previous two cases, but now ${P(L_1(n), L_2(n))}$ is the assertion that ${L_1(n)}$ is prime and ${L_2(n)}$ is an almost prime (in the sense that there are at most two prime factors).
• In the recent results establishing bounded gaps between primes, we use the linear forms ${L_i(n) = n + h_i}$ for some admissible tuple ${h_1,\dots,h_k}$, and take ${P(L_1(n),\dots,L_k(n))}$ to be the assertion that at least two of ${L_1(n),\dots,L_k(n)}$ are prime.

For these sorts of results, one can try a sieve-theoretic approach, which can broadly be formulated as follows:

1. First, one chooses a carefully selected sieve weight ${\nu: {\bf N} \rightarrow {\bf R}^+}$, which could for instance be a non-negative function having a divisor sum form

$\displaystyle \nu(n) := \sum_{d_1|L_1(n), \dots, d_k|L_k(n); d_1 \dots d_k \leq x^{1-\varepsilon}} \lambda_{d_1,\dots,d_k}$

for some coefficients ${\lambda_{d_1,\dots,d_k}}$, where ${x}$ is a natural scale parameter. The precise choice of sieve weight is often quite a delicate matter, but will not be discussed here. (In some cases, one may work with multiple sieve weights ${\nu_1, \nu_2, \dots}$.)

2. Next, one uses tools from analytic number theory (such as the Bombieri-Vinogradov theorem) to obtain upper and lower bounds for sums such as

$\displaystyle \sum_n \nu(n) \ \ \ \ \ (1)$

or

$\displaystyle \sum_n \nu(n) 1_{L_i(n) \hbox{ prime}} \ \ \ \ \ (2)$

or more generally of the form

$\displaystyle \sum_n \nu(n) f(L_i(n)) \ \ \ \ \ (3)$

where ${f(L_i(n))}$ is some “arithmetic” function involving the prime factorisation of ${L_i(n)}$ (we will be a bit vague about what this means precisely, but a typical choice of ${f}$ might be a Dirichlet convolution ${\alpha*\beta(L_i(n))}$ of two other arithmetic functions ${\alpha,\beta}$).

3. Using some combinatorial arguments, one manipulates these upper and lower bounds, together with the non-negative nature of ${\nu}$, to conclude the existence of an ${n}$ in the support of ${\nu}$ (or of at least one of the sieve weights ${\nu_1, \nu_2, \dots}$ being considered) for which ${P( L_1(n), \dots, L_k(n) )}$ holds

For instance, in the recent results on bounded gaps between primes, one selects a sieve weight ${\nu}$ for which one has upper bounds on

$\displaystyle \sum_n \nu(n)$

and lower bounds on

$\displaystyle \sum_n \nu(n) 1_{n+h_i \hbox{ prime}}$

so that one can show that the expression

$\displaystyle \sum_n \nu(n) (\sum_{i=1}^k 1_{n+h_i \hbox{ prime}} - 1)$

is strictly positive, which implies the existence of an ${n}$ in the support of ${\nu}$ such that at least two of ${n+h_1,\dots,n+h_k}$ are prime. As another example, to prove Chen’s theorem to find ${n}$ such that ${L_1(n)}$ is prime and ${L_2(n)}$ is almost prime, one uses a variety of sieve weights to produce a lower bound for

$\displaystyle S_1 := \sum_{n \leq x} 1_{L_1(n) \hbox{ prime}} 1_{L_2(n) \hbox{ rough}}$

and an upper bound for

$\displaystyle S_2 := \sum_{z \leq p < x^{1/3}} \sum_{n \leq x} 1_{L_1(n) \hbox{ prime}} 1_{p|L_2(n)} 1_{L_2(n) \hbox{ rough}}$

and

$\displaystyle S_3 := \sum_{n \leq x} 1_{L_1(n) \hbox{ prime}} 1_{L_2(n)=pqr \hbox{ for some } z \leq p \leq x^{1/3} < q \leq r},$

where ${z}$ is some parameter between ${1}$ and ${x^{1/3}}$, and “rough” means that all prime factors are at least ${z}$. One can observe that if ${S_1 - \frac{1}{2} S_2 - \frac{1}{2} S_3 > 0}$, then there must be at least one ${n}$ for which ${L_1(n)}$ is prime and ${L_2(n)}$ is almost prime, since for any rough number ${m}$, the quantity

$\displaystyle 1 - \frac{1}{2} \sum_{z \leq p < x^{1/3}} 1_{p|m} - \frac{1}{2} \sum_{z \leq p \leq x^{1/3} < q \leq r} 1_{m = pqr}$

is only positive when ${m}$ is an almost prime (if ${m}$ has three or more factors, then either it has at least two factors less than ${x^{1/3}}$, or it is of the form ${pqr}$ for some ${p \leq x^{1/3} < q \leq r}$). The upper and lower bounds on ${S_1,S_2,S_3}$ are ultimately produced via asymptotics for expressions of the form (1), (2), (3) for various divisor sums ${\nu}$ and various arithmetic functions ${f}$.

Unfortunately, there is an obstruction to sieve-theoretic techniques working for certain types of properties ${P(L_1(n),\dots,L_k(n))}$, which Zeb Brady and I recently formalised at an AIM workshop this week. To state the result, we recall the Liouville function ${\lambda(n)}$, defined by setting ${\lambda(n) = (-1)^j}$ whenever ${n}$ is the product of exactly ${j}$ primes (counting multiplicity). Define a sign pattern to be an element ${(\epsilon_1,\dots,\epsilon_k)}$ of the discrete cube ${\{-1,+1\}^k}$. Given a property ${P(l_1,\dots,l_k)}$ of ${k}$ natural numbers ${l_1,\dots,l_k}$, we say that a sign pattern ${(\epsilon_1,\dots,\epsilon_k)}$ is forbidden by ${P}$ if there does not exist any natural numbers ${l_1,\dots,l_k}$ obeying ${P(l_1,\dots,l_k)}$ for which

$\displaystyle (\lambda(l_1),\dots,\lambda(l_k)) = (\epsilon_1,\dots,\epsilon_k).$

Example 1 Let ${P(l_1,l_2,l_3)}$ be the property that at least two of ${l_1,l_2,l_3}$ are prime. Then the sign patterns ${(+1,+1,+1)}$, ${(+1,+1,-1)}$, ${(+1,-1,+1)}$, ${(-1,+1,+1)}$ are forbidden, because prime numbers have a Liouville function of ${-1}$, so that ${P(l_1,l_2,l_3)}$ can only occur when at least two of ${\lambda(l_1),\lambda(l_2), \lambda(l_3)}$ are equal to ${-1}$.

Example 2 Let ${P(l_1,l_2)}$ be the property that ${l_1}$ is prime and ${l_2}$ is almost prime. Then the only forbidden sign patterns are ${(+1,+1)}$ and ${(+1,-1)}$.

Example 3 Let ${P(l_1,l_2)}$ be the property that ${l_1}$ and ${l_2}$ are both prime. Then ${(+1,+1), (+1,-1), (-1,+1)}$ are all forbidden sign patterns.

We then have a parity obstruction as soon as ${P}$ has “too many” forbidden sign patterns, in the following (slightly informal) sense:

Claim 1 (Parity obstruction) Suppose ${P(l_1,\dots,l_k)}$ is such that that the convex hull of the forbidden sign patterns of ${P}$ contains the origin. Then one cannot use the above sieve-theoretic approach to establish the existence of an ${n}$ such that ${P(L_1(n),\dots,L_k(n))}$ holds.

Thus for instance, the property in Example 3 is subject to the parity obstruction since ${0}$ is a convex combination of ${(+1,-1)}$ and ${(-1,+1)}$, whereas the properties in Examples 1, 2 are not. One can also check that the property “at least ${j}$ of the ${k}$ numbers ${l_1,\dots,l_k}$ is prime” is subject to the parity obstruction as soon as ${j \geq \frac{k}{2}+1}$. Thus, the largest number of elements of a ${k}$-tuple that one can force to be prime by purely sieve-theoretic methods is ${k/2}$, rounded up.

This claim is not precisely a theorem, because it presumes a certain “Liouville pseudorandomness conjecture” (a very close cousin of the more well known “Möbius pseudorandomness conjecture”) which is a bit difficult to formalise precisely. However, this conjecture is widely believed by analytic number theorists, see e.g. this blog post for a discussion. (Note though that there are scenarios, most notably the “Siegel zero” scenario, in which there is a severe breakdown of this pseudorandomness conjecture, and the parity obstruction then disappears. A typical instance of this is Heath-Brown’s proof of the twin prime conjecture (which would ordinarily be subject to the parity obstruction) under the hypothesis of a Siegel zero.) The obstruction also does not prevent the establishment of an ${n}$ such that ${P(L_1(n),\dots,L_k(n))}$ holds by introducing additional sieve axioms beyond upper and lower bounds on quantities such as (1), (2), (3). The proof of the Friedlander-Iwaniec theorem is a good example of this latter scenario.

Now we give a (slightly nonrigorous) proof of the claim.

Proof: (Nonrigorous) Suppose that the convex hull of the forbidden sign patterns contain the origin. Then we can find non-negative numbers ${p_{\epsilon_1,\dots,\epsilon_k}}$ for sign patterns ${(\epsilon_1,\dots,\epsilon_k)}$, which sum to ${1}$, are non-zero only for forbidden sign patterns, and which have mean zero in the sense that

$\displaystyle \sum_{(\epsilon_1,\dots,\epsilon_k)} p_{\epsilon_1,\dots,\epsilon_k} \epsilon_i = 0$

for all ${i=1,\dots,k}$. By Fourier expansion (or Lagrange interpolation), one can then write ${p_{\epsilon_1,\dots,\epsilon_k}}$ as a polynomial

$\displaystyle p_{\epsilon_1,\dots,\epsilon_k} = 1 + Q( \epsilon_1,\dots,\epsilon_k)$

where ${Q(t_1,\dots,t_k)}$ is a polynomial in ${k}$ variables that is a linear combination of monomials ${t_{i_1} \dots t_{i_r}}$ with ${i_1 < \dots < i_r}$ and ${r \geq 2}$ (thus ${Q}$ has no constant or linear terms, and no monomials with repeated terms). The point is that the mean zero condition allows one to eliminate the linear terms. If we now consider the weight function

$\displaystyle w(n) := 1 + Q( \lambda(L_1(n)), \dots, \lambda(L_k(n)) )$

then ${w}$ is non-negative, is supported solely on ${n}$ for which ${(\lambda(L_1(n)),\dots,\lambda(L_k(n)))}$ is a forbidden pattern, and is equal to ${1}$ plus a linear combination of monomials ${\lambda(L_{i_1}(n)) \dots \lambda(L_{i_r}(n))}$ with ${r \geq 2}$.

The Liouville pseudorandomness principle then predicts that sums of the form

$\displaystyle \sum_n \nu(n) Q( \lambda(L_1(n)), \dots, \lambda(L_k(n)) )$

and

$\displaystyle \sum_n \nu(n) Q( \lambda(L_1(n)), \dots, \lambda(L_k(n)) ) 1_{L_i(n) \hbox{ prime}}$

or more generally

$\displaystyle \sum_n \nu(n) Q( \lambda(L_1(n)), \dots, \lambda(L_k(n)) ) f(L_i(n))$

should be asymptotically negligible; intuitively, the point here is that the prime factorisation of ${L_i(n)}$ should not influence the Liouville function of ${L_j(n)}$, even on the short arithmetic progressions that the divisor sum ${\nu}$ is built out of, and so any monomial ${\lambda(L_{i_1}(n)) \dots \lambda(L_{i_r}(n))}$ occurring in ${Q( \lambda(L_1(n)), \dots, \lambda(L_k(n)) )}$ should exhibit strong cancellation for any of the above sums. If one accepts this principle, then all the expressions (1), (2), (3) should be essentially unchanged when ${\nu(n)}$ is replaced by ${\nu(n) w(n)}$.

Suppose now for sake of contradiction that one could use sieve-theoretic methods to locate an ${n}$ in the support of some sieve weight ${\nu(n)}$ obeying ${P( L_1(n),\dots,L_k(n))}$. Then, by reweighting all sieve weights by the additional multiplicative factor of ${w(n)}$, the same arguments should also be able to locate ${n}$ in the support of ${\nu(n) w(n)}$ for which ${P( L_1(n),\dots,L_k(n))}$ holds. But ${w}$ is only supported on those ${n}$ whose Liouville sign pattern is forbidden, a contradiction. $\Box$

Claim 1 is sharp in the following sense: if the convex hull of the forbidden sign patterns of ${P}$ do not contain the origin, then by the Hahn-Banach theorem (in the hyperplane separation form), there exist real coefficients ${c_1,\dots,c_k}$ such that

$\displaystyle c_1 \epsilon_1 + \dots + c_k \epsilon_k < -c$

for all forbidden sign patterns ${(\epsilon_1,\dots,\epsilon_k)}$ and some ${c>0}$. On the other hand, from Liouville pseudorandomness one expects that

$\displaystyle \sum_n \nu(n) (c_1 \lambda(L_1(n)) + \dots + c_k \lambda(L_k(n)))$

is negligible (as compared against ${\sum_n \nu(n)}$ for any reasonable sieve weight ${\nu}$. We conclude that for some ${n}$ in the support of ${\nu}$, that

$\displaystyle c_1 \lambda(L_1(n)) + \dots + c_k \lambda(L_k(n)) > -c \ \ \ \ \ (4)$

and hence ${(\lambda(L_1(n)),\dots,\lambda(L_k(n)))}$ is not a forbidden sign pattern. This does not actually imply that ${P(L_1(n),\dots,L_k(n))}$ holds, but it does not prevent ${P(L_1(n),\dots,L_k(n))}$ from holding purely from parity considerations. Thus, we do not expect a parity obstruction of the type in Claim 1 to hold when the convex hull of forbidden sign patterns does not contain the origin.

Example 4 Let ${G}$ be a graph on ${k}$ vertices ${\{1,\dots,k\}}$, and let ${P(l_1,\dots,l_k)}$ be the property that one can find an edge ${\{i,j\}}$ of ${G}$ with ${l_i,l_j}$ both prime. We claim that this property is subject to the parity problem precisely when ${G}$ is two-colourable. Indeed, if ${G}$ is two-colourable, then we can colour ${\{1,\dots,k\}}$ into two colours (say, red and green) such that all edges in ${G}$ connect a red vertex to a green vertex. If we then consider the two sign patterns in which all the red vertices have one sign and the green vertices have the opposite sign, these are two forbidden sign patterns which contain the origin in the convex hull, and so the parity problem applies. Conversely, suppose that ${G}$ is not two-colourable, then it contains an odd cycle. Any forbidden sign pattern then must contain more ${+1}$s on this odd cycle than ${-1}$s (since otherwise two of the ${-1}$s are adjacent on this cycle by the pigeonhole principle, and this is not forbidden), and so by convexity any tuple in the convex hull of this sign pattern has a positive sum on this odd cycle. Hence the origin is not in the convex hull, and the parity obstruction does not apply. (See also this previous post for a similar obstruction ultimately coming from two-colourability).

Example 5 An example of a parity-obstructed property (supplied by Zeb Brady) that does not come from two-colourability: we let ${P( l_{\{1,2\}}, l_{\{1,3\}}, l_{\{1,4\}}, l_{\{2,3\}}, l_{\{2,4\}}, l_{\{3,4\}} )}$ be the property that ${l_{A_1},\dots,l_{A_r}}$ are prime for some collection ${A_1,\dots,A_r}$ of pair sets that cover ${\{1,\dots,4\}}$. For instance, this property holds if ${l_{\{1,2\}}, l_{\{3,4\}}}$ are both prime, or if ${l_{\{1,2\}}, l_{\{1,3\}}, l_{\{1,4\}}}$ are all prime, but not if ${l_{\{1,2\}}, l_{\{1,3\}}, l_{\{2,3\}}}$ are the only primes. An example of a forbidden sign pattern is the pattern where ${\{1,2\}, \{2,3\}, \{1,3\}}$ are given the sign ${-1}$, and the other three pairs are given ${+1}$. Averaging over permutations of ${1,2,3,4}$ we see that zero lies in the convex hull, and so this example is blocked by parity. However, there is no sign pattern such that it and its negation are both forbidden, which is another formulation of two-colourability.

Of course, the absence of a parity obstruction does not automatically mean that the desired claim is true. For instance, given an admissible ${5}$-tuple ${h_1,\dots,h_5}$, parity obstructions do not prevent one from establishing the existence of infinitely many ${n}$ such that at least three of ${n+h_1,\dots,n+h_5}$ are prime, however we are not yet able to actually establish this, even assuming strong sieve-theoretic hypotheses such as the generalised Elliott-Halberstam hypothesis. (However, the argument giving (4) does easily give the far weaker claim that there exist infinitely many ${n}$ such that at least three of ${n+h_1,\dots,n+h_5}$ have a Liouville function of ${-1}$.)

Remark 1 Another way to get past the parity problem in some cases is to take advantage of linear forms that are constant multiples of each other (which correlates the Liouville functions to each other). For instance, on GEH we can find two ${E_3}$ numbers (products of exactly three primes) that differ by exactly ${60}$; a direct sieve approach using the linear forms ${n,n+60}$ fails due to the parity obstruction, but instead one can first find ${n}$ such that two of ${n,n+4,n+10}$ are prime, and then among the pairs of linear forms ${(15n,15n+60)}$, ${(6n,6n+60)}$, ${(10n+40,10n+100)}$ one can find a pair of ${E_3}$ numbers that differ by exactly ${60}$. See this paper of Goldston, Graham, Pintz, and Yildirim for more examples of this type.

I thank John Friedlander and Sid Graham for helpful discussions and encouragement.

Two of the most famous open problems in additive prime number theory are the twin prime conjecture and the binary Goldbach conjecture. They have quite similar forms:

• Twin prime conjecture The equation ${p_1 - p_2 = 2}$ has infinitely many solutions with ${p_1,p_2}$ prime.
• Binary Goldbach conjecture The equation ${p_1 + p_2 = N}$ has at least one solution with ${p_1,p_2}$ prime for any given even ${N \geq 4}$.

In view of this similarity, it is not surprising that the partial progress on these two conjectures have tracked each other fairly closely; the twin prime conjecture is generally considered slightly easier than the binary Goldbach conjecture, but broadly speaking any progress made on one of the conjectures has also led to a comparable amount of progress on the other. (For instance, Chen’s theorem has a version for the twin prime conjecture, and a version for the binary Goldbach conjecture.) Also, the notorious parity obstruction is present in both problems, preventing a solution to either conjecture by almost all known methods (see this previous blog post for more discussion).

In this post, I would like to note a divergence from this general principle, with regards to bounded error versions of these two conjectures:

• Twin prime with bounded error The inequalities ${0 < p_1 - p_2 < H}$ has infinitely many solutions with ${p_1,p_2}$ prime for some absolute constant ${H}$.
• Binary Goldbach with bounded error The inequalities ${N \leq p_1+p_2 \leq N+H}$ has at least one solution with ${p_1,p_2}$ prime for any sufficiently large ${N}$ and some absolute constant ${H}$.

The first of these statements is now a well-known theorem of Zhang, and the Polymath8b project hosted on this blog has managed to lower ${H}$ to ${H=246}$ unconditionally, and to ${H=6}$ assuming the generalised Elliott-Halberstam conjecture. However, the second statement remains open; the best result that the Polymath8b project could manage in this direction is that (assuming GEH) at least one of the binary Goldbach conjecture with bounded error, or the twin prime conjecture with no error, had to be true.

All the known proofs of Zhang’s theorem proceed through sieve-theoretic means. Basically, they take as input equidistribution results that control the size of discrepancies such as

$\displaystyle \Delta(f; a\ (q)) := \sum_{x \leq n \leq 2x; n=a\ (q)} f(n) - \frac{1}{\phi(q)} \sum_{x \leq n \leq 2x} f(n) \ \ \ \ \ (1)$

for various congruence classes ${a\ (q)}$ and various arithmetic functions ${f}$, e.g. ${f(n) = \Lambda(n+h_i)}$ (or more generaly ${f(n) = \alpha * \beta(n+h_i)}$ for various ${\alpha,\beta}$). After taking some carefully chosen linear combinations of these discrepancies, and using the trivial positivity lower bound

$\displaystyle a_n \geq 0 \hbox{ for all } n \implies \sum_n a_n \geq 0 \ \ \ \ \ (2)$

one eventually obtains (for suitable ${H}$) a non-trivial lower bound of the form

$\displaystyle \sum_{x \leq n \leq 2x} \nu(n) 1_A(n) > 0$

where ${\nu}$ is some weight function, and ${A}$ is the set of ${n}$ such that there are at least two primes in the interval ${[n,n+H]}$. This implies at least one solution to the inequalities ${0 < p_1 - p_2 < H}$ with ${p_1,p_2 \sim x}$, and Zhang’s theorem follows.

In a similar vein, one could hope to use bounds on discrepancies such as (1) (for ${x}$ comparable to ${N}$), together with the trivial lower bound (2), to obtain (for sufficiently large ${N}$, and suitable ${H}$) a non-trivial lower bound of the form

$\displaystyle \sum_{n \leq N} \nu(n) 1_B(n) > 0 \ \ \ \ \ (3)$

for some weight function ${\nu}$, where ${B}$ is the set of ${n}$ such that there is at least one prime in each of the intervals ${[n,n+H]}$ and ${[N-n-H,n]}$. This would imply the binary Goldbach conjecture with bounded error.

However, the parity obstruction blocks such a strategy from working (for much the same reason that it blocks any bound of the form ${H \leq 4}$ in Zhang’s theorem, as discussed in the Polymath8b paper.) The reason is as follows. The sieve-theoretic arguments are linear with respect to the ${n}$ summation, and as such, any such sieve-theoretic argument would automatically also work in a weighted setting in which the ${n}$ summation is weighted by some non-negative weight ${\omega(n) \geq 0}$. More precisely, if one could control the weighted discrepancies

$\displaystyle \Delta(f\omega; a\ (q)) = \sum_{x \leq n \leq 2x; n=a\ (q)} f(n) \omega(n) - \frac{1}{\phi(q)} \sum_{x \leq n \leq 2x} f(n) \omega(n)$

to essentially the same accuracy as the unweighted discrepancies (1), then thanks to the trivial weighted version

$\displaystyle a_n \geq 0 \hbox{ for all } n \implies \sum_n a_n \omega(n) \geq 0$

of (2), any sieve-theoretic argument that was capable of proving (3) would also be capable of proving the weighted estimate

$\displaystyle \sum_{n \leq N} \nu(n) 1_B(n) \omega(n) > 0. \ \ \ \ \ (4)$

However, (4) may be defeated by a suitable choice of weight ${\omega}$, namely

$\displaystyle \omega(n) := \prod_{i=1}^H (1 + \lambda(n) \lambda(n+i)) \times \prod_{j=0}^H (1 - \lambda(n) \lambda(N-n-j))$

where ${n \mapsto \lambda(n)}$ is the Liouville function, which counts the parity of the number of prime factors of a given number ${n}$. Since ${\lambda(n)^2 = 1}$, one can expand out ${\omega(n)}$ as the sum of ${1}$ and a finite number of other terms, each of which consists of the product of two or more translates (or reflections) of ${\lambda}$. But from the Möbius randomness principle (or its analogue for the Liouville function), such products of ${\lambda}$ are widely expected to be essentially orthogonal to any arithmetic function ${f(n)}$ that is arising from a single multiplicative function such as ${\Lambda}$, even on very short arithmetic progressions. As such, replacing ${1}$ by ${\omega(n)}$ in (1) should have a negligible effect on the discrepancy. On the other hand, in order for ${\omega(n)}$ to be non-zero, ${\lambda(n+i)}$ has to have the same sign as ${\lambda(n)}$ and hence the opposite sign to ${\lambda(N-n-j)}$ cannot simultaneously be prime for any ${0 \leq i,j \leq H}$, and so ${1_B(n) \omega(n)}$ vanishes identically, contradicting (4). This indirectly rules out any modification of the Goldston-Pintz-Yildirim/Zhang method for establishing the binary Goldbach conjecture with bounded error.

The above argument is not watertight, and one could envisage some ways around this problem. One of them is that the Möbius randomness principle could simply be false, in which case the parity obstruction vanishes. A good example of this is the result of Heath-Brown that shows that if there are infinitely many Siegel zeroes (which is a strong violation of the Möbius randomness principle), then the twin prime conjecture holds. Another way around the obstruction is to start controlling the discrepancy (1) for functions ${f}$ that are combinations of more than one multiplicative function, e.g. ${f(n) = \Lambda(n) \Lambda(n+2)}$. However, controlling such functions looks to be at least as difficult as the twin prime conjecture (which is morally equivalent to obtaining non-trivial lower-bounds for ${\sum_{x \leq n \leq 2x} \Lambda(n) \Lambda(n+2)}$). A third option is not to use a sieve-theoretic argument, but to try a different method (e.g. the circle method). However, most other known methods also exhibit linearity in the “${n}$” variable and I would suspect they would be vulnerable to a similar obstruction. (In any case, the circle method specifically has some other difficulties in tackling binary problems, as discussed in this previous post.)

This post is a continuation of the previous post on sieve theory, which is an ongoing part of the Polymath8 project. As the previous post was getting somewhat full, we are rolling the thread over to the current post. We also take the opportunity to correct some errors in the treatment of the truncated GPY sieve from this previous post.

As usual, we let ${x}$ be a large asymptotic parameter, and ${w}$ a sufficiently slowly growing function of ${x}$. Let ${0 < \varpi < 1/4}$ and ${0 < \delta < 1/4+\varpi}$ be such that ${MPZ[\varpi,\delta]}$ holds (see this previous post for a definition of this assertion). We let ${{\mathcal H}}$ be a fixed admissible ${k_0}$-tuple, let ${I := [w,x^\delta]}$, let ${{\mathcal S}_I}$ be the square-free numbers with prime divisors in ${I}$, and consider the truncated GPY sieve

$\displaystyle \nu(n) := \lambda(n)^2$

where

$\displaystyle \lambda(n) := \sum_{d \in {\mathcal S}_I: d|P(n)} \mu(d) g(\frac{\log d}{\log R})$

where ${R := x^{1/4+\varpi}}$, ${P}$ is the polynomial

$\displaystyle P(n) := \prod_{h \in {\mathcal H}} (n+h),$

and ${g: {\bf R} \rightarrow {\bf R}}$ is a fixed smooth function supported on ${[-1,1]}$. As discussed in the previous post, we are interested in obtaining an upper bound of the form

$\displaystyle \sum_{x \leq n \leq 2x} \nu(n) \leq (\alpha+o(1)) (\frac{W}{\phi(W)})^{k_0} \frac{x}{W \log^{k_0} R}$

as well as a lower bound of the form

$\displaystyle \sum_{x \leq n \leq 2x} \nu(n) \theta(n+h) \geq (\beta+o(1)) (\frac{W}{\phi(W)})^{k_0} \frac{x}{W \log^{k_0-1} R}$

for all ${h \in {\mathcal H}}$ (where ${\theta(n) = \log n}$ when ${n}$ is prime and ${\theta(n)=0}$ otherwise), since this will give the conjecture ${DHL[k_0,2]}$ (i.e. infinitely many prime gaps of size at most ${k_0}$) whenever

$\displaystyle 1+4\varpi > \frac{4\alpha}{k_0 \beta}. \ \ \ \ \ (1)$

It turns out we in fact have precise asymptotics

$\displaystyle \sum_{x \leq n \leq 2x} \nu(n) = (\alpha+o(1)) (\frac{W}{\phi(W)})^{k_0} \frac{x}{W \log^{k_0} R} \ \ \ \ \ (2)$

and

$\displaystyle \sum_{x \leq n \leq 2x} \nu(n) \theta(n+h) = (\beta+o(1)) (\frac{W}{\phi(W)})^{k_0} \frac{x}{W \log^{k_0} R} \ \ \ \ \ (3)$

although the exact formulae for ${\alpha,\beta}$ are a little complicated. (The fact that ${\alpha,\beta}$ could be computed exactly was already anticipated in Zhang’s paper; see the remark on page 24.) We proceed as in the previous post. Indeed, from the arguments in that post, (2) is equivalent to

$\displaystyle \sum_{d_1,d_2 \in {\mathcal S}_I} \mu(d_1) g(\frac{\log d_1}{\log R}) \mu(d_2) g(\frac{\log d_2}{\log R}) \frac{k_0^{\Omega([d_1,d_2])}}{[d_1,d_2]} \ \ \ \ \ (4)$

$\displaystyle = (\alpha + o(1)) (\frac{W}{\phi(W)})^{k_0} \log^{-k_0} R$

and (3) is similarly equivalent to

$\displaystyle \sum_{d_1,d_2 \in {\mathcal S}_I} \mu(d_1) g(\frac{\log d_1}{\log R}) \mu(d_2) g(\frac{\log d_2}{\log R}) \frac{(k_0-1)^{\Omega([d_1,d_2])}}{[d_1,d_2]} \ \ \ \ \ (5)$

$\displaystyle = (\beta + o(1)) (\frac{W}{\phi(W)})^{k_0-1} \log^{-k_0+1} R.$

Here ${\Omega(d)}$ is the number of prime factors of ${d}$.

We will work for now with (4), as the treatment of (5) is almost identical.

We would now like to replace the truncated interval ${I = [w,x^\delta]}$ with the untruncated interval ${I \cup J = [w,\infty)}$, where ${J = (x^\delta,\infty)}$. Unfortunately this replacement was not quite done correctly in the previous post, and this will now be corrected here. We first observe that if ${F(d_1,d_2)}$ is any finitely supported function, then by Möbius inversion we have

$\displaystyle \sum_{d_1,d_2 \in {\mathcal S}_I} F(d_1,d_2) = \sum_{d_1,d_2 \in {\mathcal S}_{I \cup J}} F(d_1,d_2) \sum_{a \in {\mathcal S}_J} \mu(a) 1_{a|[d_1,d_2]}.$

Note that ${a|[d_1,d_2]}$ if and only if we have a factorisation ${d_1 = a_1 d'_1}$, ${d_2 = a_2 d'_2}$ with ${[a_1,a_2] = a}$ and ${d'_1 d'_2}$ coprime to ${a_1 a_2}$, and that this factorisation is unique. From this, we see that we may rearrange the previous expression as

$\displaystyle \sum_{a_1,a_2 \in {\mathcal S}_J} \mu( [a_1,a_2] ) \sum_{d'_1,d'_2 \in {\mathcal S}_{I \cup J}: (d'_1 d'_2, a_1 a_2) = 1} F( a_1 d'_1, a_2 d'_2 ).$

Applying this to (4), and relabeling ${d'_1,d'_2}$ as ${d_1,d_2}$, we conclude that the left-hand side of (4) is equal to

$\displaystyle \sum_{a_1,a_2 \in {\mathcal S}_J} \mu( [a_1,a_2] ) \sum_{d_1,d_2 \in {\mathcal S}_{I \cup J}: (d_1d_2,a_1a_2)=1}$

$\displaystyle \mu(a_1d_1) g(\frac{\log a_1d_1}{\log R}) \mu(a_2d_2) g(\frac{\log a_2d_2}{\log R}) \frac{k_0^{\Omega([a_1 d_1,a_2 d_2])}}{[a_1 d_1,a_2 d_2]}$

which may be rearranged as

$\displaystyle \sum_{a_1,a_2 \in {\mathcal S}_J} \frac{\mu( (a_1,a_2) ) k_0^{\Omega([a_1,a_2])}}{[a_1,a_2]} \sum_{d_1,d_2\in {\mathcal S}_{I \cup J}: (d_1d_2,a_1a_2)=1} \ \ \ \ \ (6)$

$\displaystyle \mu(d_1) g(\frac{\log a_1d_1}{\log R}) \mu(d_2) g(\frac{\log a_1 d_2}{\log R}) \frac{k_0^{\Omega([d_1,d_2])}}{[d_1, d_2]}.$

This is almost the same formula that we had in the previous post, except that the Möbius function ${\mu((a_1,a_2))}$ of the greatest common divisor ${(a_1,a_2)}$ of ${a_1,a_2}$ was missing, and also the coprimality condition ${(d_1d_2,a_1a_2)=1}$ was not handled properly in the previous post.

We may now eliminate the condition ${(d_1d_2,a_1a_2)=1}$ as follows. Suppose that there is a prime ${p_* \in J}$ that divides both ${d_1d_2}$ and ${a_1a_2}$. The expression

$\displaystyle \sum_{a_1,a_2 \in {\mathcal S}_J} \frac{k_0^{\Omega([a_1,a_2])}}{[a_1,a_2]} \sum_{d_1,d_2 \in {\mathcal S}_{I \cup J}: p_* | (d_1d_2,a_1a_2)}$

$\displaystyle |g(\frac{\log a_1d_1}{\log R})| |g(\frac{\log a_1 d_2}{\log R})| \frac{k_0^{\Omega([d_1,d_2])}}{[d_1, d_2]}$

can then be bounded by

$\displaystyle \ll \sum_{a_1,a_2} \sum_{d_1,d_2: p_* | (d_1d_2,a_1a_2)} \frac{k_0^{\Omega([a_1,a_2])}}{[a_1,a_2]} \frac{k_0^{\Omega([d_1,d_2])}}{[d_1, d_2]} (a_1 a_2 d_1 d_2)^{-1/\log R}$

which may be factorised as

$\displaystyle \ll \frac{1}{p_*^2} \prod_p (1 + \frac{O(1)}{p^{1+1/\log R}})$

which by Mertens’ theorem (or the simple pole of ${\zeta(s)}$ at ${s=1}$) is

$\displaystyle \ll \frac{\log^{O(1)} R}{p_*^2}.$

Summing over all ${p_* > x^\varpi}$ gives a negligible contribution to (6) for the purposes of (4). Thus we may effectively replace (6) by

$\displaystyle \sum_{a_1,a_2 \in {\mathcal S}_J} \frac{\mu( (a_1,a_2) ) k_0^{\Omega([a_1,a_2])}}{[a_1,a_2]} \sum_{d_1,d_2\in {\mathcal S}_{I \cup J}}$

$\displaystyle \mu(d_1) g(\frac{\log a_1d_1}{\log R}) \mu(d_2) g(\frac{\log a_1 d_2}{\log R}) \frac{k_0^{\Omega([d_1,d_2])}}{[d_1, d_2]}.$

The inner summation can be treated using Proposition 10 of the previous post. We can then reduce (4) to

$\displaystyle \sum_{a_1,a_2 \in {\mathcal S}_J} \frac{\mu( (a_1,a_2) ) k_0^{\Omega([a_1,a_2])}}{[a_1,a_2]} G_{k_0}( \frac{\log a_1}{\log R}, \frac{\log a_2}{\log R} ) = \alpha+o(1) \ \ \ \ \ (7)$

where ${G_{k_0}}$ is the function

$\displaystyle G_{k_0}(t_1,t_2) := \int_0^1 g^{(k_0)}(t+t_1) g^{(k_0)}(t+t_2) \frac{t^{k_0-1}}{(k_0-1)!}\ dt.$

Note that ${G}$ vanishes if ${t_1 \geq 1}$ or ${t_2 \geq 1}$. In practice, we will work with functions ${g}$ in which ${g^{(k_0)}}$ has a definite sign (in our normalisations, ${g^{(k_0)}}$ will be non-positive), making ${G_{k_0}}$ non-negative.

We rewrite the left-hand side of (7) as

$\displaystyle \sum_{a \in {\mathcal S}_J} \frac{k_0^{\Omega(a)}}{a} \sum_{a_1,a_2: [a_1,a_2] = a} \mu((a_1,a_2)) G_{k_0}( \frac{\log a_1}{\log R}, \frac{\log a_2}{\log R} ).$

We may factor ${a = p_1 \ldots p_n}$ for some ${x^\delta < p_1 < \ldots < p_n}$ with ${p_1 \ldots p_n \leq R}$; in particular, ${n < \frac{1 + 4\varpi}{4\delta}}$. The previous expression now becomes

$\displaystyle \sum_{0 \leq n < \frac{1+4\varpi}{4\delta}} k_0^n \sum_{x^\delta < p_1 < \ldots < p_n} \frac{1}{p_1 \ldots p_n}$

$\displaystyle \sum_{\{1,\ldots,n\} = S \cup T} (-1)^{|S \cap T|} G_{k_0}( \sum_{i \in S} \frac{\log p_i}{\log R}, \sum_{j \in T} \frac{\log p_j}{\log R} ).$

Using Mertens’ theorem, we thus conclude an exact formula for ${\alpha}$, and similarly for ${\beta}$:

Proposition 1 (Exact formula) We have

$\displaystyle \alpha = \sum_{0 \leq n < \frac{1+4\varpi}{4\delta}} k_0^n \int_{\frac{4\delta}{1+4\varpi} < t_1 < \ldots < t_n} G_{k_0,n}(t_1,\ldots,t_n) \frac{dt_1 \ldots dt_n}{t_1 \ldots t_n}$

where

$\displaystyle G_{k_0,n}(t_1,\ldots,t_n) := \sum_{\{1,\ldots,n\} = S \cup T} (-1)^{|S \cap T|} G_{k_0}( \sum_{i \in S} t_i, \sum_{j \in T} t_j ).$

Similarly we have

$\displaystyle \beta = \sum_{0 \leq n < \frac{1+4\varpi}{4\delta}} (k_0-1)^n \int_{\frac{4\delta}{1+4\varpi} < t_1 < \ldots < t_n} G_{k_0-1,n}(t_1,\ldots,t_n) \frac{dt_1 \ldots dt_n}{t_1 \ldots t_n}$

where ${G_{k_0-1}}$ and ${G_{k_0-1,n}}$ are defined similarly to ${G_{k_0}}$ and ${G_{k_0,n}}$ by replacing all occurrences of ${k_0}$ with ${k_0-1}$.

These formulae are unwieldy. However if we make some monotonicity hypotheses, namely that ${g^{(k_0-1)}}$ is positive, ${g^{(k_0)}}$ is negative, and ${g^{(k_0+1)}}$ is positive on ${[0,1)}$, then we can get some good estimates on the ${G_{k_0}, G_{k_0-1}}$ (which are now non-negative functions) and hence on ${\alpha,\beta}$. Namely, if ${g^{(k_0)}}$ is negative but increasing then we have

$\displaystyle -g^{(k_0)}(t+t_1) \leq -g^{(k_0)}(\frac{t}{1-t_1})$

for ${0 \leq t_1 < 1}$ and ${t \in [0,1]}$, which implies that

$\displaystyle G_{k_0}(t_1,t_1) \leq (1-t_1)_+^{k_0} G_{k_0}(0,0)$

for any ${t_1 \geq 0}$. A similar argument in fact gives

$\displaystyle G_{k_0}(t_1+t_2,t_1+t_2) \leq (1-t_1)_+^{k_0} G_{k_0}(t_2,t_2)$

for any ${t_1,t_2 \geq 0}$. Iterating this we conclude that

$\displaystyle G_{k_0}(\sum_{i \in S} t_i, \sum_{i \in S} t_i) \leq (\prod_{i \in S} (1-t_i)_+^{k_0}) G_{k_0}(0,0)$

and similarly

$\displaystyle G_{k_0}(\sum_{i \in T} t_i, \sum_{i \in T} t_i) \leq (\prod_{i \in T} (1-t_i)_+^{k_0}) G_{k_0}(0,0).$

From Cauchy-Schwarz we thus have

$\displaystyle G_{k_0}( \sum_{i \in S} t_i, \sum_{i \in T} t_i ) \leq (\prod_{i=1}^n (1 - t_i)_+^{k_0/2}) G_{k_0}(0,0).$

Observe from the binomial formula that of the ${3^n}$ pairs ${(S,T)}$ with ${S \cup T = \{1,\ldots,n\}}$, ${\frac{3^n+1}{2}}$ of them have ${|S \cap T|}$ even, and ${\frac{3^n-1}{2}}$ of them have ${|S \cap T|}$ odd. We thus have

$\displaystyle -\frac{3^n-1}{2} (\prod_{i=1}^n (1 - t_i)_+^{k_0/2}) G_{k_0}(0,0) \leq G_{k_0,n}(t_1,\ldots,t_n) \ \ \ \ \ (8)$

$\displaystyle \leq \frac{3^n+1}{2} (\prod_{i=1}^n (1 - t_i)_+^{k_0/2}) G_{k_0}(0,0).$

We have thus established the upper bound

$\displaystyle \alpha \leq G_{k_0}(0,0) (1 + \kappa) \ \ \ \ \ (9)$

where

$\displaystyle \kappa := \sum_{1 \leq n < \frac{1+4\varpi}{4\delta}} \frac{3^n+1}{2} k_0^n \int_{\frac{4\delta}{1+4\varpi} < t_1 < \ldots < t_n} (\prod_{i=1}^n (1 - t_i)_+^{k_0/2}) \frac{dt_1 \ldots dt_n}{t_1 \ldots t_n}.$

By symmetry we may factorise

$\displaystyle \kappa := \sum_{1 \leq n < \frac{1+4\varpi}{4\delta}} \frac{3^n+1}{2} \frac{k_0^n}{n!} ( \int_{\frac{4\delta}{1+4\varpi} < t \leq 1} (1-t)^{k_0/2}\ \frac{dt}{t})^n.$

The expression ${\kappa}$ is explicitly computable for any given ${\varpi,\delta,k_0}$. Following the recent preprint of Pintz, one can get a slightly looser, but cleaner, bound by using the upper bound

$\displaystyle 1-t \leq \exp(-t)$

and so

$\displaystyle \kappa \leq \sum_{1 \leq n < \frac{1+4\varpi}{4\delta}} \frac{3^n+1}{2} \frac{k_0^n}{n!} (\int_{4\delta/(1+4\varpi)}^\infty \exp( - \frac{k_0}{2} t )\ \frac{dt}{t})^n.$

Note that

$\displaystyle \int_{4\delta/(1+4\varpi)}^\infty \exp( - \frac{k_0}{2} t )\ \frac{dt}{t} = \int_1^\infty \exp( - \frac{2k_0 \delta}{1+4\varpi} t)\ \frac{dt}{t}$

$\displaystyle < \int_1^\infty \exp( - \frac{2k_0 \delta}{1+4\varpi} t)\ dt$

$\displaystyle = \frac{1+4\varpi}{2k_0\delta} \exp( - \frac{2k_0 \delta}{1+4\varpi} )$

and hence

$\displaystyle \kappa \leq \tilde \kappa$

where

$\displaystyle \tilde \kappa := \sum_{1 \leq n < \frac{1+4\varpi}{4\delta}} \frac{1}{n!} \frac{3^n+1}{2} (\frac{1+4\varpi}{2\delta} \exp( - \frac{2k_0 \delta}{1+4\varpi} ))^n.$

In practice we expect the ${n=1}$ term to dominate, thus we have the heuristic approximation

$\displaystyle \kappa \lessapprox \frac{1+4\varpi}{\delta} \exp( - \frac{2k_0 \delta}{1+4\varpi} ).$

Now we turn to the estimation of ${\beta}$. We have an analogue of (8), namely

$\displaystyle -\frac{3^n-1}{2} (\prod_{i=1}^n (1-t_i)^{(k_0-1)/2}) G_{k_0-1}(0,0) \leq G_{k_0-1,n}(t_1,\ldots,t_n)$

$\displaystyle \leq \frac{3^n+1}{2} (\prod_{i=1}^n (1-t_i)^{(k_0-1)/2}) G_{k_0-1}(0,0).$

But we have an improvment in the lower bound in the ${n=1}$ case, because in this case we have

$\displaystyle G_{k_0-1,n}(t) = G_{k_0-1}(t,0) + G_{k_0-1}(0,t) - G_{k_0-1}(t,t).$

From the positive decreasing nature of ${g^{(k_0-1)}}$ we see that ${G_{k_0-1}(t,t) \leq G_{k_0-1}(t,0)}$ and so ${G_{k_0-1,n}(t)}$ is non-negative and can thus be ignored for the purposes of lower bounds. (There are similar improvements available for higher ${n}$ but this seems to only give negligible improvements and will not be pursued here.) Thus we obtain

$\displaystyle \beta \geq G_{k_0-1}(0,0) (1-\kappa') \ \ \ \ \ (10)$

where

$\displaystyle \kappa' := \sum_{2 \leq n < \frac{1+4\varpi}{4\delta}} \frac{3^n-1}{2} \frac{(k_0-1)^n}{n!}$

$\displaystyle (\int_{4\delta/(1+4\varpi)}^1 (1-t)^{(k_0-1)/2}\ \frac{dt}{t})^n.$

Estimating ${\kappa'}$ similarly to ${\kappa}$ we conclude that

$\displaystyle \kappa' \leq \tilde \kappa'$

where

$\displaystyle \tilde \kappa' := \sum_{2 \leq n < \frac{1+4\varpi}{4\delta}} \frac{1}{n!} \frac{3^n-1}{2} (\frac{1+4\varpi}{2\delta} \exp( - \frac{2(k_0-1) \delta}{1+4\varpi} ))^n.$

By (9), (10), we see that the condition (1) is implied by

$\displaystyle (1+4\varpi) (1-\kappa') > \frac{4G_{k_0}(0,0)}{k_0 G_{k_0-1}(0,0)} (1+\kappa).$

By Theorem 14 and Lemma 15 of this previous post, we may take the ratio ${\frac{4G_{k_0}(0,0)}{k_0 G_{k_0-1}(0,0)}}$ to be arbitrarily close to ${\frac{j_{k_0-2}^2}{k_0(k_0-1)}}$. We conclude the following theorem.

Theorem 2 Let ${0 < \varpi < 1/4}$ and ${0 < \delta < 1/4 + \varpi}$ be such that ${MPZ[\varpi,\delta]}$ holds. Let ${k_0 \geq 2}$ be an integer, define

$\displaystyle \kappa := \sum_{1 \leq n < \frac{1+4\varpi}{4\delta}} \frac{3^n+1}{2} \frac{k_0^n}{n!} (\int_{4\delta/(1+4\varpi)}^1 (1-t)^{k_0/2}\ \frac{dt}{t})^n$

and

$\displaystyle \kappa' := \sum_{2 \leq n < \frac{1+4\varpi}{4\delta}} \frac{3^n-1}{2} \frac{(k_0-1)^n}{n!}$

$\displaystyle (\int_{4\delta/(1+4\varpi)}^1 (1-t)^{(k_0-1)/2}\ \frac{dt}{t})^n$

and suppose that

$\displaystyle (1+4\varpi) (1-\kappa') > \frac{j_{k_0-2}^2}{k_0(k_0-1)} (1+\kappa).$

Then ${DHL[k_0,2]}$ holds.

As noted earlier, we heuristically have

$\displaystyle \tilde \kappa \approx \frac{1+4\varpi}{\delta} \exp( - \frac{2k_0 \delta}{1+4\varpi} )$

and ${\tilde \kappa'}$ is negligible. This constraint is a bit better than the previous condition, in which ${\tilde \kappa'}$ was not present and ${\tilde \kappa}$ was replaced by a quantity roughly of the form ${2 \log(2) k_0 \exp( - \frac{2k_0 \delta}{1+4\varpi})}$.

This post is a continuation of the previous post on sieve theory, which is an ongoing part of the Polymath8 project to improve the various parameters in Zhang’s proof that bounded gaps between primes occur infinitely often. Given that the comments on that page are getting quite lengthy, this is also a good opportunity to “roll over” that thread.

We will continue the notation from the previous post, including the concept of an admissible tuple, the use of an asymptotic parameter ${x}$ going to infinity, and a quantity ${w}$ depending on ${x}$ that goes to infinity sufficiently slowly with ${x}$, and ${W := \prod_{p (the ${W}$-trick).

The objective of this portion of the Polymath8 project is to make as efficient as possible the connection between two types of results, which we call ${DHL[k_0,2]}$ and ${MPZ[\varpi,\delta]}$. Let us first state ${DHL[k_0,2]}$, which has an integer parameter ${k_0 \geq 2}$:

Conjecture 1 (${DHL[k_0,2]}$) Let ${{\mathcal H}}$ be a fixed admissible ${k_0}$-tuple. Then there are infinitely many translates ${n+{\mathcal H}}$ of ${{\mathcal H}}$ which contain at least two primes.

Zhang was the first to prove a result of this type with ${k_0 = 3,500,000}$. Since then the value of ${k_0}$ has been lowered substantially; at this time of writing, the current record is ${k_0 = 26,024}$.

There are two basic ways known currently to attain this conjecture. The first is to use the Elliott-Halberstam conjecture ${EH[\theta]}$ for some ${\theta>1/2}$:

Conjecture 2 (${EH[\theta]}$) One has

$\displaystyle \sum_{1 \leq q \leq x^\theta} \sup_{a \in ({\bf Z}/q{\bf Z})^\times} |\sum_{n < x: n = a\ (q)} \Lambda(n) - \frac{1}{\phi(q)} \sum_{n < x} \Lambda(n)|$

$\displaystyle = O( \frac{x}{\log^A x} )$

for all fixed ${A>0}$. Here we use the abbreviation ${n=a\ (q)}$ for ${n=a \hbox{ mod } q}$.

Here of course ${\Lambda}$ is the von Mangoldt function and ${\phi}$ the Euler totient function. It is conjectured that ${EH[\theta]}$ holds for all ${0 < \theta < 1}$, but this is currently only known for ${0 < \theta < 1/2}$, an important result known as the Bombieri-Vinogradov theorem.

In a breakthrough paper, Goldston, Yildirim, and Pintz established an implication of the form

$\displaystyle EH[\theta] \implies DHL[k_0,2] \ \ \ \ \ (1)$

for any ${1/2 < \theta < 1}$, where ${k_0 = k_0(\theta)}$ depends on ${\theta}$. This deduction was very recently optimised by Farkas, Pintz, and Revesz and also independently in the comments to the previous blog post, leading to the following implication:

Theorem 3 (EH implies DHL) Let ${1/2 < \theta < 1}$ be a real number, and let ${k_0 \geq 2}$ be an integer obeying the inequality

$\displaystyle 2\theta > \frac{j_{k_0-2}^2}{k_0(k_0-1)}, \ \ \ \ \ (2)$

where ${j_n}$ is the first positive zero of the Bessel function ${J_n(x)}$. Then ${EH[\theta]}$ implies ${DHL[k_0,2]}$.

Note that the right-hand side of (2) is larger than ${1}$, but tends asymptotically to ${1}$ as ${k_0 \rightarrow \infty}$. We give an alternate proof of Theorem 3 below the fold.

Implications of the form Theorem 3 were modified by Motohashi and Pintz, which in our notation replaces ${EH[\theta]}$ by an easier conjecture ${MPZ[\varpi,\delta]}$ for some ${0 < \varpi < 1/4}$ and ${0 < \delta < 1/4+\varpi}$, at the cost of degrading the sufficient condition (2) slightly. In our notation, this conjecture takes the following form for each choice of parameters ${\varpi,\delta}$:

Conjecture 4 (${MPZ[\varpi,\delta]}$) Let ${{\mathcal H}}$ be a fixed ${k_0}$-tuple (not necessarily admissible) for some fixed ${k_0 \geq 2}$, and let ${b\ (W)}$ be a primitive residue class. Then

$\displaystyle \sum_{q \in {\mathcal S}_I: q< x^{1/2+2\varpi}} \sum_{a \in C(q)} |\Delta_{b,W}(\Lambda; q,a)| = O( x \log^{-A} x) \ \ \ \ \ (3)$

for any fixed ${A>0}$, where ${I = (w,x^{\delta})}$, ${{\mathcal S}_I}$ are the square-free integers whose prime factors lie in ${I}$, and ${\Delta_{b,W}(\Lambda;q,a)}$ is the quantity

$\displaystyle \Delta_{b,W}(\Lambda;q,a) := | \sum_{x \leq n \leq 2x: n=b\ (W); n = a\ (q)} \Lambda(n) \ \ \ \ \ (4)$

$\displaystyle - \frac{1}{\phi(q)} \sum_{x \leq n \leq 2x: n = b\ (W)} \Lambda(n)|.$

and ${C(q)}$ is the set of congruence classes

$\displaystyle C(q) := \{ a \in ({\bf Z}/q{\bf Z})^\times: P(a) = 0 \}$

and ${P}$ is the polynomial

$\displaystyle P(a) := \prod_{h \in {\mathcal H}} (a+h).$

This is a weakened version of the Elliott-Halberstam conjecture:

Proposition 5 (EH implies MPZ) Let ${0 < \varpi < 1/4}$ and ${0 < \delta < 1/4+\varpi}$. Then ${EH[1/2+2\varpi+\epsilon]}$ implies ${MPZ[\varpi,\delta]}$ for any ${\epsilon>0}$. (In abbreviated form: ${EH[1/2+2\varpi+]}$ implies ${MPZ[\varpi,\delta]}$.)

In particular, since ${EH[\theta]}$ is conjecturally true for all ${0 < \theta < 1/2}$, we conjecture ${MPZ[\varpi,\delta]}$ to be true for all ${0 < \varpi < 1/4}$ and ${0<\delta<1/4+\varpi}$.

Proof: Define

$\displaystyle E(q) := \sup_{a \in ({\bf Z}/q{\bf Z})^\times} |\sum_{x \leq n \leq 2x: n = a\ (q)} \Lambda(n) - \frac{1}{\phi(q)} \sum_{x \leq n \leq 2x} \Lambda(n)|$

then the hypothesis ${EH[1/2+2\varpi+\epsilon]}$ (applied to ${x}$ and ${2x}$ and then subtracting) tells us that

$\displaystyle \sum_{1 \leq q \leq Wx^{1/2+2\varpi}} E(q) \ll x \log^{-A} x$

for any fixed ${A>0}$. From the Chinese remainder theorem and the Siegel-Walfisz theorem we have

$\displaystyle \sup_{a \in ({\bf Z}/q{\bf Z})^\times} \Delta_{b,W}(\Lambda;q,a) \ll E(qW) + \frac{1}{\phi(q)} x \log^{-A} x$

for any ${q}$ coprime to ${W}$ (and in particular for ${q \in {\mathcal S}_I}$). Since ${|C(q)| \leq k_0^{\Omega(q)}}$, where ${\Omega(q)}$ is the number of prime divisors of ${q}$, we can thus bound the left-hand side of (3) by

$\displaystyle \ll \sum_{q \in {\mathcal S}_I: q< x^{1/2+2\varpi}} k_0^{\Omega(q)} E(qW) + k_0^{\Omega(q)} \frac{1}{\phi(q)} x \log^{-A} x.$

The contribution of the second term is ${O(x \log^{-A+O(1)} x)}$ by standard estimates (see Proposition 8 below). Using the very crude bound

$\displaystyle E(q) \ll \frac{1}{\phi(q)} x \log x$

and standard estimates we also have

$\displaystyle \sum_{q \in {\mathcal S}_I: q< x^{1/2+2\varpi}} k_0^{2\Omega(q)} E(qW) \ll x \log^{O(1)} A$

and the claim now follows from the Cauchy-Schwarz inequality. $\Box$

In practice, the conjecture ${MPZ[\varpi,\delta]}$ is easier to prove than ${EH[1/2+2\varpi+]}$ due to the restriction of the residue classes ${a}$ to ${C(q)}$, and also the restriction of the modulus ${q}$ to ${x^\delta}$-smooth numbers. Zhang proved ${MPZ[\varpi,\varpi]}$ for any ${0 < \varpi < 1/1168}$. More recently, our Polymath8 group has analysed Zhang’s argument (using in part a corrected version of the analysis of a recent preprint of Pintz) to obtain ${MPZ[\varpi,\delta]}$ whenever ${\delta, \varpi > 0}$ are such that

$\displaystyle 207\varpi + 43\delta < \frac{1}{4}.$

The work of Motohashi and Pintz, and later Zhang, implicitly describe arguments that allow one to deduce ${DHL[k_0,2]}$ from ${MPZ[\varpi,\delta]}$ provided that ${k_0}$ is sufficiently large depending on ${\varpi,\delta}$. The best implication of this sort that we have been able to verify thus far is the following result, established in the previous post:

Theorem 6 (MPZ implies DHL) Let ${0 < \varpi < 1/4}$, ${0 < \delta < 1/4+\varpi}$, and let ${k_0 \geq 2}$ be an integer obeying the constraint

$\displaystyle 1+4\varpi > \frac{j_{k_0-2}^2}{k_0(k_0-1)} (1+\kappa) \ \ \ \ \ (5)$

where ${\kappa}$ is the quantity

$\displaystyle \kappa := \sum_{1 \leq n < \frac{1+4\varpi}{2\delta}} (1 - \frac{2n \delta}{1 + 4\varpi})^{k_0/2} \prod_{j=1}^{n} (1 + 3k_0 \log(1+\frac{1}{j})) ).$

Then ${MPZ[\varpi,\delta]}$ implies ${DHL[k_0,2]}$.

This complicated version of ${\kappa}$ is roughly of size ${3 \log(2) k_0 \exp( - k_0 \delta)}$. It is unlikely to be optimal; the work of Motohashi-Pintz and Pintz suggests that it can essentially be improved to ${\frac{1}{\delta} \exp(-k_0 \delta)}$, but currently we are unable to verify this claim. One of the aims of this post is to encourage further discussion as to how to improve the ${\kappa}$ term in results such as Theorem 6.

We remark that as (5) is an open condition, it is unaffected by infinitesimal modifications to ${\varpi,\delta}$, and so we do not ascribe much importance to such modifications (e.g. replacing ${\varpi}$ by ${\varpi-\epsilon}$ for some arbitrarily small ${\epsilon>0}$).

The known deductions of ${DHL[k_0,2]}$ from claims such as ${EH[\theta]}$ or ${MPZ[\varpi,\delta]}$ rely on the following elementary observation of Goldston, Pintz, and Yildirim (essentially a weighted pigeonhole principle), which we have placed in “${W}$-tricked form”:

Lemma 7 (Criterion for DHL) Let ${k_0 \geq 2}$. Suppose that for each fixed admissible ${k_0}$-tuple ${{\mathcal H}}$ and each congruence class ${b\ (W)}$ such that ${b+h}$ is coprime to ${W}$ for all ${h \in {\mathcal H}}$, one can find a non-negative weight function ${\nu: {\bf N} \rightarrow {\bf R}^+}$, fixed quantities ${\alpha,\beta > 0}$, a quantity ${A>0}$, and a fixed positive power ${R}$ of ${x}$ such that one has the upper bound

$\displaystyle \sum_{x \leq n \leq 2x: n = b\ (W)} \nu(n) \leq (\alpha+o(1)) A\frac{x}{W}, \ \ \ \ \ (6)$

the lower bound

$\displaystyle \sum_{x \leq n \leq 2x: n = b\ (W)} \nu(n) \theta(n+h_i) \geq (\beta-o(1)) A\frac{x}{W} \log R \ \ \ \ \ (7)$

for all ${h_i \in {\mathcal H}}$, and the key inequality

$\displaystyle \frac{\log R}{\log x} > \frac{1}{k_0} \frac{\alpha}{\beta} \ \ \ \ \ (8)$

holds. Then ${DHL[k_0,2]}$ holds. Here ${\theta(n)}$ is defined to equal ${\log n}$ when ${n}$ is prime and ${0}$ otherwise.

Proof: Consider the quantity

$\displaystyle \sum_{x \leq n \leq 2x: n = b\ (W)} \nu(n) (\sum_{h \in {\mathcal H}} \theta(n+h) - \log(3x)). \ \ \ \ \ (9)$

By (6), (7), this quantity is at least

$\displaystyle k_0 \beta A\frac{x}{W} \log R - \alpha \log(3x) A\frac{x}{W} - o(A\frac{x}{W} \log x).$

By (8), this expression is positive for all sufficiently large ${x}$. On the other hand, (9) can only be positive if at least one summand is positive, which only can happen when ${n+{\mathcal H}}$ contains at least two primes for some ${x \leq n \leq 2x}$ with ${n=b\ (W)}$. Letting ${x \rightarrow \infty}$ we obtain ${DHL[k_0,2]}$ as claimed. $\Box$

In practice, the quantity ${R}$ (referred to as the sieve level) is a power of ${x}$ such as ${x^{\theta/2}}$ or ${x^{1/4+\varpi}}$, and reflects the strength of the distribution hypothesis ${EH[\theta]}$ or ${MPZ[\varpi,\delta]}$ that is available; the quantity ${R}$ will also be a key parameter in the definition of the sieve weight ${\nu}$. The factor ${A}$ reflects the order of magnitude of the expected density of ${\nu}$ in the residue class ${b\ (W)}$; it could be absorbed into the sieve weight ${\nu}$ by dividing that weight by ${A}$, but it is convenient to not enforce such a normalisation so as not to clutter up the formulae. In practice, ${A}$ will some combination of ${\frac{\phi(W)}{W}}$ and ${\log R}$.

Once one has decided to rely on Lemma 7, the next main task is to select a good weight ${\nu}$ for which the ratio ${\alpha/\beta}$ is as small as possible (and for which the sieve level ${R}$ is as large as possible. To ensure non-negativity, we use the Selberg sieve

$\displaystyle \nu = \lambda^2, \ \ \ \ \ (10)$

where ${\lambda(n)}$ takes the form

$\displaystyle \lambda(n) = \sum_{d \in {\mathcal S}_I: d|P(n)} \mu(d) a_d$

for some weights ${a_d \in {\bf R}}$ vanishing for ${d>R}$ that are to be chosen, where ${I \subset (w,+\infty)}$ is an interval and ${P}$ is the polynomial ${P(n) := \prod_{h \in {\mathcal H}} (n+h)}$. If the distribution hypothesis is ${EH[\theta]}$, one takes ${R := x^{\theta/2}}$ and ${I := (w,+\infty)}$; if the distribution hypothesis is instead ${MPZ[\varpi,\delta]}$, one takes ${R := x^{1/4+\varpi}}$ and ${I := (w,x^\delta)}$.

One has a useful amount of flexibility in selecting the weights ${a_d}$ for the Selberg sieve. The original work of Goldston, Pintz, and Yildirim, as well as the subsequent paper of Zhang, the choice

$\displaystyle a_d := \log(\frac{R}{d})_+^{k_0+\ell_0}$

is used for some additional parameter ${\ell_0 > 0}$ to be optimised over. More generally, one can take

$\displaystyle a_d := g( \frac{\log d}{\log R} )$

for some suitable (in particular, sufficiently smooth) cutoff function ${g: {\bf R} \rightarrow {\bf R}}$. We will refer to this choice of sieve weights as the “analytic Selberg sieve”; this is the choice used in the analysis in the previous post.

However, there is a slight variant choice of sieve weights that one can use, which I will call the “elementary Selberg sieve”, and it takes the form

$\displaystyle a_d := \frac{1}{\Phi(d) \Delta(d)} \sum_{q \in {\mathcal S}_I: (q,d)=1} \frac{1}{\Phi(q)} f'( \frac{\log dq}{\log R}) \ \ \ \ \ (11)$

for a sufficiently smooth function ${f: {\bf R} \rightarrow {\bf R}}$, where

$\displaystyle \Phi(d) := \prod_{p|d} \frac{p-k_0}{k_0}$

for ${d \in {\mathcal S}_I}$ is a ${k_0}$-variant of the Euler totient function, and

$\displaystyle \Delta(d) := \prod_{p|d} \frac{k_0}{p} = \frac{k_0^{\Omega(d)}}{d}$

for ${d \in {\mathcal S}_I}$ is a ${k_0}$-variant of the function ${1/d}$. (The derivative on the ${f}$ cutoff is convenient for computations, as will be made clearer later in this post.) This choice of weights ${a_d}$ may seem somewhat arbitrary, but it arises naturally when considering how to optimise the quadratic form

$\displaystyle \sum_{d_1,d_2 \in {\mathcal S}_I} \mu(d_1) a_{d_1} \mu(d_2) a_{d_2} \Delta([d_1,d_2])$

(which arises naturally in the estimation of ${\alpha}$ in (6)) subject to a fixed value of ${a_1}$ (which morally is associated to the estimation of ${\beta}$ in (7)); this is discussed in any sieve theory text as part of the general theory of the Selberg sieve, e.g. Friedlander-Iwaniec.

The use of the elementary Selberg sieve for the bounded prime gaps problem was studied by Motohashi and Pintz. Their arguments give an alternate derivation of ${DHL[k_0,2]}$ from ${MPZ[\varpi,\theta]}$ for ${k_0}$ sufficiently large, although unfortunately we were not able to confirm some of their calculations regarding the precise dependence of ${k_0}$ on ${\varpi,\theta}$, and in particular we have not yet been able to improve upon the specific criterion in Theorem 6 using the elementary sieve. However it is quite plausible that such improvements could become available with additional arguments.

Below the fold we describe how the elementary Selberg sieve can be used to reprove Theorem 3, and discuss how they could potentially be used to improve upon Theorem 6. (But the elementary Selberg sieve and the analytic Selberg sieve are in any event closely related; see the appendix of this paper of mine with Ben Green for some further discussion.) For the purposes of polymath8, either developing the elementary Selberg sieve or continuing the analysis of the analytic Selberg sieve from the previous post would be a relevant topic of conversation in the comments to this post.

Suppose one is given a ${k_0}$-tuple ${{\mathcal H} = (h_1,\ldots,h_{k_0})}$ of ${k_0}$ distinct integers for some ${k_0 \geq 1}$, arranged in increasing order. When is it possible to find infinitely many translates ${n + {\mathcal H} =(n+h_1,\ldots,n+h_{k_0})}$ of ${{\mathcal H}}$ which consists entirely of primes? The case ${k_0=1}$ is just Euclid’s theorem on the infinitude of primes, but the case ${k_0=2}$ is already open in general, with the ${{\mathcal H} = (0,2)}$ case being the notorious twin prime conjecture.

On the other hand, there are some tuples ${{\mathcal H}}$ for which one can easily answer the above question in the negative. For instance, the only translate of ${(0,1)}$ that consists entirely of primes is ${(2,3)}$, basically because each translate of ${(0,1)}$ must contain an even number, and the only even prime is ${2}$. More generally, if there is a prime ${p}$ such that ${{\mathcal H}}$ meets each of the ${p}$ residue classes ${0 \hbox{ mod } p, 1 \hbox{ mod } p, \ldots, p-1 \hbox{ mod } p}$, then every translate of ${{\mathcal H}}$ contains at least one multiple of ${p}$; since ${p}$ is the only multiple of ${p}$ that is prime, this shows that there are only finitely many translates of ${{\mathcal H}}$ that consist entirely of primes.

To avoid this obstruction, let us call a ${k_0}$-tuple ${{\mathcal H}}$ admissible if it avoids at least one residue class ${\hbox{ mod } p}$ for each prime ${p}$. It is easy to check for admissibility in practice, since a ${k_0}$-tuple is automatically admissible in every prime ${p}$ larger than ${k_0}$, so one only needs to check a finite number of primes in order to decide on the admissibility of a given tuple. For instance, ${(0,2)}$ or ${(0,2,6)}$ are admissible, but ${(0,2,4)}$ is not (because it covers all the residue classes modulo ${3}$). We then have the famous Hardy-Littlewood prime tuples conjecture:

Conjecture 1 (Prime tuples conjecture, qualitative form) If ${{\mathcal H}}$ is an admissible ${k_0}$-tuple, then there exists infinitely many translates of ${{\mathcal H}}$ that consist entirely of primes.

This conjecture is extremely difficult (containing the twin prime conjecture, for instance, as a special case), and in fact there is no explicitly known example of an admissible ${k_0}$-tuple with ${k_0 \geq 2}$ for which we can verify this conjecture (although, thanks to the recent work of Zhang, we know that ${(0,d)}$ satisfies the conclusion of the prime tuples conjecture for some ${0 < d < 70,000,000}$, even if we can’t yet say what the precise value of ${d}$ is).

Actually, Hardy and Littlewood conjectured a more precise version of Conjecture 1. Given an admissible ${k_0}$-tuple ${{\mathcal H} = (h_1,\ldots,h_{k_0})}$, and for each prime ${p}$, let ${\nu_p = \nu_p({\mathcal H}) := |{\mathcal H} \hbox{ mod } p|}$ denote the number of residue classes modulo ${p}$ that ${{\mathcal H}}$ meets; thus we have ${1 \leq \nu_p \leq p-1}$ for all ${p}$ by admissibility, and also ${\nu_p = k_0}$ for all ${p>h_{k_0}-h_1}$. We then define the singular series ${{\mathfrak G} = {\mathfrak G}({\mathcal H})}$ associated to ${{\mathcal H}}$ by the formula

$\displaystyle {\mathfrak G} := \prod_{p \in {\mathcal P}} \frac{1-\frac{\nu_p}{p}}{(1-\frac{1}{p})^{k_0}}$

where ${{\mathcal P} = \{2,3,5,\ldots\}}$ is the set of primes; by the previous discussion we see that the infinite product in ${{\mathfrak G}}$ converges to a finite non-zero number.

We will also need some asymptotic notation (in the spirit of “cheap nonstandard analysis“). We will need a parameter ${x}$ that one should think of going to infinity. Some mathematical objects (such as ${{\mathcal H}}$ and ${k_0}$) will be independent of ${x}$ and referred to as fixed; but unless otherwise specified we allow all mathematical objects under consideration to depend on ${x}$. If ${X}$ and ${Y}$ are two such quantities, we say that ${X = O(Y)}$ if one has ${|X| \leq CY}$ for some fixed ${C}$, and ${X = o(Y)}$ if one has ${|X| \leq c(x) Y}$ for some function ${c(x)}$ of ${x}$ (and of any fixed parameters present) that goes to zero as ${x \rightarrow \infty}$ (for each choice of fixed parameters).

Conjecture 2 (Prime tuples conjecture, quantitative form) Let ${k_0 \geq 1}$ be a fixed natural number, and let ${{\mathcal H}}$ be a fixed admissible ${k_0}$-tuple. Then the number of natural numbers ${n < x}$ such that ${n+{\mathcal H}}$ consists entirely of primes is ${({\mathfrak G} + o(1)) \frac{x}{\log^{k_0} x}}$.

Thus, for instance, if Conjecture 2 holds, then the number of twin primes less than ${x}$ should equal ${(2 \Pi_2 + o(1)) \frac{x}{\log^2 x}}$, where ${\Pi_2}$ is the twin prime constant

$\displaystyle \Pi_2 := \prod_{p \in {\mathcal P}: p>2} (1 - \frac{1}{(p-1)^2}) = 0.6601618\ldots.$

As this conjecture is stronger than Conjecture 1, it is of course open. However there are a number of partial results on this conjecture. For instance, this conjecture is known to be true if one introduces some additional averaging in ${{\mathcal H}}$; see for instance this previous post. From the methods of sieve theory, one can obtain an upper bound of ${(C_{k_0} {\mathfrak G} + o(1)) \frac{x}{\log^{k_0} x}}$ for the number of ${n < x}$ with ${n + {\mathcal H}}$ all prime, where ${C_{k_0}}$ depends only on ${k_0}$. Sieve theory can also give analogues of Conjecture 2 if the primes are replaced by a suitable notion of almost prime (or more precisely, by a weight function concentrated on almost primes).

Another type of partial result towards Conjectures 1, 2 come from the results of Goldston-Pintz-Yildirim, Motohashi-Pintz, and of Zhang. Following the notation of this recent paper of Pintz, for each ${k_0>2}$, let ${DHL[k_0,2]}$ denote the following assertion (DHL stands for “Dickson-Hardy-Littlewood”):

Conjecture 3 (${DHL[k_0,2]}$) Let ${{\mathcal H}}$ be a fixed admissible ${k_0}$-tuple. Then there are infinitely many translates ${n+{\mathcal H}}$ of ${{\mathcal H}}$ which contain at least two primes.

This conjecture gets harder as ${k_0}$ gets smaller. Note for instance that ${DHL[2,2]}$ would imply all the ${k_0=2}$ cases of Conjecture 1, including the twin prime conjecture. More generally, if one knew ${DHL[k_0,2]}$ for some ${k_0}$, then one would immediately conclude that there are an infinite number of pairs of consecutive primes of separation at most ${H(k_0)}$, where ${H(k_0)}$ is the minimal diameter ${h_{k_0}-h_1}$ amongst all admissible ${k_0}$-tuples ${{\mathcal H}}$. Values of ${H(k_0)}$ for small ${k_0}$ can be found at this link (with ${H(k_0)}$ denoted ${w}$ in that page). For large ${k_0}$, the best upper bounds on ${H(k_0)}$ have been found by using admissible ${k_0}$-tuples ${{\mathcal H}}$ of the form

$\displaystyle {\mathcal H} = ( - p_{m+\lfloor k_0/2\rfloor - 1}, \ldots, - p_{m+1}, -1, +1, p_{m+1}, \ldots, p_{m+\lfloor (k_0+1)/2\rfloor - 1} )$

where ${p_n}$ denotes the ${n^{th}}$ prime and ${m}$ is a parameter to be optimised over (in practice it is an order of magnitude or two smaller than ${k_0}$); see this blog post for details. The upshot is that one can bound ${H(k_0)}$ for large ${k_0}$ by a quantity slightly smaller than ${k_0 \log k_0}$ (and the large sieve inequality shows that this is sharp up to a factor of two, see e.g. this previous post for more discussion).

In a key breakthrough, Goldston, Pintz, and Yildirim were able to establish the following conditional result a few years ago:

Theorem 4 (Goldston-Pintz-Yildirim) Suppose that the Elliott-Halberstam conjecture ${EH[\theta]}$ is true for some ${1/2 < \theta < 1}$. Then ${DHL[k_0,2]}$ is true for some finite ${k_0}$. In particular, this establishes an infinite number of pairs of consecutive primes of separation ${O(1)}$.

The dependence of constants between ${k_0}$ and ${\theta}$ given by the Goldston-Pintz-Yildirim argument is basically of the form ${k_0 \sim (\theta-1/2)^{-2}}$. (UPDATE: as recently observed by Farkas, Pintz, and Revesz, this relationship can be improved to ${k_0 \sim (\theta-1/2)^{-3/2}}$.)

Unfortunately, the Elliott-Halberstam conjecture (which we will state properly below) is only known for ${\theta<1/2}$, an important result known as the Bombieri-Vinogradov theorem. If one uses the Bombieri-Vinogradov theorem instead of the Elliott-Halberstam conjecture, Goldston, Pintz, and Yildirim were still able to show the highly non-trivial result that there were infinitely many pairs ${p_{n+1},p_n}$ of consecutive primes with ${(p_{n+1}-p_n) / \log p_n \rightarrow 0}$ (actually they showed more than this; see e.g. this survey of Soundararajan for details).

Actually, the full strength of the Elliott-Halberstam conjecture is not needed for these results. There is a technical specialisation of the Elliott-Halberstam conjecture which does not presently have a commonly accepted name; I will call it the Motohashi-Pintz-Zhang conjecture ${MPZ[\varpi]}$ in this post, where ${0 < \varpi < 1/4}$ is a parameter. We will define this conjecture more precisely later, but let us remark for now that ${MPZ[\varpi]}$ is a consequence of ${EH[\frac{1}{2}+2\varpi]}$.

We then have the following two theorems. Firstly, we have the following strengthening of Theorem 4:

Theorem 5 (Motohashi-Pintz-Zhang) Suppose that ${MPZ[\varpi]}$ is true for some ${0 < \varpi < 1/4}$. Then ${DHL[k_0,2]}$ is true for some ${k_0}$.

A version of this result (with a slightly different formulation of ${MPZ[\varpi]}$) appears in this paper of Motohashi and Pintz, and in the paper of Zhang, Theorem 5 is proven for the concrete values ${\varpi = 1/1168}$ and ${k_0 = 3,500,000}$. We will supply a self-contained proof of Theorem 5 below the fold, the constants upon those in Zhang’s paper (in particular, for ${\varpi = 1/1168}$, we can take ${k_0}$ as low as ${341,640}$, with further improvements on the way). As with Theorem 4, we have an inverse quadratic relationship ${k_0 \sim \varpi^{-2}}$.

In his paper, Zhang obtained for the first time an unconditional advance on ${MPZ[\varpi]}$:

Theorem 6 (Zhang) ${MPZ[\varpi]}$ is true for all ${0 < \varpi \leq 1/1168}$.

This is a deep result, building upon the work of Fouvry-Iwaniec, Friedlander-Iwaniec and Bombieri-Friedlander-Iwaniec which established results of a similar nature to ${MPZ[\varpi]}$ but simpler in some key respects. We will not discuss this result further here, except to say that they rely on the (higher-dimensional case of the) Weil conjectures, which were famously proven by Deligne using methods from l-adic cohomology. Also, it was believed among at least some experts that the methods of Bombieri, Fouvry, Friedlander, and Iwaniec were not quite strong enough to obtain results of the form ${MPZ[\varpi]}$, making Theorem 6 a particularly impressive achievement.

Combining Theorem 6 with Theorem 5 we obtain ${DHL[k_0,2]}$ for some finite ${k_0}$; Zhang obtains this for ${k_0 = 3,500,000}$ but as detailed below, this can be lowered to ${k_0 = 341,640}$. This in turn gives infinitely many pairs of consecutive primes of separation at most ${H(k_0)}$. Zhang gives a simple argument that bounds ${H(3,500,000)}$ by ${70,000,000}$, giving his famous result that there are infinitely many pairs of primes of separation at most ${70,000,000}$; by being a bit more careful (as discussed in this post) one can lower the upper bound on ${H(3,500,000)}$ to ${57,554,086}$, and if one instead uses the newer value ${k_0 = 341,640}$ for ${k_0}$ one can instead use the bound ${H(341,640) \leq 4,982,086}$. (Many thanks to Scott Morrison for these numerics.) UPDATE: These values are now obsolete; see this web page for the latest bounds.

In this post we would like to give a self-contained proof of both Theorem 4 and Theorem 5, which are both sieve-theoretic results that are mainly elementary in nature. (But, as stated earlier, we will not discuss the deepest new result in Zhang’s paper, namely Theorem 6.) Our presentation will deviate a little bit from the traditional sieve-theoretic approach in a few places. Firstly, there is a portion of the argument that is traditionally handled using contour integration and properties of the Riemann zeta function; we will present a “cheaper” approach (which Ben Green and I used in our papers, e.g. in this one) using Fourier analysis, with the only property used about the zeta function ${\zeta(s)}$ being the elementary fact that blows up like ${\frac{1}{s-1}}$ as one approaches ${1}$ from the right. To deal with the contribution of small primes (which is the source of the singular series ${{\mathfrak G}}$), it will be convenient to use the “${W}$-trick” (introduced in this paper of mine with Ben), passing to a single residue class mod ${W}$ (where ${W}$ is the product of all the small primes) to end up in a situation in which all small primes have been “turned off” which leads to better pseudorandomness properties (for instance, once one eliminates all multiples of small primes, almost all pairs of remaining numbers will be coprime).

In this final set of course notes, we discuss how (a generalisation of) the expansion results obtained in the preceding notes can be used for some nnumber-theoretic applications, and in particular to locate almost primes inside orbits of thin groups, following the work of Bourgain, Gamburd, and Sarnak. We will not attempt here to obtain the sharpest or most general results in this direction, but instead focus on the simplest instances of these results which are still illustrative of the ideas involved.

One of the basic general problems in analytic number theory is to locate tuples of primes of a certain form; for instance, the famous (and still unsolved) twin prime conjecture asserts that there are infinitely many pairs ${(n_1,n_2)}$ in the line ${\{ (n_1,n_2) \in {\bf Z}^2: n_2-n_1=2\}}$ in which both entries are prime. In a similar spirit, one of the Landau conjectures (also still unsolved) asserts that there are infinitely many primes in the set ${\{ n^2+1: n \in {\bf Z} \}}$. The Mersenne prime conjecture (also unsolved) asserts that there are infinitely many primes in the set ${\{ 2^n - 1: n \in {\bf Z} \}}$, and so forth.

More generally, given some explicit subset ${V}$ in ${{\bf R}^d}$ (or ${{\bf C}^d}$, if one wishes), such as an algebraic variety, one can ask the question of whether there are infinitely many integer lattice points ${(n_1,\ldots,n_d)}$ in ${V \cap {\bf Z}^d}$ in which all the coefficients ${n_1,\ldots,n_d}$ are simultaneously prime; let us refer to such points as prime points.

At this level of generality, this problem is impossibly difficult. Indeed, even the much simpler problem of deciding whether the set ${V \cap {\bf Z}^d}$ is non-empty (let alone containing prime points) when ${V}$ is a hypersurface ${\{ x \in {\bf R}^d: P(x) = 0 \}}$ cut out by a polynomial ${P}$ is essentially Hilbert’s tenth problem, which is known to be undecidable in general by Matiyasevich’s theorem. So one needs to restrict attention to a more special class of sets ${V}$, in which the question of finding integer points is not so difficult. One model case is to consider orbits ${V = \Gamma b}$, where ${b \in {\bf Z}^d}$ is a fixed lattice vector and ${\Gamma}$ is some discrete group that acts on ${{\bf Z}^d}$ somehow (e.g. ${\Gamma}$ might be embedded as a subgroup of the special linear group ${SL_d({\bf Z})}$, or on the affine group ${SL_d({\bf Z}) \ltimes {\bf Z}^d}$). In such a situation it is then quite easy to show that ${V = V \cap {\bf Z}^d}$ is large; for instance, ${V}$ will be infinite precisely when the stabiliser of ${b}$ in ${\Gamma}$ has infinite index in ${\Gamma}$.

Even in this simpler setting, the question of determining whether an orbit ${V = \Gamma b}$ contains infinitely prime points is still extremely difficult; indeed the three examples given above of the twin prime conjecture, Landau conjecture, and Mersenne prime conjecture are essentially of this form (possibly after some slight modification of the underlying ring ${{\bf Z}}$, see this paper of Bourgain-Gamburd-Sarnak for details), and are all unsolved (and generally considered well out of reach of current technology). Indeed, the list of non-trivial orbits ${V = \Gamma b}$ which are known to contain infinitely many prime points is quite slim; Euclid’s theorem on the infinitude of primes handles the case ${V = {\bf Z}}$, Dirichlet’s theorem handles infinite arithmetic progressions ${V = a{\bf Z} + r}$, and a somewhat complicated result of Green, Tao, and Ziegler handles “non-degenerate” affine lattices in ${{\bf Z}^d}$ of rank two or more (such as the lattice of length ${d}$ arithmetic progressions), but there are few other positive results known that are not based on the above cases (though we will note the remarkable theorem of Friedlander and Iwaniec that there are infinitely many primes of the form ${a^2+b^4}$, and the related result of Heath-Brown that there are infinitely many primes of the form ${a^3+2b^3}$, as being in a kindred spirit to the above results, though they are not explicitly associated to an orbit of a reasonable action as far as I know).

On the other hand, much more is known if one is willing to replace the primes by the larger set of almost primes – integers with a small number of prime factors (counting multiplicity). Specifically, for any ${r \geq 1}$, let us call an ${r}$-almost prime an integer which is the product of at most ${r}$ primes, and possibly by the unit ${-1}$ as well. Many of the above sorts of questions which are open for primes, are known for ${r}$-almost primes for ${r}$ sufficiently large. For instance, with regards to the twin prime conjecture, it is a result of Chen that there are infinitely many pairs ${p,p+2}$ where ${p}$ is a prime and ${p+2}$ is a ${2}$-almost prime; in a similar vein, it is a result of Iwaniec that there are infinitely many ${2}$-almost primes of the form ${n^2+1}$. On the other hand, it is still open for any fixed ${r}$ whether there are infinitely many Mersenne numbers ${2^n-1}$ which are ${r}$-almost primes. (For the superficially similar situation with the numbers ${2^n+1}$, it is in fact believed (but again unproven) that there are only finitely many ${r}$-almost primes for any fixed ${r}$ (the Fermat prime conjecture).)

The main tool that allows one to count almost primes in orbits is sieve theory. The reason for this lies in the simple observation that in order to ensure that an integer ${n}$ of magnitude at most ${x}$ is an ${r}$-almost prime, it suffices to guarantee that ${n}$ is not divisible by any prime less than ${x^{1/(r+1)}}$. Thus, to create ${r}$-almost primes, one can start with the integers up to some large threshold ${x}$ and remove (or “sieve out”) all the integers that are multiples of any prime ${p}$ less than ${x^{1/(r+1)}}$. The difficulty is then to ensure that a sufficiently non-trivial quantity of integers remain after this process, for the purposes of finding points in the given set ${V}$.

The most basic sieve of this form is the sieve of Eratosthenes, which when combined with the inclusion-exclusion principle gives the Legendre sieve (or exact sieve), which gives an exact formula for quantities such as the number ${\pi(x,z)}$ of natural numbers less than or equal to ${x}$ that are not divisible by any prime less than or equal to a given threshold ${z}$. Unfortunately, when one tries to evaluate this formula, one encounters error terms which grow exponentially in ${z}$, rendering this sieve useful only for very small thresholds ${z}$ (of logarithmic size in ${x}$). To improve the sieve level up to a small power of ${x}$ such as ${x^{1/(r+1)}}$, one has to replace the exact sieve by upper bound sieves and lower bound sieves which only seek to obtain upper or lower bounds on quantities such as ${\pi(x,z)}$, but contain a polynomial number of terms rather than an exponential number. There are a variety of such sieves, with the two most common such sieves being combinatorial sieves (such as the beta sieve), based on various combinatorial truncations of the inclusion-exclusion formula, and the Selberg upper bound sieve, based on upper bounds that are the square of a divisor sum. (There is also the large sieve, which is somewhat different in nature and based on ${L^2}$ almost orthogonality considerations, rather than on any actual sieving, to obtain upper bounds.) We will primarily work with a specific sieve in this notes, namely the beta sieve, and we will not attempt to optimise all the parameters of this sieve (which ultimately means that the almost primality parameter ${r}$ in our results will be somewhat large). For a more detailed study of sieve theory, see the classic text of Halberstam and Richert, or the more recent texts of Iwaniec-Kowalski or of Friedlander-Iwaniec.

Very roughly speaking, the end result of sieve theory is that excepting some degenerate and “exponentially thin” settings (such as those associated with the Mersenne primes), all the orbits which are expected to have a large number of primes, can be proven to at least have a large number of ${r}$-almost primes for some finite ${r}$. (Unfortunately, there is a major obstruction, known as the parity problem, which prevents sieve theory from lowering ${r}$ all the way to ${1}$; see this blog post for more discussion.) One formulation of this principle was established by Bourgain, Gamburd, and Sarnak:

Theorem 1 (Bourgain-Gamburd-Sarnak) Let ${\Gamma}$ be a subgroup of ${SL_2({\bf Z})}$ which is not virtually solvable. Let ${f: {\bf Z}^4 \rightarrow {\bf Z}}$ be a polynomial with integer coefficients obeying the following primitivity condition: for any positive integer ${q}$, there exists ${A \in \Gamma \subset {\bf Z}^4}$ such that ${f(A)}$ is coprime to ${q}$. Then there exists an ${r \geq 1}$ such that there are infinitely many ${A \in \Gamma}$ with ${f(A)}$ non-zero and ${r}$-almost prime.

This is not the strongest version of the Bourgain-Gamburd-Sarnak theorem, but it captures the general flavour of their results. Note that the theorem immediately implies an analogous result for orbits ${\Gamma b \subset {\bf Z}^2}$, in which ${f}$ is now a polynomial from ${{\bf Z}^2}$ to ${{\bf Z}}$, and one uses ${f(Ab)}$ instead of ${f(A)}$. It is in fact conjectured that one can set ${r=1}$ here, but this is well beyond current technology. For the purpose of reaching ${r=1}$, it is very natural to impose the primitivity condition, but as long as one is content with larger values of ${r}$, it is possible to relax the primitivity condition somewhat; see the paper of Bourgain, Gamburd, and Sarnak for more discussion.

By specialising to the polynomial ${f: \begin{pmatrix} a & b \\ c & d \end{pmatrix} \rightarrow abcd}$, we conclude as a corollary that as long as ${\Gamma}$ is primitive in the sense that it contains matrices with all coefficients coprime to ${q}$ for any given ${q}$, then ${\Gamma}$ contains infinitely many matrices whose elements are all ${r}$-almost primes for some ${r}$ depending only on ${\Gamma}$. For further applications of these sorts of results, for instance to Appolonian packings, see the paper of Bourgain, Gamburd, and Sarnak.

It turns out that to prove Theorem 1, the Cayley expansion results in ${SL_2(F_p)}$ from the previous set of notes are not quite enough; one needs a more general Cayley expansion result in ${SL_2({\bf Z}/q{\bf Z})}$ where ${q}$ is square-free but not necessarily prime. The proof of this expansion result uses the same basic methods as in the ${SL_2(F_p)}$ case, but is significantly more complicated technically, and we will only discuss it briefly here. As such, we do not give a complete proof of Theorem 1, but hopefully the portion of the argument presented here is still sufficient to give an impression of the ideas involved.

One of the most fundamental principles in Fourier analysis is the uncertainty principle. It does not have a single canonical formulation, but one typical informal description of the principle is that if a function ${f}$ is restricted to a narrow region of physical space, then its Fourier transform ${\hat f}$ must be necessarily “smeared out” over a broad region of frequency space. Some versions of the uncertainty principle are discussed in this previous blog post.

In this post I would like to highlight a useful instance of the uncertainty principle, due to Hugh Montgomery, which is useful in analytic number theory contexts. Specifically, suppose we are given a complex-valued function ${f: {\bf Z} \rightarrow {\bf C}}$ on the integers. To avoid irrelevant issues at spatial infinity, we will assume that the support ${\hbox{supp}(f) := \{ n \in {\bf Z}: f(n) \neq 0 \}}$ of this function is finite (in practice, we will only work with functions that are supported in an interval ${[M+1,M+N]}$ for some natural numbers ${M,N}$). Then we can define the Fourier transform ${\hat f: {\bf R}/{\bf Z} \rightarrow {\bf C}}$ by the formula

$\displaystyle \hat f(\xi) := \sum_{n \in {\bf Z}} f(n) e(-n\xi)$

where ${e(x) := e^{2\pi i x}}$. (In some literature, the sign in the exponential phase is reversed, but this will make no substantial difference to the arguments below.)

The classical uncertainty principle, in this context, asserts that if ${f}$ is localised in an interval of length ${N}$, then ${\hat f}$ must be “smeared out” at a scale of at least ${1/N}$ (and essentially constant at scales less than ${1/N}$). For instance, if ${f}$ is supported in ${[M+1,M+N]}$, then we have the Plancherel identity

$\displaystyle \int_{{\bf R}/{\bf Z}} |\hat f(\xi)|^2\ d\xi = \| f \|_{\ell^2({\bf Z})}^2 \ \ \ \ \ (1)$

while from the Cauchy-Schwarz inequality we have

$\displaystyle |\hat f(\xi)| \leq N^{1/2} \| f \|_{\ell^2({\bf Z})}$

for each frequency ${\xi}$, and in particular that

$\displaystyle \int_I |\hat f(\xi)|^2\ d\xi \leq N |I| \| f \|_{\ell^2({\bf Z})}^2$

for any arc ${I}$ in the unit circle (with ${|I|}$ denoting the length of ${I}$). In particular, an interval of length significantly less than ${1/N}$ can only capture a fraction of the ${L^2}$ energy of the Fourier transform of ${\hat f}$, which is consistent with the above informal statement of the uncertainty principle.

Another manifestation of the classical uncertainty principle is the large sieve inequality. A particularly nice formulation of this inequality is due independently to Montgomery and Vaughan and Selberg: if ${f}$ is supported in ${[M+1,M+N]}$, and ${\xi_1,\ldots,\xi_J}$ are frequencies in ${{\bf R}/{\bf Z}}$ that are ${\delta}$-separated for some ${\delta>0}$, thus ${\| \xi_i-\xi_j\|_{{\bf R}/{\bf Z}} \geq \delta}$ for all ${1 \leq i (where ${\|\xi\|_{{\bf R}/{\bf Z}}}$ denotes the distance of ${\xi}$ to the origin in ${{\bf R}/{\bf Z}}$), then

$\displaystyle \sum_{j=1}^J |\hat f(\xi_j)|^2 \leq (N + \frac{1}{\delta}) \| f \|_{\ell^2({\bf Z})}^2. \ \ \ \ \ (2)$

The reader is encouraged to see how this inequality is consistent with the Plancherel identity (1) and the intuition that ${\hat f}$ is essentially constant at scales less than ${1/N}$. The factor ${N + \frac{1}{\delta}}$ can in fact be amplified a little bit to ${N + \frac{1}{\delta} - 1}$, which is essentially optimal, by using a neat dilation trick of Paul Cohen, in which one dilates ${[M+1,M+N]}$ to ${[MK+K, MK+NK]}$ (and replaces each frequency ${\xi_j}$ by their ${K^{th}}$ roots), and then sending ${K \rightarrow \infty}$ (cf. the tensor product trick); see this survey of Montgomery for details. But we will not need this refinement here.

In the above instances of the uncertainty principle, the concept of narrow support in physical space was formalised in the Archimedean sense, using the standard Archimedean metric ${d_\infty(n,m) := |n-m|}$ on the integers ${{\bf Z}}$ (in particular, the parameter ${N}$ is essentially the Archimedean diameter of the support of ${f}$). However, in number theory, the Archimedean metric is not the only metric of importance on the integers; the ${p}$-adic metrics play an equally important role; indeed, it is common to unify the Archimedean and ${p}$-adic perspectives together into a unified adelic perspective. In the ${p}$-adic world, the metric balls are no longer intervals, but are instead residue classes modulo some power of ${p}$. Intersecting these balls from different ${p}$-adic metrics together, we obtain residue classes with respect to various moduli ${q}$ (which may be either prime or composite). As such, another natural manifestation of the concept of “narrow support in physical space” is “vanishes on many residue classes modulo ${q}$“. This notion of narrowness is particularly common in sieve theory, when one deals with functions supported on thin sets such as the primes, which naturally tend to avoid many residue classes (particularly if one throws away the first few primes).

In this context, the uncertainty principle is this: the more residue classes modulo ${q}$ that ${f}$ avoids, the more the Fourier transform ${\hat f}$ must spread out along multiples of ${1/q}$. To illustrate a very simple example of this principle, let us take ${q=2}$, and suppose that ${f}$ is supported only on odd numbers (thus it completely avoids the residue class ${0 \mod 2}$). We write out the formulae for ${\hat f(\xi)}$ and ${\hat f(\xi+1/2)}$:

$\displaystyle \hat f(\xi) = \sum_n f(n) e(-n\xi)$

$\displaystyle \hat f(\xi+1/2) = \sum_n f(n) e(-n\xi) e(-n/2).$

If ${f}$ is supported on the odd numbers, then ${e(-n/2)}$ is always equal to ${-1}$ on the support of ${f}$, and so we have ${\hat f(\xi+1/2)=-\hat f(\xi)}$. Thus, whenever ${\hat f}$ has a significant presence at a frequency ${\xi}$, it also must have an equally significant presence at the frequency ${\xi+1/2}$; there is a spreading out across multiples of ${1/2}$. Note that one has a similar effect if ${f}$ was supported instead on the even integers instead of the odd integers.

A little more generally, suppose now that ${f}$ avoids a single residue class modulo a prime ${p}$; for sake of argument let us say that it avoids the zero residue class ${0 \mod p}$, although the situation for the other residue classes is similar. For any frequency ${\xi}$ and any ${j=0,\ldots,p-1}$, one has

$\displaystyle \hat f(\xi+j/p) = \sum_n f(n) e(-n\xi) e(-jn/p).$

From basic Fourier analysis, we know that the phases ${e(-jn/p)}$ sum to zero as ${j}$ ranges from ${0}$ to ${p-1}$ whenever ${n}$ is not a multiple of ${p}$. We thus have

$\displaystyle \sum_{j=0}^{p-1} \hat f(\xi+j/p) = 0.$

In particular, if ${\hat f(\xi)}$ is large, then one of the other ${\hat f(\xi+j/p)}$ has to be somewhat large as well; using the Cauchy-Schwarz inequality, we can quantify this assertion in an ${\ell^2}$ sense via the inequality

$\displaystyle \sum_{j=1}^{p-1} |\hat f(\xi+j/p)|^2 \geq \frac{1}{p-1} |\hat f(\xi)|^2. \ \ \ \ \ (3)$

Let us continue this analysis a bit further. Now suppose that ${f}$ avoids ${\omega(p)}$ residue classes modulo a prime ${p}$, for some ${0 \leq \omega(p) < p}$. (We exclude the case ${\omega(p)=p}$ as it is clearly degenerates by forcing ${f}$ to be identically zero.) Let ${\nu(n)}$ be the function that equals ${1}$ on these residue classes and zero away from these residue classes, then

$\displaystyle \sum_n f(n) e(-n\xi) \nu(n) = 0.$

Using the periodic Fourier transform, we can write

$\displaystyle \nu(n) = \sum_{j=0}^{p-1} c(j) e(-jn/p)$

for some coefficients ${c(j)}$, thus

$\displaystyle \sum_{j=0}^{p-1} \hat f(\xi+j/p) c(j) = 0.$

Some Fourier-analytic computations reveal that

$\displaystyle c(0) = \frac{\omega(p)}{p}$

and

$\displaystyle \sum_{j=0}^{p-1} |c(j)|^2 = \frac{\omega(p)}{p}$

and so after some routine algebra and the Cauchy-Schwarz inequality, we obtain a generalisation of (3):

$\displaystyle \sum_{j=1}^{p-1} |\hat f(\xi+j/p)|^2 \geq \frac{\omega(p)}{p-\omega(p)} |\hat f(\xi)|^2.$

Thus we see that the more residue classes mod ${p}$ we exclude, the more Fourier energy has to disperse along multiples of ${1/p}$. It is also instructive to consider the extreme case ${\omega(p)=p-1}$, in which ${f}$ is supported on just a single residue class ${a \mod p}$; in this case, one clearly has ${\hat f(\xi+j/p) = e(-aj/p) \hat f(\xi)}$, and so ${\hat f}$ spreads its energy completely evenly along multiples of ${1/p}$.

In 1968, Montgomery observed the following useful generalisation of the above calculation to arbitrary modulus:

Proposition 1 (Montgomery’s uncertainty principle) Let ${f: {\bf Z} \rightarrow{\bf C}}$ be a finitely supported function which, for each prime ${p}$, avoids ${\omega(p)}$ residue classes modulo ${p}$ for some ${0 \leq \omega(p) < p}$. Then for each natural number ${q}$, one has

$\displaystyle \sum_{1 \leq a \leq q: (a,q)=1} |\hat f(\xi+\frac{a}{q})|^2 \geq h(q) |\hat f(\xi)|^2$

where ${h(q)}$ is the quantity

$\displaystyle h(q) := \mu(q)^2 \prod_{p|q} \frac{\omega(p)}{p-\omega(p)} \ \ \ \ \ (4)$

where ${\mu}$ is the Möbius function.

We give a proof of this proposition below the fold.

Following the “adelic” philosophy, it is natural to combine this uncertainty principle with the large sieve inequality to take simultaneous advantage of localisation both in the Archimedean sense and in the ${p}$-adic senses. This leads to the following corollary:

Corollary 2 (Arithmetic large sieve inequality) Let ${f: {\bf Z} \rightarrow {\bf C}}$ be a function supported on an interval ${[M+1,M+N]}$ which, for each prime ${p}$, avoids ${\omega(p)}$ residue classes modulo ${p}$ for some ${0 \leq \omega(p) < p}$. Let ${\xi_1,\ldots,\xi_J \in {\bf R}/{\bf Z}}$, and let ${{\mathcal Q}}$ be a finite set of natural numbers. Suppose that the frequencies ${\xi_j + \frac{a}{q}}$ with ${1 \leq j \leq J}$, ${q \in {\mathcal Q}}$, and ${(a,q)=1}$ are ${\delta}$-separated. Then one has

$\displaystyle \sum_{j=1}^J |\hat f(\xi_j)|^2 \leq \frac{N + \frac{1}{\delta}}{\sum_{q \in {\mathcal Q}} h(q)} \| f \|_{\ell^2({\bf Z})}^2$

where ${h(q)}$ was defined in (4).

Indeed, from the large sieve inequality one has

$\displaystyle \sum_{j=1}^J \sum_{q \in {\mathcal Q}} \sum_{1 \leq a \leq q: (a,q)=1} |\hat f(\xi_j+\frac{a}{q})|^2 \leq (N + \frac{1}{\delta}) \| f \|_{\ell^2({\bf Z})}^2$

while from Proposition 1 one has

$\displaystyle \sum_{q \in {\mathcal Q}} \sum_{1 \leq a \leq q: (a,q)=1} |\hat f(\xi_j+\frac{a}{q})|^2 \geq (\sum_{q \in {\mathcal Q}} h(q)) |\hat f(\xi_j)|^2$

whence the claim.

There is a great deal of flexibility in the above inequality, due to the ability to select the set ${{\mathcal Q}}$, the frequencies ${\xi_1,\ldots,\xi_J}$, the omitted classes ${\omega(p)}$, and the separation parameter ${\delta}$. Here is a typical application concerning the original motivation for the large sieve inequality, namely in bounding the size of sets which avoid many residue classes:

Corollary 3 (Large sieve) Let ${A}$ be a set of integers contained in ${[M+1,M+N]}$ which avoids ${\omega(p)}$ residue classes modulo ${p}$ for each prime ${p}$, and let ${R>0}$. Then

$\displaystyle |A| \leq \frac{N+R^2}{G(R)}$

where

$\displaystyle G(R) := \sum_{q \leq R} h(q).$

Proof: We apply Corollary 2 with ${f = 1_A}$, ${J=1}$, ${\delta=1/R^2}$, ${\xi_1=0}$, and ${{\mathcal Q} := \{ q: q \leq R\}}$. The key point is that the Farey sequence of fractions ${a/q}$ with ${q \leq R}$ and ${(a,q)=1}$ is ${1/R^2}$-separated, since

$\displaystyle \| \frac{a}{q}-\frac{a'}{q'} \|_{{\bf R}/{\bf Z}} \geq \frac{1}{qq'} \geq \frac{1}{R^2}$

whenever ${a/q, a'/q'}$ are distinct fractions in this sequence. $\Box$

If, for instance, ${A}$ is the set of all primes in ${[M+1,M+N]}$ larger than ${R}$, then one can set ${\omega(p)=1}$ for all ${p \leq R}$, which makes ${h(q) = \frac{\mu^2(q)}{\phi(q)}}$, where ${\phi}$ is the Euler totient function. It is a classical estimate that

$\displaystyle G(R) = \log R + O(1).$

Using this fact and optimising in ${R}$, we obtain (a special case of) the Brun-Titchmarsh inequality

$\displaystyle \pi(M+N)-\pi(M) \leq (2+o_{N \rightarrow \infty}(1)) \frac{N}{\log N}$

where ${\pi(x)}$ is the prime counting function; a variant of the same argument gives the more general Brun-Titchmarsh inequality

$\displaystyle \pi(M+N;a,q)-\pi(M;a,q) \leq (2+o_{N \rightarrow \infty;q}(1)) \frac{q}{\phi(q)} \frac{N}{\log N}$

for any primitive residue class ${a \mod q}$, where ${\pi(M;a,q)}$ is the number of primes less than or equal to ${M}$ that are congruent to ${a \mod q}$. By performing a more careful optimisation using a slightly sharper version of the large sieve inequality (2) that exploits the irregular spacing of the Farey sequence, Montgomery and Vaughan were able to delete the error term in the Brun-Titchmarsh inequality, thus establishing the very nice inequality

$\displaystyle \pi(M+N;a,q)-\pi(M;a,q) \leq 2 \frac{q}{\phi(q)} \frac{N}{\log N}$

for any natural numbers ${M,N,a,q}$ with ${N>1}$. This is a particularly useful inequality in non-asymptotic analytic number theory (when one wishes to study number theory at explicit orders of magnitude, rather than the number theory of sufficiently large numbers), due to the absence of asymptotic notation.

I recently realised that Corollary 2 also establishes a stronger version of the “restriction theorem for the Selberg sieve” that Ben Green and I proved some years ago (indeed, one can view Corollary 2 as a “restriction theorem for the large sieve”). I’m placing the details below the fold.

This is my final Milliman lecture, in which I talk about the sum-product phenomenon in arithmetic combinatorics, and some selected recent applications of this phenomenon to uniform distribution of exponentials, expander graphs, randomness extractors, and detecting (sieving) almost primes in group orbits, particularly as developed by Bourgain and his co-authors.
Read the rest of this entry »