You are currently browsing the category archive for the ‘math.CA’ category.

This is the third “research” thread of the Polymath15 project to upper bound the de Bruijn-Newman constant , continuing this previous thread. Discussion of the project of a non-research nature can continue for now in the existing proposal thread. Progress will be summarised at this Polymath wiki page.

We are making progress on the following test problem: can one show that whenever , , and ? This would imply that

which would be the first quantitative improvement over the de Bruijn bound of (or the Ki-Kim-Lee refinement of ). Of course we can try to lower the two parameters of later on in the project, but this seems as good a place to start as any. One could also potentially try to use finer analysis of dynamics of zeroes to improve the bound further, but this seems to be a less urgent task.

Probably the hardest case is , as there is a good chance that one can then recover the case by a suitable use of the argument principle. Here we appear to have a workable Riemann-Siegel type formula that gives a tractable approximation for . To describe this formula, first note that in the case we have

and the Riemann-Siegel formula gives

for any natural numbers , where is a contour from to that winds once anticlockwise around the zeroes of but does not wind around any other zeroes. A good choice of to use here is

In this case, a classical steepest descent computation (see wiki) yields the approximation

where

Thus we have

where

with and given by (1).

Heuristically, we have derived (see wiki) the more general approximation

for (and in particular for ), where

In practice it seems that the term is negligible once the real part of is moderately large, so one also has the approximation

For large , and for fixed , e.g. , the sums converge fairly quickly (in fact the situation seems to be significantly better here than the much more intensively studied case), and we expect the first term

of the series to dominate. Indeed, analytically we know that (or ) as (holding fixed), and it should also be provable that as well. Numerically with , it seems in fact that (or ) stay within a distance of about of once is moderately large (e.g. ). This raises the hope that one can solve the toy problem of showing for by numerically controlling for small (e.g. ), numerically controlling and analytically bounding the error for medium (e.g. ), and analytically bounding both and for large (e.g. ). (These numbers and are arbitrarily chosen here and may end up being optimised to something else as the computations become clearer.)

Thus, we now have four largely independent tasks (for suitable ranges of “small”, “medium”, and “large” ):

- Numerically computing for small (with enough accuracy to verify that there are no zeroes)
- Numerically computing for medium (with enough accuracy to keep it away from zero)
- Analytically bounding for large (with enough accuracy to keep it away from zero); and
- Analytically bounding for medium and large (with a bound that is better than the bound away from zero in the previous two tasks).

Note that tasks 2 and 3 do not directly require any further understanding of the function .

Below we will give a progress report on the numeric and analytic sides of these tasks.

** — 1. Numerics report (contributed by Sujit Nair) — **

There is some progress on the code side but not at the pace I was hoping. Here are a few things which happened (rather, mistakes which were taken care of).

- We got rid of code which wasn’t being used. For example, @dhjpolymath computed based on an old version but only realized it after the fact.
- We implemented tests to catch human/numerical bugs before a computation starts. Again, we lost some numerical cycles but moving forward these can be avoided.
- David got set up on GitHub and he is able to compare his output (in C) with the Python code. That is helping a lot.

Two areas which were worked on were

- Computing and zeroes for around
- Computing quantities like , , , etc. with the goal of understanding the zero free regions.

Some observations for , , include:

- does seem to avoid the negative real axis
- (based on the oscillations and trends in the plots)
- seems to be settling around range.

See the figure below. The top plot is on the complex plane and the bottom plot is the absolute value. The code to play with this is here.

** — 2. Analysis report — **

The Riemann-Siegel formula and some manipulations (see wiki) give , where

where is a contour that goes from to staying a bounded distance away from the upper imaginary and right real axes, and is the complex conjugate of . (In each of these sums, it is the first term that should dominate, with the second one being about as large.) One can then evolve by the heat flow to obtain , where

Steepest descent heuristics then predict that , , and . For the purposes of this project, we will need effective error estimates here, with explicit error terms.

A start has been made towards this goal at this wiki page. Firstly there is a “effective Laplace method” lemma that gives effective bounds on integrals of the form if the real part of is either monotone with large derivative, or has a critical point and is decreasing on both sides of that critical point. In principle, all one has to do is manipulate expressions such as , , by change of variables, contour shifting and integration by parts until it is of the form to which the above lemma can be profitably applied. As one may imagine though the computations are messy, particularly for the term. As a warm up, I have begun by trying to estimate integrals of the form

for smallish complex numbers , as these sorts of integrals appear in the form of . As of this time of writing, there are effective bounds for the case, and I am currently working on extending them to the case, which should give enough control to approximate and . The most complicated task will be that of upper bounding , but it also looks eventually doable.

This is the second “research” thread of the Polymath15 project to upper bound the de Bruijn-Newman constant , continuing this previous thread. Discussion of the project of a non-research nature can continue for now in the existing proposal thread. Progress will be summarised at this Polymath wiki page.

We now have the following proposition (see this page for a proof sketch) that looks like it can give a numerically feasible approach to bound :

Proposition 1Suppose that one has parameters obeying the following properties:

- All the zeroes of with are real.
- There are no zeroes with in the region .
- There are no zeroes with and .
Then one has .

The first hypothesis is already known for up to about (we should find out exactly what we can reach here). Preliminary calculations suggest that we can obtain the third item provided that . The second hypothesis requires good numerical calculation for , to which we now turn.

The initial definition of is given by the formula

where

This formula has proven numerically computable to acceptable error up until about the first hundred zeroes of , but degrades after that, and so other exact or approximate formulae for are needed. One possible exact formula that could be useful is

where

and

and can be chosen arbitrarily. We are still trying to see if this can be implemented numerically to give better accuracy than the previous formula.

It seems particularly promising to develop a generalisation of the Riemann-Siegel approximate functional equation for . Preliminary computations suggest in particular that we have the approximation

where

Some very preliminary numerics suggest that this formula is reasonably accurate even for moderate values of , though further numerical verification is needed. As a proof of concept, one could take this approximation as exact for the purposes of seeing what ranges of one can feasibly compute with (and for extremely large values of , we will presumably have to introduce some version of the Odlyzko-Schönhage algorithm. Of course, to obtain a rigorous result, we will eventually need a rigorous version of this formula with explicit error bounds. It may also be necessary to add more terms to the approximation to reduce the size of the error.

Sujit Nair has kindly summarised for me the current state of affairs with the numerics as follows:

—

- We need a real milestone and work backward to set up intermediate goals. This will definitely help bring in focus!
- So far, we have some utilities to compute zeroes of with a nonlinear solver which uses roots of as an initial condition. The solver is a wrapper around MINPACK’s implementation of Powell’s method. There is some room for optimization. For example, we aren’t providing the solver with the analytical Jacobian which speeds up the computation and increases accuracy.
- We have some results in the output folder which contains the first 1000 roots of for some small values of , etc. They need some more organization and visualization.

We have a decent initial start but we have some ways to go. Moving forward, here is my proposition for some areas of focus. We should expand and prioritize after some open discussion.

- Short term Optimize the existing framework and target to have the first million zeros of (for a reasonable range of ) and the corresponding plots. With better engineering practice and discipline, I am confident we can get to a few tens of millions range. Some things which will help include parallelization, iterative approaches (using zeroes of to compute zeroes of ), etc.
- Medium term We need to explore better ways to represent the zeros and compute them. An analogy is the computation of Riemann zeroes up to height . It is computed by computing the sign changes of (page 119 of Edwards) and by exploiting the speed up with the Riemann-Siegel formulation (over Euler-Maclaurin). For larger values of , I am not sure the root solver based approach is going to work to understand the gaps between zeroes.
- Long term We also need a better understanding of the errors involved in the computation — truncation, hardware/software, etc.

This is the first official “research” thread of the Polymath15 project to upper bound the de Bruijn-Newman constant . Discussion of the project of a non-research nature can continue for now in the existing proposal thread. Progress will be summarised at this Polymath wiki page.

The proposal naturally splits into at least three separate (but loosely related) topics:

- Numerical computation of the entire functions , with the ultimate aim of establishing zero-free regions of the form for various .
- Improved understanding of the dynamics of the zeroes of .
- Establishing the zero-free nature of when and is sufficiently large depending on and .

Below the fold, I will present each of these topics in turn, to initiate further discussion in each of them. (I thought about splitting this post into three to have three separate discussions, but given the current volume of comments, I think we should be able to manage for now having all the comments in a single post. If this changes then of course we can split up some of the discussion later.)

To begin with, let me present some formulae for computing (inspired by similar computations in the Ki-Kim-Lee paper) which may be useful. The initial definition of is

where

is a variant of the Jacobi theta function. We observe that in fact extends analytically to the strip

as has positive real part on this strip. One can use the Poisson summation formula to verify that is even, (see this previous post for details). This lets us obtain a number of other formulae for . Most obviously, one can unfold the integral to obtain

In my previous paper with Brad, we used this representation, combined with Fubini’s theorem to swap the sum and integral, to obtain a useful series representation for in the case because expressions such as diverge as approaches . Nevertheless we can still perform the following contour integration manipulation. Let be fixed. The function decays super-exponentially fast (much faster than , in particular) as with ; as is even, we also have this decay as with (this is despite each of the summands in having much slower decay in this direction – there is considerable cancellation!). Hence by the Cauchy integral formula we have

Splitting the horizontal line from to at and using the even nature of , we thus have

Using the functional equation , we thus have the representation

where is the oscillatory integral

The formula (2) is valid for any . Naively one would think that it would be simplest to take ; however, when and is large (with bounded), it seems asymptotically better to take closer to , in particular something like seems to be a reasonably good choice. This is because the integrand in (3) becomes significantly less oscillatory and also much lower in amplitude; the term in (3) now generates a factor roughly comparable to (which, as we will see below, is the main term in the decay asymptotics for ), while the term still exhibits a reasonable amount of decay as . We will use the representation (2) in the asymptotic analysis of below, but it may also be a useful representation to use for numerical purposes.

Brad Rodgers and I have uploaded to the arXiv our paper “The De Bruijn-Newman constant is non-negative“. This paper affirms a conjecture of Newman regarding to the extent to which the Riemann hypothesis, if true, is only “barely so”. To describe the conjecture, let us begin with the Riemann xi function

where is the Gamma function and is the Riemann zeta function. Initially, this function is only defined for , but, as was already known to Riemann, we can manipulate it into a form that extends to the entire complex plane as follows. Firstly, in view of the standard identity , we can write

and hence

By a rescaling, one may write

and similarly

and thus (after applying Fubini’s theorem)

We’ll make the change of variables to obtain

If we introduce the mild renormalisation

of , we then conclude (at least for ) that

which one can verify to be rapidly decreasing both as and as , with the decrease as faster than any exponential. In particular extends holomorphically to the upper half plane.

If we normalize the Fourier transform of a (Schwartz) function as , it is well known that the Gaussian is its own Fourier transform. The creation operator interacts with the Fourier transform by the identity

Since , this implies that the function

is its own Fourier transform. (One can view the polynomial as a renormalised version of the fourth Hermite polynomial.) Taking a suitable linear combination of this with , we conclude that

is also its own Fourier transform. Rescaling by and then multiplying by , we conclude that the Fourier transform of

is

and hence by the Poisson summation formula (using symmetry and vanishing at to unfold the summation in (2) to the integers rather than the natural numbers) we obtain the functional equation

which implies that and are even functions (in particular, now extends to an entire function). From this symmetry we can also rewrite (1) as

which now gives a convergent expression for the entire function for all complex . As is even and real-valued on , is even and also obeys the functional equation , which is equivalent to the usual functional equation for the Riemann zeta function. The Riemann hypothesis is equivalent to the claim that all the zeroes of are real.

De Bruijn introduced the family of deformations of , defined for all and by the formula

From a PDE perspective, one can view as the evolution of under the backwards heat equation . As with , the are all even entire functions that obey the functional equation , and one can ask an analogue of the Riemann hypothesis for each such , namely whether all the zeroes of are real. De Bruijn showed that these hypotheses were monotone in : if had all real zeroes for some , then would also have all zeroes real for any . Newman later sharpened this claim by showing the existence of a finite number , now known as the *de Bruijn-Newman constant*, with the property that had all zeroes real if and only if . Thus, the Riemann hypothesis is equivalent to the inequality . Newman then conjectured the complementary bound ; in his words, this conjecture asserted that if the Riemann hypothesis is true, then it is only “barely so”, in that the reality of all the zeroes is destroyed by applying heat flow for even an arbitrarily small amount of time. Over time, a significant amount of evidence was established in favour of this conjecture; most recently, in 2011, Saouter, Gourdon, and Demichel showed that .

In this paper we finish off the proof of Newman’s conjecture, that is we show that . The proof is by contradiction, assuming that (which among other things, implies the truth of the Riemann hypothesis), and using the properties of backwards heat evolution to reach a contradiction.

Very roughly, the argument proceeds as follows. As observed by Csordas, Smith, and Varga (and also discussed in this previous blog post, the backwards heat evolution of the introduces a nice ODE dynamics on the zeroes of , namely that they solve the ODE

for all (one has to interpret the sum in a principal value sense as it is not absolutely convergent, but let us ignore this technicality for the current discussion). Intuitively, this ODE is asserting that the zeroes repel each other, somewhat like positively charged particles (but note that the dynamics is first-order, as opposed to the second-order laws of Newtonian mechanics). Formally, a steady state (or equilibrium) of this dynamics is reached when the are arranged in an arithmetic progression. (Note for instance that for any positive , the functions obey the same backwards heat equation as , and their zeroes are on a fixed arithmetic progression .) The strategy is to then show that the dynamics from time to time creates a *convergence to local equilibrium*, in which the zeroes locally resemble an arithmetic progression at time . This will be in contradiction with known results on pair correlation of zeroes (or on related statistics, such as the fluctuations on gaps between zeroes), such as the results of Montgomery (actually for technical reasons it is slightly more convenient for us to use related results of Conrey, Ghosh, Goldston, Gonek, and Heath-Brown). Another way of thinking about this is that even very slight deviations from local equilibrium (such as a small number of gaps that are slightly smaller than the average spacing) will almost immediately lead to zeroes colliding with each other and leaving the real line as one evolves backwards in time (i.e., under the *forward* heat flow). This is a refinement of the strategy used in previous lower bounds on , in which “Lehmer pairs” (pairs of zeroes of the zeta function that were unusually close to each other) were used to limit the extent to which the evolution continued backwards in time while keeping all zeroes real.

How does one obtain this convergence to local equilibrium? We proceed by broad analogy with the “local relaxation flow” method of Erdos, Schlein, and Yau in random matrix theory, in which one combines some initial control on zeroes (which, in the case of the Erdos-Schlein-Yau method, is referred to with terms such as “local semicircular law”) with convexity properties of a relevant Hamiltonian that can be used to force the zeroes towards equilibrium.

We first discuss the initial control on zeroes. For , we have the classical Riemann-von Mangoldt formula, which asserts that the number of zeroes in the interval is as . (We have a factor of here instead of the more familiar due to the way is normalised.) This implies for instance that for a fixed , the number of zeroes in the interval is . Actually, because we get to assume the Riemann hypothesis, we can sharpen this to , a result of Littlewood (see this previous blog post for a proof). Ideally, we would like to obtain similar control for the other , , as well. Unfortunately we were only able to obtain the weaker claims that the number of zeroes of in is , and that the number of zeroes in is , that is to say we only get good control on the distribution of zeroes at scales rather than at scales . Ultimately this is because we were only able to get control (and in particular, lower bounds) on with high precision when (whereas has good estimates as soon as is larger than (say) ). This control is obtained by the expressing in terms of some contour integrals and using the method of steepest descent (actually it is slightly simpler to rely instead on the Stirling approximation for the Gamma function, which can be proven in turn by steepest descent methods). Fortunately, it turns out that this weaker control is still (barely) enough for the rest of our argument to go through.

Once one has the initial control on zeroes, we now need to force convergence to local equilibrium by exploiting convexity of a Hamiltonian. Here, the relevant Hamiltonian is

ignoring for now the rather important technical issue that this sum is not actually absolutely convergent. (Because of this, we will need to truncate and renormalise the Hamiltonian in a number of ways which we will not detail here.) The ODE (3) is formally the gradient flow for this Hamiltonian. Furthermore, this Hamiltonian is a convex function of the (because is a convex function on ). We therefore expect the Hamiltonian to be a decreasing function of time, and that the derivative should be an increasing function of time. As time passes, the derivative of the Hamiltonian would then be expected to converge to zero, which should imply convergence to local equilibrium.

Formally, the derivative of the above Hamiltonian is

Again, there is the important technical issue that this quantity is infinite; but it turns out that if we renormalise the Hamiltonian appropriately, then the energy will also become suitably renormalised, and in particular will vanish when the are arranged in an arithmetic progression, and be positive otherwise. One can also formally calculate the derivative of to be a somewhat complicated but manifestly non-negative quantity (a sum of squares); see this previous blog post for analogous computations in the case of heat flow on polynomials. After flowing from time to time , and using some crude initial bounds on and in this region (coming from the Riemann-von Mangoldt type formulae mentioned above and some further manipulations), we can eventually show that the (renormalisation of the) energy at time zero is small, which forces the to locally resemble an arithmetic progression, which gives the required convergence to local equilibrium.

There are a number of technicalities involved in making the above sketch of argument rigorous (for instance, justifying interchanges of derivatives and infinite sums turns out to be a little bit delicate). I will highlight here one particular technical point. One of the ways in which we make expressions such as the energy finite is to truncate the indices to an interval to create a truncated energy . In typical situations, we would then expect to be decreasing, which will greatly help in bounding (in particular it would allow one to control by time-averaged quantities such as , which can in turn be controlled using variants of (4)). However, there are boundary effects at both ends of that could in principle add a large amount of energy into , which is bad news as it could conceivably make undesirably large even if integrated energies such as remain adequately controlled. As it turns out, such boundary effects are negligible as long as there is a large gap between adjacent zeroes at boundary of – it is only narrow gaps that can rapidly transmit energy across the boundary of . Now, narrow gaps can certainly exist (indeed, the GUE hypothesis predicts these happen a positive fraction of the time); but the pigeonhole principle (together with the Riemann-von Mangoldt formula) can allow us to pick the endpoints of the interval so that no narrow gaps appear at the boundary of for any given time . However, there was a technical problem: this argument did not allow one to find a single interval that avoided gaps for *all* times simultaneously – the pigeonhole principle could produce a different interval for each time ! Since the number of times was uncountable, this was a serious issue. (In physical terms, the problem was that there might be very fast “longitudinal waves” in the dynamics that, at each time, cause some gaps between zeroes to be highly compressed, but the specific gap that was narrow changed very rapidly with time. Such waves could, in principle, import a huge amount of energy into by time .) To resolve this, we borrowed a PDE trick of Bourgain’s, in which the pigeonhole principle was coupled with local conservation laws. More specifically, we use the phenomenon that very narrow gaps take a nontrivial amount of time to expand back to a reasonable size (this can be seen by comparing the evolution of this gap with solutions of the scalar ODE , which represents the fastest at which a gap such as can expand). Thus, if a gap is reasonably large at some time , it will also stay reasonably large at slightly earlier times for some moderately small . This lets one locate an interval that has manageable boundary effects during the times in , so in particular is basically non-increasing in this time interval. Unfortunately, this interval is a little bit too short to cover all of ; however it turns out that one can iterate the above construction and find a nested sequence of intervals , with each non-increasing in a different time interval , and with all of the time intervals covering . This turns out to be enough (together with the obvious fact that is monotone in ) to still control for some reasonably sized interval , as required for the rest of the arguments.

ADDED LATER: the following analogy (involving functions with just two zeroes, rather than an infinite number of zeroes) may help clarify the relation between this result and the Riemann hypothesis (and in particular why this result does not make the Riemann hypothesis any easier to prove, in fact it confirms the delicate nature of that hypothesis). Suppose one had a quadratic polynomial of the form , where was an unknown real constant. Suppose that one was for some reason interested in the analogue of the “Riemann hypothesis” for , namely that all the zeroes of are real. A priori, there are three scenarios:

- (Riemann hypothesis false) , and has zeroes off the real axis.
- (Riemann hypothesis true, but barely so) , and both zeroes of are on the real axis; however, any slight perturbation of in the positive direction would move zeroes off the real axis.
- (Riemann hypothesis true, with room to spare) , and both zeroes of are on the real axis. Furthermore, any slight perturbation of will also have both zeroes on the real axis.

The analogue of our result in this case is that , thus ruling out the third of the three scenarios here. In this simple example in which only two zeroes are involved, one can think of the inequality as asserting that if the zeroes of are real, then they must be repeated. In our result (in which there are an infinity of zeroes, that become increasingly dense near infinity), and in view of the convergence to local equilibrium properties of (3), the analogous assertion is that if the zeroes of are real, then they do not behave locally as if they were in arithmetic progression.

Apoorva Khare and I have updated our paper “On the sign patterns of entrywise positivity preservers in fixed dimension“, announced at this post from last month. The quantitative results are now sharpened using a new monotonicity property of ratios of Schur polynomials, namely that such ratios are monotone non-decreasing in each coordinate of if is in the positive orthant, and the partition is larger than that of . (This monotonicity was also independently observed by Rachid Ait-Haddou, using the theory of blossoms.) In the revised version of the paper we give two proofs of this monotonicity. The first relies on a deep positivity result of Lam, Postnikov, and Pylyavskyy, which uses a representation-theoretic positivity result of Haiman to show that the polynomial combination

of skew-Schur polynomials is Schur-positive for any partitions (using the convention that the skew-Schur polynomial vanishes if is not contained in , and where and denotes the pointwise min and max of and respectively). It is fairly easy to derive the monotonicity of from this, by using the expansion

of Schur polynomials into skew-Schur polynomials (as was done in this previous post).

The second proof of monotonicity avoids representation theory by a more elementary argument establishing the weaker claim that the above expression (1) is non-negative on the positive orthant. In fact we prove a more general determinantal log-supermodularity claim which may be of independent interest:

Theorem 1Let be any totally positive matrix (thus, every minor has a non-negative determinant). Then for any -tuples of increasing elements of , one haswhere denotes the minor formed from the rows in and columns in .

For instance, if is the matrix

for some real numbers , one has

(corresponding to the case , ), or

(corresponding to the case , , , , ). It turns out that this claim can be proven relatively easy by an induction argument, relying on the Dodgson and Karlin identities from this previous post; the difficulties are largely notational in nature. Combining this result with the Jacobi-Trudi identity for skew-Schur polynomials (discussed in this previous post) gives the non-negativity of (1); it can also be used to directly establish the monotonicity of ratios by applying the theorem to a generalised Vandermonde matrix.

(Log-supermodularity also arises as the natural hypothesis for the FKG inequality, though I do not know of any interesting application of the FKG inequality in this current setting.)

In one of the earliest posts on this blog, I talked about the ability to “arbitrage” a disparity of symmetry in an inequality, and in particular to “amplify” such an inequality into a stronger one. (The principle can apply to other mathematical statements than inequalities, with the “hypothesis” and “conclusion” of that statement generally playing the role of the “right-hand side” and “left-hand side” of an inequality, but for sake of discussion I will restrict attention here to inequalities.) One can formalise this principle as follows. Many inequalities in analysis can be expressed in the form

for all in some space (in many cases will be a function space, and a function in that space), where and are some functionals of (that is to say, real-valued functions of ). For instance, might be some function space norm of (e.g. an norm), and might be some function space norm of some transform of . In addition, we assume we have some group of symmetries acting on the underlying space. For instance, if is a space of functions on some spatial domain, the group might consist of translations (e.g. for some shift ), or perhaps dilations with some normalisation (e.g. for some dilation factor and some normalisation exponent , which can be thought of as the dimensionality of length one is assigning to ). If we have

for all symmetries and all , we say that is *invariant* with respect to the symmetries in ; otherwise, it is not.

Suppose we know that the inequality (1) holds for all , but that there is an imbalance of symmetry: either is -invariant and is not, or vice versa. Suppose first that is -invariant and is not. Substituting by in (1) and taking infima, we can then amplify (1) to the stronger inequality

In particular, it is often the case that there is a way to send off to infinity in such a way that the functional has a limit , in which case we obtain the amplification

of (1). Note that these amplified inequalities will now be -invariant on both sides (assuming that the way in which we take limits as is itself -invariant, which it often is in practice). Similarly, if is -invariant but is not, we may instead amplify (1) to

and in particular (if has a limit as )

If neither nor has a -symmetry, one can still use the -symmetry by replacing by and taking a limit to conclude that

though now this inequality is not obviously stronger than the original inequality (1) (for instance it could well be trivial). In some cases one can also average over instead of taking a limit as , thus averaging a non-invariant inequality into an invariant one.

As discussed in the previous post, this use of amplification gives rise to a general principle about inequalities: *the most efficient inequalities are those in which the left-hand side and right-hand side enjoy the same symmetries*. It is certainly possible to have true inequalities that have an imbalance of symmetry, but as shown above, such inequalities can always be amplified to more efficient and more symmetric inequalities. In the case when limits such as and exist, the limiting functionals and are often simpler in form, or more tractable analytically, than their non-limiting counterparts and (this is one of the main reasons *why* we take limits at infinity in the first place!), and so in many applications there is really no reason to use the weaker and more complicated inequality (1), when stronger, simpler, and more symmetric inequalities such as (2), (3) are available. Among other things, this explains why many of the most useful and natural inequalities one sees in analysis are dimensionally consistent.

One often tries to prove inequalities (1) by directly chaining together simpler inequalities. For instance, one might attempt to prove (1) by by first bounding by some auxiliary quantity , and then bounding by , thus obtaining (1) by chaining together two inequalities

A variant of the above principle then asserts that *when proving inequalities by such direct methods, one should, whenever possible, try to maintain the symmetries that are present in both sides of the inequality*. Why? Well, suppose that we ignored this principle and tried to prove (1) by establishing (4) for some that is *not* -invariant. Assuming for sake of argument that (4) were actually true, we could amplify the first half of this inequality to conclude that

and also amplify the second half of the inequality to conclude that

and hence (4) amplifies to

Let’s say for sake of argument that all the quantities involved here are positive numbers (which is often the case in analysis). Then we see in particular that

Informally, (6) asserts that in order for the strategy (4) used to prove (1) to work, the extent to which fails to be -invariant cannot exceed the amount of “room” present in (1). In particular, when dealing with those “extremal” for which the left and right-hand sides of (1) are comparable to each other, one can only have a bounded amount of non--invariance in the functional . If fails so badly to be -invariant that one does not expect the left-hand side of (6) to be at all bounded in such extremal situations, then the strategy of proving (1) using the intermediate quantity is doomed to failure – even if one has already produced some clever proof of one of the two inequalities or needed to make this strategy work. And even if it did work, one could amplify (4) to a simpler inequality

(assuming that the appropriate limit existed) which would likely also be easier to prove (one can take whatever proofs one had in mind of the inequalities in (4), conjugate them by , and take a limit as to extract a proof of (7)).

Here are some simple (but somewhat contrived) examples to illustrate these points. Suppose one wishes to prove the inequality

for all . Both sides of this inequality are invariant with respect to interchanging with , so the principle suggests that when proving this inequality directly, one should only use sub-inequalities that are also invariant with respect to this interchange. However, in this particular case there is enough “room” in the inequality that it is possible (though somewhat unnatural) to violate this principle. For instance, one could decide (for whatever reason) to start with the inequality

to conclude that

and then use the obvious inequality to conclude the proof. Here, the intermediate quantity is not invariant with respect to interchange of and , but the failure is fairly mild (changing and only modifies the quantity by a multiplicative factor of at most), and disappears completely in the most extremal case , which helps explain why one could get away with using this quantity in the proof here. But it would be significantly harder (though still not impossible) to use non-symmetric intermediaries to prove the sharp version

of (8) (that is to say, the arithmetic mean-geometric mean inequality). Try it!

Similarly, consider the task of proving the triangle inequality

for complex numbers . One could try to leverage the triangle inequality for real numbers by using the crude estimate

and then use the real triangle inequality to obtain

and

and then finally use the inequalities

but when one puts this all together at the end of the day, one loses a factor of two:

One can “blame” this loss on the fact that while the original inequality (9) was invariant with respect to phase rotation , the intermediate expressions we tried to use when proving it were not, leading to inefficient estimates. One can try to be smarter than this by using Pythagoras’ theorem ; this reduces the loss from to but does not eliminate it completely, which is to be expected as one is still using non-invariant estimates in the proof. But one can remove the loss completely by using amplification; see the previous blog post for details (we also give a reformulation of this amplification below).

Here is a slight variant of the above example. Suppose that you had just learned in class to prove the triangle inequality

for (say) real square-summable sequences , , and was tasked to conclude the corresponding inequality

for doubly infinite square-summable sequences . The quickest way to do this is of course to exploit a bijection between the natural numbers and the integers, but let us say for sake of argument that one was unaware of such a bijection. One could then proceed instead by splitting the integers into the positive integers and the non-positive integers, and use (12) on each component separately; this is very similar to the strategy of proving (9) by splitting a complex number into real and imaginary parts, and will similarly lose a factor of or . In this case, one can “blame” this loss on the abandonment of translation invariance: both sides of the inequality (13) are invariant with respect to shifting the sequences , by some shift to arrive at , but the intermediate quantities caused by splitting the integers into two subsets are not invariant. Another way of thinking about this is that the splitting of the integers gives a privileged role to the origin , whereas the inequality (13) treats all values of equally thanks to the translation invariance, and so using such a splitting is unnatural and not likely to lead to optimal estimates. On the other hand, one can deduce (13) from (12) by sending this symmetry to infinity; indeed, after applying a shift to (12) we see that

for any , and on sending we obtain (13) (one could invoke the monotone convergence theorem here to justify the limit, though in this case it is simple enough that one can just use first principles).

Note that the principle of preserving symmetry only applies to *direct* approaches to proving inequalities such as (1). There is a complementary approach, discussed for instance in this previous post, which is to *spend* the symmetry to place the variable “without loss of generality” in a “normal form”, “convenient coordinate system”, or a “good gauge”. Abstractly: suppose that there is some subset of with the property that every can be expressed in the form for some and (that is to say, ). Then, if one wishes to prove an inequality (1) for all , and one knows that both sides of this inequality are -invariant, then it suffices to check (1) just for those in , as this together with the -invariance will imply the same inequality (1) for all in . By restricting to those in , one has given up (or *spent*) the -invariance, as the set will in typical not be preserved by the group action . But by the same token, by eliminating the invariance, one also eliminates the prohibition on using non-invariant proof techniques, and one is now free to use a wider range of inequalities in order to try to establish (1). Of course, such inequalities should make crucial use of the restriction , for if they did not, then the arguments would work in the more general setting , and then the previous principle would again kick in and warn us that the use of non-invariant inequalities would be inefficient. Thus one should “spend” the symmetry wisely to “buy” a restriction that will be of maximal utility in calculations (for instance by setting as many annoying factors and terms in one’s analysis to be or as possible).

As a simple example of this, let us revisit the complex triangle inequality (9). As already noted, both sides of this inequality are invariant with respect to the phase rotation symmetry . This seems to limit one to using phase-rotation-invariant techniques to establish the inequality, in particular ruling out the use of real and imaginary parts as discussed previously. However, we can instead *spend* the phase rotation symmetry to restrict to a special class of and . It turns out that the most efficient way to spend the symmetry is to achieve the normalisation of being a nonnegative real; this is of course possible since any complex number can be turned into a nonnegative real by multiplying by an appropriate phase . Once is a nonnegative real, the imaginary part disappears and we have

and the triangle inequality (9) is now an immediate consequence of (10), (11). (But note that if one had unwisely spent the symmetry to normalise, say, to be a non-negative real, then one is no closer to establishing (9) than before one had spent the symmetry.)

Apoorva Khare and I have just uploaded to the arXiv our paper “On the sign patterns of entrywise positivity preservers in fixed dimension“. This paper explores the relationship between positive definiteness of Hermitian matrices, and entrywise operations on these matrices. The starting point for this theory is the Schur product theorem, which asserts that if and are two Hermitian matrices that are positive semi-definite, then their Hadamard product

is also positive semi-definite. (One should caution that the Hadamard product is *not* the same as the usual matrix product.) To prove this theorem, first observe that the claim is easy when and are rank one positive semi-definite matrices, since in this case is also a rank one positive semi-definite matrix. The general case then follows by noting from the spectral theorem that a general positive semi-definite matrix can be expressed as a non-negative linear combination of rank one positive semi-definite matrices, and using the bilinearity of the Hadamard product and the fact that the set of positive semi-definite matrices form a convex cone. A modification of this argument also lets one replace “positive semi-definite” by “positive definite” in the statement of the Schur product theorem.

One corollary of the Schur product theorem is that any polynomial with non-negative coefficients is *entrywise positivity preserving* on the space of positive semi-definite Hermitian matrices, in the sense that for any matrix in , the entrywise application

of to is also positive semi-definite. (As before, one should caution that is *not* the application of to by the usual functional calculus.) Indeed, one can expand

where is the Hadamard product of copies of , and the claim now follows from the Schur product theorem and the fact that is a convex cone.

A slight variant of this argument, already observed by Pólya and Szegö in 1925, shows that if is any subset of and

is a power series with non-negative coefficients that is absolutely and uniformly convergent on , then will be entrywise positivity preserving on the set of positive definite matrices with entries in . (In the case that is of the form , such functions are precisely the absolutely monotonic functions on .)

In the work of Schoenberg and of Rudin, we have a converse: if is a function that is entrywise positivity preserving on for all , then it must be of the form (1) with . Variants of this result, with replaced by other domains, appear in the work of Horn, Vasudeva, and Guillot-Khare-Rajaratnam.

This gives a satisfactory classification of functions that are entrywise positivity preservers in all dimensions simultaneously. However, the question remains as to what happens if one fixes the dimension , in which case one may have a larger class of entrywise positivity preservers. For instance, in the trivial case , a function would be entrywise positivity preserving on if and only if is non-negative on . For higher , there is a necessary condition of Horn (refined slightly by Guillot-Khare-Rajaratnam) which asserts (at least in the case of smooth ) that all derivatives of at zero up to order must be non-negative in order for to be entrywise positivity preserving on for some . In particular, if is of the form (1), then must be non-negative. In fact, a stronger assertion can be made, namely that the first non-zero coefficients in (1) (if they exist) must be positive, or equivalently any negative term in (1) must be preceded (though not necessarily immediately) by at least positive terms. If is of the form (1) is entrywise positivity preserving on the larger set , one can furthermore show that any negative term in (1) must also be *followed* (though not necessarily immediately) by at least positive terms.

The main result of this paper is that these sign conditions are the *only* constraints for entrywise positivity preserving power series. More precisely:

Theorem 1For each , let be a sign.

- Suppose that any negative sign is preceded by at least positive signs (thus there exists with ). Then, for any , there exists a convergent power series (1) on , with each having the sign of , which is entrywise positivity preserving on .
- Suppose in addition that any negative sign is followed by at least positive signs (thus there exists with ). Then there exists a convergent power series (1) on , with each having the sign of , which is entrywise positivity preserving on .

One can ask the same question with or replaced by other domains such as , or the complex disk , but it turns out that there are far fewer entrywise positivity preserving functions in those cases basically because of the non-trivial zeroes of Schur polynomials in these ranges; see the paper for further discussion. We also have some quantitative bounds on how negative some of the coefficients can be compared to the positive coefficients, but they are a bit technical to state here.

The heart of the proofs of these results is an analysis of the determinants of polynomials applied entrywise to rank one matrices ; the positivity of these determinants can be used (together with a continuity argument) to establish the positive definiteness of for various ranges of and . Using the Cauchy-Binet formula, one can rewrite such determinants as linear combinations of squares of magnitudes of generalised Vandermonde determinants

where and the signs of the coefficients in the linear combination are determined by the signs of the coefficients of . The task is then to find upper and lower bounds for the magnitudes of such generalised Vandermonde determinants. These determinants oscillate in sign, which makes the problem look difficult; however, an algebraic miracle intervenes, namely the factorisation

of the generalised Vandermonde determinant into the ordinary Vandermonde determinant

and a Schur polynomial applied to , where the weight of the Schur polynomial is determined by in a simple fashion. The problem then boils down to obtaining upper and lower bounds for these Schur polynomials. Because we are restricting attention to matrices taking values in or , the entries of can be taken to be non-negative. One can then take advantage of the *total positivity* of the Schur polynomials to compare these polynomials with a monomial, at which point one can obtain good criteria for to be positive definite when is a rank one positive definite matrix .

If we allow the exponents to be real numbers rather than integers (thus replacing polynomials or power series by Pusieux series or Hahn series), then we lose the above algebraic miracle, but we can replace it with a geometric miracle, namely the *Harish-Chandra-Itzykson-Zuber identity*, which I discussed in this previous blog post. This factors the above generalised Vandermonde determinant as the product of the ordinary Vandermonde determinant and an integral of a positive quantity over the orthogonal group, which one can again compare with a monomial after some fairly elementary estimates.

It remains to understand what happens for more general positive semi-definite matrices . Here we use a trick of FitzGerald and Horn to amplify the rank one case to the general case, by expressing a general positive semi-definite matrix as a linear combination of a rank one matrix and another positive semi-definite matrix that vanishes on the last row and column (and is thus effectively a positive definite matrix). Using the fundamental theorem of calculus to continuously deform the rank one matrix to in the direction , one can then obtain positivity results for from positivity results for combined with an induction hypothesis on .

The complete homogeneous symmetric polynomial of variables and degree can be defined as

thus for instance

and

One can also define all the complete homogeneous symmetric polynomials of variables simultaneously by means of the generating function

We will think of the variables as taking values in the real numbers. When one does so, one might observe that the degree two polynomial is a positive definite quadratic form, since it has the sum of squares representation

In particular, unless . This can be compared against the superficially similar quadratic form

where are independent randomly chosen signs. The *Wigner semicircle law* says that for large , the eigenvalues of this form will be mostly distributed in the interval using the semicircle distribution, so in particular the form is quite far from being positive definite despite the presence of the first positive terms. Thus the positive definiteness is coming from the finer algebraic structure of , and not just from the magnitudes of its coefficients.

One could ask whether the same positivity holds for other degrees than two. For odd degrees, the answer is clearly no, since in that case. But one could hope for instance that

also has a sum of squares representation that demonstrates positive definiteness. This turns out to be true, but is remarkably tedious to establish directly. Nevertheless, we have a nice result of Hunter that gives positive definiteness for all even degrees . In fact, a modification of his argument gives a little bit more:

Theorem 1Let , let be even, and let be reals.

- (i) (Positive definiteness) One has , with strict inequality unless .
- (ii) (Schur convexity) One has whenever majorises , with equality if and only if is a permutation of .
- (iii) (Schur-Ostrowski criterion for Schur convexity) For any , one has , with strict inequality unless .

*Proof:* We induct on (allowing to be arbitrary). The claim is trivially true for , and easily verified for , so suppose that and the claims (i), (ii), (iii) have already been proven for (and for arbitrary ).

If we apply the differential operator to using the product rule, one obtains after a brief calculation

Using (1) and extracting the coefficient, we obtain the identity

The claim (iii) then follows from (i) and the induction hypothesis.

To obtain (ii), we use the more general statement (known as the *Schur-Ostrowski criterion*) that (ii) is implied from (iii) if we replace by an arbitrary symmetric, continuously differentiable function. To establish this criterion, we induct on (this argument can be made independently of the existing induction on ). If is majorised by , it lies in the permutahedron of . If lies on a face of this permutahedron, then after permuting both the and we may assume that is majorised by , and is majorised by for some , and the claim then follows from two applications of the induction hypothesis. If instead lies in the interior of the permutahedron, one can follow it to the boundary by using one of the vector fields , and the claim follows from the boundary case.

Finally, to obtain (i), we observe that majorises , where is the arithmetic mean of . But is clearly a positive multiple of , and the claim now follows from (ii).

If the variables are restricted to be nonnegative, the same argument gives Schur convexity for odd degrees also.

The proof in Hunter of positive definiteness is arranged a little differently than the one above, but still relies ultimately on the identity (2). I wonder if there is a genuinely different way to establish positive definiteness that does not go through this identity.

Suppose is a continuous (but nonlinear) map from one normed vector space to another . The continuity means, roughly speaking, that if are such that is small, then is also small (though the precise notion of “smallness” may depend on or , particularly if is not known to be uniformly continuous). If is known to be differentiable (in, say, the Fréchet sense), then we in fact have a linear bound of the form

for some depending on , if is small enough; one can of course make independent of (and drop the smallness condition) if is known instead to be Lipschitz continuous.

In many applications in analysis, one would like more explicit and quantitative bounds that estimate quantities like in terms of quantities like . There are a number of ways to do this. First of all, there is of course the trivial estimate arising from the triangle inequality:

This estimate is usually not very good when and are close together. However, when and are far apart, this estimate can be more or less sharp. For instance, if the magnitude of varies so much from to that is more than (say) twice that of , or vice versa, then (1) is sharp up to a multiplicative constant. Also, if is oscillatory in nature, and the distance between and exceeds the “wavelength” of the oscillation of at (or at ), then one also typically expects (1) to be close to sharp. Conversely, if does not vary much in magnitude from to , and the distance between and is less than the wavelength of any oscillation present in , one expects to be able to improve upon (1).

When is relatively simple in form, one can sometimes proceed simply by substituting . For instance, if is the squaring function in a commutative ring , one has

and thus

or in terms of the original variables one has

If the ring is not commutative, one has to modify this to

Thus, for instance, if are matrices and denotes the operator norm, one sees from the triangle inequality and the sub-multiplicativity of operator norm that

If involves (or various components of ) in several places, one can sometimes get a good estimate by “swapping” with at each of the places in turn, using a telescoping series. For instance, if we again use the squaring function in a non-commutative ring, we have

which for instance leads to a slight improvement of (2):

More generally, for any natural number , one has the identity

in a commutative ring, while in a non-commutative ring one must modify this to

and for matrices one has

Exercise 1If and are unitary matrices, show that the commutator obeys the inequality(

Hint:first control .)

Now suppose (for simplicity) that is a map between Euclidean spaces. If is continuously differentiable, then one can use the fundamental theorem of calculus to write

where is any continuously differentiable path from to . For instance, if one uses the straight line path , one has

In the one-dimensional case , this simplifies to

Among other things, this immediately implies the factor theorem for functions: if is a function for some that vanishes at some point , then factors as the product of and some function . Another basic consequence is that if is uniformly bounded in magnitude by some constant , then is Lipschitz continuous with the same constant .

Applying (4) to the power function , we obtain the identity

which can be compared with (3). Indeed, for and close to , one can use logarithms and Taylor expansion to arrive at the approximation , so (3) behaves a little like a Riemann sum approximation to (5).

Exercise 2For each , let and be random variables taking values in a measurable space , and let be a bounded measurable function.

- (i) (Lindeberg exchange identity) Show that
- (ii) (Knowles-Yin exchange identity) Show that
where is a mixture of and , with uniformly drawn from independently of each other and of the .

- (iii) Discuss the relationship between the identities in parts (i), (ii) with the identities (3), (5).
(The identity in (i) is the starting point for the

Lindeberg exchange methodin probability theory, discussed for instance in this previous post. The identity in (ii) can also be used in the Lindeberg exchange method; the terms in the right-hand side are slightly more symmetric in the indices , which can be a technical advantage in some applications; see this paper of Knowles and Yin for an instance of this.)

Exercise 3If is continuously times differentiable, establish Taylor’s theorem with remainderIf is bounded, conclude that

For real scalar functions , the average value of the continuous real-valued function must be attained at some point in the interval . We thus conclude the mean-value theorem

for some (that can depend on , , and ). This can for instance give a second proof of fact that continuously differentiable functions with bounded derivative are Lipschitz continuous. However it is worth stressing that the mean-value theorem is only available for *real scalar* functions; it is false for instance for complex scalar functions. A basic counterexample is given by the function ; there is no for which . On the other hand, as has magnitude , we still know from (4) that is Lipschitz of constant , and when combined with (1) we obtain the basic bounds

which are already very useful for many applications.

Exercise 4Let be matrices, and let be a non-negative real.

- (i) Establish the Duhamel formula
where denotes the matrix exponential of . (

Hint:Differentiate or in .)- (ii) Establish the
iterated Duhamel formulafor any .

- (iii) Establish the infinitely iterated Duhamel formula
- (iv) If is an matrix depending in a continuously differentiable fashion on , establish the variation formula
where is the adjoint representation applied to , and is the function

(thus for non-zero ), with defined using functional calculus.

We remark that further manipulation of (iv) of the above exercise using the fundamental theorem of calculus eventually leads to the Baker-Campbell-Hausdorff-Dynkin formula, as discussed in this previous blog post.

Exercise 5Let be positive definite matrices, and let be an matrix. Show that there is a unique solution to the Sylvester equationwhich is given by the formula

In the above examples we had applied the fundamental theorem of calculus along linear curves . However, it is sometimes better to use other curves. For instance, the circular arc can be useful, particularly if and are “orthogonal” or “independent” in some sense; a good example of this is the proof by Maurey and Pisier of the gaussian concentration inequality, given in Theorem 8 of this previous blog post. In a similar vein, if one wishes to compare a scalar random variable of mean zero and variance one with a Gaussian random variable of mean zero and variance one, it can be useful to introduce the intermediate random variables (where and are independent); note that these variables have mean zero and variance one, and after coupling them together appropriately they evolve by the Ornstein-Uhlenbeck process, which has many useful properties. For instance, one can use these ideas to establish monotonicity formulae for entropy; see e.g. this paper of Courtade for an example of this and further references. More generally, one can exploit curves that flow according to some geometrically natural ODE or PDE; several examples of this occur famously in Perelman’s proof of the Poincaré conjecture via Ricci flow, discussed for instance in this previous set of lecture notes.

In some cases, it is difficult to compute or the derivative directly, but one can instead proceed by implicit differentiation, or some variant thereof. Consider for instance the matrix inversion map (defined on the open dense subset of matrices consisting of invertible matrices). If one wants to compute for close to , one can write temporarily write , thus

Multiplying both sides on the left by to eliminate the term, and on the right by to eliminate the term, one obtains

and thus on reversing these steps we arrive at the basic identity

For instance, if are matrices, and we consider the resolvents

then we have the *resolvent identity*

as long as does not lie in the spectrum of or (for instance, if , are self-adjoint then one can take to be any strictly complex number). One can iterate this identity to obtain

for any natural number ; in particular, if has operator norm less than one, one has the Neumann series

Similarly, if is a family of invertible matrices that depends in a continuously differentiable fashion on a time variable , then by implicitly differentiating the identity

in using the product rule, we obtain

and hence

(this identity may also be easily derived from (6)). One can then use the fundamental theorem of calculus to obtain variants of (6), for instance by using the curve we arrive at

assuming that the curve stays entirely within the set of invertible matrices. While this identity may seem more complicated than (6), it is more symmetric, which conveys some advantages. For instance, using this identity it is easy to see that if are positive definite with in the sense of positive definite matrices (that is, is positive definite), then . (Try to prove this using (6) instead!)

Exercise 6If is an invertible matrix and are vectors, establish the Sherman-Morrison formulawhenever is a scalar such that is non-zero. (See also this previous blog post for more discussion of these sorts of identities.)

One can use the Cauchy integral formula to extend these identities to other functions of matrices. For instance, if is an entire function, and is a counterclockwise contour that goes around the spectrum of both and , then we have

and similarly

and hence by (7) one has

similarly, if depends on in a continuously differentiable fashion, then

as long as goes around the spectrum of .

Exercise 7If is an matrix depending continuously differentiably on , and is an entire function, establish the tracial chain rule

In a similar vein, given that the logarithm function is the antiderivative of the reciprocal, one can express the matrix logarithm of a positive definite matrix by the fundamental theorem of calculus identity

(with the constant term needed to prevent a logarithmic divergence in the integral). Differentiating, we see that if is a family of positive definite matrices depending continuously on , that

This can be used for instance to show that is a monotone increasing function, in the sense that whenever in the sense of positive definite matrices. One can of course integrate this formula to obtain some formulae for the difference of the logarithm of two positive definite matrices .

To compare the square root of two positive definite matrices is trickier; there are multiple ways to proceed. One approach is to use contour integration as before (but one has to take some care to avoid branch cuts of the square root). Another to express the square root in terms of exponentials via the formula

where is the gamma function; this formula can be verified by first diagonalising to reduce to the scalar case and using the definition of the Gamma function. Then one has

and one can use some of the previous identities to control . This is pretty messy though. A third way to proceed is via implicit differentiation. If for instance is a family of positive definite matrices depending continuously differentiably on , we can differentiate the identity

to obtain

This can for instance be solved using Exercise 5 to obtain

and this can in turn be integrated to obtain a formula for . This is again a rather messy formula, but it does at least demonstrate that the square root is a monotone increasing function on positive definite matrices: implies .

Several of the above identities for matrices can be (carefully) extended to operators on Hilbert spaces provided that they are sufficiently well behaved (in particular, if they have a good functional calculus, and if various spectral hypotheses are obeyed). We will not attempt to do so here, however.

Suppose one has a bounded sequence of real numbers. What kinds of limits can one form from this sequence?

Of course, we have the usual notion of limit , which in this post I will refer to as the *classical limit* to distinguish from the other limits discussed in this post. The classical limit, if it exists, is the unique real number such that for every , one has for all sufficiently large . We say that a sequence is (classically) convergent if its classical limit exists. The classical limit obeys many useful *limit laws* when applied to classically convergent sequences. Firstly, it is linear: if and are classically convergent sequences, then is also classically convergent with

and similarly for any scalar , is classically convergent with

It is also an algebra homomorphism: is also classically convergent with

We also have shift invariance: if is classically convergent, then so is with

and more generally in fact for any injection , is classically convergent with

The classical limit of a sequence is unchanged if one modifies any finite number of elements of the sequence. Finally, we have boundedness: for any classically convergent sequence , one has

One can in fact show without much difficulty that these laws uniquely determine the classical limit functional on convergent sequences.

One would like to extend the classical limit notion to more general bounded sequences; however, when doing so one must give up one or more of the desirable limit laws that were listed above. Consider for instance the sequence . On the one hand, one has for all , so if one wishes to retain the homomorphism property (3), any “limit” of this sequence would have to necessarily square to , that is to say it must equal or . On the other hand, if one wished to retain the shift invariance property (4) as well as the homogeneity property (2), any “limit” of this sequence would have to equal its own negation and thus be zero.

Nevertheless there are a number of useful generalisations and variants of the classical limit concept for non-convergent sequences that obey a significant portion of the above limit laws. For instance, we have the limit superior

and limit inferior

which are well-defined real numbers for any bounded sequence ; they agree with the classical limit when the sequence is convergent, but disagree otherwise. They enjoy the shift-invariance property (4), and the boundedness property (6), but do not in general obey the homomorphism property (3) or the linearity property (1); indeed, we only have the subadditivity property

for the limit superior, and the superadditivity property

for the limit inferior. The homogeneity property (2) is only obeyed by the limits superior and inferior for non-negative ; for negative , one must have the limit inferior on one side of (2) and the limit superior on the other, thus for instance

The limit superior and limit inferior are examples of limit points of the sequence, which can for instance be defined as points that are limits of at least one subsequence of the original sequence. Indeed, the limit superior is always the largest limit point of the sequence, and the limit inferior is always the smallest limit point. However, limit points can be highly non-unique (indeed they are unique if and only if the sequence is classically convergent), and so it is difficult to sensibly interpret most of the usual limit laws in this setting, with the exception of the homogeneity property (2) and the boundedness property (6) that are easy to state for limit points.

Another notion of limit are the Césaro limits

if this limit exists, we say that the sequence is Césaro convergent. If the sequence already has a classical limit, then it also has a Césaro limit that agrees with the classical limit; but there are additional sequences that have a Césaro limit but not a classical one. For instance, the non-classically convergent sequence discussed above is Césaro convergent, with a Césaro limit of . However, there are still bounded sequences that do not have Césaro limit, such as (exercise!). The Césaro limit is linear, bounded, and shift invariant, but not an algebra homomorphism and also does not obey the rearrangement property (5).

Using the Hahn-Banach theorem, one can extend the classical limit functional to *generalised limit functionals* , defined to be bounded linear functionals from the space of bounded real sequences to the real numbers that extend the classical limit functional (defined on the space of convergent sequences) without any increase in the operator norm. (In some of my past writings I made the slight error of referring to these generalised limit functionals as Banach limits, though as discussed below, the latter actually refers to a subclass of generalised limit functionals.) It is not difficult to see that such generalised limit functionals will range between the limit inferior and limit superior. In fact, for any specific sequence and any number lying in the closed interval , there exists at least one generalised limit functional that takes the value when applied to ; for instance, for any number in , there exists a generalised limit functional that assigns that number as the “limit” of the sequence . This claim can be seen by first designing such a limit functional on the vector space spanned by the convergent sequences and by , and then appealing to the Hahn-Banach theorem to extend to all sequences. This observation also gives a necessary and sufficient criterion for a bounded sequence to classically converge to a limit , namely that all generalised limits of this sequence must equal .

Because of the reliance on the Hahn-Banach theorem, the existence of generalised limits requires the axiom of choice (or some weakened version thereof); there are presumably models of set theory without the axiom of choice in which no generalised limits exist, but I do not know of an explicit reference for this.

Generalised limits can obey the shift-invariance property (4) or the algebra homomorphism property (2), but as the above analysis of the sequence shows, they cannot do both. Generalised limits that obey the shift-invariance property (4) are known as Banach limits; one can for instance construct them by applying the Hahn-Banach theorem to the Césaro limit functional; alternatively, if is any generalised limit, then the Césaro-type functional will be a Banach limit. The existence of Banach limits can be viewed as a demonstration of the amenability of the natural numbers (or integers); see this previous blog post for further discussion.

Generalised limits that obey the algebra homomorphism property (2) are known as *ultrafilter limits*. If one is given a generalised limit functional that obeys (2), then for any subset of the natural numbers , the generalised limit must equal its own square (since ) and is thus either or . If one defines to be the collection of all subsets of for which , one can verify that obeys the axioms of a non-principal ultrafilter. Conversely, if is a non-principal ultrafilter, one can define the associated generalised limit of any bounded sequence to be the unique real number such that the sets lie in for all ; one can check that this does indeed give a well-defined generalised limit that obeys (2). Non-principal ultrafilters can be constructed using Zorn’s lemma. In fact, they do not quite need the full strength of the axiom of choice; see the Wikipedia article on the ultrafilter lemma for examples.

We have previously noted that generalised limits of a sequence can converge to any point between the limit inferior and limit superior. The same is not true if one restricts to Banach limits or ultrafilter limits. For instance, by the arguments already given, the only possible Banach limit for the sequence is zero. Meanwhile, an ultrafilter limit must converge to a limit point of the original sequence, but conversely every limit point can be attained by at least one ultrafilter limit; we leave these assertions as an exercise to the interested reader. In particular, a bounded sequence converges classically to a limit if and only if all ultrafilter limits converge to .

There is no generalisation of the classical limit functional to any space that includes non-classically convergent sequences that obeys the subsequence property (5), since any non-classically-convergent sequence will have one subsequence that converges to the limit superior, and another subsequence that converges to the limit inferior, and one of these will have to violate (5) since the limit superior and limit inferior are distinct. So the above limit notions come close to the best generalisations of limit that one can use in practice.

We summarise the above discussion in the following table:

Limit | Always defined | Linear | Shift-invariant | Homomorphism | Constructive |

Classical | No | Yes | Yes | Yes | Yes |

Superior | Yes | No | Yes | No | Yes |

Inferior | Yes | No | Yes | No | Yes |

Césaro | No | Yes | Yes | No | Yes |

Generalised | Yes | Yes | Depends | Depends | No |

Banach | Yes | Yes | Yes | No | No |

Ultrafilter | Yes | Yes | No | Yes | No |

## Recent Comments