Suppose one has a bounded sequence of real numbers. What kinds of limits can one form from this sequence?

Of course, we have the usual notion of limit , which in this post I will refer to as the *classical limit* to distinguish from the other limits discussed in this post. The classical limit, if it exists, is the unique real number such that for every , one has for all sufficiently large . We say that a sequence is (classically) convergent if its classical limit exists. The classical limit obeys many useful *limit laws* when applied to classically convergent sequences. Firstly, it is linear: if and are classically convergent sequences, then is also classically convergent with

and similarly for any scalar , is classically convergent with

It is also an algebra homomorphism: is also classically convergent with

We also have shift invariance: if is classically convergent, then so is with

and more generally in fact for any injection , is classically convergent with

The classical limit of a sequence is unchanged if one modifies any finite number of elements of the sequence. Finally, we have boundedness: for any classically convergent sequence , one has

One can in fact show without much difficulty that these laws uniquely determine the classical limit functional on convergent sequences.

One would like to extend the classical limit notion to more general bounded sequences; however, when doing so one must give up one or more of the desirable limit laws that were listed above. Consider for instance the sequence . On the one hand, one has for all , so if one wishes to retain the homomorphism property (3), any “limit” of this sequence would have to necessarily square to , that is to say it must equal or . On the other hand, if one wished to retain the shift invariance property (4) as well as the homogeneity property (2), any “limit” of this sequence would have to equal its own negation and thus be zero.

Nevertheless there are a number of useful generalisations and variants of the classical limit concept for non-convergent sequences that obey a significant portion of the above limit laws. For instance, we have the limit superior

and limit inferior

which are well-defined real numbers for any bounded sequence ; they agree with the classical limit when the sequence is convergent, but disagree otherwise. They enjoy the shift-invariance property (4), and the boundedness property (6), but do not in general obey the homomorphism property (3) or the linearity property (1); indeed, we only have the subadditivity property

for the limit superior, and the superadditivity property

for the limit inferior. The homogeneity property (2) is only obeyed by the limits superior and inferior for non-negative ; for negative , one must have the limit inferior on one side of (2) and the limit superior on the other, thus for instance

The limit superior and limit inferior are examples of limit points of the sequence, which can for instance be defined as points that are limits of at least one subsequence of the original sequence. Indeed, the limit superior is always the largest limit point of the sequence, and the limit inferior is always the smallest limit point. However, limit points can be highly non-unique (indeed they are unique if and only if the sequence is classically convergent), and so it is difficult to sensibly interpret most of the usual limit laws in this setting, with the exception of the homogeneity property (2) and the boundedness property (6) that are easy to state for limit points.

Another notion of limit are the Césaro limits

if this limit exists, we say that the sequence is Césaro convergent. If the sequence already has a classical limit, then it also has a Césaro limit that agrees with the classical limit; but there are additional sequences that have a Césaro limit but not a classical one. For instance, the non-classically convergent sequence discussed above is Césaro convergent, with a Césaro limit of . However, there are still bounded sequences that do not have Césaro limit, such as (exercise!), basically because such sequences oscillate too slowly for the Césaro averaging to be of much use in accelerating the convergence. The Césaro limit is linear, bounded, and shift invariant, but not an algebra homomorphism and also does not obey the rearrangement property (5).

Using the Hahn-Banach theorem, one can extend the classical limit functional to *generalised limit functionals* , defined to be bounded linear functionals from the space of bounded real sequences to the real numbers that extend the classical limit functional (defined on the space of convergent sequences) without any increase in the operator norm. (In some of my past writings I made the slight error of referring to these generalised limit functionals as Banach limits, though as discussed below, the latter actually refers to a subclass of generalised limit functionals.) It is not difficult to see that such generalised limit functionals will range between the limit inferior and limit superior. In fact, for any specific sequence and any number lying in the closed interval , there exists at least one generalised limit functional that takes the value when applied to ; for instance, for any number in , there exists a generalised limit functional that assigns that number as the “limit” of the sequence . This claim can be seen by first designing such a limit functional on the vector space spanned by the convergent sequences and by , and then appealing to the Hahn-Banach theorem to extend to all sequences. This observation also gives a necessary and sufficient criterion for a bounded sequence to classically converge to a limit , namely that all generalised limits of this sequence must equal .

Because of the reliance on the Hahn-Banach theorem, the existence of generalised limits requires the axiom of choice (or some weakened version thereof); there are models of set theory without the axiom of choice in which no generalised limits exist. For instance, consider a Solovay model in which all subsets of the real numbers are measurable. If one lets denote the function that extracts the ternary digit past the decimal point (thus , and lets be a generalised limit functional, then the function is non-constant (e.g. and ), but also invariant almost everywhere with respect to translation by ternary rationals , and hence cannot be measurable (due to the continuity of translation in the strong operator topology, or the Steinhaus lemma), and so generalised limit functionals cannot exist.

Generalised limits can obey the shift-invariance property (4) or the algebra homomorphism property (3), but as the above analysis of the sequence shows, they cannot do both. Generalised limits that obey the shift-invariance property (4) are known as Banach limits; one can for instance construct them by applying the Hahn-Banach theorem to the Césaro limit functional; alternatively, if is any generalised limit, then the Césaro-type functional will be a Banach limit. The existence of Banach limits can be viewed as a demonstration of the amenability of the natural numbers (or integers); see this previous blog post for further discussion.

Generalised limits that obey the algebra homomorphism property (3) are known as *ultrafilter limits*. If one is given a generalised limit functional that obeys (3), then for any subset of the natural numbers , the generalised limit must equal its own square (since ) and is thus either or . If one defines to be the collection of all subsets of for which , one can verify that obeys the axioms of a non-principal ultrafilter. Conversely, if is a non-principal ultrafilter, one can define the associated generalised limit of any bounded sequence to be the unique real number such that the sets lie in for all ; one can check that this does indeed give a well-defined generalised limit that obeys (3). Non-principal ultrafilters can be constructed using Zorn’s lemma. In fact, they do not quite need the full strength of the axiom of choice; see the Wikipedia article on the ultrafilter lemma for examples.

We have previously noted that generalised limits of a sequence can converge to any point between the limit inferior and limit superior. The same is not true if one restricts to Banach limits or ultrafilter limits. For instance, by the arguments already given, the only possible Banach limit for the sequence is zero. Meanwhile, an ultrafilter limit must converge to a limit point of the original sequence, but conversely every limit point can be attained by at least one ultrafilter limit; we leave these assertions as an exercise to the interested reader. In particular, a bounded sequence converges classically to a limit if and only if all ultrafilter limits converge to .

There is no generalisation of the classical limit functional to any space that includes non-classically convergent sequences that obeys the subsequence property (5), since any non-classically-convergent sequence will have one subsequence that converges to the limit superior, and another subsequence that converges to the limit inferior, and one of these will have to violate (5) since the limit superior and limit inferior are distinct. So the above limit notions come close to the best generalisations of limit that one can use in practice.

(Added after comments) If denotes the Stone-Cech compactification of the natural numbers, then can be canonically identified with the continuous functions on , and hence by the Riesz representation theorem, bounded linear functionals on can be identified with finite measures on this space. From this it is not difficult to show that generalised limit functionals can be canonically identified with probability measures on the compact Hausdorff space , that ultrafilter limits correspond to those probability measures that are Dirac measures (i.e. they can be canonically identified with points in ), and Banach limits correspond to those probability measures that are invariant with respect to the translation action of the integers on .

We summarise (some of) the above discussion in the following table:

Limit | Always defined | Linear | Shift-invariant | Homomorphism | Constructive |

Classical | No | Yes | Yes | Yes | Yes |

Superior | Yes | No | Yes | No | Yes |

Inferior | Yes | No | Yes | No | Yes |

Césaro | No | Yes | Yes | No | Yes |

Generalised | Yes | Yes | Depends | Depends | No |

Banach | Yes | Yes | Yes | No | No |

Ultrafilter | Yes | Yes | No | Yes | No |

## 39 comments

Comments feed for this article

11 May, 2017 at 12:49 pm

AryehI think the inequality sign in the superadditive property of liminf is backwards.

[Corrected, thanks – T.]11 May, 2017 at 3:39 pm

Xiangui ZhaoI think there is a redundant word “the” in the sentence “Indeed, the limit superior is the always the largest limit point of the sequence, and… ”

[Corrected, thanks – T.]11 May, 2017 at 4:17 pm

Fred LunnonSomething pear-shaped about the definition of Césaro limit (singular?),

which currently reads “C minus lim … := …” ?

WFL

11 May, 2017 at 5:43 pm

CrustI think it’s correct. You should think of “-” here as “dash” not “minus” as in “p-norm” or “semi-ring”, etc.

11 May, 2017 at 10:11 pm

AnonymousWhat is the meaning of “maybe” in the table?

[Reworded to “Depends” for clarity – T.]12 May, 2017 at 1:08 am

AnonymousIn the fifth line, it seems that “this point” should be “this post”.

[Corrected, thanks – T.]12 May, 2017 at 2:38 am

Sean EberhardOne can identify with , and hence the dual of with the space of bounded measures on . Stipulating that nonnegative sequences map to nonnegative limits and 1 maps to 1 forces the measure to be a probability measure, so you can think of an arbitrary linear generalized limit as a “convex combination” of ultrafilter limits.

[Thanks, I’ve added this remark to the post. -T]12 May, 2017 at 5:11 am

AnonymousCan Ramanujan’s example be considered as a generalized limit of its partial sums in the sense of one of the above generalized limit methods?

12 May, 2017 at 8:03 am

Lior SilbermanThis example well predates Ramanujan (it’s basically due to Riemann). And belongs to a different notion of limits, “regularization”. The sequence of partial sums is truly divergent and beyond the reach of this kind of generalized limits.

The value comes from interpreting the sum formally as the value at of the series and using the analytical continuation from the usual domain of convergence .

It is more in the spirit of setting by analytical continuation of the formula for the sum of geometric series [though this geometric series does converge with respect to the -adic absolute value].

12 May, 2017 at 8:19 am

Lior SilbermanSorry – I now see that Ramanujan did independently discover this result, through manipulation of infinite series. My comment otherwise stands: while some of the approaches above (e.g. Cesàro summation) can sometimes deal with unbounded sequences, this sum is beyond their reach.

12 May, 2017 at 8:50 am

AnonymousThanks for the information!

In fact, I credited Ramanujan for this example because he stated it in one of his early letters to Hardy.

Meanwhile, I found more information on this type of “series regularization” in the Wikipedia articles “Ramanujan summation” and “Abel – Plana formula”.

12 May, 2017 at 6:15 am

allenknutsonI like to define “continuous” as “commutes with the limit functional”, for people who have recently learned the definition of “linear” as “commutes with addition and scalar multiplications”.

Do you really want to expand sup/inf to superior/inferior not supremum/infimum?

12 May, 2017 at 9:35 am

Terence TaoFrom a nonstandard analysis point of view, one can precisely describe the continuous functions as the standard functions whose nonstandard extension , when restricted to the bounded nonstandard reals , commutes with the standard part ring homomorphism (which, incidentally, is part of the short exact sequence relating the bounded nonstandard reals to the standard reals and the infinitesimal reals ). So one can indeed describe continuity in purely algebraic terms if one is willing to use nonstandard analysis.

Limit superior and limit inferior are common expansions of and , though as noted on that Wikipedia page there are some other expansions such as “supremum limit”, etc.. I suppose the idea is that “limit” is the noun and “superior/inferior” is the modifying (Latin) adjective.

14 May, 2017 at 3:58 am

Mikhail KatzIndeed, is continuous at iff for all infinitely close to .

15 May, 2017 at 12:18 am

VincentWhy does the Latex in Mikhail Katz’s post not compile while that in other posts above clearly does? I can’t see anything wrong with it.

[LaTeX now fixed. See https://terrytao.wordpress.com/about/ for how to enter LaTeX into posts. -T.]12 May, 2017 at 12:02 pm

Maths studentOddly enough, it seems as though the boundedness condition (6), together with shift invariance (4), already suffice to determine the limit functional on the space of convergent sequences.

14 May, 2017 at 1:15 am

Mikhail KatzThanks for a nice post, Terry. In earlier postings you referred to such “generalized ultrafilter limits” as ULTRALIMITS. That seems like a useful term. Why have you avoided it in this particular post?

15 May, 2017 at 8:01 am

Terence TaoI like to use the term ultralimit to denote the nonstandard element associated to a sequence of standard elements (up to equivalence in the ultrafilter). What I call an ultrafilter limit here is the standard part of the ultralimit, but is not the ultralimit itself. (But the term ultralimit is also used in the literature in a slightly different sense – for something that I would call a nonstandard hull.)

15 May, 2017 at 6:21 am

AnonymousZF + the Hahn-Banach theorem are enough to prove the Banach-Tarski paradox (reference in the Wikipedia article about BT). While by a famous theorem of Solovay, there are models of ZF in which all sets of reals are Lebesgue measurable. So the Hahn-Banach theorem can’t hold in such models. But does the existence of generalised limits really require the Hahn-Banach theorem, in the sense that their existence (plus ZF) proves the Hahn-Banach theorem?

15 May, 2017 at 6:23 am

AnonymousWait, I don’t know if the above is right, that the existence of the Solovay model applies to R**3 and not just R. Hmm.

[Comment added to post on how one can use Solovay models to deny the existence of generalised limits. I’m pretty sure that existence of generalised limits is weaker than the full strength of Hahn-Banach, since it only uses one special case of that theorem, but do not have an immediate reference for this. But see https://mathoverflow.net/questions/45844/hahn-banach-without-choice for some related discussion.-T.]16 May, 2017 at 1:10 pm

BanacheroConcerning the sentence “the only possible Banach limit for the sequence {a_n = (-1)^n} is zero”, it comes to mind the notion of “Almost convergence” which traces back to the work of Lorentz. See https://en.wikipedia.org/wiki/Almost_convergent_sequence

8 June, 2017 at 4:35 am

Shahrooz JanbazIs it possible we see the different definitions of limits as various type of measures? Thanks for your interesting post. I will use a simplified version of your post in my calculus lecture. I am sure that students will enjoy of these different views.

25 June, 2017 at 6:32 pm

Aubrey WiederinIt seems like a lot of this could be realized through considering distributions on the range of the function, i.e. applying the function to a random variable on the domain, and then possibly reducing that to a statistic.

Banach limits seem to be when using the limit of a uniform distribution and then applying a weighted mean, and specifically the Cesaro mean is with the expected value, or a uniform weighting. If you don’t mind losing linearity you could also use generalized means, which I believe would get you the inf and sup limits. Haven’t checked to see what the geometric or harmonic means would produce, but I wouldn’t be surprised if they are already well-known.

30 July, 2017 at 11:15 am

TorsteinGoing to make a total ass of myself here, but I thought the Hahn-Banach theorem for spaces with a countable basis was proved by ordinary induction.

1 August, 2017 at 10:50 am

Terence TaoOne would still need the axiom of countable choice in this case. But in any event, does not have a countable (Hamel) basis, as can be seen for instance from the Baire category theorem.

9 August, 2017 at 4:35 pm

The structure of logarithmically averaged correlations of multiplicative functions, with applications to the Chowla and Elliott conjectures | What's new[…] all limits exist. (In the paper, we instead use the device of generalised limits, as discussed in this previous post.) The idea is to combine together two rather different ways to control this function . The first […]

17 August, 2017 at 2:32 am

Jesús LópezOne can also say ultrafilter and classic limits are cases of limits along a filter as in planetmath, where in the classic case the filter is Fréchet’s. If one thinks of the sequence as defined on the ordinal omega plus one, the topological definition of limit of function at a point, here omega, matches the one of limit of a sequence at inifinity, picturalizing the expression ‘neighborhood of inifinity’ and allowing one to think of local and sequential limits at once. Local concepts such as stalk or germ should be predicable also at that point omega also.

28 September, 2017 at 6:54 pm

Daniel SmaniaTeaching Functional Analysis this year we learned that with the analytic version of HB theorem with Minkowski functionals (like in Brezis’s book) we can obtain no trivial and interesting linear functionals from the null functional and trivial space, provided the Minkowski functional also assume negative values.

My students asked me about an exercise in Maddox’s book to construct a linear functional g that satisfies

\liminf_n x_n\leq g(x)\leq \limsup_n x_n.

I started to construct a Banach limit extending the functional that gives the Cesaro limit on the subspace of Cesaro convergent sequences.

The students knew better: they told me to take the Minkowski functional p(x)= limsup_n x_n and the subspace of convergent sequences and apply the analytic HB theorem. My solution was an overkill…

Soon we realized that we can consider the Minkowski functional p(x)= limsup_n x_n and Y as the trivial subspace Y={0}, where 0 denotes the constant sequence (0,0,0,…). Then of course the null functional f satisfies f(x)\leq p(x) on Y, so there is an extension to a function g on l_\infty that satisfies

g(x)\leq limsup_n x_n \leq |x|_\infty

and consequently g is bounded and satisfies (considering g(-x) )

\liminf_n x_n\leq g(x)\leq \limsup_n x_n

I was very shocked by this, that we can start from the trial subspace and functional and construct such interesting bounded functional.

3 March, 2018 at 2:33 am

Jesús LópezCorrect me if wrong, but in “Generalised limits can obey the shift-invariance property (4) or the algebra homomorphism property (2)”, “Generalised limits that obey the algebra homomorphism property (2) are known as ultrafilter limits” and in next sentence, (3) is meant instead of (2).

[Corrected, thanks – T.]19 March, 2018 at 11:14 am

Corey LevinsonThis question is based on completely https://en.wikipedia.org/wiki/Cesàro_summation

I’m a little confused. You write the sequence has Césaro limit of , but going by the definition we have Césaro limit is . So how do we find this?

Well, the sequence of partial sums is (-1, 0, -1, 0, -1, 0, …) which is not classically convergent. However, this sum is Césaro summable. Let be the sequence of arithmetic means of the first n partial sums:

(, , , , , , …)

Then

So has Césaro limit of instead of 0?

19 March, 2018 at 11:20 am

Corey LevinsonUnless actually I just found the Césaro sum, and so the limit is . So actually it should be 0! Nevermind then, I guess

14 March, 2019 at 7:41 am

jonbannonWhen you construct the nonprincipal ultrafilter from the generalized limit functional satisfying (2), it is not crystal clear here why the generalized limit functional is the corresponding ultrafilter limit. How is this most easily seen?

14 March, 2019 at 7:55 am

jonbannonI see, perhaps it would be pedantic. One can show this for the functions 1_A with A in the ultrafilter, and the other functionals are linear combinations of these…

14 March, 2019 at 8:08 am

jonbannonNo, that wasn’t quite right. This is standard, but your exposition above is so nice I feel it needs a sentence to finish that. Given the character one constructs the nonprincipal ultrafilter. The limit along this ultrafilter is also a character and will have the same kernel as the original character, so they are identical.

15 June, 2019 at 7:27 am

Abstracting induction on scales arguments | What's new[…] arguments here would also apply with one of the other generalised limit functionals discussed in this previous post, such as ultrafilter […]

28 November, 2019 at 2:18 am

Math loverSo I still do not understand how to prove this claim: “there exists at least one generalised limit functional {\mathop{\widetilde \lim}_{n \rightarrow \infty}} that takes the value {L} when applied to {a_n}”? Why consider the span of convergent sequences and {a_n}? Any hints?

28 November, 2019 at 7:50 am

Terence TaoFirst construct a linear functional from the space spanned by the convergent sequences and that agrees with the limit functional on convergent sequences, and maps to . (This linear functional will be unique, although the generalised limit functional one then constructs from it using Hahn-Banach will not be.)

2 December, 2019 at 5:52 pm

Math loverThanks for your answer. I have another question. Is there any applicatio of the generalized limit or Banach limit? Thanks.

2 January, 2020 at 11:09 am

ChrisReally interesting. Should (2) be (3) in the statement “If one is given a generalised limit functional {p\!-\!\lim_{n \rightarrow \infty}} that obeys (2)…” And again further down?

[Corrected, thanks – T.]