Suppose one has a bounded sequence ${(a_n)_{n=1}^\infty = (a_1, a_2, \dots)}$ of real numbers. What kinds of limits can one form from this sequence?

Of course, we have the usual notion of limit ${\lim_{n \rightarrow \infty} a_n}$, which in this post I will refer to as the classical limit to distinguish from the other limits discussed in this post. The classical limit, if it exists, is the unique real number ${L}$ such that for every ${\varepsilon>0}$, one has ${|a_n-L| \leq \varepsilon}$ for all sufficiently large ${n}$. We say that a sequence is (classically) convergent if its classical limit exists. The classical limit obeys many useful limit laws when applied to classically convergent sequences. Firstly, it is linear: if ${(a_n)_{n=1}^\infty}$ and ${(b_n)_{n=1}^\infty}$ are classically convergent sequences, then ${(a_n+b_n)_{n=1}^\infty}$ is also classically convergent with

$\displaystyle \lim_{n \rightarrow \infty} (a_n + b_n) = (\lim_{n \rightarrow \infty} a_n) + (\lim_{n \rightarrow \infty} b_n) \ \ \ \ \ (1)$

and similarly for any scalar ${c}$, ${(ca_n)_{n=1}^\infty}$ is classically convergent with

$\displaystyle \lim_{n \rightarrow \infty} (ca_n) = c \lim_{n \rightarrow \infty} a_n. \ \ \ \ \ (2)$

It is also an algebra homomorphism: ${(a_n b_n)_{n=1}^\infty}$ is also classically convergent with

$\displaystyle \lim_{n \rightarrow \infty} (a_n b_n) = (\lim_{n \rightarrow \infty} a_n) (\lim_{n \rightarrow \infty} b_n). \ \ \ \ \ (3)$

We also have shift invariance: if ${(a_n)_{n=1}^\infty}$ is classically convergent, then so is ${(a_{n+1})_{n=1}^\infty}$ with

$\displaystyle \lim_{n \rightarrow \infty} a_{n+1} = \lim_{n \rightarrow \infty} a_n \ \ \ \ \ (4)$

and more generally in fact for any injection ${\phi: {\bf N} \rightarrow {\bf N}}$, ${(a_{\phi(n)})_{n=1}^\infty}$ is classically convergent with

$\displaystyle \lim_{n \rightarrow \infty} a_{\phi(n)} = \lim_{n \rightarrow \infty} a_n. \ \ \ \ \ (5)$

The classical limit of a sequence is unchanged if one modifies any finite number of elements of the sequence. Finally, we have boundedness: for any classically convergent sequence ${(a_n)_{n=1}^\infty}$, one has

$\displaystyle \inf_n a_n \leq \lim_{n \rightarrow \infty} a_n \leq \sup_n a_n. \ \ \ \ \ (6)$

One can in fact show without much difficulty that these laws uniquely determine the classical limit functional on convergent sequences.

One would like to extend the classical limit notion to more general bounded sequences; however, when doing so one must give up one or more of the desirable limit laws that were listed above. Consider for instance the sequence ${a_n = (-1)^n}$. On the one hand, one has ${a_n^2 = 1}$ for all ${n}$, so if one wishes to retain the homomorphism property (3), any “limit” of this sequence ${a_n}$ would have to necessarily square to ${1}$, that is to say it must equal ${+1}$ or ${-1}$. On the other hand, if one wished to retain the shift invariance property (4) as well as the homogeneity property (2), any “limit” of this sequence would have to equal its own negation and thus be zero.

Nevertheless there are a number of useful generalisations and variants of the classical limit concept for non-convergent sequences that obey a significant portion of the above limit laws. For instance, we have the limit superior

$\displaystyle \limsup_{n \rightarrow \infty} a_n := \inf_N \sup_{n \geq N} a_n$

$\displaystyle \liminf_{n \rightarrow \infty} a_n := \sup_N \inf_{n \geq N} a_n$

which are well-defined real numbers for any bounded sequence ${(a_n)_{n=1}^\infty}$; they agree with the classical limit when the sequence is convergent, but disagree otherwise. They enjoy the shift-invariance property (4), and the boundedness property (6), but do not in general obey the homomorphism property (3) or the linearity property (1); indeed, we only have the subadditivity property

$\displaystyle \limsup_{n \rightarrow \infty} (a_n + b_n) \leq (\limsup_{n \rightarrow \infty} a_n) + (\limsup_{n \rightarrow \infty} b_n)$

for the limit superior, and the superadditivity property

$\displaystyle \liminf_{n \rightarrow \infty} (a_n + b_n) \geq (\liminf_{n \rightarrow \infty} a_n) + (\liminf_{n \rightarrow \infty} b_n)$

for the limit inferior. The homogeneity property (2) is only obeyed by the limits superior and inferior for non-negative ${c}$; for negative ${c}$, one must have the limit inferior on one side of (2) and the limit superior on the other, thus for instance

$\displaystyle \limsup_{n \rightarrow \infty} (-a_n) = - \liminf_{n \rightarrow \infty} a_n.$

The limit superior and limit inferior are examples of limit points of the sequence, which can for instance be defined as points that are limits of at least one subsequence of the original sequence. Indeed, the limit superior is always the largest limit point of the sequence, and the limit inferior is always the smallest limit point. However, limit points can be highly non-unique (indeed they are unique if and only if the sequence is classically convergent), and so it is difficult to sensibly interpret most of the usual limit laws in this setting, with the exception of the homogeneity property (2) and the boundedness property (6) that are easy to state for limit points.

Another notion of limit are the Césaro limits

$\displaystyle \mathrm{C}\!-\!\lim_{n \rightarrow \infty} a_n := \lim_{N \rightarrow \infty} \frac{1}{N} \sum_{n=1}^N a_n;$

if this limit exists, we say that the sequence is Césaro convergent. If the sequence ${(a_n)_{n=1}^\infty}$ already has a classical limit, then it also has a Césaro limit that agrees with the classical limit; but there are additional sequences that have a Césaro limit but not a classical one. For instance, the non-classically convergent sequence ${a_n= (-1)^n}$ discussed above is Césaro convergent, with a Césaro limit of ${0}$. However, there are still bounded sequences that do not have Césaro limit, such as ${a_n := \sin( \log n )}$ (exercise!). The Césaro limit is linear, bounded, and shift invariant, but not an algebra homomorphism and also does not obey the rearrangement property (5).

Using the Hahn-Banach theorem, one can extend the classical limit functional to generalised limit functionals ${\mathop{\widetilde \lim}_{n \rightarrow \infty} a_n}$, defined to be bounded linear functionals from the space ${\ell^\infty({\bf N})}$ of bounded real sequences to the real numbers ${{\bf R}}$ that extend the classical limit functional (defined on the space ${c_0({\bf N}) + {\bf R}}$ of convergent sequences) without any increase in the operator norm. (In some of my past writings I made the slight error of referring to these generalised limit functionals as Banach limits, though as discussed below, the latter actually refers to a subclass of generalised limit functionals.) It is not difficult to see that such generalised limit functionals will range between the limit inferior and limit superior. In fact, for any specific sequence ${(a_n)_{n=1}^\infty}$ and any number ${L}$ lying in the closed interval ${[\liminf_{n \rightarrow \infty} a_n, \limsup_{n \rightarrow \infty} a_n]}$, there exists at least one generalised limit functional ${\mathop{\widetilde \lim}_{n \rightarrow \infty}}$ that takes the value ${L}$ when applied to ${a_n}$; for instance, for any number ${\theta}$ in ${[-1,1]}$, there exists a generalised limit functional that assigns that number ${\theta}$ as the “limit” of the sequence ${a_n = (-1)^n}$. This claim can be seen by first designing such a limit functional on the vector space spanned by the convergent sequences and by ${(a_n)_{n=1}^\infty}$, and then appealing to the Hahn-Banach theorem to extend to all sequences. This observation also gives a necessary and sufficient criterion for a bounded sequence ${(a_n)_{n=1}^\infty}$ to classically converge to a limit ${L}$, namely that all generalised limits of this sequence must equal ${L}$.

Because of the reliance on the Hahn-Banach theorem, the existence of generalised limits requires the axiom of choice (or some weakened version thereof); there are presumably models of set theory without the axiom of choice in which no generalised limits exist, but I do not know of an explicit reference for this.

Generalised limits can obey the shift-invariance property (4) or the algebra homomorphism property (3), but as the above analysis of the sequence ${a_n = (-1)^n}$ shows, they cannot do both. Generalised limits that obey the shift-invariance property (4) are known as Banach limits; one can for instance construct them by applying the Hahn-Banach theorem to the Césaro limit functional; alternatively, if ${\mathop{\widetilde \lim}}$ is any generalised limit, then the Césaro-type functional ${(a_n)_{n=1}^\infty \mapsto \mathop{\widetilde \lim}_{N \rightarrow \infty} \frac{1}{N} \sum_{n=1}^N a_n}$ will be a Banach limit. The existence of Banach limits can be viewed as a demonstration of the amenability of the natural numbers (or integers); see this previous blog post for further discussion.

Generalised limits that obey the algebra homomorphism property (3) are known as ultrafilter limits. If one is given a generalised limit functional ${p\!-\!\lim_{n \rightarrow \infty}}$ that obeys (2), then for any subset ${A}$ of the natural numbers ${{\bf N}}$, the generalised limit ${p\!-\!\lim_{n \rightarrow \infty} 1_A(n)}$ must equal its own square (since ${1_A(n)^2 = 1_A(n)}$) and is thus either ${0}$ or ${1}$. If one defines ${p \subset 2^{2^{\bf N}}}$ to be the collection of all subsets ${A}$ of ${{\bf N}}$ for which ${p\!-\!\lim_{n \rightarrow \infty} 1_A(n) = 1}$, one can verify that ${p}$ obeys the axioms of a non-principal ultrafilter. Conversely, if ${p}$ is a non-principal ultrafilter, one can define the associated generalised limit ${p\!-\!\lim_{n \rightarrow \infty} a_n}$ of any bounded sequence ${(a_n)_{n=1}^\infty}$ to be the unique real number ${L}$ such that the sets ${\{ n \in {\bf N}: |a_n - L| \leq \varepsilon \}}$ lie in ${p}$ for all ${\varepsilon>0}$; one can check that this does indeed give a well-defined generalised limit that obeys (2). Non-principal ultrafilters can be constructed using Zorn’s lemma. In fact, they do not quite need the full strength of the axiom of choice; see the Wikipedia article on the ultrafilter lemma for examples.

We have previously noted that generalised limits of a sequence can converge to any point between the limit inferior and limit superior. The same is not true if one restricts to Banach limits or ultrafilter limits. For instance, by the arguments already given, the only possible Banach limit for the sequence ${a_n = (-1)^n}$ is zero. Meanwhile, an ultrafilter limit must converge to a limit point of the original sequence, but conversely every limit point can be attained by at least one ultrafilter limit; we leave these assertions as an exercise to the interested reader. In particular, a bounded sequence converges classically to a limit ${L}$ if and only if all ultrafilter limits converge to ${L}$.

There is no generalisation of the classical limit functional to any space that includes non-classically convergent sequences that obeys the subsequence property (5), since any non-classically-convergent sequence will have one subsequence that converges to the limit superior, and another subsequence that converges to the limit inferior, and one of these will have to violate (5) since the limit superior and limit inferior are distinct. So the above limit notions come close to the best generalisations of limit that one can use in practice.

We summarise the above discussion in the following table:

 Limit Always defined Linear Shift-invariant Homomorphism Constructive Classical No Yes Yes Yes Yes Superior Yes No Yes No Yes Inferior Yes No Yes No Yes Césaro No Yes Yes No Yes Generalised Yes Yes Depends Depends No Banach Yes Yes Yes No No Ultrafilter Yes Yes No Yes No