If one has a sequence of real numbers , it is unambiguous what it means for that sequence to converge to a limit : it means that for every , there exists an such that for all . Similarly for a sequence of complex numbers converging to a limit .
More generally, if one has a sequence of -dimensional vectors in a real vector space or complex vector space , it is also unambiguous what it means for that sequence to converge to a limit or ; it means that for every , there exists an such that for all . Here, the norm of a vector can be chosen to be the Euclidean norm , the supremum norm , or any other number of norms, but for the purposes of convergence, these norms are all equivalent; a sequence of vectors converges in the Euclidean norm if and only if it converges in the supremum norm, and similarly for any other two norms on the finite-dimensional space or .
If however one has a sequence of functions or on a common domain , and a putative limit or , there can now be many different ways in which the sequence may or may not converge to the limit . (One could also consider convergence of functions on different domains , but we will not discuss this issue at all here.) This is contrast with the situation with scalars or (which corresponds to the case when is a single point) or vectors (which corresponds to the case when is a finite set such as ). Once becomes infinite, the functions acquire an infinite number of degrees of freedom, and this allows them to approach in any number of inequivalent ways.
What different types of convergence are there? As an undergraduate, one learns of the following two basic modes of convergence:
- We say that converges to pointwise if, for every , converges to . In other words, for every and , there exists (that depends on both and ) such that whenever .
- We say that converges to uniformly if, for every , there exists such that for every , for every . The difference between uniform convergence and pointwise convergence is that with the former, the time at which must be permanently -close to is not permitted to depend on , but must instead be chosen uniformly in .
Uniform convergence implies pointwise convergence, but not conversely. A typical example: the functions defined by converge pointwise to the zero function , but not uniformly.
However, pointwise and uniform convergence are only two of dozens of many other modes of convergence that are of importance in analysis. We will not attempt to exhaustively enumerate these modes here (but see this Wikipedia page, and see also these 245B notes on strong and weak convergence). We will, however, discuss some of the modes of convergence that arise from measure theory, when the domain is equipped with the structure of a measure space , and the functions (and their limit ) are measurable with respect to this space. In this context, we have some additional modes of convergence:
- We say that converges to pointwise almost everywhere if, for (-)almost everywhere , converges to .
- We say that converges to uniformly almost everywhere, essentially uniformly, or in norm if, for every , there exists such that for every , for -almost every .
- We say that converges to almost uniformly if, for every , there exists an exceptional set of measure such that converges uniformly to on the complement of .
- We say that converges to in norm if the quantity converges to as .
- We say that converges to in measure if, for every , the measures converge to zero as .
Observe that each of these five modes of convergence is unaffected if one modifies or on a set of measure zero. In contrast, the pointwise and uniform modes of convergence can be affected if one modifies or even on a single point.
Remark 1 In the context of probability theory, in which and are interpreted as random variables, convergence in norm is often referred to as convergence in mean, pointwise convergence almost everywhere is often referred to as almost sure convergence, and convergence in measure is often referred to as convergence in probability.
- Show that converges to along one of the above seven modes of convergence if and only if converges to along the same mode.
- If converges to along one of the above seven modes of convergence, and converges to along the same mode, show that converges to along the same mode, and that converges to along the same mode for any .
- (Squeeze test) If converges to along one of the above seven modes, and pointwise for each , show that converges to along the same mode.
We have some easy implications between modes:
- If converges to uniformly, then converges to pointwise.
- If converges to uniformly, then converges to in norm. Conversely, if converges to in norm, then converges to uniformly outside of a null set (i.e. there exists a null set such that the restriction of to the complement of converges to the restriction of ).
- If converges to in norm, then converges to almost uniformly.
- If converges to almost uniformly, then converges to pointwise almost everywhere.
- If converges to pointwise, then converges to pointwise almost everywhere.
- If converges to in norm, then converges to in measure.
- If converges to almost uniformly, then converges to in measure.
The reader is encouraged to draw a diagram that summarises the logical implications between the seven modes of convergence that the above exercise describes.
We give four key examples that distinguish between these modes, in the case when is the real line with Lebesgue measure. The first three of these examples already were introduced in the previous set of notes.
Example 1 (Escape to horizontal infinity) Let . Then converges to zero pointwise (and thus, pointwise almost everywhere), but not uniformly, in norm, almost uniformly, in norm, or in measure.
Example 2 (Escape to width infinity) Let . Then converges to zero uniformly (and thus, pointwise, pointwise almost everywhere, in norm, almost uniformly, and in measure), but not in norm.
Example 3 (Escape to vertical infinity) Let . Then converges to zero pointwise (and thus, pointwise almost everywhere) and almost uniformly (and hence in measure), but not uniformly, in norm, or in norm.
Example 4 (Typewriter sequence) Let be defined by the formula
whenever and . This is a sequence of indicator functions of intervals of decreasing length, marching across the unit interval over and over again. Then converges to zero in measure and in norm, but not pointwise almost everywhere (and hence also not pointwise, not almost uniformly, nor in norm, nor uniformly).
Remark 2 The norm of a measurable function is defined to the infimum of all the quantities that are essential upper bounds for in the sense that for almost every . Then converges to in norm if and only if as . The and norms are part of the larger family of norms, which we will study in more detail in 245B.
One particular advantage of convergence is that, in the case when the are absolutely integrable, it implies convergence of the integrals,
as one sees from the triangle inequality. Unfortunately, none of the other modes of convergence automatically imply this convergence of the integral, as the above examples show.
The purpose of these notes is to compare these modes of convergence with each other. Unfortunately, the relationship between these modes is not particularly simple; unlike the situation with pointwise and uniform convergence, one cannot simply rank these modes in a linear order from strongest to weakest. This is ultimately because the different modes react in different ways to the three “escape to infinity” scenarios described above, as well as to the “typewriter” behaviour when a single set is “overwritten” many times. On the other hand, if one imposes some additional assumptions to shut down one or more of these escape to infinity scenarios, such as a finite measure hypothesis or a uniform integrability hypothesis, then one can obtain some additional implications between the different modes.
— 1. Uniqueness —
Throughout these notes, denotes a measure space. We abbreviate “-almost everywhere” as “almost everywhere” throughout.
Even though the modes of convergence all differ from each other, they are all compatible in the sense that they never disagree about which function a sequence of functions converges to, outside of a set of measure zero. More precisely:
Proposition 1 Let be a sequence of measurable functions, and let be two additional measurable functions. Suppose that converges to along one of the seven modes of convergence defined above, and converges to along another of the seven modes of convergence (or perhaps the same mode of convergence as for ). Then and agree almost everywhere.
Note that the conclusion is the best one can hope for in the case of the last five modes of convergence, since as remarked earlier, these modes of convergence are unaffected if one modifies or on a set of measure zero.
Proof: In view of Exercise 2, we may assume that converges to either pointwise almost everywhere, or in measure, and similarly that converges to either pointwise almost everywhere, or in measure.
Suppose first that converges to both and pointwise almost everywhere. Then by Exercise 1, converges to pointwise almost everywhere, which clearly implies that is zero almost everywhere, and the claim follows. A similar argument applies if converges to both and in measure.
By symmetry, the only remaining case that needs to be considered is when converges to pointwise almost everywhere, and converges to in measure. We need to show that almost everywhere. It suffices to show that for every , that for almost every , as the claim then follows by setting for and using the fact that the countable union of null sets is again a null set.
Fix , and let . This is a measurable set; our task is to show that it has measure zero. Suppose for contradiction that . We consider the sets
These are measurable sets that are increasing in . As converges to almost everywhere, we see that almost every belongs to at least one of the , thus is equal to outside of a null set. In particular,
Applying monotone convergence for sets, we conclude that
for some finite . But by the triangle inequality, we have for all and all . As a consequence, cannot converge in measure to , which gives the desired contradiction.
— 2. The case of a step function —
One way to appreciate the distinctions between the above modes of convergence is to focus on the case when , and when each of the is a step function, by which we mean a constant multiple of a measurable set . For simplicity we will assume that the are positive reals, and that the have a positive measure . We also assume the exhibit one of two modes of behaviour: either the converge to zero, or else they are bounded away from zero (i.e. there exists such that for every . It is easy to see that if a sequence does not converge to zero, then it has a subsequence that is bounded away from zero, so it does not cause too much loss of generality to restrict to one of these two cases.
Given such a sequence of step functions, we now ask, for each of the seven modes of convergence, what it means for this sequence to converge to zero along that mode. It turns out that the answer to question is controlled more or less completely by the following three quantities:
- The height of the function ;
- The width of the function ; and
- The tail support of the sequence .
Indeed, we have:
- converges uniformly to zero if and only if as .
- converges in norm to zero if and only if as .
- converges almost uniformly to zero if and only if as , or as .
- converges pointwise to zero if and only if as , or .
- converges pointwise almost everywhere to zero if and only if as , or is a null set.
- converges in measure to zero if and only if as , or as .
- converges in norm if and only if as .
To put it more informally: when the height goes to zero, then one has convergence to zero in all modes except possibly for convergence, which requires that the product of the height and the width goes to zero. If instead the height is bounded away from zero and the width is positive, then we never have uniform or convergence, but we have convergence in measure if the width goes to zero, we have almost uniform convergence if the tail support (which has larger measure than the width) has measure that goes to zero, we have pointwise almost everywhere convergence if the tail support shrinks to a null set, and pointwise convergence if the tail support shrinks to the empty set.
It is instructive to compare this exercise with Exercise 2, or with the four examples given in the introduction. In particular:
- In the escape to horizontal infinity scenario, the height and width do not shrink to zero, but the tail set shrinks to the empty set (while remaining of infinite measure throughout).
- In the escape to width infinity scenario, the height goes to zero, but the width (and tail support) go to infinity, causing the norm to stay bounded away from zero.
- In the escape to vertical infinity, the height goes to infinity, but the width (and tail support) go to zero (or the empty set), causing the norm to stay bounded away from zero.
- In the typewriter example, the width goes to zero, but the height and the tail support stay fixed (and thus bounded away from zero).
Remark 3 The monotone convergence theorem can also be specialised to this case. Observe that the are monotone increasing if and only if and for each . In such cases, observe that the converge pointwise to , where and . The monotone convergence theorem then asserts that as , which is a consequence of the monotone convergence theorem for sets.
— 3. Finite measure spaces —
The situation simplifies somewhat if the space has finite measure (and in particular, in the case when is a probability space). This shuts down two of the four examples (namely, escape to horizontal infinity or width infinity) and creates a few more equivalences. Indeed, from Egorov’s theorem (Exercise 30 from Notes 3), we now have
Theorem 2 (Egorov’s theorem, again) Let have finite measure, and let and be measurable functions. Then converges to pointwise almost everywhere if and only if converges to almost uniformly.
Another nice feature of the finite measure case is that convergence implies convergence:
Exercise 4 Let have finite measure, and let and be measurable functions. Show that if converges to in norm, then also converges to in norm.
— 4. Fast convergence —
The typewriter example shows that convergence is not strong enough to force almost uniform or pointwise almost everywhere convergence. However, this can be rectified if one assumes that the convergence is sufficiently fast:
Exercise 5 (Fast convergence) Suppose that are measurable functions such that ; thus, not only do the quantities go to zero (which would mean convergence), but they converge in an absolutely summable fashion.
- Show that converges pointwise almost everywhere to .
- Show that converges almost uniformly to .
(Hint: If you have trouble getting started, try working first in the special case in which are step functions and and use Exercise 3 in order to gain some intuition. The second part of the exercise implies the first, but the first is a little easier to prove and may thus serve as a useful warmup. The trick may come in handy for the second part.)
As a corollary, we see that convergence implies almost uniform or pointwise almost everywhere convergence if we are allowed to pass to a subsequence:
Corollary 3 Suppose that are a sequence of measurable functions that converge in norm to a limit . Then there exists a subsequence that converges almost uniformly (and hence, pointwise almost everywhere) to (while remaining convergent in norm, of course).
Proof: Since as , we can select such that (say). This is enough for the previous exercise to apply.
Actually, one can strengthen this corollary a bit by relaxing convergence to convergence in measure:
Exercise 6 Suppose that are a sequence of measurable functions that converge in measure to a limit . Then there exists a subsequence that converges almost uniformly (and hence, pointwise almost everywhere) to . (Hint: Choose the so that the sets have a suitably small measure.)
It is instructive to see how this subsequence is extracted in the case of the typewriter sequence. In general, one can view the operation of passing to a subsequence as being able to eliminate “typewriter” situations in which the tail support is much larger than the width.
— 5. Domination and uniform integrability —
Now we turn to the reverse question, of whether almost uniform convergence, pointwise almost everywhere convergence, or convergence in measure can imply convergence. The escape to vertical and width infinity examples shows that without any further hypotheses, the answer to this question is no. However, one can do better if one places some domination hypotheses on the that shut down both of these escape routes.
We say that a sequence is dominated if there exists an absolutely integrable function such that for all and almost every . For instance, if has finite measure and the are uniformly bounded, then they are dominated. Observe that the sequences in the vertical and width escape to infinity examples are not dominated (why?).
The dominated convergence theorem (Theorem 17 of Notes 3) then asserts that if converges to pointwise almost everywhere, then it necessarily converges to in norm (and hence also in measure). Here is a variant:
Exercise 7 Suppose that are a dominated sequence of measurable functions, and let be another measurable function. Show that converges in norm to if and only if converges in measure to . (Hint: one way to establish the “if” direction is first show that every subsequence of the has a further subsequence that converges in to , using Exercise 6 and the dominated convergence theorem. Alternatively, use monotone convergence to find a set of finite measure such that , and hence and , are small.)
There is a more general notion than domination, known as uniform integrability, which serves as a substitute for domination in many (but not all) contexts.
Definition 4 (Uniform integrability) A sequence of absolutely integrable functions is said to be uniformly integrable if the following three statements hold:
- (Uniform bound on norm) One has .
- (No escape to vertical infinity) One has as .
- (No escape to width infinity) One has as .
Remark 4 It is instructive to understand uniform integrability in the step function case . The uniform bound on the norm then asserts that stays bounded. The lack of escape to vertical infinity means that along any subsequence for which , must go to zero. Similarly, the lack of escape to width infinity means that along any subsequence for which , must go to zero.
- Show that if is an absolutely integrable function, then the constant sequence is uniformly integrable. (Hint: use the monotone convergence theorem.)
- Show that every dominated sequence of measurable functions is uniformly integrable.
- Give an example of a sequence that is uniformly integrable but not dominated.
In the case of a finite measure space, there is no escape to width infinity, and the criterion for uniform integrability simplifies to just that of excluding vertical infinity:
Exercise 9 Suppose that has finite measure, and let be a sequence of measurable functions. Show that is uniformly integrable if and only if as .
Exercise 10 (Uniform bound on finite measure implies uniform integrability) Suppose that have finite measure, let , an d suppose that is a sequence of measurable functions such that . Show that the sequence is uniformly integrable.
whenever and is a measurable set with .
The dominated convergence theorem does not have an analogue in the uniformly integrable setting:
Exercise 12 Give an example of a sequence of uniformly integrable functions that converge pointwise almost everywhere to zero, but do not converge almost uniformly, in measure, or in norm.
However, one does have an analogue of Exercise 7:
Theorem 5 (Uniformly integrable convergence in measure) Let be a uniformly integrable sequence of functions, and let be another function. Then converges in norm to if and only if converges to in measure.
Proof: The “only if” part follows from Exercise 2, so we establish the “if” part.
By uniform integrability, there exists a finite such that
for all . By Exercise 6, there is a subsequence of the that converges pointwise almost everywhere to . Applying Fatou’s lemma, we conclude that
thus is absolutely integrable.
In particular, by shrinking further if necessary we see that
and hence by (3)
for all .
for all .
Meanwhile, since converges in measure to , we know that there exists an (depending on ) such that
for all . Applying Exercise 11, we conclude (making smaller if necessary) that
and hence by the triangle inequality
for all . Combining this with (4) we conclude that
for all , and so converges to in norm as desired.
Finally, we recall two results from the previous notes for unsigned functions.
Exercise 13 (Monotone convergence theorem) Suppose that are measurable, monotone non-decreasing in and are such that . Show that converges in norm to . (Note that can be infinite on a null set, but the definition of convergence can be easily modified to accomodate this.)
Exercise 14 (Defect version of Fatou’s lemma) Suppose that are measurable, are such that , and converge pointwise almost everywhere to some measurable limit . Show that converges in norm to if and only if converges to . Informally, we see that in the unsigned, bounded mass case, pointwise convergence implies norm convergence if and only if there is no loss of mass.
[The following material was added after the notes were first posted, and will eventually be moved to a more appropriate location.]
Exercise 15 Let be a measure space, let be a sequence of measurable functions converging pointwise almost everywhere as to a measurable limit , and for each , let be a sequence of measurable functions converging pointwise almost everywhere as (keeping fixed) to .
- If is finite, show that there exists a sequence such that converges pointwise almost everywhere to .
- Show the same claim is true if, instead of assuming that is finite, we merely assume that is -finite, i.e. it is the countable union of sets of finite measure.
(The claim can fail if is not -finite. A counterexample is if with counting measure, and are identically zero for all , and is the indicator function the space of all sequences with .)
Exercise 16 Suppose that are a dominated sequence of measurable functions, and let be another measurable function. Show that converges pointwise almost everywhere to if and only if converges in almost uniformly to .
Exercise 17 Let be a sequence of measurable functions, and let be another measurable function. Show that the following are equivalent:
- converges in measure to .
- Every subsequence of the has a further subsequence that converges almost uniformly to .
Exercise 18 Let be a probability space. Given any real-valued measurable function , we define the cumulative distribution function of to be the function . Given another sequence of real-valued measurable functions, we say that converges in distribution to if the cumulative distribution function of converges pointwise to the cumulative distribution function of at all for which is continuous.
- Show that if converges to in any of the seven senses discussed above (uniformly, essentially uniformly, almost uniformly pointwise, pointwise almost everywhere, in , or in measure), then it converges in distribution to .
- Give an example in which converges to in distribution, but not in any of the above seven senses.
- Show that convergence in distribution is not linear, in the sense that if converges to in distribution, and converges to , then need not converge to .
- Show that a sequence can converge in distribution to two different limits , which are not equal almost everywhere.
Convergence in distribution (not to be confused with convergence in the sense of distributions, which we will study later in this sequence) is commonly used in probability; but, as the above exercise demonstrates, it is quite a weak notion of convergence, lacking many of the properties of the modes of convergence discussed here.