You are currently browsing the tag archive for the ‘free probability’ tag.

Suppose we have an ${n \times n}$ matrix ${M}$ that is expressed in block-matrix form as

$\displaystyle M = \begin{pmatrix} A & B \\ C & D \end{pmatrix}$

where ${A}$ is an ${(n-k) \times (n-k)}$ matrix, ${B}$ is an ${(n-k) \times k}$ matrix, ${C}$ is an ${k \times (n-k)}$ matrix, and ${D}$ is a ${k \times k}$ matrix for some ${1 < k < n}$. If ${A}$ is invertible, we can use the technique of Schur complementation to express the inverse of ${M}$ (if it exists) in terms of the inverse of ${A}$, and the other components ${B,C,D}$ of course. Indeed, to solve the equation

$\displaystyle M \begin{pmatrix} x & y \end{pmatrix} = \begin{pmatrix} a & b \end{pmatrix},$

where ${x, a}$ are ${(n-k) \times 1}$ column vectors and ${y,b}$ are ${k \times 1}$ column vectors, we can expand this out as a system

$\displaystyle Ax + By = a$

$\displaystyle Cx + Dy = b.$

Using the invertibility of ${A}$, we can write the first equation as

$\displaystyle x = A^{-1} a - A^{-1} B y \ \ \ \ \ (1)$

and substituting this into the second equation yields

$\displaystyle (D - C A^{-1} B) y = b - C A^{-1} a$

and thus (assuming that ${D - CA^{-1} B}$ is invertible)

$\displaystyle y = - (D - CA^{-1} B)^{-1} CA^{-1} a + (D - CA^{-1} B)^{-1} b$

and then inserting this back into (1) gives

$\displaystyle x = (A^{-1} + A^{-1} B (D - CA^{-1} B)^{-1} C A^{-1}) a - A^{-1} B (D - CA^{-1} B)^{-1} b.$

Comparing this with

$\displaystyle \begin{pmatrix} x & y \end{pmatrix} = M^{-1} \begin{pmatrix} a & b \end{pmatrix},$

we have managed to express the inverse of ${M}$ as

$\displaystyle M^{-1} =$

$\displaystyle \begin{pmatrix} A^{-1} + A^{-1} B (D - CA^{-1} B)^{-1} C A^{-1} & - A^{-1} B (D - CA^{-1} B)^{-1} \\ - (D - CA^{-1} B)^{-1} CA^{-1} & (D - CA^{-1} B)^{-1} \end{pmatrix}. \ \ \ \ \ (2)$

One can consider the inverse problem: given the inverse ${M^{-1}}$ of ${M}$, does one have a nice formula for the inverse ${A^{-1}}$ of the minor ${A}$? Trying to recover this directly from (2) looks somewhat messy. However, one can proceed as follows. Let ${U}$ denote the ${n \times k}$ matrix

$\displaystyle U := \begin{pmatrix} 0 \\ I_k \end{pmatrix}$

(with ${I_k}$ the ${k \times k}$ identity matrix), and let ${V}$ be its transpose:

$\displaystyle V := \begin{pmatrix} 0 & I_k \end{pmatrix}.$

Then for any scalar ${t}$ (which we identify with ${t}$ times the identity matrix), one has

$\displaystyle M + UtV = \begin{pmatrix} A & B \\ C & D+t \end{pmatrix},$

and hence by (2)

$\displaystyle (M+UtV)^{-1} =$

$\displaystyle \begin{pmatrix} A^{-1} + A^{-1} B (D + t - CA^{-1} B)^{-1} C A^{-1} & - A^{-1} B (D + t- CA^{-1} B)^{-1} \\ - (D + t - CA^{-1} B)^{-1} CA^{-1} & (D + t - CA^{-1} B)^{-1} \end{pmatrix}.$

noting that the inverses here will exist for ${t}$ large enough. Taking limits as ${t \rightarrow \infty}$, we conclude that

$\displaystyle \lim_{t \rightarrow \infty} (M+UtV)^{-1} = \begin{pmatrix} A^{-1} & 0 \\ 0 & 0 \end{pmatrix}.$

On the other hand, by the Woodbury matrix identity (discussed in this previous blog post), we have

$\displaystyle (M+UtV)^{-1} = M^{-1} - M^{-1} U (t^{-1} + V M^{-1} U)^{-1} V M^{-1}$

and hence on taking limits and comparing with the preceding identity, one has

$\displaystyle \begin{pmatrix} A^{-1} & 0 \\ 0 & 0 \end{pmatrix} = M^{-1} - M^{-1} U (V M^{-1} U)^{-1} V M^{-1}.$

This achieves the aim of expressing the inverse ${A^{-1}}$ of the minor in terms of the inverse of the full matrix. Taking traces and rearranging, we conclude in particular that

$\displaystyle \mathrm{tr} A^{-1} = \mathrm{tr} M^{-1} - \mathrm{tr} (V M^{-2} U) (V M^{-1} U)^{-1}. \ \ \ \ \ (3)$

In the ${k=1}$ case, this can be simplified to

$\displaystyle \mathrm{tr} A^{-1} = \mathrm{tr} M^{-1} - \frac{e_n^T M^{-2} e_n}{e_n^T M^{-1} e_n} \ \ \ \ \ (4)$

where ${e_n}$ is the ${n^{th}}$ basis column vector.

We can apply this identity to understand how the spectrum of an ${n \times n}$ random matrix ${M}$ relates to that of its top left ${n-1 \times n-1}$ minor ${A}$. Subtracting any complex multiple ${z}$ of the identity from ${M}$ (and hence from ${A}$), we can relate the Stieltjes transform ${s_M(z) := \frac{1}{n} \mathrm{tr}(M-z)^{-1}}$ of ${M}$ with the Stieltjes transform ${s_A(z) := \frac{1}{n-1} \mathrm{tr}(A-z)^{-1}}$ of ${A}$:

$\displaystyle s_A(z) = \frac{n}{n-1} s_M(z) - \frac{1}{n-1} \frac{e_n^T (M-z)^{-2} e_n}{e_n^T (M-z)^{-1} e_n} \ \ \ \ \ (5)$

At this point we begin to proceed informally. Assume for sake of argument that the random matrix ${M}$ is Hermitian, with distribution that is invariant under conjugation by the unitary group ${U(n)}$; for instance, ${M}$ could be drawn from the Gaussian Unitary Ensemble (GUE), or alternatively ${M}$ could be of the form ${M = U D U^*}$ for some real diagonal matrix ${D}$ and ${U}$ a unitary matrix drawn randomly from ${U(n)}$ using Haar measure. To fix normalisations we will assume that the eigenvalues of ${M}$ are typically of size ${O(1)}$. Then ${A}$ is also Hermitian and ${U(n)}$-invariant. Furthermore, the law of ${e_n^T (M-z)^{-1} e_n}$ will be the same as the law of ${u^* (M-z)^{-1} u}$, where ${u}$ is now drawn uniformly from the unit sphere (independently of ${M}$). Diagonalising ${M}$ into eigenvalues ${\lambda_j}$ and eigenvectors ${v_j}$, we have

$\displaystyle u^* (M-z)^{-1} u = \sum_{j=1}^n \frac{|u^* v_j|^2}{\lambda_j - z}.$

One can think of ${u}$ as a random (complex) Gaussian vector, divided by the magnitude of that vector (which, by the Chernoff inequality, will concentrate to ${\sqrt{n}}$). Thus the coefficients ${u^* v_j}$ with respect to the orthonormal basis ${v_1,\dots,v_j}$ can be thought of as independent (complex) Gaussian vectors, divided by that magnitude. Using this and the Chernoff inequality again, we see (for ${z}$ distance ${\sim 1}$ away from the real axis at least) that one has the concentration of measure

$\displaystyle u^* (M-z)^{-1} u \approx \frac{1}{n} \sum_{j=1}^n \frac{1}{\lambda_j - z}$

and thus

$\displaystyle e_n^T (M-z)^{-1} e_n \approx \frac{1}{n} \mathrm{tr} (M-z)^{-1} = s_M(z)$

(that is to say, the diagonal entries of ${(M-z)^{-1}}$ are roughly constant). Similarly we have

$\displaystyle e_n^T (M-z)^{-2} e_n \approx \frac{1}{n} \mathrm{tr} (M-z)^{-2} = \frac{d}{dz} s_M(z).$

Inserting this into (5) and discarding terms of size ${O(1/n^2)}$, we thus conclude the approximate relationship

$\displaystyle s_A(z) \approx s_M(z) + \frac{1}{n} ( s_M(z) - s_M(z)^{-1} \frac{d}{dz} s_M(z) ).$

This can be viewed as a difference equation for the Stieltjes transform of top left minors of ${M}$. Iterating this equation, and formally replacing the difference equation by a differential equation in the large ${n}$ limit, we see that when ${n}$ is large and ${k \approx e^{-t} n}$ for some ${t \geq 0}$, one expects the top left ${k \times k}$ minor ${A_k}$ of ${M}$ to have Stieltjes transform

$\displaystyle s_{A_k}(z) \approx s( t, z ) \ \ \ \ \ (6)$

where ${s(t,z)}$ solves the Burgers-type equation

$\displaystyle \partial_t s(t,z) = s(t,z) - s(t,z)^{-1} \frac{d}{dz} s(t,z) \ \ \ \ \ (7)$

with initial data ${s(0,z) = s_M(z)}$.

Example 1 If ${M}$ is a constant multiple ${M = cI_n}$ of the identity, then ${s_M(z) = \frac{1}{c-z}}$. One checks that ${s(t,z) = \frac{1}{c-z}}$ is a steady state solution to (7), which is unsurprising given that all minors of ${M}$ are also ${c}$ times the identity.

Example 2 If ${M}$ is GUE normalised so that each entry has variance ${\sigma^2/n}$, then by the semi-circular law (see previous notes) one has ${s_M(z) \approx \frac{-z + \sqrt{z^2-4\sigma^2}}{2\sigma^2} = -\frac{2}{z + \sqrt{z^2-4\sigma^2}}}$ (using an appropriate branch of the square root). One can then verify the self-similar solution

$\displaystyle s(t,z) = \frac{-z + \sqrt{z^2 - 4\sigma^2 e^{-t}}}{2\sigma^2 e^{-t}} = -\frac{2}{z + \sqrt{z^2 - 4\sigma^2 e^{-t}}}$

to (7), which is consistent with the fact that a top ${k \times k}$ minor of ${M}$ also has the law of GUE, with each entry having variance ${\sigma^2 / n \approx \sigma^2 e^{-t} / k}$ when ${k \approx e^{-t} n}$.

One can justify the approximation (6) given a sufficiently good well-posedness theory for the equation (7). We will not do so here, but will note that (as with the classical inviscid Burgers equation) the equation can be solved exactly (formally, at least) by the method of characteristics. For any initial position ${z_0}$, we consider the characteristic flow ${t \mapsto z(t)}$ formed by solving the ODE

$\displaystyle \frac{d}{dt} z(t) = s(t,z(t))^{-1} \ \ \ \ \ (8)$

with initial data ${z(0) = z_0}$, ignoring for this discussion the problems of existence and uniqueness. Then from the chain rule, the equation (7) implies that

$\displaystyle \frac{d}{dt} s( t, z(t) ) = s(t,z(t))$

and thus ${s(t,z(t)) = e^t s(0,z_0)}$. Inserting this back into (8) we see that

$\displaystyle z(t) = z_0 + s(0,z_0)^{-1} (1-e^{-t})$

and thus (7) may be solved implicitly via the equation

$\displaystyle s(t, z_0 + s(0,z_0)^{-1} (1-e^{-t}) ) = e^t s(0, z_0) \ \ \ \ \ (9)$

for all ${t}$ and ${z_0}$.

Remark 3 In practice, the equation (9) may stop working when ${z_0 + s(0,z_0)^{-1} (1-e^{-t})}$ crosses the real axis, as (7) does not necessarily hold in this region. It is a cute exercise (ultimately coming from the Cauchy-Schwarz inequality) to show that this crossing always happens, for instance if ${z_0}$ has positive imaginary part then ${z_0 + s(0,z_0)^{-1}}$ necessarily has negative or zero imaginary part.

Example 4 Suppose we have ${s(0,z) = \frac{1}{c-z}}$ as in Example 1. Then (9) becomes

$\displaystyle s( t, z_0 + (c-z_0) (1-e^{-t}) ) = \frac{e^t}{c-z_0}$

for any ${t,z_0}$, which after making the change of variables ${z = z_0 + (c-z_0) (1-e^{-t}) = c - e^{-t} (c - z_0)}$ becomes

$\displaystyle s(t, z ) = \frac{1}{c-z}$

as in Example 1.

Example 5 Suppose we have

$\displaystyle s(0,z) = \frac{-z + \sqrt{z^2-4\sigma^2}}{2\sigma^2} = -\frac{2}{z + \sqrt{z^2-4\sigma^2}}.$

as in Example 2. Then (9) becomes

$\displaystyle s(t, z_0 - \frac{z_0 + \sqrt{z_0^2-4\sigma^2}}{2} (1-e^{-t}) ) = e^t \frac{-z_0 + \sqrt{z_0^2-4\sigma^2}}{2\sigma^2}.$

If we write

$\displaystyle z := z_0 - \frac{z_0 + \sqrt{z_0^2-4\sigma^2}}{2} (1-e^{-t})$

$\displaystyle = \frac{(1+e^{-t}) z_0 - (1-e^{-t}) \sqrt{z_0^2-4\sigma^2}}{2}$

one can calculate that

$\displaystyle z^2 - 4 \sigma^2 e^{-t} = (\frac{(1-e^{-t}) z_0 - (1+e^{-t}) \sqrt{z_0^2-4\sigma^2}}{2})^2$

and hence

$\displaystyle \frac{-z + \sqrt{z^2 - 4\sigma^2 e^{-t}}}{2\sigma^2 e^{-t}} = e^t \frac{-z_0 + \sqrt{z_0^2-4\sigma^2}}{2\sigma^2}$

which gives

$\displaystyle s(t,z) = \frac{-z + \sqrt{z^2 - 4\sigma^2 e^{-t}}}{2\sigma^2 e^{-t}}. \ \ \ \ \ (10)$

One can recover the spectral measure ${\mu}$ from the Stieltjes transform ${s(z)}$ as the weak limit of ${x \mapsto \frac{1}{\pi} \mathrm{Im} s(x+i\varepsilon)}$ as ${\varepsilon \rightarrow 0}$; we write this informally as

$\displaystyle d\mu(x) = \frac{1}{\pi} \mathrm{Im} s(x+i0^+)\ dx.$

In this informal notation, we have for instance that

$\displaystyle \delta_c(x) = \frac{1}{\pi} \mathrm{Im} \frac{1}{c-x-i0^+}\ dx$

which can be interpreted as the fact that the Cauchy distributions ${\frac{1}{\pi} \frac{\varepsilon}{(c-x)^2+\varepsilon^2}}$ converge weakly to the Dirac mass at ${c}$ as ${\varepsilon \rightarrow 0}$. Similarly, the spectral measure associated to (10) is the semicircular measure ${\frac{1}{2\pi \sigma^2 e^{-t}} (4 \sigma^2 e^{-t}-x^2)_+^{1/2}}$.

If we let ${\mu_t}$ be the spectral measure associated to ${s(t,\cdot)}$, then the curve ${e^{-t} \mapsto \mu_t}$ from ${(0,1]}$ to the space of measures is the high-dimensional limit ${n \rightarrow \infty}$ of a Gelfand-Tsetlin pattern (discussed in this previous post), if the pattern is randomly generated amongst all matrices ${M}$ with spectrum asymptotic to ${\mu_0}$ as ${n \rightarrow \infty}$. For instance, if ${\mu_0 = \delta_c}$, then the curve is ${\alpha \mapsto \delta_c}$, corresponding to a pattern that is entirely filled with ${c}$‘s. If instead ${\mu_0 = \frac{1}{2\pi \sigma^2} (4\sigma^2-x^2)_+^{1/2}}$ is a semicircular distribution, then the pattern is

$\displaystyle \alpha \mapsto \frac{1}{2\pi \sigma^2 \alpha} (4\sigma^2 \alpha -x^2)_+^{1/2},$

thus at height ${\alpha}$ from the top, the pattern is semicircular on the interval ${[-2\sigma \sqrt{\alpha}, 2\sigma \sqrt{\alpha}]}$. The interlacing property of Gelfand-Tsetlin patterns translates to the claim that ${\alpha \mu_\alpha(-\infty,\lambda)}$ (resp. ${\alpha \mu_\alpha(\lambda,\infty)}$) is non-decreasing (resp. non-increasing) in ${\alpha}$ for any fixed ${\lambda}$. In principle one should be able to establish these monotonicity claims directly from the PDE (7) or from the implicit solution (9), but it was not clear to me how to do so.

An interesting example of such a limiting Gelfand-Tsetlin pattern occurs when ${\mu_0 = \frac{1}{2} \delta_{-1} + \frac{1}{2} \delta_1}$, which corresponds to ${M}$ being ${2P-I}$, where ${P}$ is an orthogonal projection to a random ${n/2}$-dimensional subspace of ${{\bf C}^n}$. Here we have

$\displaystyle s(0,z) = \frac{1}{2} \frac{1}{-1-z} + \frac{1}{2} \frac{1}{1-z} = \frac{z}{1-z^2}$

and so (9) in this case becomes

$\displaystyle s(t, z_0 + \frac{1-z_0^2}{z_0} (1-e^{-t}) ) = \frac{e^t z_0}{1-z_0^2}$

A tedious calculation then gives the solution

$\displaystyle s(t,z) = \frac{(2e^{-t}-1)z + \sqrt{z^2 - 4e^{-t}(1-e^{-t})}}{2e^{-t}(1-z^2)}. \ \ \ \ \ (11)$

For ${\alpha = e^{-t} > 1/2}$, there are simple poles at ${z=-1,+1}$, and the associated measure is

$\displaystyle \mu_\alpha = \frac{2\alpha-1}{2\alpha} \delta_{-1} + \frac{2\alpha-1}{2\alpha} \delta_1 + \frac{1}{2\pi \alpha(1-x^2)} (4\alpha(1-\alpha)-x^2)_+^{1/2}\ dx.$

This reflects the interlacing property, which forces ${\frac{2\alpha-1}{2\alpha} \alpha n}$ of the ${\alpha n}$ eigenvalues of the ${\alpha n \times \alpha n}$ minor to be equal to ${-1}$ (resp. ${+1}$). For ${\alpha = e^{-t} \leq 1/2}$, the poles disappear and one just has

$\displaystyle \mu_\alpha = \frac{1}{2\pi \alpha(1-x^2)} (4\alpha(1-\alpha)-x^2)_+^{1/2}\ dx.$

For ${\alpha=1/2}$, one has an inverse semicircle distribution

$\displaystyle \mu_{1/2} = \frac{1}{\pi} (1-x^2)_+^{-1/2}.$

There is presumably a direct geometric explanation of this fact (basically describing the singular values of the product of two random orthogonal projections to half-dimensional subspaces of ${{\bf C}^n}$), but I do not know of one off-hand.

The evolution of ${s(t,z)}$ can also be understood using the ${R}$-transform and ${S}$-transform from free probability. Formally, letlet ${z(t,s)}$ be the inverse of ${s(t,z)}$, thus

$\displaystyle s(t,z(t,s)) = s$

for all ${t,s}$, and then define the ${R}$-transform

$\displaystyle R(t,s) := z(t,-s) - \frac{1}{s}.$

The equation (9) may be rewritten as

$\displaystyle z( t, e^t s ) = z(0,s) + s^{-1} (1-e^{-t})$

and hence

$\displaystyle R(t, -e^t s) = R(0, -s)$

or equivalently

$\displaystyle R(t,s) = R(0, e^{-t} s). \ \ \ \ \ (12)$

See these previous notes for a discussion of free probability topics such as the ${R}$-transform.

Example 6 If ${s(t,z) = \frac{1}{c-z}}$ then the ${R}$ transform is ${R(t,s) = c}$.

Example 7 If ${s(t,z)}$ is given by (10), then the ${R}$ transform is

$\displaystyle R(t,s) = \sigma^2 e^{-t} s.$

Example 8 If ${s(t,z)}$ is given by (11), then the ${R}$ transform is

$\displaystyle R(t,s) = \frac{-1 + \sqrt{1 + 4 s^2 e^{-2t}}}{2 s e^{-t}}.$

This simple relationship (12) is essentially due to Nica and Speicher (thanks to Dima Shylakhtenko for this reference). It has the remarkable consequence that when ${\alpha = 1/m}$ is the reciprocal of a natural number ${m}$, then ${\mu_{1/m}}$ is the free arithmetic mean of ${m}$ copies of ${\mu}$, that is to say ${\mu_{1/m}}$ is the free convolution ${\mu \boxplus \dots \boxplus \mu}$ of ${m}$ copies of ${\mu}$, pushed forward by the map ${\lambda \rightarrow \lambda/m}$. In terms of random matrices, this is asserting that the top ${n/m \times n/m}$ minor of a random matrix ${M}$ has spectral measure approximately equal to that of an arithmetic mean ${\frac{1}{m} (M_1 + \dots + M_m)}$ of ${m}$ independent copies of ${M}$, so that the process of taking top left minors is in some sense a continuous analogue of the process of taking freely independent arithmetic means. There ought to be a geometric proof of this assertion, but I do not know of one. In the limit ${m \rightarrow \infty}$ (or ${\alpha \rightarrow 0}$), the ${R}$-transform becomes linear and the spectral measure becomes semicircular, which is of course consistent with the free central limit theorem.

In a similar vein, if one defines the function

$\displaystyle \omega(t,z) := \alpha \int_{\bf R} \frac{zx}{1-zx}\ d\mu_\alpha(x) = e^{-t} (- 1 - z^{-1} s(t, z^{-1}))$

and inverts it to obtain a function ${z(t,\omega)}$ with

$\displaystyle \omega(t, z(t,\omega)) = \omega$

for all ${t, \omega}$, then the ${S}$-transform ${S(t,\omega)}$ is defined by

$\displaystyle S(t,\omega) := \frac{1+\omega}{\omega} z(t,\omega).$

Writing

$\displaystyle s(t,z) = - z^{-1} ( 1 + e^t \omega(t, z^{-1}) )$

for any ${t}$, ${z}$, we have

$\displaystyle z_0 + s(0,z_0)^{-1} (1-e^{-t}) = z_0 \frac{\omega(0,z_0^{-1})+e^{-t}}{\omega(0,z_0^{-1})+1}$

and so (9) becomes

$\displaystyle - z_0^{-1} \frac{\omega(0,z_0^{-1})+1}{\omega(0,z_0^{-1})+e^{-t}} (1 + e^{t} \omega(t, z_0^{-1} \frac{\omega(0,z_0^{-1})+1}{\omega(0,z_0^{-1})+e^{-t}}))$

$\displaystyle = - e^t z_0^{-1} (1 + \omega(0, z_0^{-1}))$

which simplifies to

$\displaystyle \omega(t, z_0^{-1} \frac{\omega(0,z_0^{-1})+1}{\omega(0,z_0^{-1})+e^{-t}})) = \omega(0, z_0^{-1});$

replacing ${z_0}$ by ${z(0,\omega)^{-1}}$ we obtain

$\displaystyle \omega(t, z(0,\omega) \frac{\omega+1}{\omega+e^{-t}}) = \omega$

and thus

$\displaystyle z(0,\omega)\frac{\omega+1}{\omega+e^{-t}} = z(t, \omega)$

and hence

$\displaystyle S(0, \omega) = \frac{\omega+e^{-t}}{\omega+1} S(t, \omega).$

One can compute ${\frac{\omega+e^{-t}}{\omega+1}}$ to be the ${S}$-transform of the measure ${(1-\alpha) \delta_0 + \alpha \delta_1}$; from the link between ${S}$-transforms and free products (see e.g. these notes of Guionnet), we conclude that ${(1-\alpha)\delta_0 + \alpha \mu_\alpha}$ is the free product of ${\mu_1}$ and ${(1-\alpha) \delta_0 + \alpha \delta_1}$. This is consistent with the random matrix theory interpretation, since ${(1-\alpha)\delta_0 + \alpha \mu_\alpha}$ is also the spectral measure of ${PMP}$, where ${P}$ is the orthogonal projection to the span of the first ${\alpha n}$ basis elements, so in particular ${P}$ has spectral measure ${(1-\alpha) \delta_0 + \alpha \delta_1}$. If ${M}$ is unitarily invariant then (by a fundamental result of Voiculescu) it is asymptotically freely independent of ${P}$, so the spectral measure of ${PMP = P^{1/2} M P^{1/2}}$ is asymptotically the free product of that of ${M}$ and of ${P}$.

In the foundations of modern probability, as laid out by Kolmogorov, the basic objects of study are constructed in the following order:

1. Firstly, one selects a sample space ${\Omega}$, whose elements ${\omega}$ represent all the possible states that one’s stochastic system could be in.
2. Then, one selects a ${\sigma}$-algebra ${{\mathcal B}}$ of events ${E}$ (modeled by subsets of ${\Omega}$), and assigns each of these events a probability ${{\bf P}(E) \in [0,1]}$ in a countably additive manner, so that the entire sample space has probability ${1}$.
3. Finally, one builds (commutative) algebras of random variables ${X}$ (such as complex-valued random variables, modeled by measurable functions from ${\Omega}$ to ${{\bf C}}$), and (assuming suitable integrability or moment conditions) one can assign expectations ${\mathop{\bf E} X}$ to each such random variable.

In measure theory, the underlying measure space ${\Omega}$ plays a prominent foundational role, with the measurable sets and measurable functions (the analogues of the events and the random variables) always being viewed as somehow being attached to that space. In probability theory, in contrast, it is the events and their probabilities that are viewed as being fundamental, with the sample space ${\Omega}$ being abstracted away as much as possible, and with the random variables and expectations being viewed as derived concepts. See Notes 0 for further discussion of this philosophy.

However, it is possible to take the abstraction process one step further, and view the algebra of random variables and their expectations as being the foundational concept, and ignoring both the presence of the original sample space, the algebra of events, or the probability measure.

There are two reasons for wanting to shed (or abstract away) these previously foundational structures. Firstly, it allows one to more easily take certain types of limits, such as the large ${n}$ limit ${n \rightarrow \infty}$ when considering ${n \times n}$ random matrices, because quantities built from the algebra of random variables and their expectations, such as the normalised moments of random matrices tend to be quite stable in the large ${n}$ limit (as we have seen in previous notes), even as the sample space and event space varies with ${n}$. (This theme of using abstraction to facilitate the taking of the large ${n}$ limit also shows up in the application of ergodic theory to combinatorics via the correspondence principle; see this previous blog post for further discussion.)

Secondly, this abstract formalism allows one to generalise the classical, commutative theory of probability to the more general theory of non-commutative probability theory, which does not have a classical underlying sample space or event space, but is instead built upon a (possibly) non-commutative algebra of random variables (or “observables”) and their expectations (or “traces”). This more general formalism not only encompasses classical probability, but also spectral theory (with matrices or operators taking the role of random variables, and the trace taking the role of expectation), random matrix theory (which can be viewed as a natural blend of classical probability and spectral theory), and quantum mechanics (with physical observables taking the role of random variables, and their expected value on a given quantum state being the expectation). It is also part of a more general “non-commutative way of thinking” (of which non-commutative geometry is the most prominent example), in which a space is understood primarily in terms of the ring or algebra of functions (or function-like objects, such as sections of bundles) placed on top of that space, and then the space itself is largely abstracted away in order to allow the algebraic structures to become less commutative. In short, the idea is to make algebra the foundation of the theory, as opposed to other possible choices of foundations such as sets, measures, categories, etc..

[Note that this foundational preference is to some extent a metamathematical one rather than a mathematical one; in many cases it is possible to rewrite the theory in a mathematically equivalent form so that some other mathematical structure becomes designated as the foundational one, much as probability theory can be equivalently formulated as the measure theory of probability measures. However, this does not negate the fact that a different choice of foundations can lead to a different way of thinking about the subject, and thus to ask a different set of questions and to discover a different set of proofs and solutions. Thus it is often of value to understand multiple foundational perspectives at once, to get a truly stereoscopic view of the subject.]

It turns out that non-commutative probability can be modeled using operator algebras such as ${C^*}$-algebras, von Neumann algebras, or algebras of bounded operators on a Hilbert space, with the latter being accomplished via the Gelfand-Naimark-Segal construction. We will discuss some of these models here, but just as probability theory seeks to abstract away its measure-theoretic models, the philosophy of non-commutative probability is also to downplay these operator algebraic models once some foundational issues are settled.

When one generalises the set of structures in one’s theory, for instance from the commutative setting to the non-commutative setting, the notion of what it means for a structure to be “universal”, “free”, or “independent” can change. The most familiar example of this comes from group theory. If one restricts attention to the category of abelian groups, then the “freest” object one can generate from two generators ${e,f}$ is the free abelian group of commutative words ${e^n f^m}$ with ${n,m \in {\bf Z}}$, which is isomorphic to the group ${{\bf Z}^2}$. If however one generalises to the non-commutative setting of arbitrary groups, then the “freest” object that can now be generated from two generators ${e,f}$ is the free group ${{\Bbb F}_2}$ of non-commutative words ${e^{n_1} f^{m_1} \ldots e^{n_k} f^{m_k}}$ with ${n_1,m_1,\ldots,n_k,m_k \in {\bf Z}}$, which is a significantly larger extension of the free abelian group ${{\bf Z}^2}$.

Similarly, when generalising classical probability theory to non-commutative probability theory, the notion of what it means for two or more random variables to be independent changes. In the classical (commutative) setting, two (bounded, real-valued) random variables ${X, Y}$ are independent if one has

$\displaystyle \mathop{\bf E} f(X) g(Y) = 0$

whenever ${f, g: {\bf R} \rightarrow {\bf R}}$ are well-behaved functions (such as polynomials) such that ${\mathop{\bf E} f(X)}$, ${\mathop{\bf E} g(Y)}$ both vanish. In the non-commutative setting, one can generalise the above definition to two commuting bounded self-adjoint variables; this concept is useful for instance in quantum probability, which is an abstraction of the theory of observables in quantum mechanics. But for two (bounded, self-adjoint) non-commutative random variables ${X, Y}$, the notion of classical independence no longer applies. As a substitute, one can instead consider the notion of being freely independent (or free for short), which means that

$\displaystyle \mathop{\bf E} f_1(X) g_1(Y) \ldots f_k(X) g_k(Y) = 0$

whenever ${f_1,g_1,\ldots,f_k,g_k: {\bf R} \rightarrow {\bf R}}$ are well-behaved functions such that all of ${\mathop{\bf E} f_1(X), \mathop{\bf E} g_1(Y), \ldots, \mathop{\bf E} f_k(X), \mathop{\bf E} g_k(Y)}$ vanish.

The concept of free independence was introduced by Voiculescu, and its study is now known as the subject of free probability. We will not attempt a systematic survey of this subject here; for this, we refer the reader to the surveys of Speicher and of Biane. Instead, we shall just discuss a small number of topics in this area to give the flavour of the subject only.

The significance of free probability to random matrix theory lies in the fundamental observation that random matrices which are independent in the classical sense, also tend to be independent in the free probability sense, in the large ${n}$ limit ${n \rightarrow \infty}$. (This is only possible because of the highly non-commutative nature of these matrices; as we shall see, it is not possible for non-trivial commuting independent random variables to be freely independent.) Because of this, many tedious computations in random matrix theory, particularly those of an algebraic or enumerative combinatorial nature, can be done more quickly and systematically by using the framework of free probability, which by design is optimised for algebraic tasks rather than analytical ones.

Much as free groups are in some sense “maximally non-commutative”, freely independent random variables are about as far from being commuting as possible. For instance, if ${X, Y}$ are freely independent and of expectation zero, then ${\mathop{\bf E} XYXY}$ vanishes, but ${\mathop{\bf E} XXYY}$ instead factors as ${(\mathop{\bf E} X^2) (\mathop{\bf E} Y^2)}$. As a consequence, the behaviour of freely independent random variables can be quite different from the behaviour of their classically independent commuting counterparts. Nevertheless there is a remarkably strong analogy between the two types of independence, in that results which are true in the classically independent case often have an interesting analogue in the freely independent setting. For instance, the central limit theorem (Notes 2) for averages of classically independent random variables, which roughly speaking asserts that such averages become gaussian in the large ${n}$ limit, has an analogue for averages of freely independent variables, the free central limit theorem, which roughly speaking asserts that such averages become semicircular in the large ${n}$ limit. One can then use this theorem to provide yet another proof of Wigner’s semicircle law (Notes 4).

Another important (and closely related) analogy is that while the distribution of sums of independent commutative random variables can be quickly computed via the characteristic function (i.e. the Fourier transform of the distribution), the distribution of sums of freely independent non-commutative random variables can be quickly computed using the Stieltjes transform instead (or with closely related objects, such as the ${R}$-transform of Voiculescu). This is strongly reminiscent of the appearance of the Stieltjes transform in random matrix theory, and indeed we will see many parallels between the use of the Stieltjes transform here and in Notes 4.

As mentioned earlier, free probability is an excellent tool for computing various expressions of interest in random matrix theory, such as asymptotic values of normalised moments in the large ${n}$ limit ${n \rightarrow \infty}$. Nevertheless, as it only covers the asymptotic regime in which ${n}$ is sent to infinity while holding all other parameters fixed, there are some aspects of random matrix theory to which the tools of free probability are not sufficient by themselves to resolve (although it can be possible to combine free probability theory with other tools to then answer these questions). For instance, questions regarding the rate of convergence of normalised moments as ${n \rightarrow \infty}$ are not directly answered by free probability, though if free probability is combined with tools such as concentration of measure (Notes 1) then such rate information can often be recovered. For similar reasons, free probability lets one understand the behaviour of ${k^{th}}$ moments as ${n \rightarrow \infty}$ for fixed ${k}$, but has more difficulty dealing with the situation in which ${k}$ is allowed to grow slowly in ${n}$ (e.g. ${k = O(\log n)}$). Because of this, free probability methods are effective at controlling the bulk of the spectrum of a random matrix, but have more difficulty with the edges of that spectrum (as well as with related concepts such as the operator norm, Notes 3) as well as with fine-scale structure of the spectrum. Finally, free probability methods are most effective when dealing with matrices that are Hermitian with bounded operator norm, largely because the spectral theory of bounded self-adjoint operators in the infinite-dimensional setting of the large ${n}$ limit is non-pathological. (This is ultimately due to the stable nature of eigenvalues in the self-adjoint setting; see this previous blog post for discussion.) For non-self-adjoint operators, free probability needs to be augmented with additional tools, most notably by bounds on least singular values, in order to recover the required stability for the various spectral data of random matrices to behave continuously with respect to the large ${n}$ limit. We will discuss this latter point in a later set of notes.