The classical formulation of Hilbert’s fifth problem asks whether topological groups that have the topological structure of a manifold, are necessarily Lie groups. This is indeed, the case, thanks to following theorem of Gleason and Montgomery-Zippin:

Theorem 1 (Hilbert’s fifth problem) Let ${G}$ be a topological group which is locally Euclidean. Then ${G}$ is isomorphic to a Lie group.

We have discussed the proof of this result, and of related results, in previous posts. There is however a generalisation of Hilbert’s fifth problem which remains open, namely the Hilbert-Smith conjecture, in which it is a space acted on by the group which has the manifold structure, rather than the group itself:

Conjecture 2 (Hilbert-Smith conjecture) Let ${G}$ be a locally compact topological group which acts continuously and faithfully (or effectively) on a connected finite-dimensional manifold ${X}$. Then ${G}$ is isomorphic to a Lie group.

Note that Conjecture 2 easily implies Theorem 1 as one can pass to the connected component ${G^\circ}$ of a locally Euclidean group (which is clearly locally compact), and then look at the action of ${G^\circ}$ on itself by left-multiplication.

The hypothesis that the action is faithful (i.e. each non-identity group element ${g \in G \backslash \{\hbox{id}\}}$ acts non-trivially on ${X}$) cannot be completely eliminated, as any group ${G}$ will have a trivial action on any space ${X}$. The requirement that ${G}$ be locally compact is similarly necessary: consider for instance the diffeomorphism group ${\hbox{Diff}(S^1)}$ of, say, the unit circle ${S^1}$, which acts on ${S^1}$ but is infinite dimensional and is not locally compact (with, say, the uniform topology). Finally, the connectedness of ${X}$ is also important: the infinite torus ${G = ({\bf R}/{\bf Z})^{\bf N}}$ (with the product topology) acts faithfully on the disconnected manifold ${X := {\bf R}/{\bf Z} \times {\bf N}}$ by the action

$\displaystyle (g_n)_{n \in {\bf N}} (\theta, m) := (\theta + g_m, m).$

The conjecture in full generality remains open. However, there are a number of partial results. For instance, it was observed by Montgomery and Zippin that the conjecture is true for transitive actions, by a modification of the argument used to establish Theorem 1. This special case of the Hilbert-Smith conjecture (or more precisely, a generalisation thereof in which “finite-dimensional manifold” was replaced by “locally connected locally compact finite-dimensional”) was used in Gromov’s proof of his famous theorem on groups of polynomial growth. I record the argument of Montgomery and Zippin below the fold.

Another partial result is the reduction of the Hilbert-Smith conjecture to the ${p}$-adic case. Indeed, it is known that Conjecture 2 is equivalent to

Conjecture 3 (Hilbert-Smith conjecture for ${p}$-adic actions) It is not possible for a ${p}$-adic group ${{\bf Z}_p}$ to act continuously and effectively on a connected finite-dimensional manifold ${X}$.

The reduction to the ${p}$-adic case follows from the structural theory of locally compact groups (specifically, the Gleason-Yamabe theorem discussed in previous posts) and some results of Newman that sharply restrict the ability of periodic actions on a manifold ${X}$ to be close to the identity. I record this argument (which appears for instance in this paper of Lee) below the fold also.

— 1. Transitive actions —

We now show the transitive case of the Hilbert-Smith conjecture, following Montgomery and Zippin:

Proposition 4 Let ${G}$ be a locally compact ${\sigma}$-compact group that acts transitively, faithfully, and continuously on a connected manifold ${X}$. Then ${G}$ is isomorphic to a Lie group.

The ${\sigma}$-compact hypothesis is a technical one, and can likely be dropped, but we retain it for this discussion (as in most applications we can reduce to this case).

The advantage of transitivity is that one can now view ${X}$ as a homogeneous space ${X=G/H}$ of ${G}$, where ${H = \hbox{Stab}(x_0)}$ is the stabiliser of a point ${x_0}$ (and is thus a closed subgroup of ${G}$). Note that a priori, we only know that ${X}$ and ${G/H}$ are identifiable as sets, with the identification map ${\iota: G/H \rightarrow X}$ defined by setting ${\iota( g H ) := g x_0}$ being continuous; but thanks to the ${\sigma}$-compact hypothesis, we can upgrade ${\iota}$ to a homeomorphism. Indeed, as ${G}$ is ${\sigma}$-compact, ${G/H}$ is also; and so given any compact neighbourhood of the identity ${K}$ in ${G}$, ${G/H}$ can be covered by countably many translates of ${KH/H}$. By the Baire category theorem, one of these translates ${gK}$ has an image ${\iota(gKH/H) = gKx_0}$ in ${X}$ with non-empty interior, which implies that ${K^{-1} K x_0}$ has ${x_0}$ as an interior point. From this it is not hard to see that the map ${\iota}$ is open; as it is also a continuous bijection, it is therefore a homeomorphism.

By the Gleason-Yamabe theorem, ${G}$ has an open subgroup ${G'}$ that is the inverse limit of Lie groups. (Note that ${G}$ is Hausdorff because it acts faithfully on the Hausdorff space ${X}$.) ${G'}$ acts transitively on ${G'H/H}$, which is an open subset of ${G/H}$ and thus also a manifold. Thus, we may assume without loss of generality that ${G}$ is itself the inverse limit of Lie groups ${G_n = G/K_n}$, where ${K_1 \supset K_2 \supset \ldots}$ is a sequence of normal compact groups shrinking to the identity.

As ${G}$ is ${\sigma}$-compact, the manifold ${X}$ is also. As ${G}$ acts faithfully on ${X}$, this makes ${G}$ first countable; and so ${G}$ is the inverse limit of a sequence of Lie groups ${G_n}$, with each ${G_{n+1}}$ projecting surjectively onto ${G_n}$.

Let ${H_n}$ be the projection of ${H}$ onto ${G_n}$; this is a closed subgroup of the Lie group ${G_n}$, and each ${H_{n+1}}$ projects surjectively onto ${H_n}$. Then ${G_n/H_n}$ are manifolds, and ${G/H}$ is the inverse limit of the ${G_n/H_n = G / HK_n}$. By invariance of domain, the dimensions of the ${G_n/H_n}$ must be non-decreasing, and bounded above by the dimension of ${G/H}$. Thus, for ${n}$ large enough, the dimensions of ${G_n/H_n}$ must be constant; by renumbering, we may assume that all the ${G_n/H_n}$ have the same dimension. As each ${G_{n+1}/H_{n+1}}$ is a cover of ${G_n/H_n}$ with structure group ${K_n/K_{n+1}}$, we conclude that the ${K_n/K_{n+1}}$ are zero-dimensional and compact, and thus finite. On the other hand, ${G/H}$ is locally connected, which implies that the ${K_n/K_{n+1}}$ are eventually trivial. Indeed, if we pick a simply connected neighbbourhood ${U_1}$ of the identity in ${G_1/H_1}$, then by local connectedness of ${G/H}$, there exists a connected neighbourhood ${U}$ of the identity in ${G/H}$ whose projection to ${G_1/H_1}$ is contained in ${U_1}$. Being open, ${U}$ must contain one of the ${K_n}$. If ${K_n/K_m}$ is non-trivial for any ${m>n}$, then the projection of ${U}$ to ${G_m/H_m}$ will then be disconnected (as this projection will be contained in a neighbourhood with the topological structure of ${U \times K_1/K_m}$, and its intersection with the latter fibre is at least as large as ${K_n/K_m}$. We conclude that ${K_n}$ is trivial for ${n}$ large enough, and so ${G = G_n}$ is a Lie group as required.

— 2. Periodic actions of prime order —

We now study periodic actions ${T: X \rightarrow X}$ on a manifold ${X}$ of some prime order ${p}$, thus ${T^p = \hbox{id}}$.

The basic observation to exploit here is that of rigidity: a periodic action (or more precisely, the orbits of this action) cannot be too close to the identity, without actually being the identity. More precisely, we have the following theorem of Newman:

Theorem 5 (Newman’s first theorem) Let ${U}$ be an open subset of ${{\bf R}^n}$ containing the closed unit ball ${B}$, and let ${T: U \rightarrow U}$ be a homeomorphism of some prime period ${p \geq 1}$. Suppose that for every ${x \in U}$, the orbit ${\{ T^n x: n = 0,1,\ldots,p-1\}}$ has diameter strictly less than ${1}$. Then ${T(0)=0}$.

Note that some result like this must be needed in order to establish the Hilbert-Smith conjecture. Suppose for instance that one could find a non-trivial transformation ${T}$ of some period ${p}$ on the unit ball ${B}$ that preserved the boundary of that ball. Then by placing infinitely many disjoint copies of that ball into ${{\bf R}^n}$, and considering maps that are equal to some power of ${T}$ on each such ball, and on the identity outside all the balls, we can obtain a faithful action of ${({\bf Z}/p{\bf Z})^{\bf N}}$ on ${{\bf R}^n}$, contradicting the Hilbert-Smith conjecture.

To prove the theorem we will need some basic degree theory. Given a continuously differentiable map ${\Phi: B \rightarrow {\bf R}^n}$, we know from Sard’s theorem that almost every point ${x}$ in ${{\bf R}^n}$ is a regular point, in that the preimage ${\Phi^{-1}(\{x\})}$ is finite and avoids the boundary ${\partial B}$ of ${B}$, with ${\nabla \Phi(x)}$ being non-degenerate at each ${x}$. We can define the degree of ${\Phi}$ at the regular point ${x}$ to be the number of preimages with ${\nabla \Phi}$ orientation-preserving, minus the number of preimages with ${\nabla \Phi}$ orientation-reversing. One can show that this degree extends to a constant integer-valued function on each connected component ${U}$ of ${{\bf R}^n \backslash \Phi(\partial B)}$; indeed, one can define the degree ${\hbox{deg}(U)}$ on such a component analytically by the formula

$\displaystyle \hbox{deg}(U) = \int_B \Phi^* \omega$

for any volume form ${\omega}$ on ${U}$ of total mass ${1}$ (one can show that this definition is independent of the choice of ${\omega}$). This definition is stable under uniform convergence of ${\Phi}$, and thus can be used to also define the degree for maps ${\Phi}$ that are merely continuous rather than continuously differentiable.

Proof: Suppose for contradiction that ${T(0) \neq 0}$. We use an averaging argument, combined with degree theory. Let ${\Phi: U \rightarrow {\bf R}^n}$ be the map

$\displaystyle \Phi(x) := \frac{1}{p} \sum_{n=0}^{p-1} T^n x.$

Then from construction, ${\Phi}$ is continuous and ${T}$-invariant (thus ${\Phi(x)=\Phi(Tx)}$ for all ${x \in U}$) and we have ${|\Phi(x)-x| < 1}$ for all ${x \in U}$. In particular, ${\Phi}$ is non-zero on the boundary ${\partial B}$ of ${B}$, and ${\Phi(\partial B)}$ is contractible to ${\partial B}$ in ${{\bf R}^n \backslash 0}$ (by taking convex combinations of ${\Phi}$ and the identity). As such, the degree of ${\Phi}$ on ${B}$ near the origin is equal to ${1}$.

A similar argument shows that the homeomorphism ${T}$ has degree ${1}$ on ${B}$ near the origin, and in particular is orientation-preserving rather than orientation reversing.

On the other hand, it can be shown that the degree of ${\Phi}$ must be divisible by ${q}$, leading to a contradiction. This is easiest to see in the case when ${\Phi}$ is continuously differentiable, for then by Sard’s theorem we may find a regular point ${x}$ arbitrarily close to ${0}$. On the other hand, since ${T(0)}$ is a positive distance away from ${0}$, we see that for ${x}$ sufficiently close to ${0}$, ${x}$ is not a fixed point of ${T}$, and thus (as ${T}$ has prime order) all elements in the preimage ${\Phi^{-1}(\{x\})}$ are not fixed points of ${T}$ either. Thus ${\Phi^{-1}(\{x\})}$ can be partitioned into a finite number of disjoint orbits of ${T}$ of cardinality ${p}$. As ${T}$ is orientation preserving and ${x}$ is a regular point of ${\Phi}$, each of these orbits contributes ${+p}$ or ${-p}$ to the degree, giving the claim.

The case when ${\Phi}$ is not continuously differentiable is trickier, as the degree is not as easily computed in this case. One way to proceed is to perturb ${T}$ (or more precisely, the graph ${\{ (x, Tx, \ldots, T^{p-1} x): x \in U \}}$ in ${U^p}$) to be piecewise linear near the preimage of ${0}$ (while preserving the periodicity properties of the graph), so that degree can be computed by hand; this is the approach taken in Newman’s original paper. Another is to use the machinery of singular homology, which is more general and flexible than degree theory; this is the approach taken by Smith and by Dress. $\Box$

Note that the above argument shows not only that ${T}$ fixes the origin ${0}$, but must also fix an open neighbourhood of the origin, by translating ${B}$ slightly. One can then extend this open neighbourhood to the entire space by the following variant of Theorem 5.

Theorem 6 (Newman’s second theorem) Let ${X}$ be a connected manifold, and let ${T: X \rightarrow X}$ be a homeomorphism of some prime order ${p}$ that fixes a non-empty open set ${U}$. Then ${T}$ is the identity.

Proof: We need to show that ${T}$ fixes all points in ${X}$, and not just ${U}$. Suppose for sake of contradiction that ${T}$ just fixes some of the points in ${X}$ and not others. By a continuity argument, and applying a homeomorphic change of variables if necessary, we can find a coordinate chart containing the ball ${B}$, where ${T}$ fixes all ${x = (x_1,\ldots,x_n) \in B}$ with ${x_1 \leq 0}$, but does not fix all ${x \in B}$ with ${x_1 > 0}$. By shrinking ${B}$ we may assume that the entire orbit ${B, TB, \ldots, T^{p-1} B}$ stays inside the coordinate chart (and can thus be viewed as a subset of ${{\bf R}^n}$). Then the map ${\Phi}$ can be defined as before. This map is the identity on the left hemisphere ${\{ x \in B: x_1 \leq 0 \}}$. On the right hemisphere ${\{ x \in B: x_1 > 0\}}$, one observes for ${x}$ small enough that ${x, Tx, \ldots, T^{p-1} x}$ must all stay on the right-hemisphere (as they must lie in ${B}$ and cannot enter the left-hemisphere, where ${T}$ is the identity) and so ${\Phi}$ stays on the right. This implies that ${\Phi}$ has degree ${1}$ near ${0}$ on a small ball around the origin, but as before one can argue that the degree must in fact be divisible by ${p}$, leading again to a contradiction. $\Box$

— 3. Reduction to the ${p}$-adic case —

We are now ready to prove Conjecture 2 assuming Conjecture 3. Let ${G}$ be a locally compact group acting continuously and faithfully on a connected manifold ${X}$; we wish to show that ${G}$ is Lie.

We first make some basic reductions. From the Gleason-Yamabe theorem (as discussed in this post) every locally compact group ${G}$ contains an open subgroup which is an extension of a Lie group by a compact subgroup of ${G}$. Since a group with a Lie group as an open subgroup is again Lie (because all outer automorphisms of Lie groups are smooth), and an extension of a Lie group by a Lie group is again Lie (this latter result, first proved by Gleason, follows for instance from the fact that a locally compact group is Lie if and only if it is NSS), it thus suffices to prove the claim when ${G}$ is compact. In particular, all orbits of ${G}$ on ${X}$ are also compact.

Let ${B}$ be a small ball in chart of ${X}$ around some origin ${x_0}$. By continuity, there is some neighbourhood ${U}$ of the identity in ${G}$ such that ${gB \ni x_0}$ for all ${g \in U}$. By the Peter-Weyl theorem, there is a compact normal subgroup ${G'}$ of ${G}$ in ${U}$ with ${G/G'}$ linear (and hence Lie). The set ${G' B}$ is then a ${G'}$-invariant manifold, which is precompact and connected (because all the shifts ${gB}$ are connected and share a common point). If we let ${G''}$ be the subgroup of ${G'}$ that fixes ${G'B}$, then ${G''}$ is a compact normal subgroup of ${G'}$, and ${G'/G''}$ acts faithfully on ${G'B}$. By Newman’s first theorem (Theorem 5), we see that if ${U}$ is small enough, then ${G'/G''}$ cannot contain any elements of prime order, and hence cannot contain any non-trivial periodic elements whatsoever; by Conjecture 3, it also cannot contain a continuously embedded copy of ${{\bf Z}_p}$ for any ${p}$. We claim that this forces ${G'/G''}$ to be trivial.

As ${G'B}$ is precompact, the space of ${C(\overline{G'B} \rightarrow \overline{G'B})}$ of continuous maps from ${G'/G''}$ to itself (with the compact-open topology) is first countable, which makes ${G'/G''}$ first-countable as well (since ${G'/G''}$ is homeomorphic to a subspace of ${C(\overline{G'B} \rightarrow \overline{G'B})}$). The claim now follows from

Lemma 7 Let ${G}$ be a compact first-countable group which does not contain any non-trivial periodic elements or a continuously embedded copy of ${{\bf Z}_p}$ for any ${p}$. Then ${G}$ is trivial.

Proof: Every element ${g}$ of ${G}$ is contained in a compact abelian subgroup of ${G}$, namely the closed group ${\overline{\langle g \rangle}}$ generated by ${g}$. Thus we may assume without loss of generality that ${G}$ is abelian.

As ${G}$ is both compact and first countable, it can be written (using the Peter-Weyl theorem) as the inverse limit ${\lim_{\leftarrow} G_n}$ of a countable sequence of compact abelian Lie groups ${G_n}$, with surjective continuous projection homomorphisms ${\pi_{n+1 \rightarrow n}: G_{n+1} \rightarrow G_n}$ between these Lie groups. (Note that without first countability, one might only be an inverse limit of a net of Lie groups rather than a sequence, as one can see for instance with the example ${({\bf R}/{\bf Z})^{\bf R}}$.)

Suppose for contradiction that at least one of the ${G_n}$, say ${G_1}$, is non-trivial. It is a standard fact that every compact abelian Lie group is isomorphic to the direct product of a torus and a finite group. (Indeed, in the connected case one can inspect the kernel of the exponential map, and then one can extend to the general case by viewing a compact Lie group as an extension of a finite group by a connected compact Lie group.) in particular, the periodic points (i.e. points of finite order) are dense, and so there exists an element ${g_1}$ of ${G_1}$ of finite non-trivial order. By raising ${g_1}$ to a suitable power, we may assume that ${g_1}$ has some prime order ${p}$.

We now claim inductively that for each ${n=1,2,\ldots}$, ${g_1}$ can be lifted to an element ${g_n \in G_n}$ of some order ${p^{k_n}}$, where ${1 = k_1 \leq k_2 \leq \ldots}$ are a non-decreasing set of integers. Indeed, suppose inductively that we have already lifted ${g_1}$ up to ${g_n \in G_n}$ with order ${p^{k_n}}$. The preimage of ${g_n}$ in ${G_{n+1}}$ is then a dense subset of the preimage of ${\langle g_n \rangle}$, which is a compact abelian Lie group, and thus contains an element ${g'_{n+1}}$ of some finite order. As ${g_n}$ has order ${p^{k_n}}$, ${g'_{n+1}}$ must have an order divisible by ${p^{k_n}}$, and thus of the form ${p^{k_{n+1}} q}$ for some ${q}$ coprime to ${p}$ and some ${k_{n+1} \geq k_n}$. By raising ${g'_{n+1}}$ to a multiple of ${q}$ that equals ${1}$ mod ${p^{k_n}}$, we may eliminate ${q}$ and obtain a preimage ${g_{n+1}}$ of order ${p^{k_{n+1}}}$, and the claim follows.

There are now two cases, depending on whether ${k_n}$ goes to infinity or not. If the ${k_n}$ stay bounded, then they converge to a limit ${k}$, and the inverse limit of the ${g_n}$ is then an element ${g}$ of ${G}$ of finite order ${p^k}$, a contradiction. But if the ${k_n}$ are unbounded, then ${G}$ contains a continuously embedded copy of the inverse limit of the ${{\bf Z}/p^{k_n}{\bf Z}}$, which is ${{\bf Z}_p}$, and again we have a contradiction. $\Box$

Since ${G'/G''}$ and ${G/G'}$ are both Lie groups, ${G/G''}$ is Lie too. Thus it suffices to show that ${G''}$ is Lie. Without loss of generality, we may therefore replace ${G}$ by ${G''}$ and assume that ${B}$ is fixed by ${G}$.

Now let ${\Sigma}$ be the closure of the interior of the set of fixed points of ${G}$. Then ${\Sigma}$ is non-empty, and is also clearly closed. We claim that ${\Sigma}$ is open; by connectedness of ${X}$, this implies that ${\Sigma=X}$, which by faithfulness of ${G}$ implies that ${G}$ is trivial, giving the claim. Indeed, let ${x \in \Sigma}$, and let ${B}$ be a small ball around ${x}$. As before, ${GB}$ is then a connected ${G}$-invariant ${\sigma}$-compact manifold, and so if ${G'}$ is the subgroup of ${G}$ that fixes ${GB}$, then ${G/G'}$ is a compact Lie group that acts faithfully on ${GB}$ fixing a nontrivial open subset of ${GB}$. By Newman’s theorem (Theorem 6), ${G/G'}$ cannot contain any periodic elements; by Conjecture 3, it also cannot contain any copy of ${{\bf Z}_p}$. By Lemma 7, ${G/G'}$ is trivial, and so ${G}$ fixes all of ${B}$. Thus gives the desired openness of ${\Sigma}$ as required.

Remark 1 Whereas actions of the finite group ${{\bf Z}/p{\bf Z}}$ on manifolds can be analysed by degree theory, it appears that actions of ${p}$-adic groups ${{\bf Z}_p}$ require more sophisticated homological tools; the ${p}$-adic analogues of the averaged map ${\Phi}$ now typically have infinite preimages and it is no longer obvious how to compute the degree of such maps. Nevertheless, some progress has been made along these lines under some additional regularity hypotheses on the action, such as Lipschitz continuity; see for instance this paper of Repovs and Scepin. Note that if ${T: X \rightarrow X}$ is the action of the generator of ${{\bf Z}_p}$, then the powers ${T^{p^n}: X \rightarrow X}$ will converge to the identity locally uniformly as ${n \rightarrow \infty}$. This is already enough to rule out generating maps ${T}$ that are smooth but non-trivial by use of Taylor expansion; see the text of Montgomery and Zippin. (In fact, this type of argument even works for ${C^1}$ actions.)