In mathematics, the Selberg trace formula, introduced by Selberg (1956), is an expression for the character of the unitary representation of a Lie group G on the space L²(Γ\G) of square-integrable functions, where Γ is a cofinite discrete group. The character is given by the trace of certain functions on G.

The simplest case is when Γ is cocompact, when the representation breaks up into discrete summands. Here the trace formula is an extension of the Frobenius formula for the character of an induced representation of finite groups. When Γ is the cocompact subgroup Z of the real numbers G = R, the Selberg trace formula is essentially the Poisson summation formula.

The case when Γ\G is not compact is harder, because there is a continuous spectrum, described using Eisenstein series. Selberg worked out the non-compact case when G is the group SL(2, R); the extension to higher rank groups is the Arthur–Selberg trace formula.

When Γ is the fundamental group of a Riemann surface, the Selberg trace formula describes the spectrum of differential operators such as the Laplacian in terms of geometric data involving the lengths of geodesics on the Riemann surface. In this case the Selberg trace formula is formally similar to the explicit formulas relating the zeros of the Riemann zeta function to prime numbers, with the zeta zeros corresponding to eigenvalues of the Laplacian, and the primes corresponding to geodesics. Motivated by the analogy, Selberg introduced the Selberg zeta function of a Riemann surface, whose analytic properties are encoded by the Selberg trace formula.

Cases of particular interest include those for which the space is a compact Riemann surface S. The initial publication in 1956 of Atle Selberg dealt with this case, its Laplacian differential operator and its powers. The traces of powers of a Laplacian can be used to define the Selberg zeta function. The interest of this case was the analogy between the formula obtained, and the explicit formulae of prime number theory. Here the closed geodesics on S play the role of prime numbers.

At the same time, interest in the traces of Hecke operators was linked to the Eichler–Selberg trace formula, of Selberg and Martin Eichler, for a Hecke operator acting on a vector space of cusp forms of a given weight, for a given congruence subgroup of the modular group. Here the trace of the identity operator is the dimension of the vector space, i.e. the dimension of the space of modular forms of a given type: a quantity traditionally calculated by means of the Riemann–Roch theorem.

The trace formula has applications to arithmetic geometry and number theory. For instance, using the trace theorem, Eichler and Shimura calculated the Hasse–Weil L-functions associated to modular curves; Goro Shimura's methods by-passed the analysis involved in the trace formula. The development of parabolic cohomology (from Eichler cohomology) provided a purely algebraic setting based on group cohomology, taking account of the cusps characteristic of non-compact Riemann surfaces and modular curves.

The trace formula also has purely differential-geometric applications. For instance, by a result of Buser, the length spectrum of a Riemann surface is an isospectral invariant, essentially by the trace formula.

A compact hyperbolic surface X can be written as the space of orbits $${\displaystyle \Gamma \backslash \mathbf {H} ,}$$ where Γ is a subgroup of PSL(2, R), and H is the upper half plane, and Γ acts on H by linear fractional transformations.

The Selberg trace formula for this case is easier than the general case because the surface is compact so there is no continuous spectrum, and the group Γ has no parabolic or elliptic elements (other than the identity).

Then the spectrum for the Laplace–Beltrami operator on X is discrete and real, since the Laplace operator is self adjoint with compact resolvent; that is $${\displaystyle 0=\mu _{0}<\mu _{1}\leq \mu _{2}\leq \cdots }$$ where the eigenvalues μ_n correspond to Γ-invariant eigenfunctions u in C^∞(H) of the Laplacian; in other words $${\displaystyle {\begin{cases}u(\gamma z)=u(z),\qquad \forall \gamma \in \Gamma \\y^{2}\left(u_{xx}+u_{yy}\right)+\mu _{n}u=0.\end{cases}}}$$

Using the variable substitution $${\displaystyle \mu =s(1-s),\qquad s={\tfrac {1}{2}}+ir}$$ the eigenvalues are labeled $${\displaystyle r_{n},n\geq 0.}$$

Then the Selberg trace formula is given by $${\displaystyle \sum _{n=0}^{\infty }h(r_{n})={\frac {\mu (X)}{4\pi }}\int _{-\infty }^{\infty }r\,h(r)\tanh(\pi r)\,dr+\sum _{\{T\}}{\frac {\log N(T_{0})}{N(T)^{\frac {1}{2}}-N(T)^{-{\frac {1}{2}}}}}g(\log N(T)).}$$

The right hand side is a sum over conjugacy classes of the group Γ, with the first term corresponding to the identity element and the remaining terms forming a sum over the other conjugacy classes {T } (which are all hyperbolic in this case). The function h has to satisfy the following:

• be analytic on |Im(r)| ≤ ⁠1/2⁠ + δ;
• h(−r) = h(r);
• there exist positive constants δ and M such that: $${\displaystyle \vert h(r)\vert \leq M\left(1+\left|\operatorname {Re} (r)\right|\right)^{-2-\delta }.}$$

The function g is the Fourier transform of h, that is, $${\displaystyle h(r)=\int _{-\infty }^{\infty }g(u)e^{iru}\,du.}$$

Let G be a unimodular locally compact group, and ${\displaystyle \Gamma }$ a discrete cocompact subgroup of G and ${\displaystyle \phi }$ a compactly supported continuous function on G. The trace formula in this setting is the following equality: $${\displaystyle \sum _{\gamma \in \{\Gamma \}}a_{\Gamma }^{G}(\gamma )\int _{G^{\gamma }\setminus G}\phi (x^{-1}\gamma x)\,dx=\sum _{\pi \in {\widehat {G}}}a_{\Gamma }^{G}(\pi )\operatorname {tr} \pi (\phi )}$$ where ${\displaystyle \{\Gamma \}}$ is the set of conjugacy classes in ${\displaystyle \Gamma }$, ${\displaystyle {\widehat {G}}}$ is the unitary dual of G and:

• for an element ${\displaystyle \gamma \in \Gamma }$, ${\displaystyle a_{\Gamma }^{G}(\gamma )={\text{volume}}(\Gamma ^{\gamma }\setminus G^{\gamma }).}$ with ${\displaystyle G_{\gamma },\Gamma _{\gamma }}$ the centralisers of ${\displaystyle \gamma }$ in ${\displaystyle G,\Gamma }$ respectively;
• for an irreducible unitary representation ${\displaystyle \pi }$ of ${\displaystyle G}$, ${\displaystyle a_{\Gamma }^{G}(\pi )}$ is the multiplicity of ${\displaystyle \pi }$ in the right-representation on ${\displaystyle \Gamma \backslash G}$ in ${\displaystyle L^{2}(\Gamma \backslash G}$), and ${\displaystyle \pi (\phi )}$ is the operator ${\displaystyle \int _{G}\phi (g)\pi (g)dg}$;
• all integrals and volumes are taken with respect to the Haar measure on ${\displaystyle G}$ or its quotients.

The left-hand side of the formula is called the geometric side and the right-hand side the spectral side. The terms ${\displaystyle \int _{G^{\gamma }\setminus G}\phi (x^{-1}\gamma x)\,dx}$ are orbital integrals.

Define the following operator on compactly supported functions on ${\displaystyle \Gamma \backslash G}$: $${\displaystyle R(\phi )=\int _{G}\phi (x)R(x)\,dx,}$$ It extends continuously to ${\displaystyle L^{2}(\Gamma \setminus G)}$ and for ${\displaystyle f\in L^{2}(\Gamma \setminus G)}$ we have: $${\displaystyle (R(\phi )f)(x)=\int _{G}\phi (y)f(xy)\,dy=\int _{\Gamma \setminus G}\left(\sum _{\gamma \in \Gamma }\phi (x^{-1}\gamma y)\right)f(y)\,dy}$$ after a change of variables. Assuming ${\displaystyle \Gamma \setminus G}$ is compact, the operator ${\displaystyle R(\phi )}$ is trace-class and the trace formula is the result of computing its trace in two ways as explained below.^[1]

The trace of ${\displaystyle R(\phi )}$can be expressed as the integral of the kernel ${\displaystyle K(x,y)=\sum _{\gamma \in \Gamma }\phi (x^{-1}\gamma y)}$ along the diagonal, that is: $${\displaystyle \operatorname {tr} R(\phi )=\int _{\Gamma \setminus G}\sum _{\gamma \in \Gamma }\phi (x^{-1}\gamma x)\,dx.}$$ Let ${\displaystyle \{\Gamma \}}$ denote a collection of representatives of conjugacy classes in ${\displaystyle \Gamma }$, and ${\displaystyle \Gamma ^{\gamma }}$ and ${\displaystyle G^{\gamma }}$ the respective centralizers of ${\displaystyle \gamma }$. Then the above integral can, after manipulation, be written $${\displaystyle \operatorname {tr} R(\phi )=\sum _{\gamma \in \{\Gamma \}}a_{\Gamma }^{G}(\gamma )\int _{G^{\gamma }\setminus G}\phi (x^{-1}\gamma x)\,dx.}$$ This gives the geometric side of the trace formula.

The spectral side of the trace formula comes from computing the trace of ${\displaystyle R(\phi )}$ using the decomposition of the regular representation of ${\displaystyle G}$ into its irreducible components. Thus $${\displaystyle \operatorname {tr} R(\phi )=\sum _{\pi \in {\hat {G}}}a_{\Gamma }^{G}(\pi )\operatorname {tr} \pi (\phi )}$$ where ${\displaystyle {\hat {G}}}$ is the set of irreducible unitary representations of ${\displaystyle G}$ (recall that the positive integer ${\displaystyle a_{\Gamma }^{G}(\pi )}$ is the multiplicity of ${\displaystyle \pi }$ in the unitary representation ${\displaystyle R}$ on ${\displaystyle L^{2}(\Gamma \setminus G)}$).

When ${\displaystyle G}$ is a semisimple Lie group with a maximal compact subgroup ${\displaystyle K}$ and ${\displaystyle X=G/K}$ is the associated symmetric space the conjugacy classes in ${\displaystyle \Gamma }$ can be described in geometric terms using the compact Riemannian manifold (more generally orbifold) ${\displaystyle \Gamma \backslash X}$. The orbital integrals and the traces in irreducible summands can then be computed further and in particular one can recover the case of the trace formula for hyperbolic surfaces in this way.

The general theory of Eisenstein series was largely motivated by the requirement to separate out the continuous spectrum, which is characteristic of the non-compact case.^[2]

The trace formula is often given for algebraic groups over the adeles rather than for Lie groups, because this makes the corresponding discrete subgroup Γ into an algebraic group over a field which is technically easier to work with. The case of SL₂(C) is discussed in Gel'fand, Graev & Pyatetskii-Shapiro (1990) and Elstrodt, Grunewald & Mennicke (1998). Gel'fand et al also treat SL₂(F) where F is a locally compact topological field with ultrametric norm, so a finite extension of the p-adic numbers Q_p or of the formal Laurent series F_q((T)); they also handle the adelic case in characteristic 0, combining all completions R and Q_p of the rational numbers Q.

Contemporary successors of the theory are the Arthur–Selberg trace formula applying to the case of general semisimple G, and the many studies of the trace formula in the Langlands philosophy (dealing with technical issues such as endoscopy). The Selberg trace formula can be derived from the Arthur–Selberg trace formula with some effort.

• Jacquet–Langlands correspondence

• Borel, Armand (1997). Automorphic forms on SL₂(R). Cambridge Tracts in Mathematics. Vol. 130. Cambridge University Press. ISBN 0-521-58049-8. MR 1482800.
• Chavel, Isaac; Randol, Burton (1984). "XI. The Selberg Trace Formula". Eigenvalues in Riemannian geometry. Pure and Applied Mathematics. Vol. 115. Academic Press. ISBN 0-12-170640-0. MR 0768584.
• Elstrodt, Jürgen (1981). "Die Selbergsche Spurformel für kompakte Riemannsche Flächen" (PDF). Jahresber. Deutsch. Math.-Verein. (in German). 83. Bielefeld: Deutsche Mathematiker-Vereinigung: 45–77. MR 0612411.
• Elstrodt, J.; Grunewald, F.; Mennicke, J. (1998). Groups acting on hyperbolic space: Harmonic analysis and number theory. Springer Monographs in Mathematics. Springer-Verlag. ISBN 3-540-62745-6. MR 1483315.
• Fischer, Jürgen (1987), An approach to the Selberg trace formula via the Selberg zeta-function, Lecture Notes in Mathematics, vol. 1253, Berlin, New York: Springer-Verlag, doi:10.1007/BFb0077696, ISBN 978-3-540-15208-8, MR 0892317
• Gel'fand, I. M.; Graev, M. I.; Pyatetskii-Shapiro, I. I. (1990), Representation theory and automorphic functions, Generalized Functions, vol. 6, Boston, MA: Academic Press, ISBN 978-0-12-279506-0, MR 1071179
• Godement, Roger (1966). "The decomposition of L²(G/Γ) for Γ=SL(2,Z)". Algebraic Groups and Discontinuous Subgroups. Proc. Sympos. Pure Math. Providence: American Mathematical Society. pp. 211–224. MR 0210827.
• Hejhal, Dennis A. (1976), "The Selberg trace formula and the Riemann zeta function", Duke Mathematical Journal, 43 (3): 441–482, doi:10.1215/S0012-7094-76-04338-6, ISSN 0012-7094, MR 0414490
• Hejhal, Dennis A. (1976), The Selberg trace formula for PSL(2,R). Vol. I, Lecture Notes in Mathematics, Vol. 548, vol. 548, Berlin, New York: Springer-Verlag, doi:10.1007/BFb0079608, ISBN 978-3-540-07988-0, MR 0439755
• Hejhal, Dennis A. (1983), The Selberg trace formula for PSL(2,R). Vol. 2, Lecture Notes in Mathematics, vol. 1001, Berlin, New York: Springer-Verlag, doi:10.1007/BFb0061302, ISBN 978-3-540-12323-1, MR 0711197
• Iwaniec, Henryk (2002). Spectral methods of automorphic forms. Graduate Studies in Mathematics. Vol. 53 (Second ed.). American Mathematical Society. ISBN 0-8218-3160-7. MR 1942691.
• Lax, Peter D.; Phillips, Ralph S. (1980). "Scattering theory for automorphic functions" (PDF). Bull. Amer. Math. Soc. 2: 261–295. MR 0555264.
• McKean, H. P. (1972), "Selberg's trace formula as applied to a compact Riemann surface", Communications on Pure and Applied Mathematics, 25 (3): 225–246, doi:10.1002/cpa.3160250302, ISSN 0010-3640, MR 0473166
• Selberg, Atle (1956), "Harmonic analysis and discontinuous groups in weakly symmetric Riemannian spaces with applications to Dirichlet series", J. Indian Math. Soc., New Series, 20: 47–87, MR 0088511
• Sunada, Toshikazu (1991), Trace formulae in spectral geometry, Proc. ICM-90 Kyoto, Springer-Verlag, pp. 577–585

• Selberg trace formula resource page