Sanov’s theorem in the Wasserstein distance: A necessary and sufficient condition

doi:10.1016/j.spl.2009.12.003

Statistics & Probability Letters

Volume 80, Issues 5–6, 1–15 March 2010, Pages 505-512

https://doi.org/10.1016/j.spl.2009.12.003 Get rights and content

Abstract

Let ${(X_{n})}_{n \geq 1}$ be a sequence of i.i.d.r.v.’s with values in a Polish space $(E, d)$ of law $μ$ . Consider the empirical measures $L_{n} = \frac{1}{n} \sum_{k = 1}^{n} δ_{X_{k}}, n \geq 1$ . Our purpose is to generalize Sanov’s theorem about the large deviation principle of $L_{n}$ from the weak convergence topology to the stronger Wasserstein metric $W_{p}$ . We show that $L_{n}$ satisfies the large deviation principle in the Wasserstein metric $W_{p}$ ( $p \in [1, + \infty)$ ) if and only if $\int_{E} e^{λ d^{p} (x_{0}, x)} d μ (x) < + \infty$ for all $λ > 0$ , and for some $x_{0} \in E$ .

Section snippets

Introduction and main results

Let $(E, d)$ be a complete metric and separable (say Polish) space equipped with its Borel $σ$ -field $E$ . We denote the set of all probability measures on $E$ by $M_{1} (E)$ .

Several lemmas

Lemma 2.1

Let Z be a real nonnegative random variable such that $Λ (λ) = log E e^{λ Z} < + \infty$ for all $λ > 0$ , and $Λ^{⊛}$ be semi-Legendre transform, $Λ^{⊛} (r) ≔ sup_{λ \geq 0} {r λ - log E (e^{Z λ})}$ for all $r \geq 0$ . Then $Λ^{⊛}$ is a nondecreasing lower semicontinuous convex function, and $\frac{Λ^{⊛} (r)}{r} ↗ + \infty a s r \to + \infty .$

Proof

Taking $λ = 0$ , we get $Λ^{⊛} (r) \geq 0$ . $Λ^{⊛} (r)$ is a nondecreasing lower semicontinuous convex function because it is the supremum of a class of nondecreasing continuous affine functions. For any fixed numbers $λ > 0$ , and $r > 0$ , $Λ^{⊛} (r) \geq r λ - Λ (λ), \frac{Λ^{⊛} (r)}{r} \geq λ - \frac{Λ (λ)}{r} .$ As $Λ (λ) < + \infty$ for

Another proof of the necessity in Theorem 1.1 for $p = 1$

When $p = 1$ , we have by Kantorovich–Rubinstein’s theorem (see Villani, 2003, Theorem 1.14) $W_{1} (μ, ν) = sup {\int_{E} φ d (ν - μ); φ : E \to R, {‖ φ ‖}_{L i p} \leq 1},$ where ${‖ φ ‖}_{L i p} = {sup}_{x \neq y} \frac{| φ (x) - φ (y) |}{d (x, y)}$ is the Lipschitzian seminorm. This result identifies the LDP estimation of $L_{n}$ on $(M_{1}^{1} (E), W_{1})$ as that of $L_{n} (f)$ over the class of all Lipschitz functions on $E$ , with Lipschitz constant not more than 1.

More precisely, given $ℱ ≔ {φ; φ : E \to R, {‖ φ ‖}_{L i p} \leq 1, φ (x_{0}) = 0} for some fixed x_{0} \in E,$ let $l_{\infty} (ℱ)$ be the space of all bounded real functions on $ℱ$ with

Two examples

Let us give two statistical applications.

Example 4.1

Let ${(X_{n})}_{n \geq 1}$ be real valued i.i.d.r.v.’s with common distribution function $F (μ (d x) = d F (x))$ such that $\int_{R} | x | μ (d x) < + \infty$ . Let $F_{n}^{*} (x) = \frac{1}{n} \sum_{k = 1}^{n} I_{(- \infty, x]} (X_{k}), n \geq 1$ be the empirical distribution functions. Since $W_{1} (ν_{1}, ν_{2}) = \int_{R} | F_{ν_{1}} (x) - F_{ν_{2}} (x) | d x, ν_{1}, ν_{2} \in M_{1} (R)$ (see Villani, 2003, Page 75), where $F_{ν} (x) ≔ ν (- \infty, x]$ is the distribution function of $ν$ , then $ν \to F_{ν} (x) - F (x)$ is isometric from $(M_{1}^{1} (R), W_{1})$ to $(L^{1} (R, d x), {‖ \cdot ‖}_{1})$ . By Theorem 1.1, $P (F_{n}^{*} - F \in \cdot)$ satisfies the LDP on $L^{1} (R, d x)$ if and only

Acknowledgements

The authors are very grateful to the editor and associated editor for their comments. We are particularly indebted to the referee for his attentive corrections and helpful comments. The first named author also thanks Professor F.Q. Gao, Z.L. Zhang and N. Yao for the useful discussions.

References (13)

P. Eichelsbacher et al.
Exponential approximations in completely regular topological spaces and extensions of Sanov’s theorem
Stochastic Process. Appl.
(1998)
M.A. Arcones
Large deviations of empirical processes
F. Bolley et al.
Quantitative concentration inequalities for empirical measures on non compact spaces
Probab. Theory Related Fields
(2007)
M.F. Chen
From Markov Chains to Non-equilibrium Particle Systems
(2004)
A. Dembo et al.
P. Eichelsbacher et al.
Large deviations for partial sums U-processes
Theory Probab. Appl.
(1998)

There are more references available in the full text version of this article.

Cited by (39)

An asymptotic thin shell condition and large deviations for random multidimensional projections
2022, Advances in Applied Mathematics
It is well known that fluctuations of marginals of high-dimensional random vectors that satisfy a certain concentration estimate called the thin shell condition are approximately Gaussian. In this article we identify a general condition on a sequence of high-dimensional random vectors under which one can identify the exponential decay rate of large deviation probabilities of the corresponding sequence of marginals. More precisely, consider the projection of an n-dimensional random vector onto a random $k_{n}$ -dimensional basis, $k_{n} \leq n$ , drawn uniformly from the Haar measure on the Stiefel manifold of orthonormal $k_{n}$ -frames in $R^{n}$ , in three different asymptotic regimes as $n \to \infty$ : “constant” ( $k_{n} = k$ ), “sublinear” ( $k_{n} \to \infty$ but $k_{n} / n \to 0$ ) and “linear” ( $k_{n} / n \to λ$ with $0 < λ \leq 1$ ). When the sequence of random vectors satisfies a certain “asymptotic thin shell condition”, we establish large deviation principles for the corresponding sequence of random projections in the constant regime, and for the sequence of empirical measures of the coordinates of the random projections in the sublinear and linear regimes. We also establish large deviation principles for scaled $ℓ_{q}$ norms of the random projections in all three regimes. Moreover, we show that the asymptotic thin shell condition holds for various sequences of random vectors of interest, including the uniform measure on suitably scaled $ℓ_{p}^{n}$ balls, for $p \in [1, \infty)$ , and generalized Orlicz balls defined via a superquadratic function, as well as a class of Gibbs measures with superquadratic interaction potential. Along the way, we obtain logarithmic asymptotics of volumes of high-dimensional Orlicz balls, which may be of independent interest. We also show that the decay rate of large deviation probabilities of Euclidean norms of multi-dimensional projections of $ℓ_{p}^{n}$ balls, when $p \in [1, 2)$ , exhibits an unexpected phase transition in the sublinear regime, thus disproving an earlier conjecture due to Alonso-Gutiérrez et al. Random projections of high-dimensional random vectors are of interest in a range of fields including asymptotic convex geometry and high-dimensional statistics.
Large deviations for empirical measures of mean-field Gibbs measures
2020, Stochastic Processes and their Applications
Citation Excerpt :
We first present the result of Sanov’s theorem in the Wasserstein distance by Wang et al. [21].
In this paper, we show that the empirical measure of mean-field model satisfies the large deviation principle with respect to the weak convergence topology or the stronger Wasserstein metric, under the strong exponential integrability condition on the negative part of the interaction potentials. In contrast to the known results we prove this without any continuity or boundedness condition on the interaction potentials. The proof relies mainly on the law of large numbers and the exponential decoupling inequality of de la Peña for $U$ -statistics.
The enhanced Sanov theorem and propagation of chaos
2018, Stochastic Processes and their Applications
We establish a Sanov type large deviation principle for an ensemble of interacting Brownian rough paths. As application a large deviations for the ( $k$ -layer, enhanced) empirical measure of weakly interacting diffusions is obtained. This in turn implies a propagation of chaos result in a space of rough paths and allows for a robust analysis of the particle system and its McKean–Vlasov type limit, as shown in two corollaries.
A non-exponential extension of Sanov's theorem via convex duality
2020, Advances in Applied Probability
A conditional limit theorem for high-dimensional ℓ<sup>p</sup>-spheres
2018, Journal of Applied Probability
Limit theorems for Fréchet mean sets
2024, Bernoulli

View all citing articles on Scopus

View full text

Sanov’s theorem in the Wasserstein distance: A necessary and sufficient condition

Abstract

Section snippets

Introduction and main results

Several lemmas

Another proof of the necessity in Theorem 1.1 for p=1

Two examples

Acknowledgements

Stochastic Process. Appl.

Large deviations of empirical processes

Quantitative concentration inequalities for empirical measures on non compact spaces

Probab. Theory Related Fields

From Markov Chains to Non-equilibrium Particle Systems

Large deviations for partial sums U-processes

Theory Probab. Appl.

Another proof of the necessity in Theorem 1.1 for $p = 1$