Remote Access Journal of the American Mathematical Society
Green Open Access

Journal of the American Mathematical Society

ISSN 1088-6834(online) ISSN 0894-0347(print)



Solution of Shannon's problem on the monotonicity of entropy

Authors: Shiri Artstein, Keith M. Ball, Franck Barthe and Assaf Naor
Journal: J. Amer. Math. Soc. 17 (2004), 975-982
MSC (2000): Primary 94A17
Published electronically: May 12, 2004
MathSciNet review: 2083473
Full-text PDF Free Access

Abstract | References | Similar Articles | Additional Information

Abstract: It is shown that if $X_1,X_2,\ldots$ are independent and identically distributed square-integrable random variables, then the entropy of the normalized sum

\begin{displaymath}\mathrm{Ent}\left(\frac {X_{1}+\cdots + X_{n}}{\sqrt {n}} \right) \end{displaymath}

is an increasing function of $n$.

The result also has a version for non-identically distributed random variables or random vectors.

References [Enhancements On Off] (What's this?)

  • 1. D. Bakry and Michel Émery, Diffusions hypercontractives, Séminaire de probabilités, XIX, 1983/84, Lecture Notes in Math., vol. 1123, Springer, Berlin, 1985, pp. 177–206 (French). MR 889476,
  • 2. K. Ball, F. Barthe, and A. Naor.
    Entropy jumps in the presence of a spectral gap.
    Duke Math. J., 119(1):41-63, 2003.
  • 3. Andrew R. Barron, Entropy and the central limit theorem, Ann. Probab. 14 (1986), no. 1, 336–342. MR 815975
  • 4. E. A. Carlen and A. Soffer, Entropy production by block variable summation and central limit theorems, Comm. Math. Phys. 140 (1991), no. 2, 339–371. MR 1124273
  • 5. Elliott H. Lieb, Proof of an entropy conjecture of Wehrl, Comm. Math. Phys. 62 (1978), no. 1, 35–41. MR 506364
  • 6. C. E. Shannon and W. Weaver.
    The mathematical theory of communication.
    University of Illinois Press, Urbana, IL, 1949. MR 11:258e
  • 7. A. J. Stam, Some inequalities satisfied by the quantities of information of Fisher and Shannon, Information and Control 2 (1959), 101–112. MR 0109101

Similar Articles

Retrieve articles in Journal of the American Mathematical Society with MSC (2000): 94A17

Retrieve articles in all journals with MSC (2000): 94A17

Additional Information

Shiri Artstein
Affiliation: School of Mathematical Sciences, Tel Aviv University, Ramat Aviv, Tel Aviv 69978, Israel

Keith M. Ball
Affiliation: Department of Mathematics, University College London, Gower Street, London WC1 6BT, United Kingdom

Franck Barthe
Affiliation: Institut de Mathématiques, Laboratoire de Statistique et Probabilités, CNRS UMR C5583, Université Paul Sabatier, 31062 Toulouse Cedex 4, France

Assaf Naor
Affiliation: Theory Group, Microsoft Research, One Microsoft Way, Redmond, Washington 98052-6399

Keywords: Entropy growth, Fisher information, central limit theorem
Received by editor(s): September 4, 2003
Published electronically: May 12, 2004
Additional Notes: The first author was supported in part by the EU Grant HPMT-CT-2000-00037, The Minkowski Center for Geometry and the Israel Science Foundation
The second author was supported in part by NSF Grant DMS-9796221
The third author was supported in part by EPSRC Grant GR/R37210
The last author was supported in part by the BSF, Clore Foundation and EU Grant HPMT-CT-2000-00037
Article copyright: © Copyright 2004 American Mathematical Society