Skip to main content Accessibility help

Logarithmic Sobolev inequalities in discrete product spaces

  • Katalin Marton (a1)


The aim of this paper is to prove an inequality between relative entropy and the sum of average conditional relative entropies of the following form: for a fixed probability measure q on , ( is a finite set), and any probability measure on , (*)

$$D(p||q){\rm{\le}}C \cdot \sum\limits_{i = 1}^n {{\rm{\mathbb{E}}}_p D(p_i ( \cdot |Y_1 ,{\rm{ }}...,{\rm{ }}Y_{i - 1} ,{\rm{ }}Y_{i + 1} ,...,{\rm{ }}Y_n )||q_i ( \cdot |Y_1 ,{\rm{ }}...,{\rm{ }}Y_{i - 1} ,{\rm{ }}Y_{i + 1} ,{\rm{ }}...,{\rm{ }}Y_n )),} $$
where pi(· |y1, …, yi−1, yi+1, …, yn) and qi(· |x1, …, xi−1, xi+1, …, xn) denote the local specifications for p resp. q, that is, the conditional distributions of the ith coordinate, given the other coordinates. The constant C depends on (the local specifications of) q.

The inequality (*) ismeaningful in product spaces, in both the discrete and the continuous case, and can be used to prove a logarithmic Sobolev inequality for q, provided uniform logarithmic Sobolev inequalities are available for qi(· |x1, …, xi−1, xi+1, …, xn), for all fixed i and fixed (x1, …, xi−1, xi+1, …, xn). Inequality (*) directly implies that the Gibbs sampler associated with q is a contraction for relative entropy.

In this paper we derive inequality (*), and thereby a logarithmic Sobolev inequality, in discrete product spaces, by proving inequalities for an appropriate Wasserstein-like distance.


Corresponding author


Hide All

This work was supported by grant OTKA K 105840 of the Hungarian Academy of Sciences and by National Research, Development and Innovation Office NKFIH K 120706.



Hide All
[1]Boucheron, S., Lugosi, G. and Massart, P. (2013) Concentration Inequalities, Oxford University Press.
[2]Caputo, P., Menz, G. and Tetali, P. (2015) Approximate tensorization of entropy at high temperature. Ann Fac Sci Toulouse Math Sér 6 24 691716.
[3]Cesi, F. (2001) Quasi-factorization of the entropy and logarithmic Sobolev inequalities for Gibbs random fields. Probab. Theory Rel. Fields 120 569584.
[4]Dobrushin, R. L. (1968) The description of a random field by means of conditional probabilities and condition of its regularity (in Russian). Theory Probab. Appl. 13 197224.
[5]Dobrushin, R. L. (1970) Prescribing a system of random variables by conditional distributions. Theory Probab. Appl. 15 458486.
[6]Dobrushin, R. L. and Shlosman, S. B. (1985) Constructive criterion for the uniqueness of Gibbs field. In Statistical Physics and Dynamical Systems (Fritz, J., Jaffe, A. and Szász, D., eds), Springer, pp. 371403.
[7]Dobrushin, R. L. and Shlosman, S. B. (1987) Completely analytical interactions: Constructive description. J. Statist. Phys. 46 9831014.
[8]Diaconis, P. and Saloff-Coste, L. (1996) Logarithmic Sobolev inequalities for finite Markov chains. Ann. Appl. Probab. 6 695750.
[9]Gibbs, A. L. and Su, F. E. (2002) On choosing and bounding probability metrics. Internat. Statist. Rev. 70 419435.
[10]Goldstein, S. (1979) Maximal coupling. Z. Wahrscheinlichkeitstheor. verw. Geb. 46, 193204.
[11]Gross, L. (1975) Logarithmic Sobolev inequalities. Amer. J. Math. 97 10611083.
[12]Ledoux, M. (1999) Concentration of measure and logarithmic Sobolev inequalities. In Séminaire de Probabilités XXXIII, Vol. 1709 of Lecture Notes in Mathematics, Springer, pp. 120216.
[13]Martinelli, F. and Olivieri, E. (1994) Approach to equilibrium of Glauber dynamics in the one phase region, I: The attractive case. Commun. Math. Phys. 161 447486.
[14]Martinelli, F. and Olivieri, E. (1994) Approach to equilibrium of Glauber dynamics in the one phase region, II: The general case. Commun. Math. Phys. 161 487514.
[15]Marton, K. (2013) An inequality for relative entropy and logarithmic Sobolev inequalities in Euclidean spaces. J. Funct. Anal. 264 3461.
[16]Olivieri, E. (1988) On a cluster expansion for lattice spin systems: A finite size condition for the convergence. J. Statist. Phys. 50 11791200.
[17]Olivieri, E. and Picco, P. (1990) Clustering for D-dimensional lattice systems and finite volume factorization properties. J. Statist. Phys. 59 221256.
[18]Otto, F. and Reznikoff, M. (2011) A new criterion for the logarithmic Sobolev inequality and two applications. J. Funct. Anal. 243 121157.
[19]Royer, G. (1999) Une Initiation aux Inegalités de Sobolev Logarithmiques, Société Mathématique de France.
[20]Sason, I. (2015) Tight bounds for symmetric divergence measures and a refined bound for lossless source coding. IEEE Trans. Inform. Theory 61 701707.
[21]Sason, I. (2015) On reverse Pinsker inequalities. arXiv:1503.07118v4
[22]Stroock, D. W. and Zegarlinski, B. (1992) The equivalence of the logarithmic Sobolev inequality and the Dobrushin– Shlosman mixing condition. Commun. Math. Phys. 144 303323.
[23]Stroock, D. W. and Zegarlinski, B. (1992) The logarithmic Sobolev inequality for discrete spin systems on the lattice. Comm. Math. Phys. 149 175193.
[24]Zegarlinski, B. (1992) Dobrushin uniqueness theorem and logarithmic Sobolev inequalities. J. Funct. Anal. 105 77111.

MSC classification

Logarithmic Sobolev inequalities in discrete product spaces

  • Katalin Marton (a1)


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed