Skip to main content Accessibility help
  • Print publication year: 2014
  • Online publication date: December 2013

7 - Detection/decoding based on message passing on graphical models


Probability theory and graph theory are two branches of mathematics that are widely applicable in many different domains. Graphical models combine concepts from both these branches to provide a structured framework that supports representation, inference, and learning for a broad spectrum of problems [1]. Graphical models are graphs that indicate inter-dependencies between random variables [2],[3]. Distributions that exhibit some structure can generally be represented naturally and compactly using a graphical model, even when the explicit representation of the joint distribution is very large. The structure often allows the distribution to be used effectively for inference, i.e., answering certain queries of interest using the distribution. The framework also facilitates construction of these models by learning from data.

In this chapter we consider the use of graphical models in: (1) the representation of distributions of interest in MIMO systems, (2) formulation of the MIMO detection problem as an inference problem on such models (e.g., computation of posterior probability of variables of interest), and (3) efficient algorithms for inference (e.g., low complexity algorithms for computing the posterior probabilities). Three basic graphical models that are widely used to represent distributions include Bayesian belief networks [4], Markov random fields (MRFs) [5], and factor graphs [6]. Message passing algorithms like the BP algorithm [7] are efficient tools for inference on graphical models. In this chapter, a brief survey of various graphical models and BP techniques is presented.

[1] D., Koller and N., Friedman, Probabilistic Graphical Models: Principles and Techniques. Cambridge, MA: The MIT Press, 2009.
[2] B. J., Frey, Graphical Models for Machine Learning and Digital Communication. Cambridge, MA: MIT Press, 1998.
[3] J. S., Yedidia, W. T., Freeman, and Y., Weiss, “Understand ing belief propagation and its generalizations,” in Exploring Artificial Intelligence in the New Millennium, G., Lakemeyer and B., Nebel, Eds. San Mateo, CA: Morgan Kaufmann, 2002, ch. 8.
[4] D., Heckerman and M. P., Wellman, “Bayesian networks,” Commun. ACM, vol. 38, pp. 27–30, 1990.
[5] D., Griffeath, Introduction to Markov Rand om Fields, in Denumerable Markov, Chains, J. G., Kerney, J. L., Snell and A. W., Knupp, Eds., second edition. New York: Springer-Verlag, 1976, pp. 425–485.
[6] F. R., Kschischang, B. J., Frey, and H.-A., Loeliger, “Factor graphs and the sum-product algorithm,” IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 498–519, Feb. 2001.
[7] J., Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
[8] S. M., Aji and R. J., McEliece, “The generalized distributive law,” IEEE Trans. Inform. Theory, vol. 46, no. 2, pp. 325–343, Mar. 2000.
[9] R. J., McEliece, D. J. C., MacKay, and J.-F., Cheng, “Turbo decoding as an instance of Pearl's 'belief propagation' algorithm,” IEEE J. Sel. Areas Commun., vol. 16, no. 2, pp. 140–152, Feb. 1998.
[10] N., Wiberg, “Codes and decoding on general graphs,” in Ph.D. dissertation, Linkoping University, 1996.
[11] D. J. C., MacKay and R., Neal, “Good codes based on very sparse matrices,” in 5th IMA Conf. Cryptography and Coding, vol. 1025. Berlin, Germany: Springer Lecture Notes in Computer Science. Berlin: Springer, 1995, pp. 100–111.
[12] R. G., Gallager, “Low density parity check codes,” IRE Trans. Inform. Theory, vol. IT-8, no. 2, pp. 21–28, Jan. 1962.
[13] R. M., Tanner, “A recursive approach to low complexity codes,” IEEE Trans. Inform. Theory, vol. IT-27, no. 5, pp. 533–547, Sep. 1981.
[14] G., Berrou, A., Glavieux, and P., Thitimajshima, “Near shannon limit error-correcting coding: Turbo codes,” in IEEE ICC'1993, Geneva, May 1993, pp. 1064–1070.
[15] D. J. C., MacKay, “Good error-correcting codes based on very sparse matrices,” IEEE Trans. Inform. Theory, vol. 45, no. 2, pp. 399–431, Feb. 1999.
[16] Y., Kabashima, “A CDMA multiuser detection algorithm on the basis of belief propagation,” J. Phys. A: Math. General, vol. 36, pp. 11111–11121, Oct. 2003.
[17] T., Tanaka and M., Okada, “Approximate belief propagation, density evolution, and statistical neurodynamics for CDMA multiuser detection,” IEEE Trans. Inform. Theory, vol. 51, no. 2, pp. 700–706, Feb. 2005.
[18] A., Montanari, B., Prabhakar, and D., Tse, “Belief propagation based multiuser detection,” arXiv:cs/0510044v2 [cs.IT], 22 May 2006.
[19] D., Bickson, D., Dolev, O., Shental, P. H., Siegel, and J. K., Wolf, “Gaussian belief propagation based multiuser detection,” in IEEE ISIT'2008, Toronto, Jul. 2008, pp. 1878–1882.
[20] D., Guo and C.-C., Wang, “Multiuser detection of sparsely spread CDMA,” IEEE J. Sel. Areas. Commun., vol. 26, no. 3, pp. 421–431, Apr. 2008.
[21] H., Wymeersch, Iterative Receiver Design. Cambridge, UK: Cambridge University Press, 2007.
[22] M., Tutchler, R., Koetter, and A. C., Singer, “Graphical models for coded data transmission over inter-symbol interference channels,” Eur. Trans. Telecomm., vol. 5, no. 4, pp. 307–321, Jul./Aug. 2005.
[23] O., Shental, A. J., Weiss, N., Shental, and Y., Weiss, “Generalized belief propagation receiver for near-optimal detection of two-dimensional channels with memory,” in IEEE ITW'2004, San Antonio, TX, Oct. 2004, pp. 225–229.
[24] G., Colavolpe and G., Germi, “On the application of factor graphs and the sum-product algorithm to ISI channels,” IEEE Trans. Commun., vol. 53, no. 5, pp. 818–825, May 2005.
[25] R. J., Drost and A. C., Singer, “Factor graph algorithms for equalization,” IEEE Trans. Signal Process., vol. 55, no. 5, pp. 2052–2065, May 2007.
[26] M. N., Kaynak, T. M., Duman, and E. M., Kurtas, “Belief propagation over MIMO frequency selective fading channels,” in Joint Intl. Conf. on Autonomic and Autonomous Systems and Int. Conf. on Networking and Services, Papeeti, Oct. 2005.
[27] J., Soler-Garrido, R. J., Piechocki, K., Maharatna, and D., McNamara, “Analog MIMO detection on the basis of belief propagation,” in IEEE Mid-West Symposium on Circuits and Systems, San Juan, Aug. 2006, pp. 50–54.
[28] X., Yang, Y., Xiong, and F., Wang, “An adaptive MIMO system based on unified belief propagation detection,” in IEEE ICC'2007, Jun. 2007, pp. 4156–4161.
[29] T., Wo and P. A., Hoeher, “A simple iterative Gaussian detector for severely delay-spread MIMO channels,” in IEEE ICC'2007, Glasgow, Jun. 2007, pp. 4598–4603.
[30] M., Suneel, P., Som, A., Chockalingam, and B. S., Rajan, “Belief propagation based decoding of large non-orthogonal STBCs,” in IEEE ISIT'2009, Jun.-Jul. 2008, Sead, pp. 2003–2007.
[31] P., Som, T., Datta, N., Srinidhi, A., Chockalingam, and B. S., Rajan, “Low-complexity detection in large-dimension MIMO-ISI channels using graphical models,” IEEE J. Sel. Topics Signal Process., vol. 5, no. 8, pp. 1497–1511, Dec. 2011.
[32] K., Murphy, Y., Weiss, and M., Jordan, “Loopy belief propagation for approximate inference: An empirical study,” in 15th Annual Conference on Uncertainty in Artificial Intelligence (UAI-99), K., Laskey and H., Prade, Eds., San Francisco, CA: Morgan Kaufmann, 1999, pp. 467–475.
[33] J. M., Mooij, Understand ing and improving belief propagation. Ph.D Thesis, Radboud University Nijmegen, May 2008.
[34] J. M., Mooij and H. J., Kappen, “Sufficient conditions for convergence of the sum-product algorithm,” IEEE Trans. Inform. Theory, vol. 53, no. 12, pp. 4422–4437, Dec. 2007.
[35] M., Pretti, “A message passing algorithm with damping,” J. Stat. Mech.: Theory and Practice, Nov. 2005.
[36] T., Heskes, “On the uniqueness of loopy belief propagation fixed points,” Neural Computation, vol. 16, no. 11, pp. 2379–2413, Nov. 2004.
[37] T., Heskes, K., Albers, and B., Kappen, “Approximate inference and constrained optimization,” in 19th Uncertainty in AI, Acapulco, Aug. 2003, pp. 313–320, San Francisco, CA: Morgan Kaufmann.
[38] A. L., Yuille, “A double-loop algorithm to minimize Bethe and Kikuchi free energies,” in EMMCVPR'2001, Sophic Antipolis Sep. 2001. Berlin: Springer, 2001 pp. 3–18.
[39] J., Goldberger and A., Leshem, “MIMO detection for high-order QAM based on a Gaussian tree approximation,” IEEE Trans. Inform. Theory, vol. 57, no. 8, pp. 4973–4982, Aug. 2011.
[40] C. K., Chow and C. N., Liu, “Approximating discrete probability distributions with dependence trees,” IEEE Trans. Inform. Theory, vol. 14, no. 3, pp. 462–467, May 1968.
[41] B. M., Kurkoski, P. H., Siegel, and J. K., Wolf, “Joint message-passing decoding of LDPC codes and partial-response channels,” IEEE Trans. Inform. Theory, vol. 48, no. 6, pp. 1410–1422, Jun. 2002.
[42] R., Koetter, A. C., Singer, and M., Tuchler, “Turbo equalization,” IEEE Signal Process. Mag., vol. 21, no 1, pp. 67–80, Jan. 2004.
[43] T. L., Narasimhan, A., Chockalingam, and B. S., Rajan, “Factor graph based joint detection/decoding for LDPC coded large-MIMO systems,” in IEEE VTC'2012-Spring, Yokohama, May 2012, pp. 1–5.
[44] T. L., Narasimhan and A., Chockalingam, “EXIT chart based design of irregular LDPC codes for large-MIMO systems,” IEEE Comm. Lett., vol. 17, no. 1, pp. 115–118, Jan. 2013.
[45] T., Richardson, A., Shokrollahi, and R., Urbanke, “Design of capacity-approaching irregular codes,” IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 619–637, Feb. 2001.
[46] S.-Y., Chung, J. G. D., Forney, T., Richardson, and R., Urbanke, “On the design of low-density parity-check codes within 0.0045 dB of the Shannon limit,” IEEE Commun. Lett., vol. 5, no. 2, pp. 58–60, Feb. 2001.
[47] S., ten Brink, G., Kramer, and A., Ashikhmin, “Design of low-density parity-check codes for modulation and detection,” IEEE Trans. Commun., vol. 52, no. 4, pp. 670–678, Apr. 2004.
[48] S., ten Brink, “Convergence behavior of iteratively decoded parallel concatenated codes,” IEEE Trans. Commun., vol. 49, no. 10, pp. 1727–1737, Oct. 2001.
[49] A., Ashikhmin, G., Kramer, and S., ten Brink, “Extrinsic information transfer functions: A model and two properties,” in Conf. Inform. Sci. and Sys. (CISS'2002), Prinston, Mar. 2002, pp. 742–747.
[50] D. J. C., MacKay, “Encyclopedia of sparse graph codes,” Online:
[51] “Air interface for fixed and mobile broadband systems,” in IEEE P802.16e Draft, 2005.