Skip to main content Accessibility help
Hostname: page-component-544b6db54f-5rlvm Total loading time: 0.46 Render date: 2021-10-16T13:27:08.094Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

11 - Game theory and learning techniques for self-organization in small cell networks

Published online by Cambridge University Press:  05 December 2015

Prabodini Semasinghe
University of Manitoba
Kun Zhu
University of Manitoba
Ekram Hossain
University of Manitoba
Alagan Anpalagan
Ryerson University
Alagan Anpalagan
Ryerson Polytechnic University, Toronto
Mehdi Bennis
University of Oulu, Finland
Rath Vannithamby
Intel Corporation, Portland, Oregon
Get access


Small cell networks

The tremendous increase of bandwidth-craving mobile applications (e.g., video streaming, video chatting, and online gaming) has posed enormous challenges to the design of future wireless networks. Deploying small cells (e.g., pico, micro, and femto) has been shown to be an efficient and cost-effective solution to support this constantly rising demand since the smaller cell size can provide higher link quality and more efficient spatial reuse [1]. Small cells could also deliver some other benefits such as offloading the macro network traffic, providing service to coverage holes and regions with poor signal reception (e.g., macro cell edges). Following this trend, the evolving 5G networks [2] are expected to be composed of hundreds of interconnected heterogeneous small cells.

Figure 11.1 gives an illustration of a heterogeneous network (HetNet) where a macro cell is underlaid with different types of small cells. Different from the cautiously planned traditional network, the architecture of a HetNet is more random and unpredictable due to the increased density of small cells and their impromptu way of deployment. In this case, the manual intervention and centralized control used in traditional network management will be highly inefficient, time consuming, and expensive, and therefore will be not applicable for dense heterogeneous small cell networks. Instead, self-organization has been proposed as an essential feature for future small cell networks [3, 4].

The motivations for enabling self-organization in small cell networks are explained below.

  1. • Numerous network devices with different characteristics are expected to be interconnected in future wireless networks. Also, these devices are expected to have “plug and play” capability. Therefore the initial pre-operational configuration has to be done with minimum expertise involvement.

  2. • With the emergence of small cells, the spatio-temporal dynamics of the networks has become more unpredictable than legacy systems due to the unplanned nature of small cell deployment. Therefore intelligent adaptation of the network nodes is necessary. That is, the self-organizing small cells need to learn from the environment and adapt with the network dynamics to achieve the desired performance.

Publisher: Cambridge University Press
Print publication year: 2015

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


[1] Chandrasekhar, V., “Femtocell networks: a survey,” IEEE Communications Magazine, pp. 59–67, September 2008.
[2] Demestichas, P. and Georgakopoulos, A., “5G on the horizon: key challenges for the radio-access network,” IEEE Vehicular Technology Magazine, pp. 47–53, September 2013.
[3] Aliu, O., Imran, A., Imran, M., and Evans, B., “A survey of self organisation in future cellular networks,” IEEE Communications Surveys and Tutorials, vol. 15, pp. 336–361, 2012.Google Scholar
[4] Claussen, H., Ho, L. T., and Samuel, L. G., “An overview of the femtocell concept,” Bell Labs Technical Journal, vol. 13, pp. 221–245, 2008.CrossRefGoogle Scholar
[5] Johnson, N., “Small cell forum and LTE small cells,” The Small Cell Forum, Tech. Rep., 2011.
[6] Bennis, M. and Giupponi, L., “Interference management in self-organized femtocell networks: the BeFEMTO approach,” 2nd International Conference on Wireless Communication, Vehicular Technology, Information Theory and Aerospace Electronic Systems Technology (Wireless VITAE), Chennai, 2011, pp. 5–10.Google Scholar
[7] Berg, J. V. and Litjens, R., “SOCRATES: Self-Optimisation and self-ConfiguRATion in wirelEss networkS,” Fourth ERCIM Workshop on eMobility, 2008.
[8] Jansen, T., Amirijoo, M., Turke, U., et al. “Embedding multiple self-organisation functionalities in future radio access networks,” IEEE 69th Vehicular Technology Conference. IEEE, 2009, pp. 1–5.
[9] EU FP7 project. End-to-End Efficiency (E3).
[10] Bogenfeld, E. and Gaspard, I., “Self-X in radio access networks,” E3 White Paper v1. 0, vol. 22, 2008.Google Scholar
[11] Bonabeau, E., Dorigo, M., and Theraulaz, G., Swarm Intelligence:From Natural to Artificial Systems. New York, Oxford University Press 1999, vol. 4.Google Scholar
[12] 3GPP TS 32.500, “Telecommunication management; self-organizing networks (SON); concepts and requirements.”
[13] 3GPP TR 36.902, “Evolved universal terrestrial radio access network (E-UTRAN); self-configuring and self-optimizing network (son) use cases and solutions.”
[14] NGMN Alliance, “Next generation mobile networks beyond HSPA and EVDO,” White PaperDec, vol. 5, 2006.
[15] NGMN Alliance, “Next generation mobile networks recommendation on SON and O&M requirements,” Req. Spec. v1, vol. 23, 2008.
[16] Haykin, S., “Cognitive dynamic systems,” IEEE International Conference on Acoustics, Speech and Signal Processing, 2007., vol. 4, 2007, pp. IV–1369 – IV–1372.Google Scholar
[17] Haykin, “Cognitive radio: brain-empowered wireless communications,” IEEE Journal on Selected Areas in Communications, vol. 23, pp. 201–220, 2005.
[18] Spilling, A., Nix, A., Beach, M., and Harrold, T., “Self-organisation in future mobile communications,” Electronics and Communication Engineering Journal, vol. 12, pp. 133–147, 2000.CrossRefGoogle Scholar
[19] Prehofer, C. and Bettstetter, C., “Self-organization in communication networks: principles and design paradigms,” IEEE Communications Magazine, vol. 43, no. 7, pp. 78–85, 2005.CrossRefGoogle Scholar
[20] Hämäläinen, S., Sanneck, H., Sartori, al., LTE Self-Organising Networks (SON): Network Management Automation for Operational Efficiency. John Wiley & Sons, 2012.Google Scholar
[21] Viering, I., Dottling, M., and Lobinger, A., “A mathematical perspective of self-optimizing wireless networks,” IEEE International Conference on Communications, 2009, pp. 1–6.
[22] Hu, H., Zhang, J., and Zheng, X., “Self-configuration and self-optimization for LTE networks,” IEEE Communications Magazine, pp. 94–100, February 2010.
[23] Sanneck, H., Bouwen, Y., and Troch, E., “Context based configuration management of plug and play LTE base stations,” IEEE Network Operations and Management Symposium – NOMS 2010, pp. 946–949, 2010.
[24] “Self-optimizing networks: the benefits of SON in LTE,” 4G Americas, 2013.
[25] NGMN Alliance, “Use cases related to self organising network: Overall description,” 2007.
[26] Choi, Y., Kim, C., and Bahk, S., “Flexible design of frequency reuse factor in OFDMA cellular networks,” IEEE International Conference on Communications, vol. 4, 2006, pp. 1784–1788.Google Scholar
[27] Ileri, O., Mau, S.-C., and Mandayam, N. B., “Pricing for enabling forwarding in self-configuring ad hoc networks,” Journal on Selected Areas in Communications, vol. 23, pp. 151–162, 2005.CrossRefGoogle Scholar
[28] Semasinghe, P., Hossain, E., and Zhu, K., “An evolutionary game approach for distributed resource allocation for self-organizing small cells,” IEEE Transactions of Mobile Computing, 2013.
[29] Dirani, M. and Altman, Z., “Self-organizing networks in next generation radio access networks: application to fractional power control,” Computer Networks, vol. 55, no. 2, pp. 431–438, 2011.CrossRefGoogle Scholar
[30] 3GPP TS 32.541, “Evolved universal terrestrial radio access network (E-UTRAN); self-organizing networks (son); self-healing concepts and requirements (Release 11).”
[31] Attar, A., Krishnamurthy, V., and Gharehshiran, O. N., “Interference management using cognitive base-stations for UMTS LTE,” IEEE Communications Magazine, vol. 49, pp. 152–159, 2011.CrossRefGoogle Scholar
[32] Zhu, K., Hossain, E., and Niyato, D., “Pricing, spectrum sharing, and service selection in two-tier small cell networks: a hierarchical dynamic game approach,” IEEE Transactions on Mobile Computing, 2013.
[33] Huang, X. and Beferull-Lozano, B., “Non-cooperative power allocation game with imperfect sensing information for cognitive radio,” IEEE International Conference on Communications (ICC). IEEE, 2012, pp. 1666–1671.
[34] Guruacharya, S., Niyato, D., Bennis, M., and Kim, D., “Dynamic coalition formation for network MIMO in small cell networks,” IEEE Transactions on Wireless Communications, 2013.
[35] Madan, R., Sampath, A., Bhushan, N., Khandekar, A., Borran, J., and Ji, T., “Impact of coordination delay on distributed scheduling in LTE-A femtocell networks,” in IEEE Global Telecommunications Conference (GLOBECOM 2010), 2010, pp. 1–5.
[36] Han, Z., Niyato, D., Saad, W., Basar, T., and Hjorungnes, A., Game Theory in Wireless and Communication Networks. Cambridge University Press, 2012.Google Scholar
[37] Akkarajitsakul, K., Hossain, E., Niyato, D., and Kim, D. I., “Game theoretic approaches for multiple access in wireless networks: a survey,” IEEE Communications Surveys and Tutorials, vol. 13, pp. 372–395, 2011.CrossRefGoogle Scholar
[38] Bennis, M., Perlaza, S. M., and Debbah, M., “Learning coarse correlated equilibria in two-tier wireless networks,” IEEE International Conference on Communications (ICC), 2012, pp. 1592–1596.
[39] Hong, E. J., Yun, S. Y., and Cho, D.-H., “Decentralized power control scheme in femtocell networks: a game theoretic approach,” IEEE 20th International Symposium on Personal, Indoor and Mobile Radio Communications, 2009, pp. 415–419.
[40] Topkis, D. M., Supermodularity and Complementarity. Princeton University Press, 1998.Google Scholar
[41] Altman, E. and Altman, Z., “S-modular games and power control in wireless networks,” IEEE Transactions on Automatic Control, vol. 48, pp. 839–842, 2003.CrossRefGoogle Scholar
[42] MacKenzie, A. B. and DaSilva, L. A., “Game theory for wireless engineers,” Synthesis Lectures on Communications, vol. 1, pp. 1–86, 2006.CrossRefGoogle Scholar
[43] Nisan, N., Algorithmic Game Theory. Cambridge University Press, 2007.CrossRefGoogle Scholar
[44] Yeung, M. K. H. and Kwok, Y.-K., “A game theoretic approach to power aware wireless data access,” IEEE Transactions on Mobile Computing, vol. 5, pp. 1057–1073, 2006.CrossRefGoogle Scholar
[45] Huang, J., Berry, R. A., and Honig, M. L., “Distributed interference compensation for wireless networks,” IEEE Journal on Selected Areas in Communications, vol. 24, pp. 1074–1084, 2006.Google Scholar
[46] Li, H., Gai, Y., He, Z., Niu, K., and Wu, W., “Optimal power control game algorithm for cognitive radio networks with multiple interference temperature limits,” Vehicular Technology Conference, Spring 2008. IEEE, 2008, pp. 1554–1558.
[47] Reichl, P., Tuffin, B., and Schatz, R., “Logarithmic laws in service quality perception: where microeconomics meets psychophysics and quality of experience,” Telecommunication Systems, pp. 1–14, 2011.
[48] Huang, L., Zhou, Y., Han, X., Wang, Y., Qian, M., and Shi, J., “Distributed coverage optimization for small cell clusters using game theory,” IEEE Wireless Communications and Networking Conference (WCNC). IEEE, 2013, pp. 2289–2293.
[49] Subramani, S., Basar, T., Armour, S., Kaleshi, D., and Fan, Z., “Noncooperative equilibrium solutions for spectrum access in distributed cognitive radio networks,” 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Net-works, DySPAN 2008. IEEE, 2008, pp. 1–5.
[50] Ma, W., Zhang, H., Zheng, W., and Wen, X., “Differentiated-pricing based power allocation in dense femtocell networks,” 15th International Symposium on Wireless Personal Multimedia Communications (WPMC). IEEE, 2012, pp. 599–603.
[51] Chandrasekhar, V., Andrews, J. G., Muharemovic, T., Shen, Z., and Gatherer, A., “Power control in two-tier femtocell networks,” IEEE Transactions on Wireless Communications, vol. 8, pp. 4316–4328, 2009.Google Scholar
[52] Glicksberg, I. L., “A further generalization of the kakutani fixed point theorem, with application to nash equilibrium points,” Proceedings of the American Mathematical Society, vol. 3, pp. 170–174, 1952.Google Scholar
[53] Rosen, J. B., “Existence and uniqueness of equilibrium points for concave n-person games,” Econometrica: Journal of the Econometric Society, pp. 520–534, 1965.
[54] Debreu, G., “A social equilibrium existence theorem,” Proceedings of the National Academy of Sciences of the United States of America, vol. 38, p. 886, 1952.CrossRefGoogle ScholarPubMed
[55] Yates, R. D., “A framework for uplink power control in cellular radio systems,” IEEE Journal on Selected Areas in Communications, vol. 13, pp. 1341–1347, 1995.CrossRefGoogle Scholar
[56] Mustika, I. W., Yamamoto, K., Murata, H., and Yoshida, S., “Potential game approach for self-organized interference management in closed access femtocell networks,” IEEE 73rd Vehicular Technology Conference (VTC Spring). IEEE, 2011, pp. 1–5.
[57] 3GPP TR 36.814 v9.0.0, “Evolved universal terrestrial radio access (E-UTRA); further advancement of E-UTRA physical layer aspects (Release 9),” March 2010.
[58] Giupponi, L. and Ibars, C., “Distributed interference control in OFDMA-based femtocells,” IEEE 21st International Symposium on Personal Indoor and Mobile Radio Communications (PIMRC). IEEE, 2010, pp. 1201–1206.
[59] Yun, J.-H. and Shin, K. G., “Adaptive interference management of OFDMA femtocells for co-channel deployment,” IEEE Journal on Selected Areas in Communications, vol. 29, pp. 1225–1241, 2011.CrossRefGoogle Scholar
[60] Bennis, M., Guruacharya, S., and Niyato, D., “Distributed learning strategies for interference mitigation in femtocell networks,” IEEE Global Telecommunications Conference (GLOBECOM 2011). IEEE, 2011, pp. 1–5.
[61] Semasinghe, P., Zhu, K., and Hossain, E., “Distributed resource allocation for self-organizing small cell networks: an evolutionary game approach,” IEEE GLOBECOM Workshops, 2013.
[62] Weibull, J. W., Evolutionary Game Theory. MIT Press, 1997.Google Scholar
[63] Samarakoon, S., Bennis, M., Saad, W., and Latva-aho, M., “Backhaul-aware interference management in the uplink of wireless small cell networks,” IEEE Transactions of Wireless Communications, 2013.
[64] Gur, G., Bayhan, S., and Alagoz, F., “Cognitive femtocell networks: an overlay architecture for localized dynamic spectrum access [dynamic spectrum management],” IEEE Wireless Communications, vol. 17, pp. 62–70, 2010.CrossRefGoogle Scholar
[65] Li, H., “Multi-agent Q-learning of channel selection in multi-user cognitive radio systems: a two by two case,” IEEE International Conference on Systems, Man and Cybernetics, SMC. IEEE, 2009, pp. 1893–1898.
[66] Huang, J. W. and Krishnamurthy, V., “Cognitive base stations in LTE/3GPP femtocells: a correlated equilibrium game-theoretic approach,” IEEE Transactions on Communications, vol. 59, pp. 3485–3493, 2011.CrossRefGoogle Scholar
[67] Aumann, R. J., “Correlated equilibrium as an expression of Bayesian rationality,” Econometrica: Journal of the Econometric Society, pp. 1–18, 1987.
[68] Nazir, M., Bennis, M., Ghaboosi, K., MacKenzie, A. B., and Latva-aho, M., “Learning based mechanisms for interference mitigation in self-organized femtocell networks,” Forty-Fourth Asilomar Conference on Signals, Systems and Computers, 2010, pp. 1886–1890.
[69] Pantisano, F., Bennis, M., Saad, W., Latva-aho, M., and Verdone, R., “Enabling macrocell–femtocell coexistence through interference draining,” Wireless Communications and Networking Conference Workshops, 2012, pp. 81–86.
[70] Bennis, M., Perlaza, S. M., Blasco, P., Han, Z., and Poor, H. V., “Self-organization in small cell networks: a reinforcement learning approach,” IEEE Transactions on Wireless Communications, vol. 12, pp. 3202–3212, 2013.CrossRefGoogle Scholar
[71] Sutton, R. S. and Barto, A. G., Reinforcement Learning: An Introduction. Cambridge University Press, 1998, vol. 1, no. 1.Google Scholar
[72] Bkassiny, M., Li, Y., and Jayaweera, S., “A survey on machine-learning techniques in cognitive radios,” IEEE Communications Surveys Tutorials, vol. 15, pp. 1136–1159, 2013.CrossRefGoogle Scholar
[73] Bennis, M. and Perlaza, S. M., “Decentralized cross-tier interference mitigation in cognitive femtocell networks,” IEEE International Conference on Communications (ICC). IEEE, 2011, pp. 1–5.
[74] McFadden, D. L., “Quantal choice analaysis: a survey,”Annals of Economic and Social Measurement, vol. 5, no. 4. NBER, 1976, pp. 363–390.Google Scholar
[75] McKelvey, R. D. and Palfrey, T. R., “Quantal response equilibria for normal form games,” Games and Economic Behavior, vol. 10, pp. 6–38, 1995.CrossRefGoogle Scholar
[76] Hofbauer, J. and Sandholm, W. H., “On the global convergence of stochastic fictitious play,” Econometrica, vol. 70, pp. 2265–2294, 2002.CrossRefGoogle Scholar
[77] Watkins, C. J. C. H., “Learning from delayed rewards.” PhD dissertation, University of Cambridge, 1989.
[78] Niyato, D. and Hossain, E., “Dynamics of network selection in heterogeneous wireless networks: an evolutionary game approach,” IEEE Transactions on Vehicular Technology, vol. 58, pp. 2008–2017, 2009.CrossRefGoogle Scholar
[79] Bennis, M. and Niyato, D., “A Q-learning based approach to interference avoidance in self-organized femtocell networks,” IEEE GLOBECOM Workshops. IEEE, 2010, pp. 706–710.
[80] Hart, S. and Mas-Colell, A., “A simple adaptive procedure leading to correlated equilibrium,” Econometrica, vol. 68, pp. 1127–1150, 2000.CrossRefGoogle Scholar
[81] Nie, N. and Comaniciu, C., “Adaptive channel allocation spectrum etiquette for cognitive radio networks,” Mobile Networks and Applications, vol. 11, pp. 779–797, 2006.CrossRefGoogle Scholar
[82] Ahmadabadi, M. N. and Asadpour, M., “Expertness based cooperative Q-learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 32, pp. 66–76, 2002.CrossRefGoogle ScholarPubMed
[83] Tan, M., “Multi-agent reinforcement learning: Independent vs. cooperative agents,” Proceedings of the Tenth International Conference on Machine Learning, vol. 337. Amherst, MA, 1993.Google Scholar
[84] Serrano, A. G., Giupponi, L., and Dohler, M., “Befemto's self-organized and docitive femtocells,” Future Network and Mobile Summit. IEEE, 2010, pp. 1–8.
[85] Song, Y., Wong, S. H., and Lee, K.-W., “Optimal gateway selection in multi-domain wireless networks: a potential game perspective,” Proceedings of the 17th Annual International Conference on Mobile Computing and Networking. ACM, 2011, pp. 325–336.
[86] Narendra, K. S. and Thathachar, M., “Learning automata: a survey,” IEEE Transactions on Systems, Man and Cybernetics, pp. 323–334, 1974.
[87] Xu, Y., Wang, J., Wu, Q., Anpalagan, A., and Yao, Y.-D., “Opportunistic spectrum access in unknown dynamic environment: a game-theoretic stochastic learning solution,” IEEE Transactions of Wireless Communications, pp. 1380–1391, 2012.

Send book to Kindle

To send this book to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats

Send book to Dropbox

To send content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about sending content to Dropbox.

Available formats

Send book to Google Drive

To send content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about sending content to Google Drive.

Available formats