Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-8448b6f56d-jr42d Total loading time: 0 Render date: 2024-04-24T09:05:14.951Z Has data issue: false hasContentIssue false

11 - Decentralized reinforcement learning techniques for interference management in heterogeneous networks

Published online by Cambridge University Press:  05 May 2013

By
Mehdi Bennis ,
Dusit Niyato  and
Tansu Alpcan
Edited by
Tony Q. S. Quek ,
Guillaume de la Roche ,
İsmail Güvenç  and
Marios Kountouris
Mehdi Bennis
Affiliation:
University of Oulu
Dusit Niyato
Affiliation:
Nanyang Technological University
Tansu Alpcan
Affiliation:
University of Melbourne
Tony Q. S. Quek
Affiliation:
Singapore University of Technology and Design
Guillaume de la Roche
Affiliation:
Mindspeed Technologies
İsmail Güvenç
Affiliation:
Florida International University
Marios Kountouris
Affiliation:
SUPÉLEC (Ecole Supérieure d'Electricité)
Get access

Summary

Game theory (GT) is a mathematical tool that analyzes interactions among decision makers. Game theory is seen as a natural paradigm to study and analyze wireless networks where players compete for the same resources. The importance of studying the coexistence between macrocells and femtocells from a game theoretical perspective is multi-fold. First, as illustrated in Figure 11.1, by modeling the dynamic spectrum sharing among network players (macrocell base stations (MBSs), femtocell base stations (FBSs), mobile user equipment (MUE), and home user equipment (HUE)) as games, the behaviors and actions of players can be analyzed in a formalized structure, by which the theoretical achievements in GT can be fully utilized. Second, GT equips us with different optimality criteria for various spectrum sharing problems, which are of key importance when it comes to analyzing the equilibrium of the game. Third, the application of GT enables us to derive efficient distributed algorithms for self-organized networks relying only on partial information. In order to achieve this, the theory of strategic reinforcement learning is of utmost importance by allowing players to choose their optimal strategies and gradually learn from their environment through trial and error procedures. A comprehensive source of game theoretic approaches and their application to wireless communications can be found in [1].

Type
Chapter
Information
Small Cell Networks
Deployment, PHY Techniques, and Resource Management
 , pp. 260 - 279
Publisher: Cambridge University Press
Print publication year: 2013

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] S., Lasaulce and H., Tembine, Game Theory and Learning for Wireless Networks: Fundamentals and Applications, 1st edn. Academic Press, 2011.Google Scholar
[2] D., Niyato and E., Hossain, “Dynamics of network selection in heterogeneous wireless networks: an evolutionary game approach,” IEEE Trans. Veh. Technol., vol. 58, no. 4, pp. 2008–17, May 2009.Google Scholar
[3] H., Li, “Multi-agent Q-learning of channel selection in multi-user cognitive radio systems: a two by two case,” in IEEE Int. Conf. on Systems, Man and Cybernetics (SMC), Oct. 2009, pp. 1893–8.Google Scholar
[4] M., Bennis and D., Niyato, “A Q-learning based approach to interference avoidance in self-organized femtocell networks,” in Proc. IEEE Global Telecommun. Conf. (GLOBECOM) Workshops, Dec. 2010, pp. 706–10.Google Scholar
[5] M., Galindo-Serrano, L., Giupponi, and M., Majoral, “On implementation requirements and performances of Q-learning for self-organized femtocells,” in Proc. IEEE Global Telecommun. Conf. (GLOBECOM) Workshops, Dec. 2011, pp. 706–10.Google Scholar
[6] S., Guruacharya, D., Niyato, E., Hossain, and D. I., Kim, “Hierarchical competition in femtocellbased cellular networks,” in Proc. IEEE Global Telecommun. Conf. (GLOBECOM), Dec. 2010, pp. 1–5.Google Scholar
[7] A., Feki and V., Capdevielle, “Autonomous resource allocation for dense LTE networks: a multi armed bandit formulation,” in Proc. IEEE Int. Symp. Personal, Indoor, Mobile Radio Commun. (PIMRC), Dec. 2010, pp. 706–10.Google Scholar
[8] M., Bennis and S. M., Perlaza, “Decentralized cross-tier interference mitigation in cognitive femtocell networks,” in Proc. IEEE Int. Conf. on Commun. (ICC), June 2011, pp. 1–5.Google Scholar
[9] F., Pantisano, M., Bennis, W., Saad, and M., Debbah, “Spectrum leasing as an incentive towards uplink macrocell and femtocell cooperation,” IEEE J. Sel. Areas Commun. (JSAC), vol. 30, no. 3, pp. 617–30, Apr. 2012.Google Scholar
[10] S., Bennis, M., Gurucharya and D., Niyato, “Distributed learning strategies for interference mitigation in femtocell networks,” in Proc. IEEE Global Telecommun. Conf. (GLOBECOM), June 2011, pp. 1–5.Google Scholar
[11] J., Shamma and G., Arslan, “Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria,” IEEE Trans. Autom. Control, vol. 50, no. 3, pp. 312–27, Mar. 2005.Google Scholar
[12] J. F., Nash, “Equilibrium points in n-person games,” P. Nat. Acad. Sci. USA, vol. 36, no. 1, pp. 48–9, 1950.Google Scholar
[13] M., Bennis, S., Perlaza, and M., Debbah, “Learning coarse-correlated equilibria in two-tier networks,” in Proc. IEEE Int. Conf. on Commun. (ICC), June 2012, pp. 1–5.Google Scholar
[14] J. W., Weibull, Evolutionary Game Theory. The MIT Press, 1997.Google Scholar
[15] T., Alpcan and T., Basar, Network Security: A Decision and Game Theoretic Approach, 1st edn. Cambridge: Cambridge University Press, 2011.Google Scholar
[16] S., Hart and A., Mas-Colell, “A simple adaptive procedure leading to correlated equilibrium,” Econometrica, vol. 68, no. 5, pp. 1127–50, Sep. 2000.Google Scholar
[17] A., Nedic and A., Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Trans. Autom. Control, vol. 54, no. 1, pp. 48–61, Jan. 2009.Google Scholar
[18] “3rd Generation Partnership Project; Technical Specification Group Radio Access Networks; 3G Home NodeB Study Item Technical Report (Release 8),” 3GPP, 3GPP TR 25.820, Mar. 2008.

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×