Skip to main content Accessibility help

Modeling the Social Dynamics of Moral Enhancement: Social Strategies Sold Over the Counter and the Stability of Society



How individuals tend to evaluate the combination of their own and other’s payoffs—social value orientations—is likely to be a potential target of future moral enhancers. However, the stability of cooperation in human societies has been buttressed by evolved mildly prosocial orientations. If they could be changed, would this destabilize the cooperative structure of society? We simulate a model of moral enhancement in which agents play games with each other and can enhance their orientations based on maximizing personal satisfaction. We find that given the assumption that very low payoffs lead agents to be removed from the population, there is a broadly stable prosocial attractor state. However, the balance between prosociality and individual payoff-maximization is affected by different factors. Agents maximizing their own satisfaction can produce emergent shifts in society that reduce everybody’s satisfaction. Moral enhancement considerations should take the issues of social emergence into account.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Modeling the Social Dynamics of Moral Enhancement: Social Strategies Sold Over the Counter and the Stability of Society
      Available formats

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Modeling the Social Dynamics of Moral Enhancement: Social Strategies Sold Over the Counter and the Stability of Society
      Available formats

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Modeling the Social Dynamics of Moral Enhancement: Social Strategies Sold Over the Counter and the Stability of Society
      Available formats



Hide All


1. Van Lange, PA, Joireman, J, Parks, CD, Van Dijk, E. The psychology of social dilemmas: A review. Organizational Behavior and Human Decision Processes 2013;120(2):125–41.

2. For example, Van Lange, PA. The pursuit of joint outcomes and equality in outcomes: An integrative model of social value orientation. Journal of Personality and Social Psychology 1999;77(2):337.

3. Murphy, RO, Ackermann, KA, Handgraaf, M. Measuring social value orientation. Judgment and Decision Making 2011;6(8):771–81.

4. Persson, I, Savulescu, J. The perils of cognitive enhancement and the urgent imperative to enhance the moral character of humanity. Journal of Applied Philosophy 2008;25(3):162–77; Persson, I, Savulescu, J. Unfit for the Future: The Need for Moral Enhancement. Oxford: Oxford University Press; 2012.

5. Douglas, T. Moral enhancement. Journal of Applied Philosophy 2008;25(3):228–45.

6. Bilderbeck, AC, Brown, GD, Read, J, Woolrich, M, Cowen, PJ, Behrens, TE, et al. Serotonin and social norms tryptophan depletion impairs social comparison and leads to resource depletion in a multiplayer harvesting game. Psychological Science 2014;25(7):1303–13.

7. Kosfeld, M, Heinrichs, M, Zak, PJ, Fischbacher, U, Fehr, E. Oxytocin increases trust in humans. Nature 2005;435(7042):673–6.

8. Zak, PJ, Stanton, AA, Ahmadi, S. Oxytocin increases generosity in humans. PLoS One 2007;2(11):e1128.

9. Shamay–Tsoory, SG, Fischer, M, Dvash, J, Harari, H, Perach–Bloom, N, Levkovitz, Y. Intranasal administration of oxytocin increases envy and schadenfreude (gloating). Biological Psychiatry 2009;66(9):864–70.

10. Kirsch P, Esslinger C, Chen Q, Mier D, Lis S, Siddhanti S, et al. Oxytocin modulates neural circuitry for social cognition and fear in humans. The Journal of Neuroscience 2005;25(49):11,489–93; Ditzen B, Schaer M, Gabriel B, Bodenmann G, Ehlert U, Heinrichs M. Intranasal oxytocin increases positive communication and reduces cortisol levels during couple conflict. Biological Psychiatry 2009;65(9):728–31; Domes G, Heinrichs M, Michel A, Berger C, Herpertz SC. Oxytocin improves “mind-reading” in humans. Biological Psychiatry 2007;61(6):731–3; Krueger F, Parasuraman R, Moody L, Twieg P, de Visser E, McCabe K, et al. Oxytocin selectively increases perceptions of harm for victims but not the desire to punish offenders of criminal offenses. Social Cognitive and Affective Neuroscience 2013;8(5):494–8; Guastella AJ, Mitchell PB, Mathews F. Oxytocin enhances the encoding of positive social memories in humans. Biological Psychiatry 2008;64(3):256–8; Unkelbach C, Guastella AJ, Forgas JP. Oxytocin selectively facilitates recognition of positive sex and relationship words. Psychological Science 2008;19(11):1092–4; Fischer–Shofty M, Levkovitz Y, Shamay–Tsoory SG. Oxytocin facilitates accurate perception of competition in men and kinship in women. Social Cognitive and Affective Neuroscience 2013;8:313–7.

11. Israel S, Lerer E, Shalev I, Uzefovsky F, Riebold M, et al. The oxytocin receptor (OXTR) contributes to prosocial fund allocations in the dictator game and the social value orientations task. PloS One 2009;4(5):e5535.

12. Shook JR. Neuroethics and the possible types of moral enhancement. AJOB Neuroscience. 2012;3(4):3–14.

13. de Melo–Martin I, Salles A. Moral bioenhancement: Much ado about nothing? Bioethics 2014;9702:124–31; Murphy, T. F. Preventing Ultimate Harm as the Justification for Biomoral Modification. Bioethics 2014;9702; Harris J. Moral progress and moral enhancement. Bioethics 2013;27(5):285–90.

14. Wasserman, D. When bad people do good things: Will moral enhancement make the world a better place? Journal of Medical Ethics 2014;40(6):374–5.

15. Morioka M. Some Remarks on Moral Bioenhancement. In: Akabayashi, A ed. The Future of Bioethics: International Dialogues. Oxford: Oxford Scholarship Online; 2014.

16. Rogers EM. Diffusion of Innovations, 4 th Edition [e-book]. London: Simon and Schuster; 2010.

17. See note 16, Rogers 2010.

18. See note 1, Van Lange et al. 2013

19. Griesinger DW, Livingston JW. Toward a model of interpersonal motivation in experimental games. Behavioral Science 1973;18(3):173–88.

20. See note 3, Murphy et al. 2011.

21. Poole ME, Langan-Fox J, Omodei M. Contrasting subjective and objective criteria as determinants of perceived career success: A longitudinal study. Journal of Occupational and Organizational Psychology 1993;66(1):39–54; Abele AE, Spurk D. How do objective and subjective career success interrelate over time? Journal of Occupational and Organizational Psychology 2009;82(4):803–24.

22. Joireman JA, Shelley GP, Teta PD, Wilding J, Kuhlman DM. Computer simulation of social value orientation: Vitality, satisfaction, and emergent game structures. In: Frontiers in Social Dilemmas Research. Berlin, Heidelberg: Springer; 1996;289–310.

23. In some simulations, weights for MaxDiff, |S-O|, and MinDiff, -|S+O|, were included to model (anti)egalitarian orientations.

24. Extracted from note 3, Murphy et al. 2011.

25. Using the Prisoner’s Dilemma game instead of random games also destabilizes the sadomasochist attractor. Agents trying to minimize the joint benefit in this case have a coordination problem that destabilizes the attractor.

26. Axelrod, R, Hamilton, WD. The evolution of cooperation. Science. 1981;211(4489):1390–6.

27. Technically: let the agent have expected utility function U(a,o) that gives its satisfaction given its own action a and other’s actions o (given the probabilities in its current environment). The agent will perform the optimal action a* that maximizes U(a,o). If the agent changes its utility function to U’(a,o) it will now take optimal actions a*’. These can be evaluated according to the current utility function U, and if U(a*’,o)-U(a*,o)>0, it is rational for the agent to change its utilities to match U’.

28. In general, it is NP complete (i.e., hard) to find the Nash equilibria with the highest social welfare in games between rational agents (Conitzer V, Sandholm T. New complexity results about Nash equilibria. Games and Economic Behavior 2008;63(2):621–41).

29. See note 12, Shook 2012.

30. Riis, J, Simmons, JP, Goodwin, GP. Preferences for enhancement pharmaceuticals: The reluctance to enhance fundamental traits. Journal of Consumer Research 2008;35(3):495508.

31. Fukuyama F. Trust: the Social Virtues and Creation of Prosperity. London: Free Press; 1996; Knack S, Keefer P. Does social capital have an economic payoff? A cross-country investigation. The Quarterly Journal of Economics. 1997;1251–88; Wike R, Holzwart K. Where trust is high, crime and corruption are low [Blog post]. Pew Research Center 2008. Available at (last accessed 10 Sept 2015).

32. Van Lange, PA. Generalized trust four lessons from genetics and culture. Current Directions in Psychological Science 2015;24(1):7176.

33. See note 14, Wasserman 2014.

34. Bornstein G. Intergroup conflict: Individual, group, and collective interests. Personality and Social Psychology Review: An Official Journal of the Society for Personality and Social Psychology. 2003;7(2):129–45; De Dreu CKW. Social conflict. Current Sociology 2013;61:696–713.

35. For example, Cardenas JC, Mantilla C. Between-group competition, intra-group cooperation and relative performance. Frontiers in Behavioral Neuroscience 2015;9:1–9; Puurtinen M, Mappes T. Between-group competition and human cooperation. Proceeding Biological Sciences, The Royal Society 2009;276:355–60; Burton–Chellew MN, Ross–Gillespie A, West SA. Cooperation in humans: competition between groups and proximate emotions. Evolution and Human Behavior 2010;31(2):104–8; Bornstein G. Winter E, Goren H. Experimental study of repeated team-games. European Journal of Political Economy 1996;12(4):629–39.

36. Sidanius J, Veniegas RC. Gender and race discrimination: The interactive nature of disadvantage. In: Oskamp S, ed. Reducing prejudice and discrimination (The Claremont Symposium on Applied Social Psychology). Mahwah, NJ: Routledge; 2000;47–69.

37. Bowles, S, Gintis, H. A Cooperative Species: Human Reciprocity and its Evolution. Princeton: Princeton University Press; 2011.

38. Their model revealed that: (1) groups with non-parochial cooperators have a disadvantage over other groups and, therefore, would not have evolved in the first place, however; (2) groups with parochial cooperators, which are willing to sacrifice themselves fighting against out groups in order to benefit their peers, have an evolutionary advantage and; finally, (3) merely parochial groups have a general disadvantage.

39. Boyd, R, Gintis, H, Bowles, S, Richerson, PJ. The evolution of altruistic punishment. Proceedings of the National Academy of Sciences 2003;100(6):3531–5.

40. Bostrom, N. Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press; 2014.

The ideas behind this article were first discussed in a seminar and subsequent discussion at the Centre for Research in Social Simulation in Surrey, where Jen Badham, Juan Cano, and Corinna Elsenbroich helped in properly shaping our models and assumptions. The article also benefited from discussions with Brian Earp on cooperation and individualism. Emma Bates and Paulo Salem gave much appreciated comments to improve clarity. We have also benefited from the comments and questions from the attendees of the Belgrade “Enhancing the Understanding of Enhancement” conference, October 27–28 2015.



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed