Hostname: page-component-848d4c4894-8bljj Total loading time: 0 Render date: 2024-06-16T03:23:57.709Z Has data issue: false hasContentIssue false
  • English
  • Français

Computational models of intrinsic motivation for curiosity and creativity

Published online by Cambridge University Press:  21 May 2024

Sophia Becker Open the ORCID record for Sophia Becker [Opens in a new window] ,
Alireza Modirshanechi Open the ORCID record for Alireza Modirshanechi [Opens in a new window]  and
Wulfram Gerstner Open the ORCID record for Wulfram Gerstner [Opens in a new window]
Sophia Becker
Affiliation:
Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland sophia.becker@epfl.ch alireza.modirshanechi@epfl.ch wulfram.gerstner@epfl.ch; https://lcnwww.epfl.ch/gerstner/ School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
Alireza Modirshanechi
Affiliation:
Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland sophia.becker@epfl.ch alireza.modirshanechi@epfl.ch wulfram.gerstner@epfl.ch; https://lcnwww.epfl.ch/gerstner/ School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
Wulfram Gerstner*
Affiliation:
Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland sophia.becker@epfl.ch alireza.modirshanechi@epfl.ch wulfram.gerstner@epfl.ch; https://lcnwww.epfl.ch/gerstner/ School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
*
*Corresponding author.
Get access Rights & Permissions [Opens in a new window]

Abstract

We link Ivancovsky et al.'s novelty-seeking model (NSM) to computational models of intrinsically motivated behavior and learning. We argue that dissociating different forms of curiosity, creativity, and memory based on the involvement of distinct intrinsic motivations (e.g., surprise and novelty) is essential to empirically test the conceptual claims of the NSM.

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 47 , 2024 , e94
Check for updates
Copyright
Copyright © The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akiti, K., Tsutsui-Kimura, I., Xie, Y., Mathis, A., Markowitz, J. E., Anyoha, R., ..., & Watabe-Uchida, M. (2022). Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction. Neuron, 110, 3789–3804.e9. https://doi.org/10.1016/j.neuron.2022.08.022CrossRefGoogle ScholarPubMed
Barry, M., & Gerstner, W. (2024). Fast adaptation to rule switching using neuronal surprise. To appear in: PLOS Computational Biology (2024) Early version from 2022 on bioRxiv: https://doi.org/10.1101/2022.09.13.507727Google ScholarPubMed
Barto, A., Mirolli, M., & Baldassarre, G. (2013). Novelty or surprise? Frontiers in Psychology, 4, 907. https://doi.org/10.3389/fpsyg.2013.00907CrossRefGoogle ScholarPubMed
Bogacz, R., & Brown, M. (2003). Comparison of computational models of familiarity discrimination in the perirhinal cortex. Hippocampus, 13(4), 494–524. https://doi.org/10.1002/hipo.10093CrossRefGoogle ScholarPubMed
Duszkiewicz, A. J., McNamara, C. G., Takeuchi, T., & Genzel, L., (2019). Novelty and dopaminergic modulation of memory persistence: A tale of two systems. Trends in Neurosciences, 42(2), 102–114. https://doi.org/10.1016/j.tins.2018.10.002CrossRefGoogle ScholarPubMed
Gershman, S. J., Monfils, M.-H., Norman, K. A., & Niv, Y. (2017). The computational nature of memory modification. eLife, 6, e23763. https://doi.org/10.7554/eLife.23763CrossRefGoogle ScholarPubMed
Gerstner, W., Lehmann, M., Liakoni, V., Corneil, D., & Brea, J. (2018). Eligibility traces and plasticity on behavioral time scales: Experimental support of neoHebbian three-factor learning rules. Frontiers in Neural Circuits, 12, 53. https://doi.org/10.3389/fncir.2018.00053CrossRefGoogle ScholarPubMed
Gottlieb, J., & Oudeyer, P.-Y. (2018). Towards a neuroscience of active sampling and curiosity. Nature Reviews Neuroscience, 19, 758–770. https://doi.org/10.1038/s41583-018-0078-0CrossRefGoogle ScholarPubMed
Homann, J., Koay, S. A., Chen, K. S., Tank, D. W., & Berry II, M. J., (2022). Novelty stimuli evoke excess activity in the mouse primary visual cortex. Proceedings of the National Academy of Sciences of the United States of America, 119(5), e2108882119. https://doi.org/10.1073/pnas.2108882119CrossRefGoogle ScholarPubMed
Lisman, J., Grace, A. A., & Duzel, E. (2011). A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends in Neurosciences. 34, 536–547. https://doi.org/10.1016/j.tins.2011.07.006CrossRefGoogle ScholarPubMed
Maheu, M., Dehaene, S., & Meyniel, F. (2019). Brain signatures of a multiscale process of sequence learning in humans. eLife, 8, e41541. https://doi.org/10.7554/eLife.41541CrossRefGoogle ScholarPubMed
Modirshanechi, A., Becker, S., Brea, J., & Gerstner, W. (2023a). Surprise and novelty in the brain. Current Opinion in Neurobiology, 82, 102758. https://doi.org/10.1016/j.conb.2023.102758CrossRefGoogle ScholarPubMed
Modirshanechi, A., Brea, J., & Gerstner, W. (2022). A taxonomy of surprise definitions. Journal of Mathematical Psychology, 110, 102712. https://doi.org/10.1016/j.jmp.2022.102712CrossRefGoogle Scholar
Modirshanechi, A., Kondrakiewicz, K., Gerstner, W., & Haesler, S. (2023b). Curiosity-driven exploration: Foundations in neuroscience and computational modeling. Trends in Neuroscience, 46(12), 1054–1066. https://doi.org/10.1016/j.tins.2023.10.002CrossRefGoogle ScholarPubMed
Morrens, J., Aydin, C., van Rensburg, A. J., Esquivelzeta Rabell, J., & Haesler, S. (2020). Cue-evoked dopamine promotes conditioned responding during learning. Neuron, 106(1), 142–153.e7. https://doi.org/10.1016/j.neuron.2020.01.012CrossRefGoogle ScholarPubMed
Priestley, J. B., Bowler, J. C., Rolotti, S. V., Fusi, S., & Losonczy, A. (2022). Signatures of rapid plasticity in hippocampal CA1 representations during novel experiences. Neuron, 110, 1978–1992. https://doi.org/10.1016/j.neuron.2022CrossRefGoogle ScholarPubMed
Puigdomènech Badia, A., Piot, A., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, Z. D., & Blundell, C. (2020). Agent57: Outperforming the Atari human benchmark. Proceedings of Machine Learning Research, 119, 507–517.Google Scholar
Schomaker, J., & Meeter, M. (2015). Short- and long-lasting consequences of novelty, deviance and surprise on brain and cognition. Neuroscience & Biobehavioral Reviews, 55, 268–279. https://doi.org/10.1016/j.neubiorev.2015.05.002CrossRefGoogle Scholar
Schulz, A., Miehl, C., Berry II, M. J., & Gjorgjieva, J. (2021). The generation of cortical novelty responses through inhibitory plasticity. eLife, 10, e65309. https://doi.org/10.7554/eLife.65309CrossRefGoogle ScholarPubMed
Wittmann, B. C., Bunzeck, N., Dolan, R. J., & Düzel, E. (2007). Anticipation of novelty recruits reward system and hippocampus while promoting recollection. NeuroImage, 38(1), 194–202. https://doi.org/10.1016/j.neuroimage.2007.06.038CrossRefGoogle ScholarPubMed
Xu, H. A., Modirshanechi, A., Lehmann, M. P., Gerstner, & W., Herzog, M. H. (2021). Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making. PLoS Computational Biology, 17(6), e1009070. https://doi.org/10.1371/journal.pcbi.1009070CrossRefGoogle Scholar
Yu, A., & Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46(4), 681–692. https://doi.org/10.1016/j.neuron.2005.04.026CrossRefGoogle ScholarPubMed
Zhang, K., Bromberg-Martin, E. S., Sogukpinar, F., Kocher, K., & Monosov, I. E. (2022). Surprise and recency in novelty detection in the primate brain. Current Biology, 32(10), 2160–2173.e6. https://doi.org/10.1016/j.cub.2022.03.064CrossRefGoogle ScholarPubMed