Skip to main content Accessibility help
  • Print publication year: 2019
  • Online publication date: February 2019

8 - Attention, Information-Seeking, and Active Sampling

from Part II - Rewards, Incentives, and Choice


In this chapter, we present an overview of the literature addressing the neuroscience of attention, information-seeking, and active sampling, and we discuss its potential significance for learning and learning progress. First, we review the emerging hypothesis that attention is an active mechanism for information sampling and exploration in the environment. We then turn to a discussion of how reward motivates attention and how attention can be employed to reduce uncertainty about knowledge of one's current state. We further consider the way rewards interact with other factors (including novelty, surprise, and task relevance). Throughout the review, we particularly focus on the distinction between extrinsic and intrinsic motivation, highlighting curiosity as a key example of the latter in motivating the search for intrinsically desirable information that benefits learning on both long and short timescales. Finally, we discuss the role of cognitive control in directing attention during learning, as well as the way neural systems underlying cognition and motivation have implications for informing techniques for teaching and learning in wider educational contexts.

Abuhamdeh, S. & Csikszentmihalyi, M. (2012). The importance of challenge for the enjoyment of intrinsically motivated, goal-directed activities. Personality and Social Psychology Bulletin, 38(3), 317–30. doi: 10.1177/0146167211427147.
Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. (2006). Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron, 50(3), 507–17. doi: 10.1016/j.neuron.2006.03.036.
Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences of the United States of America, 108(25), 10367–71. doi: 10.1073/pnas.1104047108.
Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press.
Baldassarre, G. & Mirolli, M. (2013). Intrinsically motivated learning systems: An overview. In Intrinsically motivated learning in natural and artificial systems (pp. 114). Berlin Heidelberg: Springer. doi: 10.1007/978-3-642-32375-1_1.
Baldi, P. & Itti, L. (2010). Of bits and wows: A Bayesian theory of surprise with applications to attention. Neural Networks, 23(5), 649–66. doi: 10.1016/j.neunet.2009.12.007.
Baranes, A. & Oudeyer, P. Y. (2013). Active learning of inverse models with intrinsically motivated goal exploration in robots. Robotics and Autonomous Systems, 61(1), 4973. doi: 10.1016/j.robot.2012.05.00.
Baranes, A., Oudeyer, P. Y., & Gottlieb, J. (2015). Eye movements reveal epistemic curiosity in human observers. Vision Research, 117, 8190. doi: 10.1016/j.visres.2015.10.009.
Barto, A. G. (2013). Intrinsic motivation and reinforcement learning. In Intrinsically motivated learning in natural and artificial systems (pp. 1747). Berlin Heidelberg: Springer. doi: 10.1007/978-3-642-32375-1_2.
Barto, A., Mirolli, M., & Baldassarre, G. (2013). Novelty or surprise? Frontiers in Psychology, 4. doi: 10.3389/fpsyg.2013.00907.
Beck, J. M., Ma, W. J., Kiani, R., Hanks, T., Churchland, A. K., Roitman, J., ... Pouget, A. (2008). Probabilistic population codes for Bayesian decision making. Neuron, 60(6), 1142–52. doi: 10.1016/j.neuron.2008.09.021.
Berlyne, D. E. (1960). Conflict, arousal, and curiosity. McGraw-Hill.
Berridge, K. C. & Kringelbach, M. L. (2008). Affective neuroscience of pleasure: Reward in humans and animals. Psychopharmacology, 199(3), 457–80. doi: 10.1007/s00213-008-1099-6.
Bialek, W., Nemenman, I., & Tishby, N. (2001). Predictability, complexity, and learning. Neural Computation, 13(11), 2409–63. doi: 10.1162/089976601753195969.
Blake, A. & Yuille, A. (1992). Active vision. Cambridge, MA: MIT Press.
Blanchard, T. C., Hayden, B. Y., & Bromberg-Martin, E. S. (2015). Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity. Neuron, 85(3), 602–14. doi: 10.1016/j.neuron.2014.12.050.
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 624. doi: 10.1037/0033-295X.108.3.624.
Britten, K. H., Shadlen, M. N., Newsome, W. T., & Movshon, J. A. (1992). The analysis of visual motion: A comparison of neuronal and psychophysical performance. Journal of Neuroscience, 12(12), 4745–65.
Brockmole, J. R. & Henderson, J. M. (2005a). Object appearance, disappearance, and attention prioritization in real-world scenes. Psychonomic Bulletin & Review, 12(6), 1061–7. doi: 10.3758/BF03206444.
Brockmole, J. R. & Henderson, J. M. (2005b). Prioritization of new objects in real-world scenes: Evidence from eye movements. Journal of Experimental Psychology–Human Perception and Performance, 31(5), 857–68. doi: 10.1037/0096-1523.31.5.857.
Bromberg-Martin, E. S. & Hikosaka, O. (2009). Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron, 63(1), 119–26. doi: 10.1016/j.neuron.2009.06.009.
Buschman, T. J. & Miller, E. K. (2007). Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science, 315(5820), 1860–2.
Castro, D. C. & Berridge, K. C. (2014). Advances in the neurobiological bases for food “liking” versus “wanting”. Physiology & Behavior, 136, 2230. doi: 10.1126/science.1138071.
Chiba, A. A., Bucci, D. J., Holland, P. C., & Gallagher, M. (1995). Basal forebrain cholinergic lesions disrupt increments but not decrements in conditioned stimulus processing. Journal of Neuroscience, 15(11), 7315–22.
Csikszentmihalyi, M. (1997). Flow and the psychology of discovery and invention. New York, NY: Harper Perennial.
Daddaoua, N., Lopes, M., & Gottlieb, J. (2016). Intrinsically motivated oculomotor exploration guided by uncertainty reduction and conditioned reinforcement in non-human primates. Scientific Reports, 6. doi: 10.1038/srep20202.
Dalley, J. W., McGaughy, J., O'Connell, M. T., Cardinal, R. N., Levita, L., & Robbins, T. W. (2001). Distinct changes in cortical acetylcholine and noradrenaline efflux during contingent and noncontingent performance of a visual attentional task. Journal of Neuroscience, 21(13), 4908–14.
Dayan, P. & Daw, N. D. (2008). Decision theory, reinforcement learning, and the brain. Cognitive, Affective, & Behavioral Neuroscience, 8(4), 429–53. doi: 10.3758/cabn.8.4.429.
Dayan, P., Niv, Y., Seymour, B., & Daw, N. D. (2006). The misbehavior of value and the discipline of the will. Neural Networks, 19(8), 1153–60. doi: 10.1016/j.neunet.2006.03.002.
Della Libera, C. & Chelazzi, L. (2009). Learning to attend and to ignore is a matter of gains and losses. Psychological Science, 20(6), 778–84. doi: 10.1111/j.1467-9280.2009.02360.x.
Dosenbach, N. U., Fair, D. A., Cohen, A. L., Schlaggar, B. L., & Petersen, S. E. (2008). A dual-networks architecture of top-down control. Trends in Cognitive Sciences, 12(3), 99105. doi: 10.1016/j.tics.2008.01.001.
Everitt, B. J. & Robbins, T. W. (1997). Central cholinergic systems and cognition. Annual Review of Psychology, 48(1), 649–84. doi: 10.1146/annurev.psych.48.1.649.
Falkenstein, M., Koshlykova, N. A., Kiroj, V. N., Hoormann, J., & Hohnsbein, J. (1995). Late ERP components in visual and auditory Go/Nogo tasks. Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section, 96(1), 3643. doi: 10.1016/0013-4694(94)00182-k.
Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., ... Akil, H., (2011). A selective role for dopamine in reward learning. Nature, 469(7328), 53. doi: 10.1038/nature09588.
Foley, N. C., Jangraw, D. C., Peck, C., & Gottlieb, J. (2014). Novelty enhances visual salience independently of reward in the parietal lobe. Journal of Neuroscience, 34(23), 7947–57. doi: 10.1523/jneurosci.4171-13.2014.
Foley, N. C., Kelly, S. P., Mhatre, H., Lopes, M., & Gottlieb, J. (2017). Parietal neurons encode expected gains in instrumental information. Proceedings of the National Academy of Sciences of the United States of America, 114(16), E3315-E3323.
Forestier, S. & Oudeyer, P. Y. (2016). Curiosity-driven development of tool use precursors: A computational model. In 38th Annual Conference of the Cognitive Science Society (CogSci 2016) (pp. 1859–64).
Friedman-Hill, S. R., Robertson, L. C., Desimone, R., & Ungerleider, L. G. (2003). Posterior parietal cortex and the filtering of distractors. Proceedings of the National Academy of Sciences of the United States of America, 100(7), 4263–8. doi: 10.1073/pnas.0730772100.
Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. (2013). The anatomy of choice: active inference and agency. Frontiers in Human Neuroscience, 7. doi: 10.3389/fnhum.2013.00598.
Gehring, W. J., Goss, B., Coles, M. G., Meyer, D. E., & Donchin, E. (1993). A neural system for error detection and compensation. Psychological Science, 4(6), 385–90. doi: 10.1111/j.1467-9280.1993.tb00586.x.
Gehring, W. J. & Knight, R. T. (2002). Lateral prefrontal damage affects processing selection but not attention switching. Cognitive Brain Research, 13(2), 267–79. doi: 10.1016/s0926-6410(01)00132-x.
Germain, C. M. & Hess, T. M. (2007). Motivational influences on controlled processing: Moderating distractibility in older adults. Aging, Neuropsychology, and Cognition, 14(5), 462–86. doi: 10.1080/13825580600611302.
Gold, J. I. & Shadlen, M. N. (2007). The neural basis of decision making. Annual Review of Neuroscience, 30, 535–74. doi: 10.1146/annurev.neuro.29.051605.113038.
Goldberg, M. E., Bisley, J. W., Powell, K. D., & Gottlieb, J. (2006). Saccades, salience and attention: the role of the lateral intraparietal area in visual behavior. Progress in Brain Research, 155, 157–75. doi: 10.1016/S0079-6123(06)55010-1.
Gottlieb, J. (2012). Attention, learning, and the value of information. Neuron, 76(2), 281–95. doi: 10.1016/j.neuron.2012.09.034.
Gottlieb, J. (2018). Understanding active sampling strategies: empirical approaches and implications for attention and decision research. Cortex, 102, 150–60. doi: 10.1016/j.cortex.2017.08.019.
Gottlieb, J. & Balan, P. (2010). Attention as a decision in information space. Trends in Cognitive Sciences, 14(6), 240–8. doi: 10.1016/j.tics.2010.03.001.
Gottlieb, J. & Goldberg, M. E. (1999). Activity of neurons in the lateral intraparietal area of the monkey during an antisaccade task. Nature Neuroscience, 2(10), 906–13. doi: 10.1038/13209.
Gottlieb, J., Hayhoe, M., Hikosaka, O., & Rangel, A. (2014). Attention, reward, and information seeking. Journal of Neuroscience, 34(46), 15497–504. doi: 10.1523/JNEUROSCI.3270-14.2014.
Gottlieb, J., Lopes, M., & Oudeyer, P. Y. (2016). Motivated cognition: Neural and computational mechanisms of curiosity, attention, and intrinsic motivation. In Kim, S., Reeve, J. & Bong, M. (Eds.), Advances in motivation and achievement: Recent developments in neuroscience research on human motivation (pp. 149–72). Bingley: Emerald Group Publishing.
Gottlieb, J., Oudeyer, P. Y., Lopes, M., & Baranes, A. (2013). Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends in Cognitive Sciences, 17(11), 585–93. doi: 10.1016/j.tics.2013.09.001.
Grossnickle, E. M. (2015). The expression and enactment of interest and curiosity in a multiple source use task (Doctoral dissertation). University of Maryland, College Park.
Gruber, M. J., Gelman, B. D., & Ranganath, C. (2014). States of curiosity modulate hippocampus-dependent learning via the dopaminergic circuit. Neuron, 84(2), 486–96. doi: 10.1016/j.neuron.2014.08.060.
Hickey, C., Chelazzi, L., & Theeuwes, J. (2010). Reward changes salience in human vision via the anterior cingulate. Journal of Neuroscience. 30, 11096–103. doi: 10.1523/JNEUROSCI.1026-10.2010.
Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of Educational Research, 60(4), 549–71. doi: 10.3102/00346543060004549.
Hidi, S. (2016). Revisiting the role of rewards in motivation and learning: Implications of neuroscientific research. Educational Psychology Review, 28(1), 6193. doi: 10.1007/s10648-015-9307-5.
Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience, 96(4), 651–6. doi: 10.1016/S0306-4522(00)00019-1.
Isaacowitz, D. M., Wadlinger, H. A., Goren, D., & Wilson, H. R. (2006). Selective preference in visual fixation away from negative images in old age? An eye-tracking study. Psychology and Aging, 21(1), 40. doi: 10.1037/0882-7974.21.1.40.
Itti, L. & Baldi, P. (2009). Bayesian surprise attracts human attention. Vision Research, 49(10), 1295–306. doi: 10.1016/j.visres.2008.09.007.
Johnson, L., Sullivan, B., Hayhoe, M., & Ballard, D. (2014). Predicting human visuomotor behavior in a driving task. Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1636), 20130044. doi: 10.1098/rstb.2013.0044.
Kable, J. W. & Glimcher, P. W. (2009). The neurobiology of decision: consensus and controversy. Neuron, 63(6), 733–45. doi: 10.1016/j.neuron.2009.09.003.
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1), 99134. doi: 10.1016/s0004-3702(98)00023-x.
Kakade, S. & Dayan, P. (2002). Dopamine: Generalization and bonuses. Neural Networks, 15(4), 549–59. doi: 10.1016/s0893-6080(02)00048-5.
Kang, M. J., Hsu, M., Krajbich, I. M., Loewenstein, G., McClure, S. M., Wang, J. T. Y., & Camerer, C. F. (2009). The wick in the candle of learning: Epistemic curiosity activates reward circuitry and enhances memory. Psychological Science, 20(8), 963–73. doi: 10.2139/ssrn.1308286.
Kaplan, F. & Oudeyer, P.-Y. (2007). In search of the neural circuits of intrinsic motivation. Frontiers in Neuroscience, 1, 17
Kaplan, F. & Oudeyer, P.-Y. (2011). From hardware and software to kernels and envelopes: A concept shift for robotics, developmental psychology, and brain sciences. In Krichmar, J. L. & Wagatsuma, H. (Eds.), Neuromorphic and brain-based robots (pp. 217–50). Cambridge: Cambridge University Press. doi: 10.1017/cbo9780511994838.011.
Keller, J. & Bless, H. (2008). Flow and regulatory compatibility: An experimental approach to the flow model of intrinsic motivation. Personality and Social Psychology Bulletin, 34(2), 196209. doi: 10.1177/0146167207310026.
Laurent, P. A. (2008). The emergence of saliency and novelty responses from reinforcement learning principles. Neural Networks, 21(10), 1493–9. doi: 10.1016/j.neunet.2008.09.004.
Litman, J. A. (2005). Curiosity and the pleasures of learning: Wanting and liking new information. Cognition & Emotion, 19(6), 793814. doi: 10.1080/026999 30541000101.
Litman, J. A. (2007). Curiosity as a feeling of interest and feeling of deprivation: The I/D model of curiosity. Issues in the Psychology of Motivation, 149–56. doi: 10.1037/t27877-000.
Litman, J. A. (2008). Interest and deprivation factors of epistemic curiosity. Personality and Individual Differences, 44(7), 1585–95. doi: 10.1016/j.paid.2008.01.014.
Loewenstein, G. (1994). The psychology of curiosity: A review and reinterpretation. Psychological Bulletin, 116(1), 75. doi: 10.1037//0033-2909.116.1.75.
Lopes, M. & Oudeyer, P. Y. (2012). The strategic student approach for life-long exploration and learning. In 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL) (pp. 18). IEEE. doi: 10.1109/devlrn.2012.6400807.
Mather, M. & Carstensen, L. L. (2003). Aging and attentional biases for emotional faces. Psychological Science, 14(5), 409–15. doi: 10.1111/1467-9280.01455.
Maunsell, J. H. (2004). Neuronal representations of cognitive state: reward or attention? Trends in Cognitive Sciences, 8(6), 261–5. doi: 10.1016/j.tics.2004.04.003.
Mazurek, M. E., Roitman, J. D., Ditterich, J., & Shadlen, M. N. (2003). A role for neural integrators in perceptual decision making. Cerebral Cortex, 13(11), 1257–69. doi: 10.1093/cercor/bhg097.
Moulin-Frier, C. & Oudeyer, P. Y. (2012). Curiosity-driven phonetic learning. In 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL) (pp. 18). IEEE. doi: 10.1109/devlrn.2012.6400583.
Mullaney, K. M., Carpenter, S. K., Grotenhuis, C., & Burianek, S. (2014). Waiting for feedback helps if you want to know the answer: The role of curiosity in the delay-of-feedback benefit. Memory & Cognition, 42(8), 1273–84. doi: 10.3758/s13421-014-0441-y.
Najemnik, J. & Geisler, W. S. (2005). Optimal eye movement strategies in visual search. Nature, 434(7031), 387. doi: 10.1016/j.ajo.2005.04.009.
Navalpakkam, V. & Itti, L. (2005). Modeling the influence of task on attention. Vision Research, 45(2), 205–31. doi: 10.1016/j.visres.2004.07.042.
Ngo, H., Luciw, M., Forster, A., & Schmidhuber, J. (2012). Learning skills from play: artificial curiosity on a katana robot arm. In 2012 International Joint Conference on Neural Networks (IJCNN) (pp. 18). IEEE. doi: 10.1109/ijcnn.2012.6252824.
Nguyen, S.M. & Oudeyer, P. Y. (2013). Socially guided intrinsic motivation for robot learning of motor skills. Autonomous Robots, 36(3), 273–94. doi: 10.1007/s10514-013-9339-y.
Norman, D. A. & Shallice, T. (1986). Attention to action. In Consciousness and self-regulation (pp. 118). NYC, NY: Springer US. doi: 10.1007/978-1-4757-0629-1_1.
Oudeyer, P. Y., Baranes, A., & Kaplan, F. (2013). Intrinsically motivated learning of real-world sensorimotor skills with developmental constraints. In M. Mirolli & G. Baldassarre (Eds.), Intrinsically motivated learning in natural and artificial systems (pp. 303–65). Berlin Heidelberg: Springer. doi: 10.1007/978-3-642-32375-1_13.
Oudeyer, P. Y., Gottlieb, J., & Lopes, M. (2016). Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies. Progress in Brain Research, 229, 257–84. doi: 10.1016/bs.pbr.2016.05.005.
Oudeyer, P. Y. & Kaplan, F. (2006). Discovering communication. Connection Science, 18(2), 189206. doi: 10.1080/09540090600768567.
Oudeyer, P. Y., Kaplan, F., & Hafner, V. V. (2007). Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation, 11(2), 265–86. doi: 10.1109/TEVC.2006.890271.
Pape, L., Oddo, C. M., Controzzi, M., Cipriani, C., Förster, A., Carrozza, M. C. & Schmidhuber, J. (2012). Learning tactile skills through curious exploration. Frontiers in Neurorobotics, 6. doi: 10.3389/fnbot.2012.00006.
Payzan-LeNestour, E. & Bossaerts, P. (2011). Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Computational Biology, 7(1), e1001048. doi: 10.1371/journal.pcbi.1001048.
Peck, C. J., Jangraw, D. C., Suzuki, M., Efem, R., & Gottlieb, J. (2009). Reward modulates attention independently of action value in posterior parietal cortex. Journal of Neuroscience, 29(36), 11182–91. doi: 10.1523/jneurosci.1929-09.2009.
Pessoa, L., Kastner, S., & Ungerleider, L. G. (2003). Neuroimaging studies of attention: From modulation of sensory processing to top-down control. Journal of Neuroscience, 23(10), 3990–8. doi: 0270-6474/03/233990-09.00/0.
Platt, M. L. & Glimcher, P. W. (1999). Neural correlates of decision variables in parietal cortex. Nature, 400(6741), 233.
Posner, M. I. (1994). Attention: The mechanisms of consciousness. Proceedings of the National Academy of Sciences of the United States of America, 91(16), 7398–403. doi: 10.1073/pnas.91. 16.7398.
Posner, M. I. & Dehaene, S. (1994). Attentional networks. Trends in Neurosciences, 17(2), 75–9. doi: 10.1016/0166-2236(94)90078-7.
Renninger, K. A. & Hidi, S. (2011). Revisiting the conceptualization, measurement, and generation of interest. Educational Psychologist, 46(3), 168–84. doi: 10.1080/00461520.2011.587723.
Renninger, K. A. & Hidi, S. & Hidi, S. (2016). The power of interest for motivation and learning. New York, NY; Routledge.
Renninger, L. W., Verghese, P., & Coughlan, J. (2007). Where to look next? Eye movements reduce local uncertainty. Journal of Vision, 7(3), 6. doi: 10.1167/7.3.6.
Risko, E. F., Anderson, N. C., Lanthier, S., & Kingstone, A. (2012). Curious eyes: Individual differences in personality predict eye movement behavior in scene-viewing. Cognition, 122(1), 8690. doi: 10.1016/j.cognition.2011.08.014.
Ryan, R. M. & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 5467. doi: 10.1006/ceps.1999.1020.
Sarter, M., Gehring, W. J., & Kozak, R. (2006). More attention must be paid: The neurobiology of attentional effort. Brain Research Reviews, 51(2), 145–60. doi: 10.1016/j.brainresrev.2005.11.002.
Satterthwaite, T. D., Ruparel, K., Loughead, J., Elliot, M. A., Gerraty, R. T., Calkins, M. E., ... Wolf, D. H. (2012). Being right is its own reward: load and performance related ventral striatum activation to correct responses during a working memory task in youth. Neuroimage, 61(3), 723–29.
Schmidhuber, J. (2006). Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science, 18(2), 173–87. doi: 10.1080/09540090600768658.
Schmidhuber, J. (2013). Maximizing fun by creating data with easily reducible subjective complexity. In Mirolli, M. & Baldassarre, G. (Eds.), Intrinsically motivated learning in natural and artificial systems (pp. 95128). Berlin Heidelberg: Springer. doi: 10.1007/978-3-642-32375-1_5.
Singh, S., James, M. R., & Rudary, M. R. (2004). Predictive state representations: A new theory for modeling dynamical systems. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (pp. 512–19). AUAI Press.
Singh, S., Lewis, R. L., Barto, A. G., & Sorg, J. (2010). Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Autonomous Mental Development, 2(2), 7082. doi: 10.1109/TAMD.2010.2051031.
Srivastava, R. K., Steunebrink, B. R., & Schmidhuber, J. (2013). First experiments with PowerPlay. Neural Networks, 41, 130–36. doi: 10.1016/j.neunet.2013.01.022.
Steels, L. (2004). The autotelic principle. Lecture Notes in Computer Science, (3139), 231–42. doi: 10.1007/978-3-540-27833-7_17.
Stuss, D. T., Shallice, T., Alexander, M. P., & Picton, T. W. (1995). A multidisciplinary approach to anterior attentional functions. Annals of the New York Academy of Sciences, 769(1), 191212. doi: 10.1111/j.1749-6632.1995.tb38140.x.
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2005). Choosing the greater of two goods: neural currencies for valuation and decision making. Nature Reviews. Neuroscience, 6(5), 363. doi: 10.1038/nrn1666.
Sullivan, B. T., Johnson, L., Rothkopf, C. A., Ballard, D., & Hayhoe, M. (2012). The role of uncertainty and reward on eye movements in a virtual driving task. Journal of Vision, 12(13), 19. doi: 10.1167/12.13.19.
Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011). Eye guidance in natural vision: Reinterpreting salience. Journal of Vision, 11(5), 5. doi: 10.1167/11.5.5.
Thorndike, E. L. (1911). Animal intelligence. New York: Macmillan.
Tsotsos, J. K. (2011). A computational perspective on visual attention. Cambridge, MA: MIT Press. doi: 10.7551/mitpress/9780262015417.001.0001.
Ullsperger, M. & von Cramon, D. Y. (2004). Neuroimaging of performance monitoring: Error detection and beyond. Cortex, 40(4), 593604. doi: 10.1016/s0010-9452(08)70155-2.
van Duijvenvoorde, A. C., Peters, S., Braams, B. R., & Crone, E. A. (2016). What motivates adolescents? Neural responses to rewards and their influence on adolescents’ risk taking, learning, and cognitive control. Neuroscience & Biobehavioral Reviews, 70, 135–47. doi: 10.1016/j.neubiorev.2016.06.037.
Vossel, S., Mathys, C., Daunizeau, J., Bauer, M., Driver, J., & Friston, K. J., & Stephan, K. E. (2013). Spatial attention, precision, and Bayesian inference: A study of saccadic response speed. Cerebral Cortex, 24(6), 1436–450. doi: 10.1093/cercor/bhs418.
Vossel, S., Mathys, C., Stephan, K. E., & Friston, K. J. (2015). Cortical coupling reflects Bayesian belief updating in the deployment of spatial attention. Journal of Neuroscience, 35(33), 11532–42. doi: 10.1523/jneurosci.1382-15.2015.
Vossel, S., Thiel, C. M., & Fink, G. R. (2006). Cue validity modulates the neural correlates of covert endogenous orienting of attention in parietal and frontal cortex. Neuroimage, 32(3), 1257–64. doi: 10.1016/j.neuroimage.2006.05.019.
Wang, X. J. (2008). Decision making in recurrent neuronal circuits. Neuron, 60(2), 215–34. doi: 10.1016/j.neuron.2008.09.034.
Wise, R. A. (2004). Dopamine, learning and motivation. Nature Reviews Neuroscience, 5(6), 483.
Wittmann, B. C., Bunzeck, N., Dolan, R. J., & Düzel, E. (2007). Anticipation of novelty recruits reward system and hippocampus while promoting recollection. Neuroimage, 38(1), 194202. doi: 10.1016/j.neuroimage.2007.06.038.
Wittmann, B. C., Daw, N. D., Seymour, B., & Dolan, R. J. (2008). Striatal activity underlies novelty-based choice in humans. Neuron, 58(6), 967–73. doi: 10.1016/j.neuron.2008.04.027.
Yang, H., Chen, X., & Zelinsky, G. J. (2009). A new look at novelty effects: Guiding search away from old distractors. Attention, Perception, & Psychophysics, 71(3), 554–64. doi: 10.3758/app.71.3.554.
Yang, S. C. H., Lengyel, M., & Wolpert, D. M. (2016). Active sensing in the categorization of visual patterns. eLife, 5, e12215. doi: 10.7554/elife.25660.
Yang, T. & Shadlen, M. N. (2007). Probabilistic reasoning by neurons. Nature, 447(7148), 1075. doi: 10.1038/nature05852.
Yu, A.J. & Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46(4), 681–92. doi: 10.1016/j.neuron.2005.04.026.