Book contents
- Frontmatter
- Contents
- Contributors
- Introduction: Modelling perception with artificial neural networks
- Part I General themes
- Part II The use of artificial neural networks to elucidate the nature of perceptual processes in animals
- 3 Correlation versus gradient type motion detectors: the pros and cons
- 4 Spatial constancy and the brain: insights from neural networks
- 5 The interplay of Pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat
- 6 Evolution, (sequential) learning and generalisation in modular and nonmodular visual neural networks
- 7 Effects of network structure on associative memory
- 8 Neural networks and neuro-oncology: the complex interplay between brain tumour, epilepsy and cognition
- Part III Artificial neural networks as models of perceptual processing in ecology and evolutionary biology
- Part IV Methodological issues in the use of simple feedforward networks
- Index
- References
5 - The interplay of Pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat
from Part II - The use of artificial neural networks to elucidate the nature of perceptual processes in animals
Published online by Cambridge University Press: 05 July 2011
Book contents
- Frontmatter
- Contents
- Contributors
- Introduction: Modelling perception with artificial neural networks
- Part I General themes
- Part II The use of artificial neural networks to elucidate the nature of perceptual processes in animals
- 3 Correlation versus gradient type motion detectors: the pros and cons
- 4 Spatial constancy and the brain: insights from neural networks
- 5 The interplay of Pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat
- 6 Evolution, (sequential) learning and generalisation in modular and nonmodular visual neural networks
- 7 Effects of network structure on associative memory
- 8 Neural networks and neuro-oncology: the complex interplay between brain tumour, epilepsy and cognition
- Part III Artificial neural networks as models of perceptual processing in ecology and evolutionary biology
- Part IV Methodological issues in the use of simple feedforward networks
- Index
- References
Summary
Introduction
The flexibility and capacity of adaptation of organisms greatly depends on their learning capabilities. For this reason, animal psychology has devoted great efforts to the study of learning processes. In particular, in the last century a huge body of empirical data has been collected around the two main experimental paradigms of ‘classical conditioning’ (Pavlov, 1927; Lieberman, 1993) and ‘instrumental conditioning’ (Thorndike, 1911; Skinner, 1938; Balleine et al., 2003; Domjan, 2006).
Classical conditioning (also called ‘Pavlovian conditioning’) refers to an experimental paradigm in which a certain basic behaviour such as salivation or approaching (the ‘unconditioned response’ – UR), which is linked to a biologically salient stimulus such as food ingestion (the ‘unconditioned stimulus’ – US), becomes associated to a neutral stimulus like the sound of a bell (the ‘conditioned stimulus’ – CS), after the neutral stimulus is repeatedly presented before the appearance of the salient stimulus. Such acquired associations are referred to as ‘CS-US’ or ‘CS-UR’ associations (Pavlov, 1927; Lieberman, 1993).
Instrumental conditioning (also called ‘operant conditioning’) refers to an experimental paradigm in which an animal, given a certain stimulus/context such as a lever in a cage (the ‘stimulus’ – S), learns to produce a particular action such as pressing the lever (the ‘response’ – R), which produces a certain outcome such as the opening of the cage (the ‘action outcome’ – O), if this outcome is consistently accompanied by a reward such as the access to food.
- Type
- Chapter
- Information
- Modelling Perception with Artificial Neural Networks , pp. 93 - 113Publisher: Cambridge University PressPrint publication year: 2010
References
, , & 1997. Stimulus generalization of fear responses: effects of auditorycortexlesions in a computational model and in rats. Cereb Cortex 7(2), 157–165.CrossRefGoogle Scholar
2008. Self-organization as phase transition in decentralized groups of robots: a study based on Boltzmann entropy. In Advances in Applied Self-Organizing Systems (ed. M. Prokopenko), pp. 127–146. Springer-Verlag.Google Scholar
, & 2003. The effect of lesions of the basolateral amygdala on instrumental conditioning. J Neurosci 23(2), 666–675.CrossRefGoogle ScholarPubMed
& 2006. Parallel incentive processing: an integrated view of amygdala function. Trends Neurosci 29(5), 272–279.CrossRefGoogle ScholarPubMed
, & 2004. Intrinsically motivated learning of hierarchical collections of skills. In International Conference on Developmental Learning (ICDL), LaJolla, CA.Google Scholar
, , & 2005. The lateral amygdala processes the value of conditioned and unconditioned aversive stimuli. Neuroscience 133(2), 561–569.CrossRefGoogle ScholarPubMed
, , & (Eds.) 2004. Abstracts of Papers Presented at the 2004 Meeting on Computational & System Neuroscience. Cold Spring Harbor Laboratory.Google Scholar
, , , , & (Ed.) 2001. Self-organization in Biological Systems. Princeton University Press.Google Scholar
, , & 2002. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26(3), 321–352.CrossRefGoogle ScholarPubMed
2007. Cortical mechanisms of action selection: the affordance competition hypothesis. Phil Trans R Soc B 362(1485), 1585–1599.CrossRefGoogle ScholarPubMed
1859. The Origin of Species. Retrieved from http://www.literature.org/authors/darwincharles/the-origin-of-species/index.html.
& 2002. Reward, motivation and reinforcement learning. Neuron 36, 285–298.CrossRefGoogle ScholarPubMed
, & 2000. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20(6), 2369–2382.CrossRefGoogle ScholarPubMed
, & 1995. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In Models of Information Processing in the Basal Ganglia (ed. , & D. G. Beiser), pp. 249–270. MIT Press.Google Scholar
, & 2005. The role of the human amygdala in the production of conditioned fear responses. Neuroimage 26(4), 1193–1200.CrossRefGoogle ScholarPubMed
& 2007. Reward prediction error computation in the pedunculopontine tegmental nucleus neurons. Ann N Y Acad Sci 1104, 310–323.
(Ed.) 1987. The First International Conference on the Simulation and Synthesis of Living Systems (ALifeI).
, & 2007. The role of amygdala in devaluation: a model tested with a simulated robot. In Proceedings of the Seventh International Conference on Epigenetic Robotics (ed. , , , & ), pp. 77–84. University of Lund.Google Scholar
, & 2008. A computational model of the amygdala nuclei's role in second order conditioning. In Proceedings of the Tenth International Conference on Simulation of Adaptive Behavior: From Amimals to Animals 10 (ed. , & ). Springer-Verlag.Google Scholar
2005. Building and burying fear memories in the brain. Neuroscientist 11(1), 89–99.CrossRefGoogle Scholar
1998. Cortical pathways to the mammalian amygdala. Prog Neurobiol 55(3), 257–332.CrossRefGoogle ScholarPubMed
& (Ed.) 1991. From Animals to Animats 1: Proceedings of the First International Conference on Simulation of Adaptive Behaviour. MIT Press.
, & 1980. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 14(2–3), 69–97.CrossRefGoogle ScholarPubMed
& 2000. A computational model of emotional learning in the amygdala. In From Animals to Animats 6: Proceedings of the 6th International Conference on the Simulation of Adaptive Behaviour (ed. , , , & ). MIT Press.Google Scholar
1973. You can't play 20 questions with nature and win: projective comments on the papers of this symposium. In Visual Information Processing (ed. ), pp. 283–308. Academic Press.
, , & 2007. Tonic dopamine: opportunity costs and the control of response vigor. J Psychopharmacol 191(3), 507–520.CrossRefGoogle ScholarPubMed
2006. Behaviour as a complex adaptive system: on the role of self-organization in the development of individual and collective behaviour. ComplexUs 2(3–4), 195–203.CrossRefGoogle Scholar
, , & 2007. PVLV: The primary value and learned value pavlovian learning algorithm. Behav Neurosci 121, 31–49.CrossRefGoogle ScholarPubMed
& 2002. Learning and memory functions of the basal ganglia. Annu Rev Neurosci 25, 563–593.CrossRefGoogle ScholarPubMed
, & 1990. Econets: Neural networks that learn in an environment. Network 1, 149–168.CrossRefGoogle Scholar
1927. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press.Google Scholar
, & 2000. Anatomic heterogeneity of the rat amygdaloid complex. Folia Morphol. (Warsz) 59(1), 1–23.Google ScholarPubMed
, , & 2003. Towards a methodology for embodied computational neuroscience. In Proceedings of the Symposium on Scientific Methods for the Analysis of Agent-Environment Interaction (AISB2003). AISB Press.Google Scholar
, , , & 2006. A robot model of the basal ganglia: behavior and intrinsic processing. Neural Netw 19(1), 31–61.CrossRefGoogle ScholarPubMed
2003. Comparative aspects of amygdala connectivity. Ann N Y Acad Sci 985(1), 50–58.CrossRefGoogle ScholarPubMed
, & 1999. The basal ganglia: a vertebrate solution to the selection problem?J Neurosci 89(4), 1009–1023.CrossRefGoogle ScholarPubMed
2005. Taste and related systems in primates including humans. Chem Senses 30 Suppl. 1, i76–i77.CrossRefGoogle ScholarPubMed
, & 2007. Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot. In Proceedings of the 6th International Conference on Development and Learning (ICDL) (ed. , , & ), pp. E1–6. Imperial College London.Google Scholar
& 1999. Pain pathways involved in fear conditioning measured with fear potentiated startle: lesion studies. J Neurosci 19(1), 420–430.CrossRefGoogle ScholarPubMed
& 1981. Toward a modern theory of adaptive networks: Expectation and prediction. Psychol Rev 88, 135–140.CrossRefGoogle Scholar
, , et al. 2001. Autonomous mental development by robots and animals. Science 291, 599–600.CrossRefGoogle ScholarPubMed
& 2006. The role of the basal ganglia in habit formation. Nat Rev Neurosci 7, 464–476.CrossRefGoogle ScholarPubMed
& 2001. Introduction: Why epigenetic robotics? In Proceedings of the First International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems (ed. , , , & ), University of Lund. pp. 1–4.Google Scholar
- 8
- Cited by