Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-8448b6f56d-42gr6 Total loading time: 0 Render date: 2024-04-19T21:36:53.808Z Has data issue: false hasContentIssue false

8 - The operant/respondent distinction: a computational neural-network analysis

Published online by Cambridge University Press:  10 January 2011

Nestor Schmajuk
By
José E. Burgos
Nestor Schmajuk
Affiliation:
Duke University Medical Center, Durham
Get access

Summary

This chapter presents an analysis of the distinction between operant (instrumental) and respondent (Pavlovian, classical) conditioning, in terms of a computational, neural-network model. The importance of this distinction cannot be overemphasized, judging by the extensive literature on it (e.g., Asratyan, 1974; Bindra, 1972; Davis & Hurwitz, 1977; Dykman, 1976; Gormezano & Tait, 1976; Gray, 1975; Guthrie, 1935; Hearst, 1975; Henton & Iversen, 1978; Hineline, 1986; Hull, 1943; Kimmel, 1976; Konorski & Miller, 1937a, 1937b; Logan, 1960; Mackintosh & Dickinson, 1979; Miller & Konorski, 1928; Mowrer, 1960; Pear & Eldrige, 1984; Ray & Brown, 1976; Rehfeldt & Hayes, 1998; Rescorla & Solomon, 1967; Schlosberg, 1937; Schoenfeld, 1966; Sheffield, 1965; Skinner, 1935, 1937; Spence, 1956; Trapold & Overmier, 1972). Obviously, this abundance of analyses is too long to summarize in a way that does them justice. Instead, I will begin with how the distinction has been treated in the field of computational modeling of conditioning.

A glance at the field reveals that most models are of respondent conditioning (e.g., Gibbon & Balsam, 1981; Klopf, 1988; Mackintosh, 1975; Pearce & Hall, 1980; Rescorla & Wagner, 1972; Schmajuk & Moore, 1986; Stout & Miller, 2007; Sutton & Barto, 1981; Wagner & Brandon 1989), or operant conditioning (e.g., Dragoi & Staddon, 1999; Killeen, 1994; Machado, 1997), with little if any communication between the two types of models. The field is thus deeply divided. If there is any validity to the motto “United we stand, divided we fall,” the field has fallen long ago.

Type
Chapter
Information
Computational Models of Conditioning , pp. 244 - 271
Publisher: Cambridge University Press
Print publication year: 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Asratyan, E. A. (1972). Conditioned reflex theory and motivational behavior. Acta Neurobiologiae Experimentalis, 34, 15–31.Google Scholar
Bindra, D. (1972). A unified account of classical and operant conditioning. In Black, A. H. and Prokasy, W. F., eds., Classical Conditioning ii: Current Research and Theory. New York: Appleton–Century–Crofts, pp. 453–481.Google Scholar
Brown, P. L. & Jenkins, H. M. (1968). Auto-shaping of the pigeon's key-peck. Journal of the Experimental Analysis of Behavior, 11, 1–8.CrossRefGoogle ScholarPubMed
Burgos, J. E. (1997). Evolving artifical neural networks in Pavlovian environments. In Donahoe, J. W. and Dorsel, V. P., eds., Neural-Network Models of Cognition: Biobehavioral Foundations. Amsterdam: Elsevier, pp. 58–79.Google Scholar
Burgos, J. E. (2003). Theoretical note: simulating latent inhibition with selection neural networks. Behavioural Processes, 62, 183–192.CrossRefGoogle ScholarPubMed
Burgos, J. E. (2005). Theoretical note: the C/T ratio in artificial neural networks. Behavioural Processes, 69, 249–256.CrossRefGoogle Scholar
Burgos, J. E. (2007). Autoshaping and automaintenance: a neural-network approach. Journal of the Experimental Analysis of Behavior, 88, 115–130.CrossRefGoogle ScholarPubMed
Burgos, J. E. & Donahoe, J. W. (2000). Structure and function in selectionism: implications for complex behavior. In Leslie, J. and Blackman, D., eds., Issues in Experimental and Applied Analyses of Human Behavior. Reno, NV: Context Press, pp. 39–57.Google Scholar
Burgos, J. E. & Murillo-Rodríguez, E. (2007). Neural-network simulations of two context-dependence phenomena. Behavioural Processes, 75, 242–249.CrossRefGoogle ScholarPubMed
Burgos, J. E., Flores, C., García, Ó., Díaz, C. & Cruz, Y. (2007). A simultaneous procedure facilitates acquisition under an optimal interstimulus interval in artificial neural networks and rats. Behavioural Processes, 78, 302–309.CrossRefGoogle Scholar
Davis, H. & Hurwitz, H. M. B. (eds., 1977). Operant-Pavlovian Interactions. Hillsdale, NJ: Erlbaum.Google Scholar
Dickinson, A. (1979). Review of Hulse, S. H., Fowler, H. & Honig, W. K. (eds.). Cognitive processes in animal behavior. Quarterly Journal of Experimental Psychology, 31, 551–554.Google Scholar
Donahoe, J. W. & Burgos, J. E. (1999). Commentary: timing without a timer. Journal of the Experimental Analysis of Behavior, 71, 257–301.CrossRefGoogle ScholarPubMed
Donahoe, J. W. & Burgos, J. E. (2000). Behavior analysis and revaluation. Journal of the Experimental Analysis of Behavior, 74, 331–346.CrossRefGoogle ScholarPubMed
Donahoe, J. W. & Palmer, D. C. (1994). Learning and Complex Behavior. Boston, MA: Allyn & Bacon.Google Scholar
Donahoe, J. W., Burgos, J. E. & Palmer, D. C. (1993). A selectionist approach to reinforcement. Journal of the Experimental Analysis of Behavior, 60, 17–40.CrossRefGoogle ScholarPubMed
Donahoe, J. W., Palmer, D. C. & Burgos, J. E. (1997a). The S-R issue: its status in behavior analysis and in Donahoe and Palmer's Learning and Complex Behavior. Journal of the Experimental Analysis of Behavior, 67, 193–211.CrossRefGoogle ScholarPubMed
Donahoe, J. W., Palmer, D. C. & Burgos, J. E. (1997b). The unit of selection: what do reinforcers reinforce?Journal of the Experimental Analysis of Behavior, 67, 259–273.CrossRefGoogle ScholarPubMed
Dragoi, V. & Staddon, J. E. R. (1999). The dynamics of operant conditioning.Psychological Review, 106, 20–61.CrossRefGoogle ScholarPubMed
Dykman, R. A. (1976). Conditioning and sensitization. Pavlovian Journal of Biological Science, 11, 24–36.Google ScholarPubMed
Fukushima, K. (1975). Cognitron: a self-organizing multilayered neural network. Biological Cybernetics, 20, 126–136.CrossRefGoogle ScholarPubMed
Gibbon, J. & Balsam, P. (1981). Spreading association in time. In Locurto, C. M., Terrace, H. S. and Gibbon, J., eds., Autoshaping and Conditioning Theory. New York: Academic Press, pp. 219–253.Google Scholar
Gonzalez, F. A. (1974). Effects of varying the percentage of key illuminations paired with food in a positive automaintenance procedure. Journal of the Experimental Analysis of Behavior, 22, 483–489.CrossRefGoogle Scholar
Gormezano, I. & Tait, R. W. (1976). The Pavlovian analysis of instrumental conditioning. Pavlovian Journal of Biological Science, 11, 37–55.Google ScholarPubMed
Gray, J. A. (1975). Elements of a Two-Process Theory of Learning. London: Academic Press.Google Scholar
Grossberg, S. (1987). The Adaptive Brain i. Amsterdam: Elsevier.Google Scholar
Guthrie, E. R. (1935). The Psychology of Learning. New York: Harper & Row.Google Scholar
Hearst, E. (1975). The classical-instrumental distinction: reflexes, voluntary behavior, and categories of associative learning. In Estes, W. K., ed., Handbook of Learning and Cognitive Processes: Vol. 2. Conditioning and Behavior Theory. Hillsdale, NJ: Erlbaum, pp. 181–223.Google Scholar
Hearst, E. & Jenkins, H. M. (1974). Sign-Tracking: The Stimulus–Reinforcer Relation and Directed Action. Monograph of the Psychonomic Society: Austin, TX.Google Scholar
Henton, W. W. & Iversen, I. H. (1978). Classical and Operant Conditioning: A Response Pattern Analysis. New York: Springer-Verlag.CrossRefGoogle Scholar
Hineline, P. N. (1986). Re-tuning the operant/respondent distinction. In Thompson, T. and Zeiler, M. D., eds., Analysis and Integration of Behavioral Units. Hillsdale, NJ: Erlbaum, pp. 55–79.Google Scholar
Hull, C. L. (1943). Principles of Behavior. New York: Appleton.Google Scholar
Killeen, P. R. (1994). Mathematical principles of reinforcement. Behavioral and Brain Sciences, 17, 105–172.CrossRefGoogle Scholar
Kimmel, H. D. (1976). Cherchez la difference!The Pavlovian Journal of Biological Science, 11, 56–65.Google ScholarPubMed
Klopf, A. H. (1988). A neuronal model of classical conditioning. Psychobiology, 16, 85–125.Google Scholar
Konorski, J. & Miller, S. (1937a). On two types of conditioned reflex. Journal of General Psychology, 16, 264–272.CrossRefGoogle Scholar
Konorski, J. & Miller, S. (1937b). Further remarks on two types of conditioned reflex. Journal of General Psychology, 17, 405–407.CrossRefGoogle Scholar
Logan, F. A. (1960). Incentive. New Haven, CT: Yale University Press.Google Scholar
Machado, A. (1997). Learning the temporal dynamics of behavior. Psychological Review, 104, 241–265.CrossRefGoogle ScholarPubMed
Mackintosh, N. J. (1975). A theory of attention: variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298.CrossRefGoogle Scholar
Mackintosh, N. J. & Dickinson, A. (1979). Instrumental (Type II) conditioning. In Dickinson, A. and Boakes, R. A., eds., Mechanisms of Learning and Motivation: A Memorial Volume to Jerzy Konorski. Hillsdale, NJ: Erlbaum, pp. 143–169.Google Scholar
Miller, S. & Konorski, J. (1928). Sur une forme particulière des reflexes conditionnels. Les Compte Rendus des Seances et Mémoires de la Société Polonaise de Biologie, 99, 1155–1157. Translated by Skinner, B. F. (1969) as “On a particular form of conditioned reflex.” Journal of the Experimental Analysis of Behavior, 12, 187–189.Google Scholar
Mowrer, H. O. (1960). Learning Theory and Behavior. New York: Wiley.CrossRefGoogle Scholar
Myer, J. S. (1971). Some effects of noncontingent aversive stimulation. In Brush, F. R., ed., Aversive Conditioning and Learning. New York: Academic Press, pp. 469–536.Google Scholar
Nutan, K. S. & Meti, B. L. (2000). Deficits in operant behavior and alteration of CA1, CA3 hippocampal dendritic arborization due to subicular lesions. Journal of Neuroscience Research, 59, 806–812.3.0.CO;2-2>CrossRefGoogle ScholarPubMed
Olds, J. & Milner, P. (1954). Positive reinforcement produced by electrical stimulation of the septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology, 47, 419–427.CrossRefGoogle ScholarPubMed
Pan, W-.X., Schmidt, R., Wickens, J. R. & Hyland, B. I. (2005). Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. The Journal of Neuroscience, 25, 6235–6242.CrossRefGoogle ScholarPubMed
Pear, J. J. & Eldridge, G. D. (1984). The operant/respondent distinction: future directions. Journal of the Experimental Analysis of Behavior, 42, 453–467.CrossRefGoogle Scholar
Pearce, J. M. & Hall, G. (1980). A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532–552.CrossRefGoogle Scholar
Ray, R. D. & Brown, D. A. (1976). The behavioral specificity of behavior: a systems approach to procedural distinctions of classical and instrumental conditioning. The Pavlovian Journal of Biological Science, 11, 3–23.Google ScholarPubMed
Rehfeldt, R. A. & Hayes, L. J. (1998). The operant/respondent distinction revisited: toward an understanding of stimulus equivalence. The Psychological Record, 48, 187−210.CrossRefGoogle Scholar
Rescorla, R. W. & Solomon, R. L. (1967). Two-process learning theory: relationships between Pavlovian conditioning and instrumental learning. Psychological Review, 74, 151–182.CrossRefGoogle ScholarPubMed
Rescorla, R. W. & Wagner, A. R. (1972). A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and noneinforcement. In Black, A. H. and Prokasy, W. F., eds., Classical Conditioning ii: Current Research and Theory. New York: Appleton, pp. 64–99.Google Scholar
Sánchez, J. M., Galeazzi, J. M. & Burgos, J. E. (2010). Some structural determinants of Pavlovian conditioning in artificial neural networks. Behavioural Processes, 84, 526–535.CrossRefGoogle ScholarPubMed
Schlosberg, H. (1937). The relationship between success and the laws of conditioning. Psychological Review, 44, 379−394.CrossRefGoogle Scholar
Schmajuk, N. A. (1994). Behavioral dynamics of escape and avoidance: a neural network approach. In Cliff, D., Husbands, P., Meyerm, J.-A. and Wilson, S., eds., From Animals to Animats 3. Cambridge, MA: MIT Press, pp. 118–127.Google Scholar
Schmajuk, N. A. & Moore, J. W. (1986). A real-time attentional-associative model for classical conditioning of the rabbit's nictitating membrane response. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum, pp. 794–807.Google Scholar
Schoenfeld, W. N. (1966). Editorial: some old work for modern conditioning theory. Conditional Reflex: A Pavlovian Journal of Research and Therapy, 1, 219−223.Google Scholar
Schwartz, B. & Gamzu, E. (1977). Pavlovian control of operant behavior: an analysis of autoshaping and its implications for operant conditioning. In Honig, W. K. and Staddon, J. E. R., eds., Handbook of Operant Behavior. Englewood Cliffs, NJ: Prentice-Hall, pp. 53–97.Google Scholar
Sharf, R., Lee, D. Y. & Ranaldi, R. (2005). Microinjections of SCH 23390 in the ventral tegmental area reduce operant responding under a progressive ratio schedule of food reinforcement in rats. Brain Research, 1033, 179–185.CrossRefGoogle Scholar
Sheffield, F. D. (1965). Relation between classical conditioning and instrumental learning. In Prokasy, W. F., ed., Classical Conditioning: A Symposium. New York: Appleton, pp. 302–322.Google Scholar
Skinner, B. F. (1935). Two types of conditioned reflex and a pseudo type. Journal of General Psychology, 12, 66–77.CrossRefGoogle Scholar
Skinner, B. F. (1937). Two types pf conditioned reflex: a reply to Konorski and Miller. Journal of General Psychology, 16, 272–279.CrossRefGoogle Scholar
Spence, K. W. (1956). Behavior Theory and Conditioning. New Haven, CT: Yale University Press.CrossRefGoogle Scholar
Staddon, J. E. R. (2001). The New Behaviorism: Mind, Mechanism, and Society. Philadelphia, PA: Taylor & Francis.Google Scholar
Stiers, M. & Silberberg, A. (1974). Lever-contact responses in rats: automaintenance with and without a negative response-reinforcer dependency. Journal of the Experimental Analysis of Behavior, 22, 497–506.CrossRefGoogle ScholarPubMed
Stout, S. C. & Miller, R. R. (2007). Sometimes competing retrieval (SOCR): a formalization of the comparator hypothesis. Psychological Review, 114, 759–783.CrossRefGoogle ScholarPubMed
Sutton, R. S. & Barto, A. G. (1981). Toward a modern theory of adaptive networks. Psychological Review, 88, 135–170.CrossRefGoogle Scholar
Trapold, M. A. & Overmier, J. B. (1972). The second learning process in instrumental learning. In Black, A. H. and Prokasy, W. F., eds., Classical Conditioning ii: Current Research and Theory. New York: Appleton, pp. 427–452.Google Scholar
Tronson, N. C., Schrick, C., Guzman, Y. F., Huh, K. H., Srivastava, D. P., Penzes, et al. (2009). Segregated populations of hippocampal principal CA1 neurons mediating conditioning and extinction of contextual fear. The Journal of Neuroscience, 29, 3387–3394.CrossRefGoogle ScholarPubMed
Wagner, A. R. & Brandon, S. E. (1989). Evolution of a structured connectionist model of Pavlovian conditioning (AESOP). In Klein, S. B. and Mowrer, R. R., eds., Contemporary Learning Theories: Pavlovian Conditioning and the Status of Traditional Learning Theory. Hillsdale, NJ: Erlbaum, pp. 149–189.Google Scholar
Wurtz, R. H. & Olds, J. (1963). Amygdaloid stimulation and operant reinforcement in the rat. Journal of Comparative and Physiological Psychology, 56, 941–949.CrossRefGoogle ScholarPubMed
Yun, I. A., Wakabayashi, K. T., Fields, H. L. & Nicola, S. M. (2004). The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. The Journal of Neuroscience, 24, 2923–2933.CrossRefGoogle ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×