In the history of the scientific study of learning and memory, a number of terms have been used to describe abstract experimental paradigms that purport to represent different kinds of learning and to reflect the distinct associative mechanisms that underlie them. We will have occasion in this book to refer to several of these experimental paradigms, but the most inclusive ones are classical (Pavlovian) conditioning and instrumental (Thorndikian) learning. As we shall see, these two paradigms are central to understanding the phenomena I call the reward-schedule effects. These phenomena of learning and memory depend on a variety of sequences of reward and nonreward, and are the basis for a family of generalizations known as frustration theory.

My major thesis, in general terms, is that inherent in such reward schedules is the buildup of primary frustration, defined simply as a temporary state that results when a response is nonreinforced (or nonrewarded in more neutral language in the appetitive case) in the presence of a reward expectancy; that this temporally labile state of frustration is subject to Pavlovian (or classical) conditioning, which is to say that a learned or anticipatory form can be elicited by an originally indifferent or neutral cue; that this conditional form of frustration, like other learned states, is therefore permanent, at least for that situation; and that, together, the primary (unlearned) form and the secondary (learned) form can account for a number of important processes in the dynamics of instrumental behavior.