Every operant conditioning experiment is a feedback system. Response and reinforcer measures are neither independent nor dependent variables – causes and effects – but interdependent variables. Because of feedback, the relations between response and reinforcer rates, for example, reveal little about true causes, the processes internal to the organism that allow it to react to the stimuli and reinforcers the environment provides. But, also because of feedback, the relations between response and reinforcer measures in these experiments are often extremely orderly. This chapter describes a very simple model for the reliable molar regularities of simultaneous and successive free operant choice (concurrent and multiple variable-interval [VI] schedules). I seek the order in the empirical laws, skipping over the molecular/procedural details necessary to obtain them, such as the effects of changeover delay, interresponse time reinforcement, etc., which were discussed in Chapter 9.

Matching and optimality

Chapter 9 showed how diminishing marginal utility could account for response functions (steady-state response rate vs. reinforcer rate functions) obtained on numerous simple schedules. Chapter 10 showed how marginal utility can explain optimal choice between food patches. A version of this approach can be easily applied to concurrent schedules (simultaneous discrimination).

Matching can be derived in a way that allows for interim activities and can be generalized so as to account for performance on multiple schedules (i.e., successive discrimination). In a two-component concurrent schedule, the components share an interim activity, which occurs in both. I assume that in each component, the balance between operant reinforcement and the interim activity adjusts so as to minimize cost (maximize value), in the sense defined in Chapter 8.

The condition for optimal (maximizing) behavior is that the marginal change in reinforcement rate be equal to the marginal change in the value of the interim activity, formally

dR (x) /dx = dV (z) /dz,

where V(z) is the value of the interim activity as a function of its level, z, x is the level of responding in one component, and R(x) is its rate of reinforcement. If we assume that the marginal value of the interim activity is constant across both components, the optimality condition is that

dR (x) /dx = dR (y) /dy,

where y is the response rate in the other component.