This chapter applies the ideas developed in the preceding chapter to a class of bounded resource learning procedures known as payoff-based models. Payoff-based models are alternatives to classical Bayesian models that reduce the complexity of a learning situation by disregarding information about states of the world. I am going to focus on one particular payoff-based model, the “basic model of reinforcement learning,” which captures in a precise and mathematically elegant way the idea that acts which are deemed more successful (according to some specific criterion) are more likely to be chosen.

What we are going to see is that the basicmodel can be derived from certain symmetry principles, analogous to the derivation of Carnap's family of inductive methods. Studying the symmetries involved in this derivation leads into a corner of decision theory that is relatively unknown in philosophy. Duncan Luce, in the late 1950s, introduced a thoroughly probabilistic theory of individual choice behavior in which preferences are replaced by choice probabilities. A basic constraint on choice probabilities, known as “Luce's choice axiom,” together with the theory of commutative learning operators, provides us with the fundamental principles governing the basic model of reinforcement learning.

Our exploration of the basic model does not, of course, exhaust the study of payoff-based and other learning models. I indicate some other possible models throughout the chapter and in the appendices. The main conclusion is that learning procedures that stay within a broadly probabilistic framework often arise from symmetry principles in a way that is analogous to Bayesian models.