THE FORMAL DERIVATION of a test and the analysis of item difficulty using algorithmic information theory establishes an embryo for universal psychometrics. However, this proves almost nothing if we cannot generalise these ideas to the computational definition of other tasks. We must aim at a well-founded universal notion of cognitive task from which task and item difficulty can be derived. In this chapter we will see a characterisation of cognitive tasks as interactive (stochastic) machines, which is mirrored by the characterisation of the subjects that take the tasks as interactive machines too. The view of tasks as stochastic, featuring many instances, is crucial for the distinction of a solution for a single instance and a policy that achieves an acceptable result for the whole task. Looking at the policies instead of the task is the key issue leading to a computational notion of difficulty, in terms of (the logarithm of) the number of computational steps required to find the simplest acceptable policy.

INTERPRETING TASKS AND INSTANCES

In Chapter 1we saw that cognitive evaluation is performed through instruments, or tests, which are composed of cognitive tasks. In subsequent chapters we also saw that some subjects are designed (by evolution for natural systems or by programming for artificial systems) to do some tasks, whereas other systems are able to learn to do new tasks, usually through several trials and some kind of feedback. A proper notion of task should accommodate the different administrations and evaluation goals seen in Chapters 3–6.

Let us start with a very simple task. Panel 8.1 explains the “relative numerousness” task, whose goal is to tell which quantity is lowest.