HERE WE WILL unfurl the unconventional and perhaps (politically) incorrect position of using intelligence as a reference, instead of humanity. This ratiocentric stance will start with the very notion of personhood and will lead us to a critical account of other issues such as ‘superintelligence’, its (existential) risks, the rights and effects of cognitive enhancement, and the implications of all this for the organisation of society. From a more sceptical position than the rest of the book, this chapter will discuss several ratiocentric scenarios, not because I can demonstrate that they are more likely or better than others, but rather because if they become a reality, the evaluation of cognitive abilities will be of utmost importance, much more than psychometrics is already today with the nowadays dominant anthropocentric or biocentric stances. For instance, should intelligence be kept shared by many independent individuals? Should some rights, such as the right to vote, depend on behavioural features? Should this be accompanied by the universal right to be (and not to be) enhanced? How necessary are measurements to keep an equilibrium? Not only are the answers to these questions uncomfortable, but the mere questions themselves.

PERSONHOOD

Human psychometrics has always been controversial. Its use to identify group differences in human populations, to select personnel in the labour market, to certify deep intellectual disabilities, to complement forensic psychiatry and to route children's education is still strongly contended, as potentially discriminative. Other uses in strategic government decisions, birth control and any explicit kind of eugenics have simply been expelled from political correctness. At this point of the book, with amoremature perspective of universal psychometrics, it is now the right time to analyse these and new controversial issues for the whole machine kingdom. We start with the root of the problem, the very concept of person.

ARTIFICIAL INTELLIGENCE is a successful discipline. New applications and breakthroughs take us by surprise day after day, fuelled by the increasing power of machine learning. But do AI artefacts of today feature intelligence? If so, how much? Of what kind? How can we know? In this chapter we will see that the field has usually evaluated its artefacts in terms of task performance, not really in terms of intelligence. With that task-oriented perspective, we can say that machines are superhuman at chess, at driving a car and, of course, at solving the Rubik's cube. Beyond these particular tasks, Turing's imitation game, still a touchstone for some, is not used in general practice, except for variants, such as CAPTCHAs. Instead, benchmarks and competitions are the regular way of evaluating AI systems. However, as systems specialise for the new benchmarks, more challenging tasks are being devised. But are these more challenging tasks demanding more intelligence?

BARING INTELLIGENCE: THE AI EFFECT

The roots of artificial intelligence can be traced back to Ramon Llull's Ars Magna, Thomas Hobbes's Artificial Man, Gottfried Leibniz's Ars Combinatoria and the tradition of building mechanical automata since the middle ages until the early twentieth century. However, it is Alan Turing's celebrated 1950 paper “Can a machine think?” that definitively connects the possibility of an artificial mind with modern computers, as per the notions of a universal Turing machine and the Church-Turing thesis (see Panel 1.2).

The name “artificial intelligence” and the official start as a discipline took place during the famous 1956 Dartmouth College's “Summer Research Project on Artificial Intelligence”, which congregated John McCarthy, Marvin Minsky, Nathaniel Rochester, Claude Shannon, Arthur Samuel, Oliver Selfridge, Ray Solomonoff, Allen Newell, Herbert Simon and Trenchard More. An excerpt of the proposal contained the following: “The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.…

THE FORMAL DERIVATION of a test and the analysis of item difficulty using algorithmic information theory establishes an embryo for universal psychometrics. However, this proves almost nothing if we cannot generalise these ideas to the computational definition of other tasks. We must aim at a well-founded universal notion of cognitive task from which task and item difficulty can be derived. In this chapter we will see a characterisation of cognitive tasks as interactive (stochastic) machines, which is mirrored by the characterisation of the subjects that take the tasks as interactive machines too. The view of tasks as stochastic, featuring many instances, is crucial for the distinction of a solution for a single instance and a policy that achieves an acceptable result for the whole task. Looking at the policies instead of the task is the key issue leading to a computational notion of difficulty, in terms of (the logarithm of) the number of computational steps required to find the simplest acceptable policy.

INTERPRETING TASKS AND INSTANCES

In Chapter 1we saw that cognitive evaluation is performed through instruments, or tests, which are composed of cognitive tasks. In subsequent chapters we also saw that some subjects are designed (by evolution for natural systems or by programming for artificial systems) to do some tasks, whereas other systems are able to learn to do new tasks, usually through several trials and some kind of feedback. A proper notion of task should accommodate the different administrations and evaluation goals seen in Chapters 3–6.

Let us start with a very simple task. Panel 8.1 explains the “relative numerousness” task, whose goal is to tell which quantity is lowest.

Bob, the senior author of this book, always thought of himself as a reasonably good communicator. When he started graduate school, he was proud of the papers he had written as an undergraduate. As a new graduate student, he thought he would get an edge in with his new graduate adviser by showing the adviser (Professor Gordon Bower at Stanford) his favorite paper he had written as an undergraduate. He asked Professor Bower to give him comments on the paper, figuring that Bower would be impressed that he had a terrific new student to advise.

Bower handed him back the paper a couple of weeks later. Bower said that he had crossed out the parts of the paper that he didn't think added much. Unfortunately, Bower had crossed out most of the paper. The experience was not the start to his graduate career that Bob had been hoping for. Bob realized that his communication skills were not what he thought they were.

Many students go through college never learning, in any reasonably formal way, how to communicate, at least in psychology and related disciplines. They receive no explicit instruction in how to write a paper, how to give a talk, or how to create a poster. (When Bob gave his first talk in graduate school, one of the senior professors fell asleep during the talk.) The purpose of this book is to give you the guidance you need to be effective in communication and to attain professional success.

In this chapter, we would like to help you get started. To get started, you need just five basic tips. Once you master these tips, you are ready to read the rest of the book and, more importantly, to get going in doing all the writing and speaking you need to do to achieve success.

There is no one foolproof way of getting ideas for papers. You have to find the ways that work for you. In the first part of this chapter, we present different ways in which you can develop ideas for your research projects. Of course, it is not enough to have ideas. To be a successful researcher, you also need the ability to evaluate your ideas and find out whether they are good ones. You do not want to waste your time on bad ideas. The second part of this chapter deals with the evaluation of your ideas. And once you have come to the conclusion that yours is a wonderful idea, it will be important for you to present and persuade others of the value of your idea. How do you convince others that your planned study is one that is worthwhile to conduct? Especially creative ideas are often hard to sell. Therefore, the third part of this chapter shows you some ways to sell your ideas.

GENERATING IDEAS

Generating Ideas by Consulting with Others

In many colleges and universities, the faculty is among the most underutilized of resources. In Bob's first semester of teaching at Yale, he set aside three hours each week for office hours. He encouraged – sometimes he practically begged – students to come see him during those hours for advice on papers, projects, and the like. He left his door wide open to encourage students to enter. For the most part, though, he sat staring at the walls or at the people scurrying by (but not in) the door. He also encouraged students to make individual appointments if they were unable to see him during his prearranged hours, but, for the most part, students also failed to take him up on this offer. Later, business picked up, although much more so among graduate than among undergraduate students. (Bob is now older; he is at Cornell; and he has the same experience. On most days, no one comes to his office hours.)

Why are students are so timid in approaching faculty? Sometimes students try once, are unsuccessful in reaching the faculty member, and give up. Sometimes they don't try at all.

NO MORE NEW measurement problems need to be concocted. There are too many already out there, and many more are coming that we cannot foresee. It is now time to resolve whether the principles and techniques in this book are going to accelerate the solutions to these problems, through actual measurement tools. To raise the likelihood of a positive answer, we recount the things that the book leaves as open questions, clarify the learnt lessons and determine the priorities and dimensions of the work that is to be done. At a more general level, we also address some other questions around the several disciplines dealing with the evaluation of behavioural features: Will universal psychometrics help to bridge them? What new possibilities and applications are envisaged from a further integration?

NEW GROUNDS

The conscientious reader who has reached this point – after going through the previous chapter –may get the wrong impression that we are aiming at far-flung problems. Universal psychometrics was not motivated by speculative scenarios of a world that does not yet exist and may never do. Neither was universal psychometrics motivated by any important disfunction of human psychometrics in their traditional applications. State-of-the-art psychometric tests, if well chosen and administered, produce accurate and valid measurements on human individuals. The real instigation behind universal psychometrics was a series of measurement needs that we are already facing, most (but not all) of them originating from new kinds of cognitive entities that were not present just a few years ago.

Of course, artificial intelligence is greatly responsible for these peremptory needs, for its creating many new kinds of entities and its immaturity about their measurement. However, other needs originate from humans, computers and collectives whose abilities are modified by the use and dependency on external devices. Cognitively enhanced organisms and biologically enhanced computers, which are created by new forms of communication and integration, add to the machine kingdom.

WHEN SEVERAL SYSTEMS are governed, momentarily or permanently, by common goals, they can work as a team. From the outside, if the team has a unified interface, its evaluation should, in principle, be the same as that of an individual. However, if the members of the team can perform different actions and have different observations, the task setting changes slightly and so must change the evaluation. In both cases, what can we say when we compare the group of individual policies versus the joint policy? How does the performance of the group depend on the behavioural features of its members? We will see that the members’ features and the way the team is arranged and communicates are key, but we will also see that diversity, mostly in terms of the psychometric profile of the group, might be equally important. The general open question, which extends to hybrids and large collaborative platforms, is what psychometric profiles must be selected for mustering a team that meets an overall desired psychometric profile.

CHARACTERISING COLLECTIVE TASKS

In Chapter 13 we analysed the way in which an agent can be analysed, and measured, in a social context. One natural setting for evaluation is a situated test, a multi-agent system, where several agents interact. In particular, in Section 13.3 we saw that those agents sharing rewards (a common utility or payoff) were considered to be in a team, having an incentive to co-operate. Of course, having a shared reward or goal does not imply that the agents will co-operate. The motivation may be insufficient (e.g., in a large group agents can become lazy, since their effort may be unnoticeable), the task may make it impossible (e.g., agents are in isolated rooms) or the cognitive abilities of the group may be inappropriate for co-operation (e.g., lack of communication). For instance, in Figure 13.4 we saw that a single random agent can be very disruptive to a set of agents working in the same team.

ABILITIES DESCRIBE WHAT a subject is able to do. Every ability is a potential. But what if abilities evolve too? Can we detect this trend? Can some cognitive abilities lead to other abilities and traits? Also, if we ignore speed, can a universal Turing machine (UTM) acquire any ability if properly ‘educated’? More interestingly, given that humans are, in a way, UTMs, can humans acquire any ability? Positive answers to these questions, at least in theory, would be comforting. Actually, we are approaching the real conundrum of intelligence, the ability of cumulative learning, or acquiring skills. The evaluation of development will be key.

EARLY SENSORIMOTOR REPRESENTATIONS

In previous chapters we have assumed that subjects are static, and so are the tests and items designed to evaluate them. This is reasonable, precisely because behavioural features are meant to describe how a subject behaves in general, and not for a particular state. Indeed, we decided to consider cognitive abilities and personality traits, but we intentionally excluded emotional states from our measurement. However, even if we consider that a characterisation of an individual has to be done with stable traits, it is short-sighted to consider that subjects are immutable and their behavioural features do not change with time. Subjects develop. In this chapter, we analyse this dynamic, developmental perspective, known as mental, intellectual or, simply, cognitive development, and discuss its assessment from the point of view of universal psychometrics.

A dynamic view of the elements of the machine kingdom must be accompanied by some dynamics about the corresponding space of behavioural features. So far, as illustrated in Figure 10.4, tasks have been arranged into abilities for each slice of difficulty. However, a different way of looking at the space of abilities, and features in general, is to do it dynamically, in terms of development.

THE TRADITIONAL DISTINCTION between humans and all the other living beings is symptomatic of the relevance of the Homo sapiens. However, this relevance led in the past to an anthropocentric stance, which has been, and still is, highly pernicious for many disciplines, such as biology. Comparative cognition and evolutionary psychology take a different perspective, with humans being the norm rather than the exception. The evaluation of behaviour is a fundamental tool in these disciplines, but the methodology and principles differ from human psychometrics. Is animal cognition so different from human cognition to justify a completely different evaluation methodology? And what about other living organisms, such as swarms and plants? Actually, when we move around the tree of life we find some awkward cases of adaptation, with an arguably minimal cognition ranging from the very small (bacteria) to the very big (ecosystems). Ultimately, we can further extend our cure for anthropocentrism to gaiacentrism as well, by thinking about extraterrestrial life. How are behavioural features evaluated in this highly diverse range of organisms?

THE BIOLOGICAL ROLE OF BEHAVIOURAL FEATURES

Juan Huarte developed Galen's purported view that the “rational powers” of men and other animals just differ in their degree: “Galen said that the difference which is found between man and brute beast is the selfe same which is found between a fool and a wise man; which is nought else than in respect of the more and the less. This (truly) is not to be doubted, that brute beasts enjoy memorie and imagination, and another power which resembles understanding, as the Ape is very like a man, and that his soule takes use of the composition of the braine, it is a matter apparant” (Huarte, 1575, pp. 29–30). However, this “apparent” gradation was only taken seriously when itwas seen in the context of the theory of evolution, three centuries later, epitomised by Charles Darwin's famous statement: “the mental powers of the higher animals do not differ in kind, though greatly in degree, from the corresponding powers of man” (Darwin, 1871, chap. XVIII). Even under an evolutionary perspective, this view was (and is still) not shared universally.