Skip to main content Accessibility help
×
Hostname: page-component-84b7d79bbc-g5fl4 Total loading time: 0 Render date: 2024-07-27T18:37:40.424Z Has data issue: false hasContentIssue false

5 - The Evaluation of Artificial Intelligence

from PART II - THE EVALUATION DISCORDANCE

Published online by Cambridge University Press:  19 January 2017

José Hernández-Orallo
Affiliation:
Universitat Politècnica de València, Spain
Get access

Summary

AI will now proceed along two parallel paths: (1) specialized systems and (2) habile systems, general, intelligent systems , having general skill.

– Nils J. Nilsson, Eye on the Prize (1995)

ARTIFICIAL INTELLIGENCE is a successful discipline. New applications and breakthroughs take us by surprise day after day, fuelled by the increasing power of machine learning. But do AI artefacts of today feature intelligence? If so, how much? Of what kind? How can we know? In this chapter we will see that the field has usually evaluated its artefacts in terms of task performance, not really in terms of intelligence. With that task-oriented perspective, we can say that machines are superhuman at chess, at driving a car and, of course, at solving the Rubik's cube. Beyond these particular tasks, Turing's imitation game, still a touchstone for some, is not used in general practice, except for variants, such as CAPTCHAs. Instead, benchmarks and competitions are the regular way of evaluating AI systems. However, as systems specialise for the new benchmarks, more challenging tasks are being devised. But are these more challenging tasks demanding more intelligence?

BARING INTELLIGENCE: THE AI EFFECT

The roots of artificial intelligence can be traced back to Ramon Llull's Ars Magna, Thomas Hobbes's Artificial Man, Gottfried Leibniz's Ars Combinatoria and the tradition of building mechanical automata since the middle ages until the early twentieth century. However, it is Alan Turing's celebrated 1950 paper “Can a machine think?” that definitively connects the possibility of an artificial mind with modern computers, as per the notions of a universal Turing machine and the Church-Turing thesis (see Panel 1.2).

The name “artificial intelligence” and the official start as a discipline took place during the famous 1956 Dartmouth College's “Summer Research Project on Artificial Intelligence”, which congregated John McCarthy, Marvin Minsky, Nathaniel Rochester, Claude Shannon, Arthur Samuel, Oliver Selfridge, Ray Solomonoff, Allen Newell, Herbert Simon and Trenchard More. An excerpt of the proposal contained the following: “The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.

Type
Chapter
Information
The Measure of All Minds
Evaluating Natural and Artificial Intelligence
, pp. 117 - 151
Publisher: Cambridge University Press
Print publication year: 2017

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×