THE PROBLEM

Chapter 9 described how psychometricians can uncover the dimensions of ability underlying performance on test batteries, such as the Scholastic Assessment Test, the Armed Services Vocational Battery, and numerous personality tests and employment assessment batteries. In this chapter, we turn our attention to the insides of individual tests within a battery. How is a test to be scored? How do we decide if a question is a good question? To take a concrete example, let us look at the problem of designing a vocabulary test.

Why would we give someone a vocabulary test at all? There are two reasons for doing so.

One is that the person may be applying for a position requiring the use of specialized vocabulary, so knowing the vocabulary is a requirement of the job. For example, in international commercial aviation all radio transmissions between controllers and aircraft are in English, albeit a very reduced version of English. Commercial pilots and air traffic controllers must display an understanding of the vocabulary and rudimentary English syntax of air control messages before receiving their licenses. Physicians and nurses have to know several hundred terms referring to parts of the body. Both these examples refer to situations in which there is an absolute standard of competence. It is easy to design a vocabulary test for this sort of situation; just identify the necessary vocabulary and see if the applicant knows it.

Suppose that instead we want to determine how broad a person's English vocabulary is.