Book contents
- Frontmatter
- Dedication
- Contents
- Preface and Acknowledgments
- Introduction
- Section I Thinking Like a Data Scientist
- Section II Communicating Like a Data Scientist
- Section III Applying the Tools of Data Science to Education
- 12 Waiting for Achilles
- 13 How Much Is Tenure Worth?
- 14 Detecting Cheating Badly: If It Could Have Been, It Must Have Been
- 15 When Nothing Is Not Zero: A True Saga of Missing Data, Adequate Yearly Progress, and a Memphis Charter School
- 16 Musing about Changes in the SAT: Is the College Board Getting Rid of the Bulldog?
- 17 For Want of a Nail: Why Worthless Subscores May Be Seriously Impeding the Progress of Western Civilization
- Section IV Conclusion: Don't Try Th is at Home
- Bibliography
- Sources
- Index
16 - Musing about Changes in the SAT: Is the College Board Getting Rid of the Bulldog?
from Section III - Applying the Tools of Data Science to Education
Published online by Cambridge University Press: 05 December 2015
- Frontmatter
- Dedication
- Contents
- Preface and Acknowledgments
- Introduction
- Section I Thinking Like a Data Scientist
- Section II Communicating Like a Data Scientist
- Section III Applying the Tools of Data Science to Education
- 12 Waiting for Achilles
- 13 How Much Is Tenure Worth?
- 14 Detecting Cheating Badly: If It Could Have Been, It Must Have Been
- 15 When Nothing Is Not Zero: A True Saga of Missing Data, Adequate Yearly Progress, and a Memphis Charter School
- 16 Musing about Changes in the SAT: Is the College Board Getting Rid of the Bulldog?
- 17 For Want of a Nail: Why Worthless Subscores May Be Seriously Impeding the Progress of Western Civilization
- Section IV Conclusion: Don't Try Th is at Home
- Bibliography
- Sources
- Index
Summary
During the first week of March 2014 the front page of most U.S. newspapers reported a story released by the College Board of big changes planned for the SAT. These were amplified and augmented by the cover story of the March 9 issue of the New York Times’ Sunday Magazine (Balf 2014).
Having spent twenty-one years as an employee of the Educational Testing Service, the College Board's principal contractor in the production, administration, and scoring of the SAT, I read these reports with great interest. In the end I was left wondering why these changes, in particular, were being proposed and, moreover, why all the hoopla.
Let me elaborate. Three principal changes described were:
The scoring would no longer have a penalty for guessing.
They would reduce the amount of arcane vocabulary used on the test.
They would discontinue the “writing” part of the exam and return to just the canonical verbal and quantitative sections, leaving writing as a stand-alone that would be optional.
By my reckoning the first change is likely to have only small effect, but it probably will increase the error variance of the test, making the scores a little less accurate. The second change addresses something that isn't really much of a problem, but it is always wise to be vigilant about including item characteristics that are unrelated to the trait being measured. I am not sanguine about any changes being implemented successfully. The third change is probably a face-saving reaction to the 2005 introduction of the writing section, which did not work as hoped, and is the one modification that is likely to yield a positive effect.
To see how I arrived at these conclusions, let's go over each of the changes in more detail.
No Penalty for Guessing
The current SAT uses what is called “formula scoring.” That is, an examinee's score is equal to the number right diminished by one-fourth the number wrong (for the kinds of five-choice items commonly used on the SAT). The idea behind this is that if examinees guess completely at random among the choices on the items for which they don't know the answer, for every four items they get wrong they will, on average, get one right by chance. So, under these circumstances, the expected gain from guessing is zero.
- Type
- Chapter
- Information
- Truth or TruthinessDistinguishing Fact from Fiction by Learning to Think Like a Data Scientist, pp. 167 - 174Publisher: Cambridge University PressPrint publication year: 2015