Hostname: page-component-848d4c4894-tn8tq Total loading time: 0 Render date: 2024-06-22T10:35:02.807Z Has data issue: false hasContentIssue false
  • English
  • Français

The Role of Theory in Experimental Design: Experiments Without Randomization

Published online by Cambridge University Press:  04 January 2017

James L. Gibson ,
Gregory A. Caldeira  and
Lester Kenyatta Spence
James L. Gibson
Affiliation:
Department of Political Science, Washington University in St. Louis, Campus Box 1063, 219 Eliot Hall, St. Louis, MO 63130-4899. e-mail: jgibson@artsci.wustl.edu
Gregory A. Caldeira
Affiliation:
Department of Political Science, Ohio State University, 2140 Derby Hall, 154 N. Oval Mall, Columbus, OH 43210-1373. e-mail: caldeira.1@osu.edu
Lester Kenyatta Spence
Affiliation:
Department of Political Science, Washington University in St. Louis, Campus Box 1063, 219 Eliot Hall, St. Louis, MO 63130-4899. e-mail: kspence@artsci.wustl.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Some who have written about the logic of experimentation argue that random assignment of subjects to treatment conditions is an essential attribute of an experiment. Others disagree. Rather than treating this as a matter of dueling definitions, we consider experiments without randomization from a theoretical perspective. Our central contention here is that, for some research questions, theory dictates systematic (not random) assignment of respondents to experimental conditions. Two such areas of inquiry are research on political tolerance and on institutional legitimacy. This article gives cursory attention to the former body of work and detailed attention to the latter, based on an experiment conducted in a survey in 2001 on the consequences of the American presidential election for institutional legitimacy. Because in both instances theory requires nonrandom assignment, the problem becomes one of identifying the costs of nonrandomization (threats to internal validity) and specifying analytical techniques that might ameliorate those costs. Consequently, we present results from a statistical approach that addresses the problem of nonrandomization. The most important claim of this article is that theory ought to specify research design, including experimental designs, and that dogmatic attachment to one definition of experiment will not serve the discipline of political science.

Type
Research Article
Information
Political Analysis , Volume 10 , Issue 4: Special Issue on Experimental Methods in Political Science , Autumn 2002 , pp. 362 - 375
Copyright
Copyright © Political Methodology Section of the American Political Science Association 2002 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Achen, Christopher H. 1986. The Statistical Analysis of Quasi-Experiments. Berkeley: University of California Press.Google Scholar
Bachman, Ronet, Paternoster, Raymond, and Ward, Sally. 1992. “The Rationality of Sexual Offending: Testing a Deterrence/Rational Choice Conception of Sexual Assault.” Law and Society Review 26:343–372.Google Scholar
Berry, William D. 1984. Nonrecursive Causal Models. Newbury Park, CA: Sage.Google Scholar
Cook, Thomas D., and Campbell, Donald T. 1979. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston: Houghton Mifflin.Google Scholar
Easton, David. 1965. A Systems Analysis of Political Life. New York: John Wiley.Google Scholar
Finkel, Steven E. Forthcoming. “Civic Education and the Mobilization of Political Participation in Developing Democracies.” The Journal of Politics.Google Scholar
Friedman, Lawrence M. 1998. American Law: An Introduction, rev. ed. New York: W. W. Norton.Google Scholar
Gibson, James L. 1992. “Alternative Measures of Political Tolerance: Must Tolerance Be ‘Least-Liked’?” American Journal of Political Science 36:560–577.Google Scholar
Gibson, James L. 1997. “The Struggle between Order and Liberty in Contemporary Russian Political Culture.” Australian Journal of Political Science 32:271–290.Google Scholar
Gibson, James L. 1998. “A Sober Second Thought: An Experiment in Persuading Russians to Tolerate.” American Journal of Political Science 42:819–850.Google Scholar
Gibson, James L. 2001. “Does Truth Lead to Reconciliation? Testing the Causal Assumptions of the South African Truth and Reconciliation Process.” Presented at the Annual Meeting of the American Political Science Association, San Francisco, August 30-September 2.Google Scholar
Gibson, James L., and Caldeira, Gregory A. 1995. “The Legitimacy of Transnational Legal Institutions: Compliance, Support, and the European Court of Justice.” American Journal of Political Science 39:459–489.CrossRefGoogle Scholar
Gibson, James L., and Caldeira, Gregory A. Forthcoming. “Defenders of Democracy? Legitimacy, Popular Acceptance, and the South African Constitutional Court.” The Journal of Politics.Google Scholar
Gibson, James L., and Gouws, Amanda. 1999. “Truth and Reconciliation in South Africa: Attributions of Blame and the Struggle over Apartheid.” American Political Science Review 93:501–517.Google Scholar
Gibson, James L., and Gouws, Amanda. 2001. “Making Tolerance Judgments: The Effects of Context, Local and National.” The Journal of Politics 63:1067–1090.CrossRefGoogle Scholar
Gibson, James L., and Gouws, Amanda. 2002. Overcoming Intolerance in South Africa: Experiments in Democratic Persuasion. New York: Cambridge University Press.Google Scholar
Greene, William H. 1993. Econometric Analysis, 2nd ed. New York: Macmillan.Google Scholar
Grosskopf, Anke, and Mondak, Jeffrey J. 1998. “Do Attitudes Toward Specific Supreme Court Decisions Matter? The Impact of Webster and Texas v. Johnson on Public Confidence in the Supreme Court.” Political Research Quarterly 51:633–654.CrossRefGoogle Scholar
Hamilton, V. Lee, and Sanders, Joseph. 1992. Everyday Justice: Responsibility and the Individual in Japan and the United States. New Haven, CT: Yale University Press.Google Scholar
Hamilton, V. Lee, and Sanders, Joseph. 1995. “Crimes and Obedience and Conformity in the Workplace: Surveys of Americans, Russians, and Japanese.” Journal of Social Issues 51:67–88.Google Scholar
Hanushek, Eric A., and Jackson, John E. 1977. Statistical Methods for Social Scientists. New York: Academic.Google Scholar
Hibbing, John R., and Theiss-Morse, Elizabeth. 2001. “Process Preferences and American Politics: What the People Want Government to Be.” American Political Science Review 95:145–153.Google Scholar
Kinder, Donald R., and Palfrey, Thomas R., eds. 1993. Experimental Foundations of Political Science. Ann Arbor: University of Michigan Press.Google Scholar
Marcus, George E., Sullivan, John L., Theiss-Morse, Elizabeth, and Wood, Sandra L. 1995. With Malice Toward Some: How People Make Civil Liberties Judgments. New York: Cambridge University Press.Google Scholar
McGraw, Kathleen M. 1996. “Political Methodology: Research Design and Experimental Methods.” In A New Handbook of Political Science, eds. Goodin, Robert E. and Klingemann, Hans-Dieter. New York: Oxford University Press, pp. 769–786.Google Scholar
Mutz, Diana C. 1998. Impersonal Influence: How Perceptions of Mass Collectives Affect Political Attitudes. New York: Cambridge University Press.Google Scholar
Robinson, Paul H., and Darley, John M. 1995. Justice, Liability and Blame: Community Views and the Criminal Law. Boulder, CO: Westview Press.Google Scholar
Sullivan, John L., Piereson, James E., and Marcus, George E. 1982. Political Tolerance and American Democracy. Chicago: University of Chicago Press.Google Scholar
Winship, Christopher, and Mare, Robert D. 1992. “Models for Sample Selection Bias.” Annual Review of Sociology 18:327–350.Google Scholar