Skip to main content Accessibility help
×
Hostname: page-component-8448b6f56d-m8qmq Total loading time: 0 Render date: 2024-04-24T22:09:18.645Z Has data issue: false hasContentIssue false

10 - Quasi-Experimentation When Random Assignment Is Not Possible

Observations from Practical Experiences in the Field

Published online by Cambridge University Press:  05 June 2014

Anthony A. Braga
Affiliation:
Harvard University
Brandon C. Welsh
Affiliation:
Northeastern University
Anthony A. Braga
Affiliation:
Rutgers University, New Jersey
Gerben J. N. Bruinsma
Affiliation:
Netherlands Institute for the Study of Crime and Law Enforcement
Get access

Summary

INTRODUCTION

Although tradition and experience often provide the only guidance for criminal justice practitioners, there is a growing consensus among scholars, practitioners, and policy makers that crime prevention practices and policies should be rooted as much as possible in scientific evidence about “what works” (Cullen and Gendreau 2000; MacKenzie 2000; Sherman 1998; Sherman et al. 2006). “Evidence-based crime prevention” is part of a larger and increasingly expanding movement in social policy to use scientific research evidence to guide program development and implementation. In general terms, this movement is dedicated to the improvement of society through the utilization of the highest quality scientific evidence on what works best (see, e.g., Sherman et al. 2006).

In an evidence-based model, the source of scientific evidence is empirical research in the form of evaluations of programs, practices, and policies. Common evaluation designs include randomized experiments, quasi-experiments, and nonexperimental (sometimes called observational) research designs. Not all evaluation designs are considered equal, however. Some evaluation designs, namely randomized controlled experiments, are considered more scientifically valid than others (Campbell and Stanley 1966; Cook and Campbell 1979; Weisburd 2003). The findings of stronger evaluation designs are privileged over the findings of weaker research designs in determining “what works” in crime and justice interventions. For instance, in their report to the United States Congress on what works in preventing crime, University of Maryland researchers developed the Maryland Scientific Methods Scale to indicate to scholars, practitioners, and policymakers that studies evaluating criminological interventions may differ in terms of methodological quality of evaluation techniques (Sherman et al. 1997).

Type
Chapter
Information
Experimental Criminology
Prospects for Advancing Science and Public Policy
, pp. 223 - 250
Publisher: Cambridge University Press
Print publication year: 2013

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Becker, Sascha, and Ichino, Andrea. 2002. “Estimation of Average Treatment Effects Based on Propensity Scores.” The Stata Journal 2: 358–77.
Berk, Richard. 2005a. “Randomized Experiments as the Bronze Standard.” Journal of Experimental Criminology 1: 417–33.CrossRefGoogle Scholar
Berk, Richard 2005b. “Knowing When to Fold ‘Em: An Essay on Evaluating the Impact of Ceasefire, Compstat, and Exile.” Criminology & Public Policy 4: 451 – 66.CrossRefGoogle Scholar
Berk, Richard, Barnes, Geoffrey, Ahlman, Lindsay, and Kurtz, Ellen. 2010. “When Second Best is Good Enough: A Comparison between a True Experiment and a Regression Discontinuity Quasi-Experiment.” Journal of Experimental Criminology 6: 159–90.CrossRefGoogle Scholar
Berk, Richard, Smyth, Gordon K., and Sherman, Lawrence W.. 1988. “When Random Assignment Fails: Some Lessons from the Minneapolis Spouse Abuse Experiment.” Journal of Quantitative Criminology 4: 209–23.CrossRefGoogle Scholar
Boruch, Robert F. 1975. “On Common Contentions about Randomized Field Experiments.” In Experimental Testing of Public Policy: The Proceedings of the 1974 Social Sciences Research Council Conference on Social Experimentation, edited by Boruch, Robert F. and Reicken, Henry L., pp. 107–45. Boulder, CO: Westview Press.Google Scholar
Braga, Anthony A. 2010. “Setting a Higher Standard for the Evaluation of Problem-Oriented Policing Initiatives.” Criminology & Public Policy 9: 173–82.CrossRefGoogle Scholar
Braga, Anthony A., and Bond, Brenda J.. 2008. “Policing Crime and Disorder Hot Spots: A Randomized Controlled Trial.” Criminology 46: 577–608.CrossRefGoogle Scholar
Braga, Anthony A., Hureau, David M., and Papachristos, Andrew V.. 2011a. “The Relevance of Micro Places to Citywide Robbery Trends: A Longitudinal Analysis of Robbery Incidents at Street Corners and Block Faces in Boston.” Journal of Research in Crime and Delinquency 48(1): 7–32.CrossRefGoogle Scholar
Braga, Anthony A., Hureau, David M., and Papachristos, Andrew V. 2011b. “An Ex-Post-Facto Evaluation Framework for Place-Based Police Interventions.” Evaluation Review 35(6): 592–626.CrossRefGoogle ScholarPubMed
Braga, Anthony A., Hureau, David M., and Papachristos, Andrew V. 2013. “Deterring Gang-Involved Gun Violence: Measuring the Impact of Boston’s Operation Ceasefire on Street Gang Behavior.” Journal of Quantitative Criminology. Available online: doi: .
Braga, Anthony A., Kennedy, David M., Waring, Elin J., and Piehl, Anne M.. 2001. “Problem-Oriented Policing, Deterrence, and Youth Violence: An Evaluation of Boston’s Operation Ceasefire.” Journal of Research in Crime and Delinquency 38: 195–225.CrossRefGoogle Scholar
Braga, Anthony A., McDevitt, Jack, and Glenn, L. Pierce. 2006. “Understanding and Preventing Gang Violence: Problem Analysis and Response Development in Lowell, Massachusetts.” Police Quarterly 9: 20–46.CrossRefGoogle Scholar
Braga, Anthony A., Papachristos, Andrew V., and David, M. Hureau. 2010. “The Concentration and Stability of Gun Violence at Micro Places in Boston, 1980–2008.” Journal of Quantitative Criminology 26: 33–53.CrossRefGoogle Scholar
Braga, Anthony A., Pierce, Glenn L., McDevitt, Jack, Bond, Brenda J., and Cronin, Shea. 2008. “The Strategic Prevention of Gun Violence among Gang-Involved Offenders.” Justice Quarterly 25: 132–62.CrossRefGoogle Scholar
Braga, Anthony A., and Weisburd, David L.. 2012. “The Effects of Focused Deterrence Strategies on Crime: A Systematic Review and Meta-Analysis of the Empirical Evidence.” Journal of Research in Crime and Delinquency, 49: 323–58.CrossRefGoogle Scholar
Braga, Anthony A., Weisburd, David L., Waring, Elin J., Mazerolle, Lorraine Green, Spelman, William, and Gajewski, Francis. 1999. “Problem-Oriented Policing in Violent Crime Places: A Randomized Controlled Experiment.” Criminology 37: 541–80.CrossRefGoogle Scholar
Butterfield, Fox. 1996. “In Boston, Nothing Is Something.” The New York Times, November 21: A20.Google Scholar
Campbell, Donald, and Stanley, Julian. 1966. Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally.Google Scholar
Cook, Philip J., and Laub, John H.. 2002. “After the Epidemic: Recent Trends in Youth Violence in the United States.” In Crime and Justice: A Review of Research, edited by Michael Tonry, pp. 27–64. Chicago: University of Chicago Press.Google Scholar
Cook, Philip J., and Ludwig, Jens. 2006. “Aiming for Evidence-Based Gun Policy.” Journal of Policy Analysis and Management 48: 691–735.CrossRefGoogle Scholar
Cook, Thomas, and Campbell, Donald. 1979. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston: Houghton Mifflin.Google Scholar
Cullen, Francis, and Gendreau, Paul. 2000. “Assessing Correctional Rehabilitation: Policy, Practice, and Prospects.” In Policies, Processes, and Decisions of the Criminal Justice System, edited byHorney, Julie (Criminal Justice 2000, Vol. 3), pp. 109–75. Washington, DC: U.S. Department of Justice, National Institute of Justice.Google Scholar
Deutsch, Stuart, and Alt, Francis. 1977. “The Effect of Massachusetts’ Gun Control Law on Gun-Related Crimes in the City of Boston.” Evaluation Quarterly 1: 543–68.CrossRefGoogle Scholar
Fagan, Jeffrey. 2002. “Policing Guns and Youth Violence.” The Future of Children 12: 133 –51.CrossRefGoogle ScholarPubMed
Farrington, David P., Gottfredson, Denise, Sherman, Lawrence W., and Welsh, Brandon C.. 2006. “The Maryland Scientific Methods Scale.” In Evidence-Based Crime Prevention (rev. ed.), edited by Sherman, Lawrence W., Farrington, David P., Welsh, Brandon C., and MacKenzie, Doris L.., pp. 12–21. New York: Routledge.Google Scholar
Hay, Richard, and McCleary, Richard. 1979. “On the Specification of Box-Tiao Time Series Models for Impact Assessment: A Comment on the Recent Work of Deutsch and Alt.” Evaluation Quarterly 3: 277–314.CrossRefGoogle Scholar
Heckman, James, and Smith, Jeffrey. 1995. “Assessing the Case for Social Experiments.” Journal of Economic Perspectives 9: 85–110.CrossRefGoogle Scholar
Kelling, George, Pate, Tony, Dieckman, Duane, and Brown, Charles. 1974. The Kansas City Preventive Patrol Experiment: A Technical Report. Washington, DC: Police Foundation.Google Scholar
Kennedy, David M. 1997. “Pulling Levers: Chronic Offenders, High-Crime Settings, and a Theory of Prevention. Valparaiso University Law Review 31: 449–84.Google Scholar
Kennedy, David M. 2006. “Old Wine in New Bottles: Policing and the Lessons of Pulling Levers.” In Police Innovation: Contrasting Perspectives, edited by Weisburd, David L. and Braga, Anthony A., pp. 155–70. New York: Cambridge University Press.Google Scholar
Kennedy, David M. 2008. Deterrence and Crime Prevention: Reconsidering the Prospect of Sanction. London: Routledge Press.Google Scholar
Kennedy, David M., Piehl, Anne M., and Anthony, A. Braga. 1996. “Youth Violence in Boston: Gun Markets, Serious Youth Offenders, and a Use-Reduction Strategy.” Law and Contemporary Problems 59:147–97.CrossRefGoogle Scholar
Lipsey, Mark. 1990. Design Sensitivity: Statistical Power for Experimental Research. Thousand Oaks, CA: SAGE.Google Scholar
Lipsey, Mark, and Wilson, David B.. 1993. Practical Meta-Analysis. Thousand Oaks, CA: SAGE.Google ScholarPubMed
Ludwig, Jens. 2005. “Better Gun Enforcement, Less Crime.” Criminology & Public Policy 4: 677–716.CrossRefGoogle Scholar
Ludwig, Jens, and Cook, Philip J.. (2000). “Homicide and Suicide Rates Associated with the Implementation of the Brady Handgun Violence Prevention Act.” JAMA 284: 585–91.CrossRefGoogle ScholarPubMed
Ludwig, Jens, and Miller, Douglas. 2007. “Does Head Start Improve Children’s Life Chances? Evidence from a Regression Discontinuity Design.” Quarterly Journal of Economics 122: 159–208.CrossRefGoogle Scholar
MacKenzie, Doris L. 2000. “Evidence-Based Corrections: Identifying What Works.” Crime & Delinquency 46: 457–71.CrossRefGoogle Scholar
McGarrell, Edmund, Chermak, Steven, Wilson, James, and Corsaro, Nicholas. 2006. “Reducing Homicide through a ‘Lever-Pulling’ Strategy.” Justice Quarterly 23: 214–29.CrossRefGoogle Scholar
Moore, Mark H. 2006. “Improving Police through Expertise, Experience, and Experiments.” In Police Innovation: Contrasting Perspectives, edited byDavid, L. Weisburd and Anthony, A. Braga, pp. 322–38. New York: Cambridge University Press.Google Scholar
Morgan, Stephen L., and Winship, Christopher. 2007. Counterfactuals and Causal Inference: Methods and Principals for Social Research. New York: Cambridge University Press.CrossRefGoogle Scholar
Petersilia, Joan. 2008. “Influencing Public Policy: An Embedded Criminologist Reflects on California Prison Reform.” Journal of Experimental Criminology 4: 335–56.CrossRefGoogle Scholar
Piehl, Anne M., Cooper, Suzanne J., Braga, Anthony A., and Kennedy, David M.. 2003. “Testing for Structural Breaks in the Evaluation of Programs.” Review of Economics and Statistics 85: 550–8.CrossRefGoogle Scholar
Pierce, Glenn L., and Bowers, William. 1981. “The Bartley-Fox Gun Law’s Short-term Impact on Crime in Boston.” Annals of the American Academy of Political and Social Science 455: 120–37.CrossRefGoogle Scholar
Rosenbaum, Paul, and Rubin, Donald. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika 70: 41–55.CrossRefGoogle Scholar
Rosenfeld, Richard, Fornango, Robert, and Baumer, Eric. 2005. “Did Ceasefire, Compstat, and Exile Reduce Homicide?Criminology & Public Policy 4: 419–50.CrossRefGoogle Scholar
Sechrest, Lee D., and Rosenblatt, Aaron. 1987. “Research Methods.” In Handbook of Juvenile Delinquency, edited by Quay, Herbert, pp. 417–50. New York: John Wiley & Sons.Google Scholar
Sherman, Lawrence W. 1998. Evidence-Based Policing. Ideas in American Policing series. Washington, DC: Police Foundation.Google Scholar
Sherman, Lawrence W., Farrington, David P., Welsh, Brandon C., and MacKenzie, Doris Layton (eds.). 2006. Evidence-Based Crime Prevention (rev. ed.). New York: Routledge.Google Scholar
Sherman, Lawrence W., Gottfredson, Denise, MacKenzie, Doris L., Eck, John E., Reuter, Peter, and Bushway, Shawn. 1997. Preventing Crime: What Works, What Doesn’t, What’s Promising. Washington, DC: U.S. Department of Justice, National Institute of Justice.Google Scholar
Sparrow, Malcolm. 2011. “Governing Science.” New Perspectives in Policing. Washington, DC: U.S. Department of Justice, National Institute of Justice.Google Scholar
Tita, George, Riley, K. Jack, Ridgeway, Greg, Grammich, Clifford, Abrahamse, Allan, and Greenwood, Peter. 2004. Reducing Gun Violence: Results from an Intervention in East Los Angeles. Santa Monica, CA: RAND Corporation.Google Scholar
Travis, Jeremy. 1998. “Crime, Justice, and Public Policy.” Plenary presentation to the American Society of Criminology, Washington, DC, November 12 Retrieved from: (Accessed August 2, 2011).
Weisburd, David L. 1993. “Design Sensitivity in Criminal Justice Experiments.” In Crime and Justice: A Review of Research, Vol. 17, edited by Tonry, Michael. Chicago: University of Chicago Press.Google Scholar
Weisburd, David L. 2003. “Ethical Practice and Evaluation of Interventions in Crime and Justice: The Moral Imperative for Randomized Trials.” Evaluation Review 27: 336–54.CrossRefGoogle ScholarPubMed
Weisburd, David L., and Britt, Chester. 2007. Statistics in Criminal Justice (3rd ed.). New York: Springer.Google Scholar
Weisburd, David L., Lum, Cynthia, and Petrosino, Anthony. 2001. “Does Research Design Affect Study Outcomes in Criminal Justice?Annals of the American Academy of Political and Social Science 578: 50–70.CrossRefGoogle Scholar
Wellford, Charles F., Pepper, John V., and Petrie, Carol V. (eds.). 2005. Firearms and Violence: A Critical Review. Committee to Improve Research Information and Data on Firearms. Washington, DC: The National Academies Press.Google Scholar
Welsh, Brandon C., Peel, Meghan E., Farrington, David P., Elffers, Henk, and Braga, Anthony A.. 2011. “Research Design Influence on Study Outcomes in Crime and Justice: A Partial Replication with Public Area Surveillance.” Journal of Experimental Criminology 7: 183–98.CrossRefGoogle Scholar
Witkin, Gordon. 1997. “Sixteen Silver Bullets: Smart Ideas to Fix the World. US News and World Report, December 29: 67.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×