The purpose of this chapter is to give readers a sense of the breadth of experimental applications in the social sciences. The chapter reviews lab, field, and survey experiments, as well as naturally-occurring experiments such as lotteries. Each type of experiment is illustrated by reviewing in detail an exemplary study, drawing from experimental literature in psychology, development economics, health, and political science. Special attention is paid to the design choices that researchers made when recruiting subjects, measuring outcomes, and allocating subjects to experimental conditions. Discussion of each study includes the analysis of its main statistical findings. By showing how experiments are designed and analyzed, this chapter lays the groundwork for the practice experiment that readers will undertake in Chapter 6.

In the civil versus uncivil comparison, the group treated with uncivil discourse and the “baseline” group exposed to civil discourse watch a debate between political candidates. In this case, the average treatment effect (ATE) represents the average effect of exposure to debates with different levels of civility. Comparing the untreated control group to the treatment group exposed to uncivil dialogue, on the other hand, captures both the effect of watching a political debate (regardless of civility) and the effect of uncivil discourse.

Before conducting an experiment with human subjects, researchers much consider a number of important ethical and regulatory constraints. This chapter reviews the leading ethical concerns that arise in the context of human subjects research in general and experimental research in particular. These ethical concerns have also set in motion regulations, such as The Common Rule, that researchers must follow before launching a study. The chapter concludes by discussing other professional norms, such as research transparency.

The purpose of this chapter is to give readers a feel for how experiments are designed, implemented, and analyzed. The chapter walks through the steps of designing a small, inexpensive experiment that can be conducted at home. We will also discuss the fine points of implementing an experiment, assembling a dataset, and preparing a statistical analysis. In order to put aside ethical and procedural issues that apply to experiments involving human participants, this chapter confines its attention to product testing. Drawing inspiration from the first field experiments conducted a century ago, my running example will test the effects of fertilizer on plant growth.] As I design and implement my experiment, I call attention to small but consequential decisions aimed at preventing violations of core assumptions. The final section of the chapter describes some illustrative experiments conducted by students, and the exercises provide their data so that you can retrace their steps.

This chapter introduces key terms used to describe experiments and, more generally, the investigation of cause and effect. Because so many different disciplines use experiments, layers of overlapping terminology have accumulated, and this chapter tries to cut through the clutter by grouping synonyms, thereby keeping jargon to a minimum. In addition to providing definitions, this chapter explains why these key concepts are important in practice. The chapter starts with the basic ingredients of an experiment (treatments, outcomes). Next, we define what we mean by a causal effect, introducing the concept of potential outcomes. The chapter culminates in the presentation of three core assumptions for unbiased causal inference. These core assumptions figure prominently throughout the book, as readers are continually encouraged to assess whether illustrative experiments satisfy these assumptions in practice.

Prior chapters relied on elementary statistical calculations and base R functions to analyze and visualize experimental results. This chapter builds on this foundation by showing how covariate adjustment using regression can be used to improve the precision with which treatment effects are estimated. Readers are shown how to apply regression to actual experimental data and to visualize multivariate regression results using R packages. This chapter also introduces the concepts of substantive and statistical “significance,” calling attention to the distinction between estimates of the average treatment effect that are large enough to be meaningful, even if they are not statistically distinguishable from zero. Examples of this distinction are provided using actual experimental data.

Social Science Experiments: A Hands-on Introduction is an accessible textbook for undergraduates. Why a hands-on approach that urges readers to roll up their sleeves and conduct their own experiments? When students design their own experiments, they must reflect on basic questions. What is the treatment … and control? Who are the participants? What is the outcome? The process of conducting an experiment builds other important skills: Creating a dataset, inspecting the results, and drawing inferences. Learning is easier when the motivation to acquire specific skills emerges organically through hands-on experience.

Having reviewed examples of social science experiments in Chapter 4 and ethical considerations in Chapter 5, this chapter walks readers through the design, implementation, and analysis of an experiment involving human participants. After laying out the ground rules for this practice experiment – most importantly, that the study poses no appreciable risks to subjects – the chapter offers examples of inexpensive and brief experiments that can be approved by an institutional review committee and completed in the context of a semester-long course. The chapter provides a checklist of items that should be described in the write-up of the experimental design and results.

Research on persuasion and social influence suggests that crafting effective persuasive and influential appeals is not only feasible but can be done fairly reliably with appropriate guidance from the relevant theories.With the advent of large-scale experiments conducted in field settings, key propositions about persuasion and social influence can be evaluated on a grand scale. In this chapter we assess whether well-known psychological insights work in practice, reviewing efforts related to political mobilisation and persuasion. We argue that in many cases field tests generate an estimated effect that is much smaller than highly influential psychological studies might lead us to expect. The implications of large-scale testing are profound, not only because of the guidance they offer for political campaigns, but also because of their implications for prominent psychological theories.