To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure firstname.lastname@example.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Causal inference and machine learning are typically introduced in the social sciences separately as theoretically distinct methodological traditions. However, applications of machine learning in causal inference are increasingly prevalent. This Element provides theoretical and practical introductions to machine learning for social scientists interested in applying such methods to experimental data. We show how machine learning can be useful for conducting robust causal inference and provide a theoretical foundation researchers can use to understand and apply new methods in this rapidly developing field. We then demonstrate two specific methods – the prediction rule ensemble and the causal random forest – for characterizing treatment effect heterogeneity in survey experiments and testing the extent to which such heterogeneity is robust to out-of-sample prediction. We conclude by discussing limitations and tradeoffs of such methods, while directing readers to additional related methods available on the Comprehensive R Archive Network (CRAN).
There has recently been an increased interest in studying the language development of non-western languages. This is not new - it began in 1960’s and continued into the 1980’s and 1990’s. The current renewed interest is much welcomed, and will benefit from many of the experimental methods and theoretical insights developed over the past decades.
This chapter tests every step of the book’s theoretical framework using survey experiments. It uses profiles of three hypothetical citizens: Left Leaner, Right Leaner, and Tuned Out. As a narrative device to show how the theory is tested, the chapter then takes these citizens through the steps of our theory using experiments, which use writing exercises, videos, and text vignettes as treatments. The results of these analyses are presented using simple graphs and figures that are relatively understandable for readers with limited statistical expertise. We find that economic discontent does significantly increase populism, regime antipathy, and conspiracism, and that these effects are mediated through cultural discontent, resentment, and (in the case of conspiracism) anxiety, as expected. The chapter further shows that economic discontent increased negative intergroup attitudes, but only among conservatives.
The years following the 2008 financial crisis produced a surge of political discontent with populism, conspiracism, and Far Right extremism rising across the world. Despite this timing, many of these movements coalesced around cultural issues rather than economic grievances. But if culture, and not economics, is the primary driver of political discontent, why did these developments emerge after a financial collapse, a pattern that repeats throughout the history of the democratic world? Using the framework of 'Affective Political Economy', The Age of Discontent demonstrates that emotions borne of economic crises produce cultural discontent, thus enflaming conflicts over values and identities. The book uses this framework to explain the rise of populism and the radical right in the US, UK, Spain, and Brazil, and the social uprising in Chile. It argues that states must fulfill their roles as providers of social insurance and channels for citizen voices if they wish to turn back the tide of political discontent.
In a series of recent experiments (Davis, Millner and Reilly, 2005, Eckel and Grossman, 2003, 2005a-c, 2006), matching subsidies generate significantly higher charity receipts than do theoretically equivalent rebate subsidies. This paper reports a laboratory experiment conducted to examine whether the higher receipts are attributable to a relative preference for matching subsidies or to an “isolation effect” (McCaffery and Baron, 2003, 2006). Some potential policy implications of isolation effects on charitable contributions are also considered.
Previous literature and conventional wisdom have led researchers to believe that boredom increases economic risk taking, but the evidence in support of this conclusion is limited and has important shortcomings. In four experiments (including more than 1,300 subjects), we systematically studied the effects of boredom on economic risk preferences. Across different risk elicitation tasks, boredom inductions, incentive schemes, subject pools, and using both reduced form and structural analyses, we consistently failed to find an effect of boredom on risky decisions. Our results disprove that boredom leads to even small increments in risk taking in one-shot elicitation tasks, and small to medium increases in multiple-choice elicitations. These findings question an important established belief, contribute to better understand the consequences of boredom, and have substantive implications for experiments on economic decision making.
In this paper, we document a violation of normative and descriptive models of decision making under risk. In contrast to uncertainty effects found by Gneezy, List and Wu (2006), some subjects in our experiments valued lotteries more than the best possible outcome. We show that the overbidding effect is more strongly related to individuals’ competitiveness traits than comprehension of the lottery’s payoff mechanism.
Subjects performed a decision task (Grether, 1980) in both a well-rested and experimentally sleep-deprived state. We found two main results: 1) final choice accuracy was unaffected by sleep deprivation, and yet 2) the estimated decision model differed significantly following sleep-deprivation. Following sleep deprivation, subjects placed significantly less weight on new information in forming their beliefs. Because the altered decision process still maintains decision accuracy, it may suggest that increased accident and error rates attributed to reduced sleep in modern society stem from reduced auxiliary function performance (e.g., slowed reaction time, reduced motor skills) or other components of decision making, rather than the inability to integrate multiple pieces of information.
We designed an experiment to test the robustness of Dana, Weber, and Kuang’s (DWK), 2007 results. DWK observed that, when participants were given a “costless” way — the click of a button — to ignore the consequences of their actions on others’ payoffs, they chose to remain ignorant and fair behavior diminished. By implementing a double-blind experiment together with a design that controls for alternative explanations for the observed behavior, we confirmed DWK’s findings.
This paper experimentally investigates a well-known anomaly in portfolio management, i.e., the fact that paper losses are realized less than paper gains (disposition effect). I confirm the existence of the disposition effect in a simple risky task in which choices are taken sequentially. However, when choices are planned ahead and a contingent plan is defined, a reversal in the disposition effect is observed.
The dominant paradigm of experiments in the social and behavioral sciences views an experiment as a test of a theory, where the theory is assumed to generalize beyond the experiment's specific conditions. According to this view, which Alan Newell once characterized as “playing twenty questions with nature,” theory is advanced one experiment at a time, and the integration of disparate findings is assumed to happen via the scientific publishing process. In this article, we argue that the process of integration is at best inefficient, and at worst it does not, in fact, occur. We further show that the challenge of integration cannot be adequately addressed by recently proposed reforms that focus on the reliability and replicability of individual findings, nor simply by conducting more or larger experiments. Rather, the problem arises from the imprecise nature of social and behavioral theories and, consequently, a lack of commensurability across experiments conducted under different conditions. Therefore, researchers must fundamentally rethink how they design experiments and how the experiments relate to theory. We specifically describe an alternative framework, integrative experiment design, which intrinsically promotes commensurability and continuous integration of knowledge. In this paradigm, researchers explicitly map the design space of possible experiments associated with a given research question, embracing many potentially relevant theories rather than focusing on just one. The researchers then iteratively generate theories and test them with experiments explicitly sampled from the design space, allowing results to be integrated across experiments. Given recent methodological and technological developments, we conclude that this approach is feasible and would generate more-reliable, more-cumulative empirical and theoretical knowledge than the current paradigm—and with far greater efficiency.
The purpose of this chapter is to give readers a sense of the breadth of experimental applications in the social sciences. The chapter reviews lab, field, and survey experiments, as well as naturally-occurring experiments such as lotteries. Each type of experiment is illustrated by reviewing in detail an exemplary study, drawing from experimental literature in psychology, development economics, health, and political science. Special attention is paid to the design choices that researchers made when recruiting subjects, measuring outcomes, and allocating subjects to experimental conditions. Discussion of each study includes the analysis of its main statistical findings. By showing how experiments are designed and analyzed, this chapter lays the groundwork for the practice experiment that readers will undertake in Chapter 6.
The purpose of this chapter is to give readers a feel for how experiments are designed, implemented, and analyzed. The chapter walks through the steps of designing a small, inexpensive experiment that can be conducted at home. We will also discuss the fine points of implementing an experiment, assembling a dataset, and preparing a statistical analysis. In order to put aside ethical and procedural issues that apply to experiments involving human participants, this chapter confines its attention to product testing. Drawing inspiration from the first field experiments conducted a century ago, my running example will test the effects of fertilizer on plant growth.] As I design and implement my experiment, I call attention to small but consequential decisions aimed at preventing violations of core assumptions. The final section of the chapter describes some illustrative experiments conducted by students, and the exercises provide their data so that you can retrace their steps.
Political scientists designing experiments often face the question of how abstract or detailed their experimental stimuli should be. Typically, this question is framed in terms of tradeoffs relating to experimental control and generalizability: the more context introduced into studies, the less control, and the more difficulty generalizing the results. Yet, we have reason to question this tradeoff, and there is relatively little systematic evidence to rely on when calibrating the degree of abstraction in studies. We make two contributions. First, we provide a theoretical framework which identifies and considers the consequences of three dimensions of abstraction in experimental design: situational hypotheticality, actor identity, and contextual detail. Second, we field a range of survey experiments, varying these levels of abstraction. We find that situational hypotheticality does not substantively change experimental results, but increased contextual detail dampens treatment effects and the salience of actor identities moderates results in specific situations.
This chapter introduces key terms used to describe experiments and, more generally, the investigation of cause and effect. Because so many different disciplines use experiments, layers of overlapping terminology have accumulated, and this chapter tries to cut through the clutter by grouping synonyms, thereby keeping jargon to a minimum. In addition to providing definitions, this chapter explains why these key concepts are important in practice. The chapter starts with the basic ingredients of an experiment (treatments, outcomes). Next, we define what we mean by a causal effect, introducing the concept of potential outcomes. The chapter culminates in the presentation of three core assumptions for unbiased causal inference. These core assumptions figure prominently throughout the book, as readers are continually encouraged to assess whether illustrative experiments satisfy these assumptions in practice.
Prior chapters relied on elementary statistical calculations and base R functions to analyze and visualize experimental results. This chapter builds on this foundation by showing how covariate adjustment using regression can be used to improve the precision with which treatment effects are estimated. Readers are shown how to apply regression to actual experimental data and to visualize multivariate regression results using R packages. This chapter also introduces the concepts of substantive and statistical “significance,” calling attention to the distinction between estimates of the average treatment effect that are large enough to be meaningful, even if they are not statistically distinguishable from zero. Examples of this distinction are provided using actual experimental data.
Social Science Experiments: A Hands-on Introduction is an accessible textbook for undergraduates. Why a hands-on approach that urges readers to roll up their sleeves and conduct their own experiments? When students design their own experiments, they must reflect on basic questions. What is the treatment … and control? Who are the participants? What is the outcome? The process of conducting an experiment builds other important skills: Creating a dataset, inspecting the results, and drawing inferences. Learning is easier when the motivation to acquire specific skills emerges organically through hands-on experience.
Having reviewed examples of social science experiments in Chapter 4 and ethical considerations in Chapter 5, this chapter walks readers through the design, implementation, and analysis of an experiment involving human participants. After laying out the ground rules for this practice experiment – most importantly, that the study poses no appreciable risks to subjects – the chapter offers examples of inexpensive and brief experiments that can be approved by an institutional review committee and completed in the context of a semester-long course. The chapter provides a checklist of items that should be described in the write-up of the experimental design and results.
This book is designed for an undergraduate, one-semester course in experimental research, primarily targeting programs in sociology, political science, environmental studies, psychology, and communications. Aimed at those with limited technical background, this introduction to social science experiments takes a practical, hands-on approach. After explaining key features of experimental designs, Green takes students through exercises designed to build appreciation for the nuances of design, implementation, analysis, and interpretation. Using applications and statistical examples from many social science fields, the textbook illustrates the breadth of what may be learned through experimental inquiry. A chapter devoted to research ethics introduces broader ethical considerations, including research transparency. The culminating chapter prepares readers for their own social science experiments, offering examples of studies that can be conducted ethically, inexpensively, and quickly. Replication datasets and R code for all examples and exercises are available online.
We consider the simultaneous propagation of two contagions over a social network. We assume a threshold model for the propagation of the two contagions and use the formal framework of discrete dynamical systems. In particular, we study an optimization problem where the goal is to minimize the total number of new infections subject to a budget constraint on the total number of available vaccinations for the contagions. While this problem has been considered in the literature for a single contagion, our work considers the simultaneous propagation of two contagions. This optimization problem is NP-hard. We present two main solution approaches for the problem, namely an integer linear programming (ILP) formulation to obtain optimal solutions and a heuristic based on a generalization of the set cover problem. We carry out a comprehensive experimental evaluation of our solution approaches using many real-world networks. The experimental results show that our heuristic algorithm produces solutions that are close to the optimal solution and runs several orders of magnitude faster than the ILP-based approach for obtaining optimal solutions. We also carry out sensitivity studies of our heuristic algorithm.