We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure coreplatform@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Measuring risk preferences using monetary incentives is costly. In the field, it might be also unfair and unsafe. The commonly used measure of Holt and Laury (2002) relies on a dozen lottery choices and payments, which make it time consuming and expensive. It also raises moral concerns as a result of the unequal payments generated by good and bad luck. Paying some but not all subjects may also create tensions between the researcher and subjects. In a pre-registered study in Honduras, Nigeria and Spain, we use a short version of Holt and Laury where we address all three concerns. We find in the field that not paying at all or paying with and without probabilistic rules makes no difference. Our hypothetical and short version makes our measurement of risk cheaper, fairer and safer.
Political elites increasingly express interest in evidence-based policymaking, but transparent research collaborations necessary to generate relevant evidence pose political risks, including the discovery of sub-par performance and misconduct. If aversion to collaboration is non-random, collaborations may produce evidence that fails to generalize. We assess selection into research collaborations in the critical policy arena of policing by sending requests to discuss research partnerships to roughly 3,000 law enforcement agencies in 48 states. A host of agency and jurisdiction attributes fail to predict affirmative responses to generic requests, alleviating concerns over generalizability. However, across two experiments, mentions of agency performance in our correspondence depressed affirmative responses – even among top-performing agencies – by roughly eight percentage points. Many agencies that initially indicate interest in transparent, evidence-based policymaking recoil once performance evaluations are made salient. We discuss several possible mechanisms for these dynamics, which can inhibit valuable policy experimentation in many communities.
The share of basic services that NGOs deliver has grown dramatically in developing countries due to increased receipt of aid and philanthropy in these countries. Many scholars and practitioners worry that NGOs reduce reliance on government services and, in turn, lower demand for government provision and undermine political engagement. Others argue that NGOs prop-up poorly performing governments that receive undeserved credit for the production, allocation, or welfare effects of NGO services. Using original surveys and a randomized health intervention, implemented in parallel to a similar universal government program, this article investigates the long-term effect of NGO provision on political attitudes and behavior. Access to NGO services increased preferences for NGO, relative to government, provision. However, political engagement and perceptions of government legitimacy were unaffected. Instead, the intervention generated political credit for the incumbent president. This study finds that citizens see NGOs as a resource that powerful government actors control, and they reward actors who they see as responsible for allocation of those resources.
Field experiments which test the application of behavioural insights to policy design have become popular to inform policy decisions. This study is the first to empirically examine who and what drives these experiments with public partners. Through a mixed-methods approach, based on a novel dataset of insights from academic researchers, behavioural insight team members and public servants, I derive three main results: First, public bodies have a considerable influence on study set-up and sample design. Second, high scientific standards are regularly not met in cooperative field experiments, mainly due to risk aversion in the public body. Third, transparency and quality control in collaborative research are low with respect to pre-analysis plans, the publication of results and medium or long-term effects. To remedy the current weaknesses, the study sketches out several promising ways forward, such as setting up a matchmaking platform for researchers and public bodies to facilitate cooperation, and using time-embargoed pre-analysis plans.
A substantial body of research has found biased recruitment in a variety of societal spheres. We study selection in the judiciary, a domain that has received less attention than the economic and political spheres. Our field experiment took place in the midst of a Swedish government campaign encouraging ordinary citizens to contact local parties, which are responsible for recruiting lay judges (jurors) and put themselves forward as lay judge candidates. Parties’ responsiveness to citizen requests does not seem to favor their own sympathizers, does not vary at all with signals of gender, and is only marginally affected by ethnicity and age. Given the potential importance of ideology and identity in judicial decision-making, the finding that there is little bias with respect to these factors at this first stage of the recruitment process is reassuring from the perspective of impartiality.
Research on persuasion and social influence suggests that crafting effective persuasive and influential appeals is not only feasible but can be done fairly reliably with appropriate guidance from the relevant theories. With the advent of large-scale experiments conducted in field settings, key propositions about persuasion and social influence can be evaluated on a grand scale. In this chapter we assess whether well-known psychological insights work in practice, reviewing efforts related to political mobilisation and persuasion. We argue that in many cases field tests generate an estimated effect that is much smaller than highly influential psychological studies might lead us to expect. The implications of large-scale testing are profound, not only because of the guidance they offer for political campaigns, but also because of their implications for prominent psychological theories.
Experimental approaches are gaining in popularity across disciplines, ranging from behavioural sciences to economics. In this chapter, we discuss the advantages and disadvantages of field experiments and review their use by scholars to study routine dynamics. Based on these, we suggest that field experiments hold further promise to study routines given their potential to develop and test theory, while achieving internal and external validity. To further the adoption of field experiments to study routines, we outline a five-step procedure, including research questions and hypotheses, context and research setting, treatment and design, measurement and statistical tests, and managing field experiments. We conclude by discussing potential research questions and contexts suitable for field experiments.
To what extent can civil rights NGOs protect ethnic minorities against unequal treatment? We study this question by combining an audit experiment of 1260 local governments in Hungary with an intervention conducted in collaboration with a major Hungarian civil rights NGO. In the audit experiment we demonstrated that Roma individuals were about 13 percent-points less likely to receive responses to information requests from local governments, and the responses they received were of substantially lower quality. The intervention that reminded a random subset of local governments of their legal responsibility of equal treatment led to a short-term reduction in their discriminatory behavior, but the effects of the intervention dissipated within a month. These findings suggest that civil rights NGOs might face substantive difficulties in trying to reduce discrimination through simple information campaigns.
We fielded an experiment on a sample of approximately 400 Black state legislators to test whether they would be more responsive to an email that mentioned the National Association for the Advancement of Colored People (NAACP) relative to an email that mentioned Black Lives Matter (BLM). The experiment tested Cohen's theory of secondary marginalization (1999), whereby relatively advantaged members of a marginalized group regulate the behavior, attitudes, and access to resources of less advantaged members of the group. We expected that Black legislators would be less responsive to an email that referenced BLM, an organization that is associated with more marginalized members of the Black community. Contrary to our hypothesis, Black legislators were as responsive to emails referencing inspiration from BLM as they were to emails referencing inspiration from the NAACP. Thus, we do not find any evidence of intragroup discrimination by Black state legislators. To our knowledge, this is the first field experiment to test Cohen's theory of secondary marginalization.1
Inoculation of symbiotic N2-fixing rhizobacteria (rhizobia) in legumes is an alternative to reduce synthetic N fertiliser input to crops. Even though common bean benefits from the biological N2 fixation carried out by native rhizobia isolates, the low efficiency of this process highlights the importance of screening new strains for plant inoculation. Two rhizobial strains (SEMIA 4108 and SEMIA 4107) previously showed great potential to improve the growth of common beans under greenhouse conditions. Thus, this study evaluated the growth and grain yield of common bean plants inoculated with those strains in field experiments. The rhizobial identification was performed by 16S rRNA sequencing and the phylogeny showed that SEMIA 4108 and SEMIA 4107 are closely related to Rhizobium phaseoli, within a clade containing other 18 Rhizobium spp. type strains. Common bean plants inoculated with SEMIA 4107 showed similar productivity to N-fertilised (N+) plants in the first experiment (2016/17) and higher productivity in the second experiment (2018/19). The development of inoculated plants was different from that observed for N+. Nonetheless, comparing inoculated treatments with N-fertilised control, no yield or productivity losses at the end of the growing process were detected. Our results showed that inoculation of the rhizobial isolates SEMIA 4108 and SEMIA 4107 improved the growth and grain yield of common bean plants. The observed agronomical performance confirms that both strains were effective and can sustain common bean growth without nitrogen fertilisation under the edaphoclimatic conditions of this study.
Informed consent has been a mainstay of all ethical research guidelines since the 1970s, but the proliferation of field experiments in the social sciences – which include audit experiments, correspondence experiments, canvasing experiments, social media experiments, and information experiments – has brought with it an increasing resistance to procuring informed consent. This essay grapples with the now common practice of denying research subjects an opportunity to voluntarily consent to participate in research. It provides a framework for thinking about virtual consent, a situation in which the researcher consents for participants. Drawing on a Rawlsian thought experiment, I argue that ethical research is that to which a reasonable person, not knowing whether she would be the subject or the scientist, would consent. This type of reasoning provides a way for thinking about potential downstream consequences not just for the individual subject, but also for society writ large. Yet, because virtual consent does not entail voluntary participation, in constitutes a bronze standard, rather than a best practice, for ethics in experiments.
We review recent experimental research on the behavior of street-level bureaucrats. These front-line government workers are tasked with implementing most government policy in both advanced democracies and developing countries, but their behavior is often difficult to observe. We highlight how experimental approaches have helped to address classic questions about street-level bureaucratic behavior, and then consider design challenges that arise in running experiments in this context. Finally, we raise several ethical concerns about experimentation on street-level bureaucrats, and propose strategies to minimize the social costs, and maximize the social benefits, of such research.
Ten years since the publication of the first edition of this handbook two things are clear: The world is no less complicated than it was a decade ago and we are better at designing, running, and analyzing experiments today than we were then. In light of these observations, in this chapter I highlight the areas in which political scientists and their collaborators have excelled and how they have done so; but I also point out the challenges –in fact, in some cases, the pure limitations – that remain. Still, the prescription is for more work, more science, and more explanation in the service of reducing the apparent chaos of the interactions between the people and institutions around us.
Experimental political science has transformed in the last decade. The use of experiments has dramatically increased throughout the discipline, and technological and sociological changes have altered how political scientists use experiments. We chart the transformation of experiments and discuss new challenges that experimentalists face. We then outline how the contributions to this volume will help scholars and practitioners conduct high-quality experiments.
Experimental political science has changed. In two short decades, it evolved from an emergent method to an accepted method to a primary method. The challenge now is to ensure that experimentalists design sound studies and implement them in ways that illuminate cause and effect. Ethical boundaries must also be respected, results interpreted in a transparent manner, and data and research materials must be shared to ensure others can build on what has been learned. This book explores the application of new designs; the introduction of novel data sources, measurement approaches, and statistical methods; the use of experiments in more substantive domains; and discipline-wide discussions about the robustness, generalizability, and ethics of experiments in political science. By exploring these novel opportunities while also highlighting the concomitant challenges, this volume enables scholars and practitioners to conduct high-quality experiments that will make key contributions to knowledge.
The chapter concludes the book by synthesizing key arguments from previous chapters and making comprehensive arguments about redesigning civil service systems. Previous chapters are examined to question if prior analysis was too optimistic. The chapter discusses processes for advancing the civil service reform agenda, including leveraging small wins to achieve incremental change and aiming for comprehensive reforms. Two examples of navigating comprehensive change, Georgia and South Africa, are discussed. Finally, research surrounding the integration of public service motivation and civil service reform is reviewed. An analysis of systematic programs of field experiments and macro-research about variations in national performance precedes a discussion of the dark side of public service motivation. The chapter concludes with a call for further scholarly scrutiny of public service motivation-related policies to be supplemented with real-world experimentation.
Social media may help civil society organize and mobilize for different campaigns. However, the extent to which social media campaigns simply recruit like-minded individuals as compared to exerting a causal impact on joiners’ attitudes is difficult to disentangle. We test both the organizational and transformative potential of a civil society campaign in a randomized field experiment deployed via Facebook or an email newsletter in collaboration with a Bulgarian environmental campaign. As expected, we find that Bulgarian Facebook users who are active in pro-environmental groups, and those who decide to follow the campaign, are more highly educated than those who decide to stay at the sidelines. Moreover, beliefs in the effectiveness of civic society, character traits and prior activism systematically predict whether a Bulgarian Facebook user decides to join the cause on Facebook, or subscribe to the email newsletter. In contrast, we find little evidence that the campaign affected opinions, knowledge, or self-reported behavior. We conclude that social media campaigns that are commonplace among civil society organizations are effective at selecting activist-types, but changing the views and behaviors of the broader social media population may be more difficult than assumed.
While concerns about the public's receptivity to factual information are widespread, much less attention has been paid to the factual receptivity, or lack thereof, of elected officials. Recent survey research has made clear that US legislators and legislative staff systematically misperceive their constituents' opinions on salient public policies. This study reports the results from two field experiments designed to correct misperceptions of sitting US legislators. The legislators (n = 2,346) were invited to access a dashboard of constituent opinion generated using the 2016 Cooperative Congressional Election Study. Despite extensive outreach efforts, only 11 per cent accessed the information in Study 1 and only 2.3 per cent did so in Study 2. More troubling for democratic norms, legislators who accessed constituent opinion data were no more accurate at perceiving their constituents' opinions. The findings underscore the challenges confronting efforts to improve the accuracy of elected officials' perceptions and suggest that elected officials may indeed resist factual information.
To promote good governance, citizens can inform governments directly and routinely about the implementation of policies and the delivery of public services. Yet citizens lack incentives to provide information when they do not expect governments to be responsive, and citizen disengagement in turn often prevents governments from providing public goods effectively. In two field experiments, we studied potential remedies to this dilemma related to solid waste services in Uganda. We randomly assigned reporters to be recruited by community nomination and to be recognized by community leaders in an attempt to select for and motivate information sharing. We also randomly assigned reporters to hear from the government about how their reports were used to make real improvements to waste services. Community nominations and public announcements did not increase reporting. However, responsiveness boosted participation over several months for reporters who had been recruited earliest and had been reporting longest, highlighting the critical role of timely government responsiveness in sustaining information flows from citizens.
Behavioral economists and social psychologists have shown that extrinsic motivations can crowd out intrinsic motivations to act. This study examines this crowding out effect in the context of legislative behavior. By exploiting the federal nature of Swiss elections, we examine if response rates to requests of voters residing inside or outside a candidate's district vary based on the electoral competition candidate legislators face. We report two main findings. First, we find a high response rate among Swiss candidates (66 percent) which remains high for voters who reside outside a candidate's district (59 percent) suggesting that intrinsic motivations are a key driver of constituency effort. Second, the response to voters who reside inside a candidate's district is more pronounced for candidates confronted with a high degree of electoral competition. This suggests that extrinsic motivations are important for constituency work, but at the same time their presence might crowd out intrinsic motivations. This evidence suggests that the relationship between electoral competition and responsiveness might be less straightforward than assumed.