To send content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about sending content to .
To send content items to your Kindle, first ensure email@example.com
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about sending to your Kindle.
Note you can select to send to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
How can we elicit honest responses in surveys? Conjoint analysis has become a popular tool to address social desirability bias (SDB), or systematic survey misreporting on sensitive topics. However, there has been no direct evidence showing its suitability for this purpose. We propose a novel experimental design to identify conjoint analysis’s ability to mitigate SDB. Specifically, we compare a standard, fully randomized conjoint design against a partially randomized design where only the sensitive attribute is varied between the two profiles in each task. We also include a control condition to remove confounding due to the increased attention to the varying attribute under the partially randomized design. We implement this empirical strategy in two studies on attitudes about environmental conservation and preferences about congressional candidates. In both studies, our estimates indicate that the fully randomized conjoint design could reduce SDB for the average marginal component effect (AMCE) of the sensitive attribute by about two-thirds of the AMCE itself. Although encouraging, we caution that our results are exploratory and exhibit some sensitivity to alternative model specifications, suggesting the need for additional confirmatory evidence based on the proposed design.
Conjoint survey experiments have become a popular method for analyzing multidimensional preferences in political science. If properly implemented, conjoint experiments can obtain reliable measures of multidimensional preferences and estimate causal effects of multiple attributes on hypothetical choices or evaluations. This chapter provides an accessible overview of the methodology for designing, implementing, and analyzing conjoint survey experiments. Specically, we begin by detailing a new substantive example: how do candidate attributes affect the support of American respondents for candidates running against President Trump in 2020? We then discuss the theoretical underpinnings and key advantages of conjoint designs. We next provide guidelines for practitioners in designing and analyzing conjoint survey experiments. We conclude by discussing further design considerations, common conjoint applications, common criticisms, and possible future directions.
One of the strongest findings across the sciences is that publication bias occurs. Of particular note is a “file drawer bias” where statistically significant results are privileged over nonsignificant results. Recognition of this bias, along with increased calls for “open science,” has led to an emphasis on replication studies. Yet, few have explored publication bias and its consequences in replication studies. We offer a model of the publication process involving an initial study and a replication. We use the model to describe three types of publication biases: (1) file drawer bias, (2) a “repeat study” bias against the publication of replication studies, and (3) a “gotcha bias” where replication results that run contrary to a prior study are more likely to be published. We estimate the model’s parameters with a vignette experiment conducted with political science professors teaching at Ph.D. granting institutions in the United States. We find evidence of all three types of bias, although those explicitly involving replication studies are notably smaller. This bodes well for the replication movement. That said, the aggregation of all of the biases increases the number of false positives in a literature. We conclude by discussing a path for future work on publication biases.
Does media choice cause polarization, or merely reflect it? We investigate a critical aspect of this puzzle: How partisan media contribute to attitude polarization among different groups of media consumers. We implement a new experimental design, called the Preference-Incorporating Choice and Assignment (PICA) design, that incorporates both free choice and forced exposure. We estimate jointly the degree of polarization caused by selective exposure and the persuasive effect of partisan media. Our design also enables us to conduct sensitivity analyses accounting for discrepancies between stated preferences and actual choice, a potential source of bias ignored in previous studies using similar designs. We find that partisan media can polarize both its regular consumers and inadvertent audiences who would otherwise not consume it, but ideologically opposing media potentially also can ameliorate the existing polarization between consumers. Taken together, these results deepen our understanding of when and how media polarize individuals.
To investigate the association between suicide death and serum cholesterol levels as measured at times close to suicide death.
We conducted a nested case-control study of 41 cases of suicide deaths and 205 matched controls with serum total cholesterol (TC) levels till 3 years before suicide death in a large cohort of Japanese workers.
Individuals in the lowest versus highest tertile/predefined category of TC in a Japanese working population had a three- to four-fold greater risk of suicide death. Each 10 mg/dl decrement of average TC was associated with an 18% increased chance of suicide death (95% confidence interval, 2–35%). Similar results were found for TC levels at each year.
These results suggest that a low serum TC level in recent past is associated with an increased risk of suicide death.
Recent years have seen a renaissance of conjoint survey designs within social science. To date, however, researchers have lacked guidance on how many attributes they can include within conjoint profiles before survey satisficing leads to unacceptable declines in response quality. This paper addresses that question using pre-registered, two-stage experiments examining choices among hypothetical candidates for US Senate or hotel rooms. In each experiment, we use the first stage to identify attributes which are perceived to be uncorrelated with the attribute of interest, so that their effects are not masked by those of the core attributes. In the second stage, we randomly assign respondents to conjoint designs with varying numbers of those filler attributes. We report the results of these experiments implemented via Amazon's Mechanical Turk and Survey Sampling International. They demonstrate that our core quantities of interest are generally stable, with relatively modest increases in survey satisficing when respondents face large numbers of attributes.
Although politicians’ personal attributes are an important component of elections and representation, few studies have rigorously investigated which attributes are most relevant in shaping voters’ preferences for politicians, or whether these preferences vary across different electoral system contexts. We investigate these questions with a conjoint survey experiment using the case of Japan’s mixed-member bicameral system. We find that the attributes preferred by voters are not entirely consistent with the observed attributes of actual politicians. Moreover, voters’ preferences do not vary when asked to consider representation under different electoral system contexts, whereas the observed attributes of politicians do vary across these contexts. These findings point to the role of factors beyond voters’ sincere preferences, such as parties’ recruitment strategies, the effect of electoral rules on the salience of the personal vote, and the availability of different types of politicians, in determining the nature of representation.
Representative democracy entails the aggregation of multiple policy issues by parties into competing bundles of policies, or “manifestos,” which are then evaluated holistically by voters in elections. This aggregation process obscures the multidimensional policy preferences underlying a voter’s single choice of party or candidate. We address this problem through a conjoint experiment based on the actual party manifestos in Japan’s 2014 House of Representatives election. By juxtaposing sets of issue positions as hypothetical manifestos and asking respondents to choose one, our study identifies the effects of specific positions on the overall assessment of manifestos, heterogeneity in preferences among subgroups of respondents, and the popularity ranking of manifestos. Our analysis uncovers important discrepancies between voter preferences and the portrayal of the election results by politicians and the media as providing a policy mandate to the Liberal Democratic Party, underscoring the potential danger of inferring public opinion from election outcomes alone.
In recent years, political and social scientists have made increasing use of conjoint survey designs to study decision-making. Here, we study a consequential question which researchers confront when implementing conjoint designs: How many choice tasks can respondents perform before survey satisficing degrades response quality? To answer the question, we run a set of experiments where respondents are asked to complete as many as 30 conjoint tasks. Experiments conducted through Amazon’s Mechanical Turk and Survey Sampling International demonstrate the surprising robustness of conjoint designs, as there are detectable but quite limited increases in survey satisficing as the number of tasks increases. Our evidence suggests that in similar study contexts researchers can assign dozens of tasks without substantial declines in response quality.
Survey experiments are a core tool for causal inference. Yet, the design of classical survey experiments prevents them from identifying which components of a multidimensional treatment are influential. Here, we show how conjoint analysis, an experimental design yet to be widely applied in political science, enables researchers to estimate the causal effects of multiple treatment components and assess several causal hypotheses simultaneously. In conjoint analysis, respondents score a set of alternatives, where each has randomly varied attributes. Here, we undertake a formal identification analysis to integrate conjoint analysis with the potential outcomes framework for causal inference. We propose a new causal estimand and show that it can be nonparametrically identified and easily estimated from conjoint data using a fully randomized design. The analysis enables us to propose diagnostic checks for the identification assumptions. We then demonstrate the value of these techniques through empirical applications to voter decision making and attitudes toward immigrants.
Social scientists are often interested in testing multiple causal mechanisms through which a treatment affects outcomes. A predominant approach has been to use linear structural equation models and examine the statistical significance of the corresponding path coefficients. However, this approach implicitly assumes that the multiple mechanisms are causally independent of one another. In this article, we consider a set of alternative assumptions that are sufficient to identify the average causal mediation effects when multiple, causally related mediators exist. We develop a new sensitivity analysis for examining the robustness of empirical findings to the potential violation of a key identification assumption. We apply the proposed methods to three political psychology experiments, which examine alternative causal pathways between media framing and public opinion. Our analysis reveals that the validity of original conclusions is highly reliant on the assumed independence of alternative causal mechanisms, highlighting the importance of proposed sensitivity analysis. All of the proposed methods can be implemented via an open source R package, mediation.
Identifying causal mechanisms is a fundamental goal of social science. Researchers seek to study not only whether one variable affects another but also how such a causal relationship arises. Yet commonly used statistical methods for identifying causal mechanisms rely upon untestable assumptions and are often inappropriate even under those assumptions. Randomizing treatment and intermediate variables is also insufficient. Despite these difficulties, the study of causal mechanisms is too important to abandon. We make three contributions to improve research on causal mechanisms. First, we present a minimum set of assumptions required under standard designs of experimental and observational studies and develop a general algorithm for estimating causal mediation effects. Second, we provide a method for assessing the sensitivity of conclusions to potential violations of a key assumption. Third, we offer alternative research designs for identifying causal mechanisms under weaker assumptions. The proposed approach is illustrated using media framing experiments and incumbency advantage studies.
Email your librarian or administrator to recommend adding this to your organisation's collection.