Over the last decade the survey environment has been rapidly changing, creating new opportunities and new challenges for survey methodologists. These include: relatively low response among all contact modes and designs; the rapid replacement of landline telephones for mobile and smart phones; the integration of new and different types of data for sampling and new sampling methods; the use of different survey interview modes; how to study behaviors and opinions about sensitive topics; and the development of survey designs for experts and opinion leaders. All have implications for data quality and for our ability to consistently and reliably answer substantive and theoretical questions about politics and society.

In this article, we present data from a three-mode survey comparison study carried out in 2010. National surveys were fielded at the same time over the Internet (using an opt-in Internet panel), by telephone with live interviews (using a national Random Digit Dialing (RDD) sample of landlines and cell phones), and by mail (using a national sample of residential addresses). Each survey utilized a nearly identical questionnaire soliciting information across a range of political and social indicators, many of which can be validated with government data. Comparing the findings from the modes using a Total Survey Error approach, we demonstrate that a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates.

We examine the quality of two probability-based polls, one interviewer administered (telephone) and one self-administered (Internet and mail mixed mode survey). The polls use the same sampling frame (registered voters) and the same questions. First, we examine the representativeness of both surveys using information known about the population, and although we find important differences between the two in terms of sampling and nonresponse bias, we also find that both surveys represent the underlying population despite low response rates. We also test for mode effects between surveys due to social desirability and how it influences nondifferentiation or satisficing. Using a variety of methods (t-tests, multivariate regression, and genetic propensity matching), we find evidence that the presence of an interviewer alters response patterns on ego-driven questions. The implications of our work are important, due to the increasing popularity of mixed mode surveys. Researchers need to be methodologically sensitive to these differences when analyzing surveys that allow for different response modes.

This article outlines a new method for surveys to study elections and voter attitudes. Pre-election surveys often suffer from an inability to identify and survey the likely electorate for the upcoming election. We propose a new and inexpensive method to conduct representative surveys of the electorate. We demonstrate the performance of our method in producing a representative sample of the future electorate that can be used to study campaign dynamics and many other issues. We compare pre-election outcome forecasts to election outcomes in seven primary and general election surveys conducted prior to the 2008 and 2010 primary and general elections in three states. The results indicate that the methodology produces representative samples, including in low-turnout elections such as primaries where traditional methods have difficulty consistently sampling the electorate. This new methodology combines Probability Proportional to Size (PPS) sampling, mailed invitation letters, and online administration of the questionnaire. The PPS sample is drawn based on a model employing variables from the publicly available voter file to produce a probability of voting score for each individual voter. The proposed method provides researchers a valuable tool to study the attitudes of the voting public.

Longitudinal or panel surveys offer unique benefits for social science research, but they typically suffer from attrition, which reduces sample size and can result in biased inferences. Previous research tends to focus on the demographic predictors of attrition, conceptualizing attrition propensity as a stable, individual-level characteristic—some individuals (e.g., young, poor, residentially mobile) are more likely to drop out of a study than others. We argue that panel attrition reflects both the characteristics of the individual respondent as well as her survey experience, a factor shaped by the design and implementation features of the study. In this article, we examine and compare the predictors of panel attrition in the 2008–2009 American National Election Study, an online panel, and the 2006–2010 General Social Survey, a face-to-face panel. In both cases, survey experience variables are predictive of panel attrition above and beyond the standard demographic predictors, but the particular measures of relevance differ across the two surveys. The findings inform statistical corrections for panel attrition bias and provide study design insights for future panel data collections.

Social scientists have increasingly turned to expert judgments to generate data for difficult-to-measure concepts, but getting access to and response from highly expert informants can be costly and challenging. We examine how informant selection and post-survey response aggregation influence the validity and reliability of measures built from informant observations. We draw upon three surveys with parallel survey questions of candidate characteristics to examine the trade-off between expanding the size of the local informant pool and the pool's level of expertise. We find that a “wisdom-of-crowds” effect trumps the benefits associated with the expertise of individual informants when the size of the rater pool is modestly increased. We demonstrate that the benefits of expertise are best realized by prescreening potential informants for expertise rather than post-survey weighting by expertise.

The development and proliferation of strategic estimators has narrowed the gap between theoretical models and empirical testing. But despite recent contributions that extend the basic strategic estimator, researchers have continued to neglect a classic social science phenomenon: selection. Compared to nonstrategic estimators, strategic models are even more prone to selection effects. First, external shocks or omitted variables can lead to correlated errors. Second, because the systematic parts of actors' utilities usually overlap on certain key variables, the two sets of explanatory variables are correlated. As a result, both the systematic and the stochastic components can be correlated. However, given that the estimates for the first mover are computed based on the potentially biased predicted probabilities of the second actor, we also generate biased estimates for the first actor. In applied work, researchers neglect the potential shortcomings due to selection bias. This article presents an alternative strategic estimator that takes selection into account and allows scholars to obtain consistent, unbiased, and efficient estimates in the presence of both selection and strategic action. I present a Monte Carlo analysis as well as a real-world application to illustrate the superior performance of this estimator relative to the standard practice.

Using a scale of child rearing preferences, scholars find that African Americans are far more authoritarian than Whites. We argue that this racial gap in authoritarianism is largely a measurement artifact. The child rearing scale now used to measure authoritarianism is cross-racially invalid because it draws heavily on a metaphor about hierarchy. Akin to someone who favors enforcing conformity in a child, the authoritarian is thought to be inclined toward enforcing conformity in social subordinates. In both cases, one's perspective is drawn from a position of relative power. We believe this metaphor is effective among members of a majority racial group because individual dominance at home meshes with group dominance in society. For members of a racial minority, we believe this metaphor breaks down. Using multi-group confirmatory factor analysis, we establish that Blacks and Whites construe the child rearing items differently. Consequently, authoritarianism correlates highly with the things it should for Whites, but rarely so for Blacks. Using an illegal immigration experiment, we then show divergent patterns of intolerance based on the same child rearing scale. Our results highlight measurement's role in producing large racial gaps in authoritarianism, while illuminating the racial boundaries of the child rearing scale.