In the analysis of causal effects in non-experimental studies, conditioning on observable covariates is one way to try to reduce unobserved confounder bias. However, a developing literature has shown that conditioning on certain covariates may increase bias, and the mechanisms underlying this phenomenon have not been fully explored. We add to the literature on bias-increasing covariates by first introducing a way to decompose omitted variable bias into three constituent parts: bias due to an unobserved confounder, bias due to excluding observed covariates, and bias due to amplification. This leads to two important findings. Although instruments have been the primary focus of the bias amplification literature to date, we identify the fact that the popular approach of adding group fixed effects can lead to bias amplification as well. This is an important finding because many practitioners think that fixed effects are a convenient way to account for any and all group-level confounding and are at worst harmless. The second finding introduces the concept of bias unmasking and shows how it can be even more insidious than bias amplification in some cases. After introducing these new results analytically, we use constructed observational placebo studies to illustrate bias amplification and bias unmasking with real data. Finally, we propose a way to add bias decomposition information to graphical displays for sensitivity analysis to help practitioners think through the potential for bias amplification and bias unmasking in actual applications.

In recent years, political science has seen a boom in the use of sophisticated methodological tools for time series analysis. One such tool is the general error correction model (GECM), originally introduced to political scientists in the pages of this journal over 20 years ago (Durr 1992; Ostrom and Smith 1992) and re-introduced by De Boef and Keele (2008), who advocate its use for a wider set of time series data than previously considered appropriate. Their article has proven quite influential, with numerous papers justifying their methodological choices with reference to De Boef and Keele's contribution.

Grant and Lebo (2016) take issue with the increasing use of the GECM in political science and argue that the methodology is widely misused and abused by practitioners. Given the recent surge of research conducted using error correction methods, there is every reason to take their suggestions seriously and provide a fuller discussion of the points they raise in their paper. The present symposium serves such a role. It consists of Grant and Lebo's critique, a detailed response by Keele, Linn, and Webb (2016b), and shorter comments by Esarey (2016), Freeman (2016), and Helgason (2016). Finally, Lebo and Grant (2016) and Keele, Linn, and Webb (2016a) reflect on the contributions made in the symposium, as well as discuss outstanding issues.

Sensitive survey techniques (SSTs) are frequently used to study sensitive behaviors. However, existing strategies for employing SSTs lead to highly variable prevalence estimates and do not permit analysts to address the question of whether the use of an SST is actually necessary. The current article presents a survey questioning strategy and corresponding statistical framework that fills this gap. By jointly analyzing survey responses generated by an SST (the crosswise model) along with direct responses about the sensitive behavior, the article's framework addresses the question of whether the use of an SST is required to study a given sensitive behavior, provides an efficient estimate of the prevalence of the sensitive behavior, and, in its extended form, efficiently estimates how individual characteristics relate to the likelihood of engaging in the behavior. The utility of the approach is demonstrated through an examination of gender differences in proclivities towards corruption in Costa Rica.

Suppose that an experimenter has collected a sample as well as baseline information about the units in the sample. How should she allocate treatments to the units in this sample? We argue that the answer does not involve randomization if we think of experimental design as a statistical decision problem. If, for instance, the experimenter is interested in estimating the average treatment effect and evaluates an estimate in terms of the squared error, then she should minimize the expected mean squared error (MSE) through choice of a treatment assignment. We provide explicit expressions for the expected MSE that lead to easily implementable procedures for experimental design.

While traditionally considered for non-stationary and cointegrated data, DeBoef and Keele suggest applying a General Error Correction Model (GECM) to stationary data with or without cointegration. The GECM has since become extremely popular in political science but practitioners have confused essential points. For one, the model is treated as perfectly flexible when, in fact, the opposite is true. Time series of various orders of integration–stationary, non-stationary, explosive, near- and fractionally integrated–should not be analyzed together but researchers consistently make this mistake. That is, without equation balance the model is misspecified and hypothesis tests and long-run-multipliers are unreliable. Another problem is that the error correction term's sampling distribution moves dramatically depending upon the order of integration, sample size, number of covariates, and the boundedness of Yt. This means that practitioners are likely to overstate evidence of error correction, especially when using a traditional t-test. We evaluate common GECM practices with six types of data, 746 simulations, and five paper replications.

Political scientists increasingly recognize that multiple imputation represents a superior strategy for analyzing missing data to the widely used method of listwise deletion. However, there has been little systematic investigation of how multiple imputation affects existing empirical knowledge in the discipline. This article presents the first large-scale examination of the empirical effects of substituting multiple imputation for listwise deletion in political science. The examination focuses on research in the major subfield of comparative and international political economy (CIPE) as an illustrative example. Specifically, I use multiple imputation to reanalyze the results of almost every quantitative CIPE study published during a recent five-year period in International Organization and World Politics, two of the leading subfield journals in CIPE. The outcome is striking: in almost half of the studies, key results “disappear” (by conventional statistical standards) when reanalyzed.

Political scientists increasingly use instrumental variable (IV) methods, and must often choose between operationalizing their endogenous treatment variable as discrete or continuous. For theoretical and data availability reasons, researchers frequently coarsen treatments with multiple intensities (e.g., treating a continuous treatment as binary). I show how such coarsening can substantially upwardly bias IV estimates by subtly violating the exclusion restriction assumption, and demonstrate that the extent of this bias depends upon the first stage and underlying causal response function. However, standard IV methods using a treatment where multiple intensities are affected by the instrument–even when fine-grained measurement at every intensity is not possible–recover a consistent causal estimate without requiring a stronger exclusion restriction assumption. These analytical insights are illustrated in the context of identifying the long-run effect of high school education on voting Conservative in Great Britain. I demonstrate that coarsening years of schooling into an indicator for completing high school upwardly biases the IV estimate by a factor of three.

We present new methods to estimate causal effects retrospectively from micro data with the assistance of a machine learning ensemble. This approach overcomes two important limitations in conventional methods like regression modeling or matching: (i) ambiguity about the pertinent retrospective counterfactuals and (ii) potential misspecification, overfitting, and otherwise bias-prone or inefficient use of a large identifying covariate set in the estimation of causal effects. Our method targets the analysis toward a well-defined “retrospective intervention effect” based on hypothetical population interventions and applies a machine learning ensemble that allows data to guide us, in a controlled fashion, on how to use a large identifying covariate set. We illustrate with an analysis of policy options for reducing ex-combatant recidivism in Colombia.

When facing small numbers of observations or rare events, political scientists often encounter separation, in which explanatory variables perfectly predict binary events or nonevents. In this situation, maximum likelihood provides implausible estimates and the researcher might want incorporate some form of prior information into the model. The most sophisticated research uses Jeffreys’ invariant prior to stabilize the estimates. While Jeffreys’ prior has the advantage of being automatic, I show that it often provides too much prior information, producing smaller point estimates and narrower confidence intervals than even highly skeptical priors. To help researchers assess the amount of information injected by the prior distribution, I introduce the concept of a partial prior distribution and develop the tools required to compute the partial prior distribution of quantities of interest, estimate the subsequent model, and summarize the results.

In this article, we highlight three points. First, we counter Grant and Lebo's claim that the error correction model (ECM) cannot be applied to stationary data. We maintain that when data are properly stationary, the ECM is an entirely appropriate model. We clarify that for a model to be properly stationary, it must be balanced. Second, we contend that while fractional integration techniques can be useful, they also have important weaknesses, especially when applied to many time series typical in political science. We also highlight two related but often ignored complications in time series: low power and overfitting. We argue that the statistical tests used in time-series analyses have little power to detect differences in many of the sample sizes typical in political science. Moreover, given the small sample sizes, many analysts overfit their time-series models. Overfitting occurs when a statical model describes random error or noise instead of the underlying relationship. We argue that the results in the Grant and Lebo replications could easily be a function of overfitting.

In this article, we evaluate the usefulness of Google Consumer Surveys (GCS) as a low-cost tool for doing rigorous social scientific work. We find that its relative strengths and weaknesses make it most useful to researchers who attempt to identify causality through randomization to treatment groups rather than selection on observables. This finding stems, in part, from the fact that the real cost advantage of GCS over other alternatives is limited to short surveys with a small number of questions. Based on our replication of four canonical social scientific experiments and one study of treatment heterogeneity, we find that the platform can be used effectively to achieve balance across treatment groups, explore treatment heterogeneity, include manipulation checks, and that the provided inferred demographics may be sufficiently sound for weighting and explorations of heterogeneity. Crucially, we successfully managed to replicate the usual directional finding in each experiment. Overall, GCS is likely to be a useful platform for survey experimentalists.

Two contributions in this issue, Grant and Lebo and Keele, Linn, and Webb, recommend using an ARFIMA model to diagnose the presence of and estimate the degree of fractional integration, then either (i) fractionally differencing the data before analysis or, (ii) for cointegrated variables, estimating a fractional error correction model. But Keele, Linn, and Webb also present evidence that ARFIMA models yield misleading indicators of the presence and degree of fractional integration in a series with fewer than 1000 observations. In a simulation study, I find evidence that the simple autodistributed lag model (ADL) or equivalent error correction model (ECM) can, without first testing or correcting for fractional integration, provide a useful estimate of the immediate and long-run effects of weakly exogenous variables in fractionally integrated (but stationary) data.

Existing methods for estimating ideal points of legislators that are comparable across time and chambers make restrictive assumptions regarding how legislators' ideal points can move over time, either by fixing some legislators' ideal points or by constraining their movement over time. These assumptions are clearly contradictory to some theories of congressional responsiveness to election dynamics and changes in constituency. Instead of using legislators as anchors, our approach relies on matching roll calls in one chamber and session with roll calls or cosponsorship decisions on identical bills introduced in a different chamber or session. By using these “bridge decisions” to achieve comparability, we can remove any assumptions about the movement of legislators' ideal points. We produce these estimates for both chambers from the 102nd (1991–92) to 111th (2009–11) Congresses, and we show that our estimates provide interesting insights into the nature of legislative behavior change.

Many political processes consist of a series of theoretically meaningful transitions across discrete phases that occur through time. Yet political scientists are often theoretically interested in studying not just individual transitions between phases, but also the duration that subjects spend within phases, as well as the effect of covariates on subjects’ trajectories through the process's multiple phases. We introduce the multistate survival model to political scientists, which is capable of modeling precisely this type of situation. The model is appealing because of its ability to accommodate multiple forms of causal complexity that unfold over time. In particular, we highlight three attractive features of multistate models: transition-specific baseline hazards, transition-specific covariate effects, and the ability to estimate transition probabilities. We provide two applications to illustrate these features.

Existing approaches to measuring political disagreement from text data perform poorly except when applied to narrowly selected texts discussing the same issues and written in the same style. We demonstrate the first viable approach for estimating legislator-specific scores from the entire speech corpus of a legislature, while also producing extensive information about the evolution of speech polarization and politically loaded language. In the Irish Dáil, we show that the dominant dimension of speech variation is government–opposition, with ministers more extreme on this dimension than backbenchers, and a second dimension distinguishing between the establishment and anti-establishment opposition parties. In the U. S. Senate, we estimate a dimension that has moderate within-party correlations with scales based on roll-call votes and campaign donation patterns; however, we observe greater overlap across parties in speech positions than roll-call positions and partisan polarization in speeches varies more clearly in response to major political events.

Models of ideal point estimation usually build on the assumption of spatial preferences. This ignores legislators' non-policy incentives and is thus likely to produce implausible results for many legislatures. We study this problem in parliamentary systems and develop a model of roll call voting that considers both the policy and the non-policy, tactical incentives of legislators. We go on to show how the relative weight of these policy and tactical incentives is influenced by the identity of the mover and characteristics of the motion. Analyses of two data sets of 2174 roll call votes in German state legislatures and 3295 roll call votes in the British House of Commons result in three main findings. First, we show that tactical incentives may be more important than policy incentives, and second, that the importance of tactical incentives varies with the importance of motions. Third, there are interesting twists: backbench private members' bills may reverse tactical incentives whereas proposals from anti-system parties are virtually always rejected by moderate parties, rendering these votes uninformative. Our findings have implications for ideal point estimation in parliamentary systems, as well as for research on separation of power systems.