Skip to main content Accessibility help

Empirical versus Theoretical Claims about Extreme Counterfactuals: A Response

  • Gary King (a1) and Langche Zeng (a2)


In response to the data-based measures of model dependence proposed in King and Zeng (2006), Sambanis and Michaelides (2008) propose alternative measures that rely upon assumptions untestable in observational data. If these assumptions are correct, then their measures are appropriate and ours, based solely on the empirical data, may be too conservative. If instead, and as is usually the case, the researcher is not certain of the precise functional form of the data generating process, the distribution from which the data are drawn, and the applicability of these modeling assumptions to new counterfactuals, then the data-based measures proposed in King and Zeng (2006) are much preferred. After all, the point of model dependence checks is to verify empirically, rather than to stipulate by assumption, the effects of modeling assumptions on counterfactual inferences.


Corresponding author

e-mail: (corresponding author)


Hide All

Author's note: Easy-to-use software to implement the methods discussed here, called “WhatIf: Software for Evaluating Counterfactuals,” is available at All information necessary to replicate the analyses herein can be found in King and Zeng (2008). Conflict of interest statement. None declared.



Hide All
Box, George E.P., Hunger, William G., and Stuart Hunter, J. 1978. Statistics for experimenters. New York: Wiley-Interscience.
Elekes, G. 1986. A geometric inequality and the complexity of computing volume. Discrete & Computational Geometry 1: 289–92.
Greene, William H. 2008. Econometric analysis. 6th ed. New York: Prentice Hall.
Hastie, Trevor, Tibshirani, Robert, and Friedman, Jerome. 2001. The elements of statistical learning: data mining, inference, and prediction. New York: Springer.
Ho, Daniel, Imai, Kosuke, King, Gary, and Stuart, Elizabeth. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15: 199236.
Imai, Kosuke, King, Gary, and Nall, Clayton. 2008. “The essential role of pair matching in cluster-randomized experiments, with application to the Mexican Universal Health Insurance Evaluation.” Unpublished manuscript, submitted to Statistical Science.
Imai, Kosuke, King, Gary, and Stuart, Elizabeth. 2008. Misunderstandings among experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society, Series A 171: 481502.
Jevons, W. Stanley. 1874. The principles of science: a treatise on logic and the scientific method. New York: MacMillen and Co.
King, Gary, Gakidou, Emmanuela, Ravishankar, Nirmala, Moore, Ryan T., Lakin, Jason, Vargas, Manett, María Téllez-Rojo, Martha, Eugenio Hernández Ávila, Juan, Hernández Ávila, Mauricio, and Hernández Llamas, Héctor. 2007. A ‘politically robust’ experimental design for public policy evaluation, with application to the Mexican universal health insurance program. Journal of Policy Analysis and Management 26: 479506.
King, Gary, and Zeng, Langche. 2006. The dangers of extreme counterfactuals. Political Analysis 14: 131–59.
King, Gary, and Zeng, Langche. 2007. When can history be our guide? The pitfalls of counterfactual inference.
International Studies Quarterly 51: 183210.
King, Gary, and Zeng, Langche. 2008. Replication data for: empirical vs. theoretical claims about extreme counterfactuals: a response. hdl:1902.1/11903, Murray Research Archive [Distributor].
Sambanis, Nicholas and Michaelides, Alexander. 2008. A Comment on Diagnostic Tools for Counterfactual Inference. Political Analysis. Advance Access published February 12, 2008, doi: 10.1093/pan/mpm032.
Twain, Mark. 1883. Life on the Mississippi. London: Chatto and Windus.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed