Skip to main content Accessibility help
×
×
Home

A Theory of Statistical Inference for Matching Methods in Causal Research

  • Stefano M. Iacus (a1), Gary King (a2) and Giuseppe Porro (a3)

Abstract

Researchers who generate data often optimize efficiency and robustness by choosing stratified over simple random sampling designs. Yet, all theories of inference proposed to justify matching methods are based on simple random sampling. This is all the more troubling because, although these theories require exact matching, most matching applications resort to some form of ex post stratification (on a propensity score, distance metric, or the covariates) to find approximate matches, thus nullifying the statistical properties these theories are designed to ensure. Fortunately, the type of sampling used in a theory of inference is an axiom, rather than an assumption vulnerable to being proven wrong, and so we can replace simple with stratified sampling, so long as we can show, as we do here, that the implications of the theory are coherent and remain true. Properties of estimators based on this theory are much easier to understand and can be satisfied without the unattractive properties of existing theories, such as assumptions hidden in data analyses rather than stated up front, asymptotics, unfamiliar estimators, and complex variance calculations. Our theory of inference makes it possible for researchers to treat matching as a simple form of preprocessing to reduce model dependence, after which all the familiar inferential techniques and uncertainty calculations can be applied. This theory also allows binary, multicategory, and continuous treatment variables from the outset and straightforward extensions for imperfect treatment assignment and different versions of treatments.

Copyright

Corresponding author

Footnotes

Hide All

Authors’ note: Our thanks to Alberto Abadie, Adam Glynn, Kosuke Imai, and Molly Roberts for helpful comments on an earlier draft. The replication code can be found in Iacus (2018).

Contributing Editor: Jonathan N. Katz

Footnotes

References

Hide All
Abadie, Alberto, and Imbens, Guido W.. 2002. Simple and bias-corrected matching estimators for average treatment effects. NBER Technical Working Paper 283.
Abadie, Alberto, and Imbens, Guido W.. 2006. Large sample properties of matching estimators for average treatment effects. Econometrica 74(1):235267.
Abadie, Alberto, and Imbens, Guido W.. 2011. Bias-corrected matching estimators for average treatment effects. Journal of Business & Economic Statistics 29(1):111.
Abadie, Alberto, and Imbens, Guido W.. 2012. A martingale representation for matching estimators. Journal of the American Statistical Association 107(498):833843.
Angrist, Joshua D., Imbens, Guido W., and Rubin, Donald B.. 1996. Identification of causal effects using instrumental variables (with discussion). Journal of the American Statistical Association 91(434):444455.
Berkson, Joseph. 1950. Are there two regressions? Journal of the American Statistical Association 45(250):164180.
Cochran, William G. 1968. The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics 24:295313.
Cochran, William G., and Rubin, Donald B.. 1973. Controlling bias in observational studies: A review. Sankhya: The Indian Journal of Statistics, Series A 35, Part 4:417466.
Cox, David R. 1958. Planning of experiments . New York: John Wiley.
Crump, Richard K., Hotz, V. Joseph, Imbens, Guido W., and Mitnik, Oscar. 2009. Dealing with limited overlap in estimation of average treatment effects. Biometrika 96(1):187.
De Crescenzo, Antonio. 1999. A Probabilistic analogue of the mean value theorem and its applications to reliability theory. Journal of Applied Probability 36:706719.
Dehejia, Rajeev H., and Wahba, Sadek. 1999. Causal effects in nonexperimental studies: Re-evaluating the evaluation of training programs. Journal of the American Statistical Association 94(448):10531062.
Ding, Peng. 2016. A paradox from randomization-based causal inference. Preprint, arXiv:1402.0142.
Heitjan, D. F., and Rubin, Donald B.. 1991. Ignorability and coarse data. The Annals of Statistics 19(4):22442253.
Ho, Daniel E., Imai, Kosuke, King, Gary, and Stuart, Elizabeth A.. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15(3):199236.
Holland, Paul W. 1986. Statistics and causal inference. Journal of the American Statistical Association 81:945960.
Hyslop, Dean R., and Imbens, Guido W.. 2001. Bias from classical and other forms of measurement error. Journal of Business and Economic Statistics 19(4):475481.
Iacus, Stefano M., King, Gary, and Porro, Giuseppe. 2011. Multivariate matching methods that are monotonic imbalance bounding. Journal of the American Statistical Association 106(493):345361.
Iacus, Stefano. 2018. Replication script for Iacus, King, Porro (2018), “A Theory of Statistical Inference for Matching Methods in Causal Research”. https://doi.org/10.7910/DVN/AOY452, Harvard Dataverse, V1, UNF:6:OEbDh6lIbV89a2sMhvelCQ==.
Imai, Kosuke. 2008. Variance identification and efficiency analysis in randomized experiments under the matched-pair design. Statistics in Medicine 27(24):4857.
Imai, Kosuke, and van Dyk, David A.. 2004. Causal inference with general treatment treatment regimes: Generalizing the propensity score. Journal of the American Statistical Association 99(467):854866.
Imai, Kosuke, King, Gary, and Nall, Clayton. 2009. The essential role of pair matching in cluster-randomized experiments, with application to the Mexican universal health insurance evaluation. Statistical Science 24(1):2953.
Imbens, Guido W. 2000. The role of the propensity score in estimating the dose-response functions. Biometrika 87(3):706710.
Imbens, Guido W. 2004. Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and Statistics 86(1):429.
Imbens, Guido W., and Wooldridge, J. M.. 2009. Recent developments in the econometrics of program evaluation. Journal of Economic Literature 47(1):586.
King, Gary, Lucas, Christopher, and Nielsen, Richard A.. 2017. The balance-sample size frontier in matching methods for causal inference. American Journal of Political Science 61(2):473489.
King, Gary, and Nielsen, Richard A.. 2017. Why propensity scores should not be used for matching. Working Paper. URL: http://j.mp/PSMnot.
King, Gary, and Zeng, Langche. 2006. The dangers of extreme counterfactuals. Political Analysis 14(2):131159.
Lalonde, Robert. 1986. Evaluating the econometric evaluations of training programs. American Economic Review 76:604620.
Lechner, Michael. 2001. Identification and estimation of causal effects of multiple treatments under the conditional independence assumption. In Econometric evaluation of labour market policies , ed. Lechner, M. and Pfeiffer, F.. Heidelberg: Physica, pp. 4358.
Lin, Winston. 2013. Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique. The Annals of Applied Statistics 7(1):295318.
Mielke, P. W., and Berry, K. J.. 2007. Permutation methods: A distance function approach . New York: Springer.
Miratrix, Luke W., Sekhon, Jasjeet S., and Yu, Bin. 2013. Adjusting treatment effect estimates by post-stratification in randomized experiments. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 75(2):369396.
Morgan, Stephen L., and Winship, Christopher. 2014. Counterfactuals and causal inference: Methods and principles for social research , 2nd edn Cambridge: Cambridge University Press.
Neyman, J. 1935. Statistical problems in agricultural experimentation. Journal of the Royal Statistical Society II 2:107154.
Robins, James M. 1986. A new approach to causal inference in mortality studies with sustained exposure period - application to control of the healthy worker survivor effect. Mathematical Modelling 7:13931512.
Rosenbaum, Paul R. 1988. Permutation tests for matched pairs with adjustments for covariates. Applied Statistics 37(3):401411.
Rosenbaum, Paul R., and Rubin, Donald B.. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70:4155.
Rubin, Donald B. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 6:688701.
Rubin, Donald B. 1977. Assignment to treatment group on the basis of a covariate. Journal of Educational Statistics 2(1–26):1.
Rubin, Donald B. 1980. Comments on “Randomization analysis of experimental data: The Fisher randomization test,” by D. Basu. Journal of the American Statistical Association 75:591593.
Rubin, Donald B. 1990. On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Comment: Neyman (1923) and causal inference in experiments and observational studies. Statistical Science 5(4):472480.
Rubin, Donald B. 1991. Practical implications of modes of statistical inference for causal effects and the critical role of the assignment mechanism. Biometrics 47:12131234.
Rubin, Donald B. 2010. On the limitations of comparative effectiveness research. Statistics in Medicine 29(19):19911995.
Smith, Jeffrey A., and Todd, Petra E.. 2005. Does matching overcome LaLonde’s critique of nonexperimental estimators? Journal of Econometrics 125(1–2):305353.
Stuart, Elizabeth A. 2010. Matching methods for causal inference: A review and a look forward. Statistical Science 25(1):121.
VanderWeele, Tyler J., and Hernan, Miguel A.. 2012. Causal inference under multiple versions of treatment. Journal of Causal Inference 1:120.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed