Skip to main content Accessibility help
×
×
Home

Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies

  • KOSUKE IMAI (a1), LUKE KEELE (a2), DUSTIN TINGLEY (a3) and TEPPEI YAMAMOTO (a4)

Abstract

Identifying causal mechanisms is a fundamental goal of social science. Researchers seek to study not only whether one variable affects another but also how such a causal relationship arises. Yet commonly used statistical methods for identifying causal mechanisms rely upon untestable assumptions and are often inappropriate even under those assumptions. Randomizing treatment and intermediate variables is also insufficient. Despite these difficulties, the study of causal mechanisms is too important to abandon. We make three contributions to improve research on causal mechanisms. First, we present a minimum set of assumptions required under standard designs of experimental and observational studies and develop a general algorithm for estimating causal mediation effects. Second, we provide a method for assessing the sensitivity of conclusions to potential violations of a key assumption. Third, we offer alternative research designs for identifying causal mechanisms under weaker assumptions. The proposed approach is illustrated using media framing experiments and incumbency advantage studies.

Copyright

Corresponding author

Kosuke Imai is Associate Professor, Department of Politics, Princeton University, Corwin Hall 036, Princeton NJ 08544 (kimai@princeton.edu).
Luke Keele is Assistant Professor, Department of Political Science, Pennsylvania State University, 211 Pond Lab, University Park, PA 16802 (ljk20@psu.edu).
Dustin Tingley is Assistant Professor, Department of Government, Harvard University, 1737 Cambridge Street, CGIS Knafel Building 208, Cambridge MA 02138 (dtingley@gov.harvard.edu).
Teppei Yamamoto is Assistant Professor, Department of Political Science, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139 (teppei@mit.edu).

References

Hide All
Albert, J. 2008. “Mediation Analysis via Potential Outcomes Models.” Statistics in Medicine 27: 12821304.
Angrist, J. D., Imbens, G. W., and Rubin, D. B.. 1996. “Identification of Causal Effects Using Instrumental Variables (with Discussion).” Journal of the American Statistical Association 91 (434): 444–55.
Ansolabehere, S., Snowberg, E. C., and Snyder, J. M.. 2006. “Television and the Incumbency Advantage in U.S. Elections.” Legislative Studies Quarterly 31 (4): 469–90.
Ansolabehere, S., Snyder, J. M., and Stewart, C.. 2000. “Old Voters, New Voters, and the Personal Vote: Using Redistricting to Measure the Incumbency Advantage.” American Journal of Political Science 44 (1): 1734.
Baron, R. M., and Kenny, D. A.. 1986. “The Moderator–Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations.” Journal of Personality and Social Psychology 51 (6): 1173–82.
Bartels, L. M. 1993. “Messages Received: The Political Impact of Media Exposure.” American Political Science Review 87 (2): 267–85.
Bertrand, M., and Mullainathan, S.. 2004. “Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination.” American Economic Review 94 (4): 9911013.
Blattman, C. 2009. “From Violence to Voting: War and Political Participation in Uganda.” American Political Science Review 103 (2): 231–47.
Brader, T., Valentino, N. A., and Suhay, E.. 2008. “What Triggers Public Opposition to Immigration? Anxiety, Group Cues, and Immigration.” American Journal of Political Science 52 (4): 959–78.
Brady, H. E., and Collier, D.. 2004. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham, MD: Rowman and Littlefield.
Bullock, J., Green, D., and Ha, S.. 2010. “Yes, But What's the Mechanism? (Don't Expect an Easy Answer).” Journal of Personality and Social Psychology 98 (4): 550–58.
Chong, D., and Druckman, J.. 2007. “Framing Theory.” Annual Review of Political Science 10: 103–26.
Cnudde, C. F., and McCrone, D. J.. 1966. “The Linkage between Constituency Attitudes and Congressional Voting Behavior: A Causal Model.” American Political Science Review 60 (1): 6672.
Cole, S. R., and Frangakis, C. E.. 2009. “The Consistency Statement in Causal Inference: A Definition or Assumption?Epidemiology 20 (1): 35.
Collier, D., Brady, H. E., and Seawright, J.. 2004. “Source of Leverage in Causal Inference: Toward an Alternative View of Methodology.” In Rethinking Social Inquiry:Diverse Tools, Shared Standards eds. Brady, H. and Collier, D., Berkeley, CA: Rowman and Littlefield.
Cox, G. W., and Katz, J. N.. 1996. “Why Did the Incumbency Advantage in U.S. House Elections Grow?American Journal of Political Science 40 (2): 478–97.
Deaton, A. 2010a. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.
Deaton, A. 2010b. “Understanding the Mechanisms of Economic Development.” Journal of Economic Perspectives 24 (3): 316.
Druckman, J. 2005. “Media Matter: How Newspapers and Television News Cover Campaigns and Influence Voters.” American Political Science Review 22: 463–81.
Erikson, R. S., and Palfrey, T. R.. 1998. “Campaign Spending and Incumbency: An Alternative Simultaneous Equations Approach.” Journal of Politics 60 (2): 355–73.
Gadarian, S. K. 2010. “The Politics of Threat: How Terrorism News Shapes Foreign Policy Attitudes.” Journal of Politics 72 (2): 469–83.
Gelman, A., and King, G.. 1990. “Estimating Incumbency Advantage without Bias.” American Journal of Political Science 34 (4): 1142–64.
Gerber, A. 1998. “Estimating the Effect of Campaign Spending on Senate Election Outcomes Using Instrumental Variables.” American Political Science Review 92 (2): 401–11.
Glynn, A. N. 2010. “The Product and Difference Fallacies for Indirect Effects.” Department of Government, Harvard University. Unpublished manuscript, Mimeo.
Green, D. P., Ha, S. E., and Bullock, J. G.. 2010. “Enough Already about Black Box Experiments: Studying Mediation Is More Difficult Than Most Scholars Suppose.” Annals of the American Academy of Political and Social Sciences 628 (1): 200–08.
Gross, J. J., and Levenson, R. W.. 1995. “Eliciting Emotions Using Films.” Cognition and Emotion 9 (1): 87108.
Haavelmo, T. 1943. “The Statistical Implications of a System of Simultaneous Equations.” Econometrica 11: 112.
Heckman, J. J., and Smith, J. A.. 1995. “Assessing the Case for Social Experiments.” Journal of Economic Perspectives 9 (2): 85110.
Hetherington, M. J. 2001. “Resurgent Mass Partisanship: The Role of Elite Polarization.” American Political Science Review 95 (3): 619–31.
Ho, D. E., Imai, K., King, G., and Stuart, E. A.. 2007. “Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference.” Political Analysis 15 (3): 199236.
Holland, P. W. 1986. “Statistics and Causal Inference.” Journal of the American Statistical Association 81: 945–60.
Holland, P. W. 1988. “Causal Inference, Path Analysis, and Recursive Structural Equations Models.” Sociological Methodology 18: 449–84.
Horiuchi, Y., Imai, K., and Taniguchi, N.. 2007. “Designing and Analyzing Randomized Experiments: Application to a Japanese Election Survey Experiment.” American Journal of Political Science 51 (3): 669–87.
Imai, K., Keele, L., and Tingley, D.. 2010. “A General Approach to Causal Mediation Analysis.” Psychological Methods 15 (4): 309–34.
Imai, K., Keele, L., Tingley, D. and Yamamoto, T.. 2010. “Causal Mediation Analysis Using R”. In Advances in Social Science Research Using R, ed. Vinod, H. D., Lecture Notes in Statistics. Springer-verlag: New York, 129–54.
Imai, K., Keele, L., Tingley, D., and Yamamoto, T.. 2011. “Replication Data for: Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies.” The Dataverse Network. hdl:1902.1/16467 (accessed September 1, 2011).
Imai, K., Keele, L., and Yamamoto, T.. 2010. “Identification, Inference, and Sensitivity Analysis for Causal Mediation Effects.” Statistical Science 25 (1): 5171.
Imai, K., King, G., and Stuart, E. A.. 2008. “Misunderstandings among Experimentalists and Observationalists about Causal Inference.” Journal of the Royal Statistical Society, Series A (Statistics in Society) 171 (2): 481502.
Imai, K., and Tingley, D.. N.d. “A Statistical Method for Empirical Testing of Competing Theories.” American Journal of Political Science. Forthcoming.
Imai, K., Tingley, D., and Yamamoto, T.. N.d. “Experimental Designs for Identifying Causal Mechanisms.” (With discussions). Journal of the Royal Statistical Society, Series A (Statistics in Society). Forthcoming.
Imai, K., and Yamamoto, T.. 2010. “Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis.” American Journal of Political Science 54 (2): 543–60.
Imai, K., and Yamamoto, T.. 2011. “Sensitivity Analysis for Causal Mediation Effects under Alternative Exogeneity Assumptions.” http://imai.princeton.edu/research/medsens.html. (accessed September 1, 2011).
Isbell, L., and Ottati, V.. 2002. “The Emotional Voter.” In The Social Psychology of Politics, ed. Ottati, V., New York: Kluwer, 5574.
Jacobson, G. C. 1987. “The Politics of Congressional Elections. Boston: Little, Brown.
Jo, B. 2008. “Causal Inference in Randomized Experiments with Mediational Processes.” Psychological Methods 13 (4): 314–36.
Jost, J. T., Napier, J. L., Thorisdottir, H., Gosling, S. D., Palfai, T. P., and Ostafin, B.. 2007. “Are Needs to Manage Uncertainty and Threat Associated With Political Conservatism or Ideological Extremity?Personality and Social Psychology Bulletin 33 (7): 9891007.
Kinder, D. R. and Sanders, L.. 1996. Divided by Color: Racial Politics and Democratic Ideals. Chicago: University of Chicago Press.
King, G., Keohane, R. O., and Verba, S.. 1994. Designing Social Inquiry. Princeton, NJ: Princeton University Press.
King, G., Tomz, M., and Wittenberg, J.. 2000. “Making the Most of Statistical Analyses: Improving Interpretation and Presentation.” American Journal of Political Science 44: 341–55.
Levitt, S. D. and Wolfram, C. D.. 1997. “Decomposing the Sources of Incumbency Advantage in the U.S. House.” Legislative Studies Quarterly 22 (1): 4560.
MacKinnon, D. 2008. Introduction to Statistical Mediation Analysis. New York: Routledge.
MacKinnon, D., Lockwood, C., Brown, C., Wang, W., and Hoffman, J.. 2007. “The Intermediate Endpoint Effect in Logistic and Probit Regression.” Clinical Trials 4: 499513.
Manski, C. F. 2007. Identification for Prediction and Decision. Cambridge, MA: Harvard University Press.
Miller, J. M., and Krosnick, J. A.. 2000. “News Media Impact on the Ingredients of Presidential Evaluations: Politically Knowledgeable Citizens Are Guided by a Trusted Source.” American Journal of Political Science 44 (2): 301–15.
Miller, W. E., and Stokes, D. W.. 1963. “Constituency Influence in Congress.” American Political Science Review 57 (1): 4546.
Nelson, T. E., Clawson, R. A., and Oxley, Z. M.. 1997. “Media Framing of a Civil Liberties Conflict and Its Effect on Tolerance.” American Political Science Review 91 (3): 567–83.
Nelson, T. E., and Kinder, D. R.. 1996. “Issue Frames and Group-centrism in American Public Opinion.” The Journal of Politics 58 (4): 1055–78.
Neyman, J. [1923] 1990. “On the Application of Probability Theory to Agricultural Experiments: Essay on Principles, Section 9.” Statistical Science 5: 465–80.
Olsson, A., Ebert, J. P., Banaji, M. R., and Phelps, E. A.. 2005. “The Role of Social Groups in the Persistence of Learned Fear.” Science 309 (5735): 785–87.
Oxley, D. R., Smith, K. B., Alford, J. R., Hibbing, M. V., Miller, J. L., Scalora, M., Hatemi, P. K., and Hibbing, J. R.. 2008. “Political Attitudes Vary with Physiological Traits.” Science 321 (5896): 1667–70.
Pearl, J. 2001. “Direct and Indirect Effects.” In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, eds. Breese, Jack S. and Koller, Daphne. San Francisco: Morgan Kaufmann, 411–20.
Pearl, J. N.d. “The Causal Mediation Formula: A Guide to the Assessment of Pathways and Mechanisms.” Prevention Science. Forthcoming.
Petersen, M. L., Sinisi, S. E., and van der Laan, M. J.. 2006. “Estimation of Direct Causal Effects.” Epidemiology 17 (3): 276–84.
Prior, M. 2006. “The Incumbent in the Living Room: The Rise of Television and the Incumbency Advantage in U.S. House Elections.” Journal of Politics 68 (3): 657–73.
Robins, J. M. 2003. “Semantics of Causal DAG Models and the Identification of Direct and Indirect Effects.” In Highly Structured Stochastic Systems, eds. Green, P. J., Hjort, N. L., and Richardson, S., Oxford: Oxford University Press, 7081.
Robins, J. M., and Greenland, S.. 1992. “Identifiability and Exchangeability for Direct and Indirect Effects.” Epidemiology 3 (2): 143–55.
Robins, J. M., and Richardson, T.. 2010. “Alternative Graphical Causal Models and the Identification of Direct Effects. In Causality and Psychopathology: Finding the Determinants of Disorders and Their Cures, eds. Shrout, P., Keyes, K., and Omstein, K., Oxford: Oxford University Press, 103–58.
Rosenbaum, P. R. 2002a. “Covariance Adjustment in Randomized Experiments and Observational Studies: Rejoinder.” Statistical Science 17 (3): 321–27.
Rosenbaum, P. R. 2002b. “Covariance Adjustment in Randomized Experiments and Observational Studies (with Discussion).” Statistical Science 17 (3): 286327.
Rubin, D. B. 1974. “Estimating Causal Effects of Treatments in Randomized and Non-randomized Studies.” Journal of Educational Psychology 66: 688701.
Shadish, W. R., Cook, T. D., and Campbell, D. T.. 2001. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.
Sjölander, A. 2009. “Bounds on Natural Direct Effects in the Presence of Confounded Intermediate Variables.” Statistics in Medicine 28 (4): 558–71.
Sobel, M. E. 1982. “Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models.” Sociological Methodology 13: 290321.
Sobel, M. E. 2008. “Identification of Causal Parameters in Randomized Studies with Mediating Variables.” Journal of Educational and Behavioral Statistics 33 (2): 230–51.
Spencer, S., Zanna, M., and Fong, G.. 2005. “Establishing a Causal Chain: Why Experiments Are Often More Effective Than Mediational Analyses in Examining Psychological Processes.” Journal of Personality and Social Psychology 89 (6): 845–51.
Tiedens, L. Z. and Linton, S.. 2001. “Judgment under Emotional Certainty and Uncertainty: The Effects of Specific Emotions on Information Processing.” Journal of Personality and Social Psychology 81 (6): 973–88.
Tomz, M. and Houweling, R. P. van. 2009. “The Electoral Implications of Candidate Ambiguity.” American Political Science Review 103 (1): 8398.
VanderWeele, T. J. 2009. “Marginal Structural Models for the Estimation of Direct and Indirect Effects.” Epidemiology 20 (1): 1826.
VanderWeele, T. J. and Robins, J. M.. 2009. “Minimal Sufficient Causation and Directed Acyclic Graphs.” Annals of Statistics 37 (3): 1437–65.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

American Political Science Review
  • ISSN: 0003-0554
  • EISSN: 1537-5943
  • URL: /core/journals/american-political-science-review
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed