Hostname: page-component-84b7d79bbc-x5cpj Total loading time: 0 Render date: 2024-07-27T21:15:49.018Z Has data issue: false hasContentIssue false
  • English
  • Français

Power and False Negatives in Qualitative Comparative Analysis: Foundations, Simulation and Estimation for Empirical Studies

Published online by Cambridge University Press:  29 January 2018

Ingo Rohlfing
Ingo Rohlfing*
Affiliation:
Cologne Center for Comparative Politics, Universität zu Köln, Albertus-Magnus-Platz, Köln 50931, Germany. Email: i.rohlfing@uni-koeln.de
*
*Email: i.rohlfing@uni-koeln.de
Get access Rights & Permissions [Opens in a new window]

Abstract

In Qualitative Comparative Analysis (QCA), empirical researchers use the consistency value as one, if not sole, criterion to decide whether an association between a term and an outcome is consistent with a set-relational claim. Braumoeller (2015) points out that the consistency value is unsuitable for this purpose. We need to know the probability of obtaining it under the null hypothesis of no systematic relation. He introduces permutation testing for estimating the $p$ value of a consistency score as a safeguard against false positives. In this paper, I introduce permutation-based power estimation as a safeguard against false-negative conclusions. Low power might lead to the false exclusion of truth table rows from the minimization procedure and the generation and interpretation of invalid solutions. For a variety of constellations between an alternative and null hypothesis and numbers of cases, simulations demonstrate that power estimates can range from 1 to 0. Ex post power analysis for 63 truth table analyses shows that even under the most favorable constellation of parameters, about half of them can be considered low-powered. This points to the value of estimating power and calculating the required number of cases before the truth table analysis.

Keywords

false negativespermutationpowerQualitative Comparative Analysisrandomization testsset theory
Type
Articles
Information
Political Analysis , Volume 26 , Issue 1 , January 2018 , pp. 72 - 89
Copyright
Copyright © The Author(s) 2018. Published by Cambridge University Press on behalf of the Society for Political Methodology. 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Contributing Editor: Jonathan N. Katz.

References

Abadie, Alberto, Athey, Susan, Imbens, Guido W., and Wooldridge, Jeffrey M.. 2014. Finite population causal standard errors. NBER Working Paper No. 20325, accessed April 26, 2017, https://www.nber.org/papers/w20325.Google Scholar
Athey, Susan, and Imbens, Guido W.. 2016. The econometrics of randomized experiments. arXiv working paper, accessed April 26, 2017, https://arxiv.org/abs/1607.00698v1.Google Scholar
Braumoeller, Bear F. 2015. Guarding against false positives in Qualitative Comparative Analysis. Political Analysis 23(4):471–487.Google Scholar
Braumoeller, Bear F., and Goertz, Gary. 2000. The methodology of necessary conditions. American Journal of Political Science 44(4):844–858.Google Scholar
Carsey, Thomas M., and Harden, Jeffrey J.. 2013. Monte Carlo simulation and resampling methods for social science . Thousand Oaks: SAGE.Google Scholar
Ellis, Paul D. 2010. The essential guide to effect sizes: Statistical power, meta-analysis, and the Interpretation of research results . Cambridge: Cambridge University Press.Google Scholar
Freitag, Markus, and Schlicht, Raphaela. 2009. Educational federalism in Germany: Foundations of social inequality in education. Governance 22(1):47–72.Google Scholar
Good, Phillip I. 1994. Permutation tests . New York: Springer-Verlag.Google Scholar
Hug, Simon. 2013. Qualitative Comparative Analysis: How inductive use and measurement error lead to problematic inference. Political Analysis 21(2):252–265.Google Scholar
Marx, Axel, Rihoux, Benoît, and Ragin, Charles C.. 2014. The origins, development, and application of Qualitative Comparative Analysis: The first 25 years. European Political Science Review 6(1):115–142.Google Scholar
McCluskey, Edward J. 1956. Minimization of Boolean functions. The Bell System Technical Journal 35(6):1417–1444.Google Scholar
Mello, Patrick A. 2012. Parliamentary peace or partisan politics? Democracies’ participation in the Iraq war. Journal of International Relations and Development 15(3):420–453.Google Scholar
Ragin, Charles C. 1987. The comparative method: Moving beyond quantitative and qualitative strategies . Berkeley: University of Berkeley Press.Google Scholar
Ragin, Charles C. 2000. Fuzzy-set social science . Chicago: University of Chicago Press.Google Scholar
Ragin, Charles C. 2006. Set relations in social research: Evaluating their consistency and coverage. Political Analysis 14(3):291–310.Google Scholar
Ragin, Charles C. 2008. Redesigning social inquiry . Chicago: Chicago University Press.Google Scholar
Ragin, Charles C. 2014. Comment: Lucas and Szatrowski in critical perspective. Sociological Methodology 44(1):80–94.Google Scholar
Rohlfing, Ingo. 2017. Replication data for: Power and false negatives in Qualitative Comparative Analysis (QCA). doi:10.7910/DVN/UNHYMP, Harvard Dataverse, V1, UNF:6:4QDrl2fEelRJmjDVYJ5hFw==.Google Scholar
Schneider, Carsten Q., and Wagemann, Claudius. 2012. Set-theoretic methods for the social sciences. A guide to qualitative comparative analysis . Cambridge: Cambridge University Press.Google Scholar
Thiem, Alrik. 2014. Navigating the complexities of Qualitative Comparative Analysis: Case numbers, necessity relations, and model ambiguities. Evaluation Review 38(6):487–513.Google Scholar
Thiem, Alrik, Spöhel, Reto, and Dusa, Adrian. 2015. Enhancing sensitivity diagnostics for Qualitative Comparative Analysis: A combinatorial approach. Political Analysis 24(1):104–120.Google Scholar
Supplementary material: File

Rohlfing supplementary material 1

Rohlfing supplementary material

Download Rohlfing supplementary material 1(File)
File 348.4 KB