Scholars and policy makers need systematic assessments of the validity of the measures produced by V-Dem. In Chapter 6, we present our approach to comparative data validation – the set of steps we take to evaluate the precision, accuracy, and reliability of our measures, both in isolation and compared to extant measures of the same concepts. Our approach assesses the degree to which measures align with shared concepts (content validation), shared rules of translation (data generation assessment), and shared realities (convergent validation). Within convergent validity, we execute two convergent validity tests. First, we examine convergent validity as it is typically conceived – examining convergence between V-Dem measures and extant measures. Second, we evaluate the level of convergence across coders, considering the individual coder and country traits that predict coder convergence. Throughout the chapter, we focus on three indices included in the V-Dem data set: polyarchy, corruption, and core civil society. These three concepts collectively provide a “hard test” for the validity of our data, representing a range of existing measurement approaches, challenges, and solutions.

This chapter sets forth the conceptual scheme for the V–Dem project. We begin by discussing the concept of democracy. Next, we lay out seven principles by which this key concept may be understood – electoral, liberal, majoritarian, consensual, participatory, deliberative, and egalitarian. Each defines a “variety“ of democracy, and together they offer a fairly comprehensive accounting of the concept as used in the world today. Next, we show how this seven-part framework fits into our overall thinking about democracy, including multiple levels of disaggregation – to components, subcomponents, and indicators. The final section of the chapter discusses several important caveats and clarifications pertaining to this ambitious taxonomic exercise.

This chapter recounts how a project of this scale came together and why it has succeeded. Five main factors were responsible for V–Dem’s success: timing, inclusion, deliberation, administrative centralization, and fund–raising. First, planning for V-Dem began at a time when both social scientists and practitioners were realizing that they needed better democracy measures. This made it possible to recruit collaborators and find funding. Second, the leaders of the project were always eager to expand the team to acquire whatever expertise they lacked and share credit with everyone who contributed. Third, the project leaders practiced an intensely deliberative decision–making style to ensure that all points of view were consulted and only decisions that won wide acceptance were adopted. Fourth, centralizing the execution of the agreed–upon tasks helped tremendously by streamlining processes and promoting standardization, documentation, professionalization, and coordination of a large number of intricate steps. Finally, successful fund–raising from a mix of both research foundations and bilateral and multilateral organizations has been critical.

In this chapter we focus on the measurement of five key principles of democracy – electoral, liberal, participatory, deliberative, and egalitarian. For each principle, we discuss (1) the theoretical rationale for the selected indicators, (2) whether these indicators are correlated strongly enough to warrant being collapsed into an index, and (3) the justification of aggregation rules for moving from indicators to components and from components to higher–level indices. In each section we also (4) highlight the top– and bottom–five countries on each principle of democracy in early (1812 or 1912) and late (2012) years of our sample period, as well as the aggregate trend over the whole time period 1789–2017 (where applicable). Finally, we (5) look at how the different principles are intercorrelated in order to assess the trade–offs involved between the conceptual parsimony achieved by aggregating to a few general concepts and the retention of useful variation permitted by aggregating less.