To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure email@example.com
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Political scientists routinely face the challenge of assessing the quality (validity and reliability) of measures in order to use them in substantive research. While stand-alone assessment tools exist, researchers rarely combine them comprehensively. Further, while a large literature informs data producers, data consumers lack guidance on how to assess existing measures for use in substantive research. We delineate a three-component practical approach to data quality assessment that integrates complementary multimethod tools to assess: (1) content validity; (2) the validity and reliability of the data generation process; and (3) convergent validity. We apply our quality assessment approach to the corruption measures from the Varieties of Democracy (V-Dem) project, both illustrating our rubric and unearthing several quality advantages and disadvantages of the V-Dem measures, compared to other existing measures of corruption.
Models for converting expert-coded data to estimates of latent concepts assume different data-generating processes (DGPs). In this paper, we simulate ecologically valid data according to different assumptions, and examine the degree to which common methods for aggregating expert-coded data (1) recover true values and (2) construct appropriate coverage intervals. We find that the mean and both hierarchical Aldrich–McKelvey (A–M) scaling and hierarchical item-response theory (IRT) models perform similarly when expert error is low; the hierarchical latent variable models (A-M and IRT) outperform the mean when expert error is high. Hierarchical A–M and IRT models generally perform similarly, although IRT models are often more likely to include true values within their coverage intervals. The median and non-hierarchical latent variable models perform poorly under most assumed DGPs.
The traditional process of peer review and publication has come under intense scrutiny in recent years. The time seems propitious for a consideration of alternatives in political science. To that end, we propose a Peer Review and Publication Consortium. The Consortium retains the virtues of the traditional peer-review process (governed by academic journals) while also mitigating some of its vices.
Scholars and policy makers need systematic assessments of the validity of the measures produced by V-Dem. In Chapter 6, we present our approach to comparative data validation – the set of steps we take to evaluate the precision, accuracy, and reliability of our measures, both in isolation and compared to extant measures of the same concepts. Our approach assesses the degree to which measures align with shared concepts (content validation), shared rules of translation (data generation assessment), and shared realities (convergent validation). Within convergent validity, we execute two convergent validity tests. First, we examine convergent validity as it is typically conceived – examining convergence between V-Dem measures and extant measures. Second, we evaluate the level of convergence across coders, considering the individual coder and country traits that predict coder convergence. Throughout the chapter, we focus on three indices included in the V-Dem data set: polyarchy, corruption, and core civil society. These three concepts collectively provide a “hard test” for the validity of our data, representing a range of existing measurement approaches, challenges, and solutions.
This chapter sets forth the conceptual scheme for the V–Dem project. We begin by discussing the concept of democracy. Next, we lay out seven principles by which this key concept may be understood – electoral, liberal, majoritarian, consensual, participatory, deliberative, and egalitarian. Each defines a “variety“ of democracy, and together they offer a fairly comprehensive accounting of the concept as used in the world today. Next, we show how this seven-part framework fits into our overall thinking about democracy, including multiple levels of disaggregation – to components, subcomponents, and indicators. The final section of the chapter discusses several important caveats and clarifications pertaining to this ambitious taxonomic exercise.
This chapter recounts how a project of this scale came together and why it has succeeded. Five main factors were responsible for V–Dem’s success: timing, inclusion, deliberation, administrative centralization, and fund–raising. First, planning for V-Dem began at a time when both social scientists and practitioners were realizing that they needed better democracy measures. This made it possible to recruit collaborators and find funding. Second, the leaders of the project were always eager to expand the team to acquire whatever expertise they lacked and share credit with everyone who contributed. Third, the project leaders practiced an intensely deliberative decision–making style to ensure that all points of view were consulted and only decisions that won wide acceptance were adopted. Fourth, centralizing the execution of the agreed–upon tasks helped tremendously by streamlining processes and promoting standardization, documentation, professionalization, and coordination of a large number of intricate steps. Finally, successful fund–raising from a mix of both research foundations and bilateral and multilateral organizations has been critical.
In this chapter we focus on the measurement of five key principles of democracy – electoral, liberal, participatory, deliberative, and egalitarian. For each principle, we discuss (1) the theoretical rationale for the selected indicators, (2) whether these indicators are correlated strongly enough to warrant being collapsed into an index, and (3) the justification of aggregation rules for moving from indicators to components and from components to higher–level indices. In each section we also (4) highlight the top– and bottom–five countries on each principle of democracy in early (1812 or 1912) and late (2012) years of our sample period, as well as the aggregate trend over the whole time period 1789–2017 (where applicable). Finally, we (5) look at how the different principles are intercorrelated in order to assess the trade–offs involved between the conceptual parsimony achieved by aggregating to a few general concepts and the retention of useful variation permitted by aggregating less.
Four characteristics of V-Dem data present distinct opportunities and challenges for explanatory analysis: (1) the large number of democracy indicators (i.e., variables), (2) the measurement of concepts by multiple coders filtered through the V-Dem measurement model, (3) the large number of years in the data set, and 4) the ex ante potential for dependence across countries (generically referred to as spatial dependence). This chapter discusses 3 challenges and 10 opportunities that are implied by these characteristics. At the end of this chapter, we also discuss three assumptions that are implicit in most analyses of observational indicators of macro-features at the national level, which aim to draw conclusions about causal relationships.
Varieties of Democracy is the essential user's guide to The Varieties of Democracy project (V-Dem), one of the most ambitious data collection efforts in comparative politics. This global research collaboration sparked a dramatic change in how we study the nature, causes, and consequences of democracy. This book is ambitious in scope: more than a reference guide, it raises standards for causal inferences in democratization research and introduces new, measurable, concepts of democracy and many political institutions. Varieties of Democracy enables anyone interested in democracy - teachers, students, journalists, activists, researchers and others - to analyze V-Dem data in new and exciting ways. This book creates opportunities for V-Dem data to be used in education, research, news analysis, advocacy, policy work, and elsewhere. V-Dem is rapidly becoming the preferred source for democracy data.
Users of V–Dem data should take care to understand how the data are generated because the data collection strategies have consequences for the validity, reliability, and proper interpretation of the values. Chapters 4 and 5 explain how we process the data after collecting the raw scores and how we aggregate the most specific indicators into more general indices. In this chapter we explain where the raw scores come from. We distinguish among the different types of data that V–Dem reports and describe the processes that produce each type and the infrastructure required to execute these processes.
V-Dem relies on country experts who code a host of ordinal variables, providing subjective ratings of latent – that is, not directly observable – regime characteristics. Sets of around five experts rate each case, and each rater works independently. Our statistical tools model patterns of disagreement between experts, who may offer divergent ratings because of differences of opinion, variation in scale conceptualization, or mistakes. These tools allow us to aggregate ratings into point estimates of latent concepts and quantify our uncertainty around these estimates. This chapter describes item response theory models that can account and adjust for differential item functioning (i.e., differences in how experts apply ordinal scales to cases) and variation in rater reliability (i.e., random error). We also discuss key challenges specific to applying item response theory to expert–coded cross-national panel data, explain how we address them, highlight potential problems with our current framework, and describe long-term plans for improving our models and estimates. Finally, we provide an overview of the end–user–accessible products of the V-Dem measurement model.
It goes without saying that “leaders rule.” And it stands to reason that the background characteristics of leaders affect the way they rule. Who are the leaders of the world? We generate a composite portrait of the global political elite with data from the Global Leadership Project (GLP), the first dataset offering biographical information on a wide array of leaders in most countries of the world. We offer comparisons across office, regions, regime types, and level of development. And we enlist the variables in the dataset in a latent class model to arrive at an empirical typology of political leaders around the world.