Naming the Indicator

The assertion that an indicator has been brought into existence and given life is typically marked by naming it. The name itself is usually a simplification of what the index purports to measure or rank. The name's constancy may mask changes over time in the indicator itself. Calling an indicator a measure of “transparency” or “human development” asserts a claim that there is such a phenomenon and that the numerical representation measures it. An indicator may even create the phenomenon it claims to measure, as intelligence quotient (IQ) tests came to define intelligence. Labeling this measure as an indicator, index, ranking, or league table implies a claim to knowing and measuring a phenomenon. As a result, the indicator represents an assertion of power to produce knowledge and to define or shape the way the world is understood.

Rank-Ordered Structure

All indicators are fundamentally comparative, and some element of ranking is a feature of the indicators we are studying. Indicators usually enable comparison of different units but in a few cases only permit comparison of the same unit at different times. However, an indicator need not rank all data points or all units in a transitive way. Influential indicators are usually cardinal (attributing separately defined values to each unit), and most use one or other of a standard menu of scaling methods (e.g., a purely ordinal scale, an equal-interval scale, or a ratio scale), but it is possible to have an indicator that does not have these attributes. Some listings using most of the attributes of indicators may merely divide units into categories described nominally, identifying difference without ranking the categories. These do not fall within our definition of an indicator. Other nominal listings may have an element of hierarchy among broad categories (e.g., red, yellow, green). These do qualify as indicators for our purposes.

Simplification

Simplification, or reductionism, is central to the appeal (and probably the impact) of indicators. They are often numerical representations of complex phenomena intended to render them more simple and comparable with other complex phenomena that have also been represented numerically. Indicators are typically aimed at policy makers and are intended to be convenient, easy to understand, and easy to use. Yet, the transformation of particularistic knowledge into numerical representations that are readily comparable strips meaning and context from the phenomenon. In this numerical form, such knowledge carries a distinctive authority that shifts configurations and uses of power and counterpower. This transformation reflects, but also contributes to, changes in decisionmaking structures and processes.

Indicators also often present the world in black and white, with few ambiguous intermediate shades. They take flawed and incomplete data that may have been collected for other purposes and merge them together to produce an apparently coherent and complete picture. Wendy Espeland and Mitchell Stevens identify this as a potential consequence of what March and Simon refer to as uncertainty absorption, which “takes place when inferences are drawn from a body of evidence, and the inferences instead of the evidence itself, are then communicated” (Reference March and SimonMarch & Simon 1958: 165). As Espeland and Stevens describe this process, “ ‘Raw’ information typically is collected and compiled by workers near the bottom of organizational hierarchies; but as it is manipulated, parsed, and moved upward, it is transformed so as to make it accessible and amenable for those near the top, who make the big decision. This ‘editing’ removes assumptions, discretion and ambiguity, a process that results in ‘uncertainty absorption’: information appears more robust than it actually is.… The premises behind the numbers disappear, with the consequence that decisions seem more obvious than they might otherwise have been. An often unintended effect of this phenomenon is numbers that appear more authoritative as they move up a chain of command. The authority of the information parallels the authority of its handlers in the hierarchy” (2008: 421–22). The degree of uncertainty beneath the surface of many of the most influential simplifying indicators in global governance is quite intensively scrutinized, but usually only in specialized scientific literature (Reference HoodHood et al. 2008; Reference HøylandHoyland et al. 2012; Reference MorseMorse 2004).

Indicators as Tools for Evaluation

We single out indicators from other collections of data based on their potential use in evaluating performance. Indicators set standards. The standard against which performance is to be measured is often suggested by the name of the indicator—corruption, protection of human rights, respect for the rule of law, and so on. To the extent that an indicator is used to evaluate performance against one standard rather than another, the use of that indicator embodies a theoretical claim about the appropriate standards for evaluating actors' conduct. Indicators often have embedded within them, or are placeholders for, a much further-reaching theory—which some might call an ideology—of what a good society is, or how governance should ideally be conducted to achieve the best possible approximation of a good society or a good policy. At a minimum they are produced as, or used as, markers for larger policy ideas. They may measure “success” directly along this axis, or they may measure what, from the standpoint of the theory or policy idea, are pathologies or problems to be overcome. More frequently they address simply some measurable elements within a wider scenario envisaged by the theory or policy idea. Often the theory or policy idea is not spelled out at all in the indicator but remains implicit.Footnote ¹

The theory or idea embedded in an indicator may be developed or reframed by its users or by other actors in ways that differ from the intentions of its producers. Indicators often express ideologies about the ideal society and the process of achieving it. But what they actually communicate, and to whom, may not be what their producers and promulgators sought to communicate. This communicative element makes it essential to consider the indicator's audience and how it is engaged by the indicator.

Use of an indicator in evaluative processes requires that its audience include active evaluators. Those evaluators may or may not exert significant governance power over the actor being evaluated. An indicator may be taken up by its audience (sometimes without any explicit intention on the audience's part) in social processes that do not directly involve evaluation, including establishing or cementing key concepts (such as human development), influencing actor identities, condensing and redefining status and hierarchies in quantified forms, framing standards or causal theories that may then be rendered in other ways (for example, in an organizational policy or a statement of best practices), influencing decisions as to what is measured or how statistics are compiled, or crudely validating or calling into question other ideas or evaluative impressions. These other roles or uses of indicators do not alter the definitional requirement that an indicator must be capable of being used for evaluation, even while some of its roles and effects do not depend on the operation of specific evaluative processes.

Topology of Governance

The idea that indicators and other quantitative ways of representing social phenomena can serve as technologies of governance has distinct implications for the topology of global governance. Indicators are one of the technologies of “government at a distance” (Miller & Reference Miller and RoseRose 2008), allowing certain actors to exercise influence over the conduct of large numbers of geographically dispersed actors. A comparable phenomenon, where power is exercised beyond the state and with or without government action, may be termed “governance at a distance.” In particular settings of global governance, using indicators as a technology calls for the expansion of ordinary political conceptions of who qualifies as a governor, while at the same time complicating models of governance premised on clear distinctions among governors, governed, and others.

Recognizing indicators as a technology of global governance implies that actors who promulgate indicators ought to be counted among the governors, even if they otherwise would not be recognized as wielders of power in global governance, or would be recognized as such only to a lesser extent. Thus indicators help constitute or embed power relations. Moreover, simple producers of indicators used in global governance, or actors whose decisions have a significant impact on the form or content of such indicators, may exercise power even where they are not the formal promulgators or users of the indicator.

Including producers of indicators in the class of governors does not mean that tracing the strands of agency and power relations is necessarily straightforward. While in some cases (such as between credit rating agencies and their clients who pay to be rated) there is a symbiotic relationship between those who measure and those who are measured, particularly when the measured entity actively consents to the measuring, in other cases the measurer unilaterally exercises power over the measured. These complex and variegated power relations do not map neatly onto the distinction between governors and governed.

Another complicating factor is that the production of the indicators used in global governance is often a collective process. In many cases promulgators attach their names to indicators whose production involves contributions from a number of other actors. For example, reports and rankings for the Programme for International Student Assessment (PISA) are promulgated by the Organisation for Economic Co-operation and Development (OECD), but are actually prepared and produced by an Australian consultancy under contract with the OECD (Reference Bogdandy and GoldmannBogdandy & Goldmann 2008). Moreover, the promulgators of indicators typically rely on data collected by a large network of independent actors including international agencies, national statistical agencies, local and national NGOs, and local villages. They also rely upon analytical techniques generated by some segment of the scientific community. Consequently, the promulgator of an indicator may or may not be the actor most involved in determining its content. Instead, the promulgator is often more like a consumer-product manufacturer, whose main contribution is to lend its brand name, and perhaps its design and marketing expertise and quality-control power, to the collective product of a global supply chain.

The production of indicators also draws into the practice of global governance, through their own use of the indicators, people who otherwise would be regarded simply as members of the public. For example, when the U.S. State Department publishes its annual glossy report with indicators of countries' compliance with antitrafficking standards, these can be read by activist groups who may influence economic agents such as prospective tourists in Toronto just as easily as they can be read by government officials. Learning of Costa Rica's low score may lead a Toronto resident to alter her perceptions of Costa Rica, a country downgraded to the Tier 2 Watch List in 2011 (United States Department of State 2011). In particular, her travel decisions, in combination with the decisions of other members of the public, may have a material effect on Costa Rica's tourism revenues.

Indicators may also play significant roles in global governance in helping to constitute actors and shape identities. Some organizations, such as Freedom House and Transparency International, depend for their prominence and influence primarily on indicators they produce. In many organizations, indicator production is important to the business model as it helps generate Web site traffic or demand for the organization's consultancy services; some indicators are even sold commercially (Reference DavisDavis et al. forthcoming). Disparate actors in different categories may become linked through an indicator that they help construct, or that measures behavior with which they are concerned. Indicators may thus play roles in shaping highly decentralized or informal governance structures such as networks. Indicators may be important in such governance modalities even where no clear delineation of governors, governed, and interested public can be made, and where clear overarching human intentionality is lacking or where structural or inanimate elements (such as technological elements) greatly shape outcomes.

The use of indicators in global governance enhances the role played by the subset of the public that comprises the scientific community. The scientific community determines the scientific authority of an indicator, which in turn may affect the extent of the indicator's influence. Producers of indicators are well aware of this fact. For example, Kaufmann and Kraay assert that their Worldwide Governance Indicators are more reliable than other indicators because they are published in peer-reviewed scientific journals (Reference KaufmannKaufmann et al. 1999: 32; see also Reference KaufmannKaufmann et al. 2009). Indicators typically rest on claims to objectivity and social science knowledge, but they differ significantly in the extent to which they reflect social science research and analysis. There are close relations between indicators developed for social science theory testing and indicators that address policy questions, with the data and analysis of one informing the other.

Standard Setting

As we have explained, indicators are standard-setting instruments. But while the processes that generate indicators ultimately result in the production of specific goals and targets against which societies are measured, they may be different from other more politically explicit standard-setting processes (Reference Büthe and MattliBüthe & Mattli 2011; Reference Lampland and StarLampland & Star 2009). Whereas political efforts to formulate norms and standards—for example, in multilateral intergovernmental negotiations conducted by diplomats—tend to involve processes such as voting, interest-group bargaining, or the exercise of material power, the processes in specialist agencies and expert meetings at which the standards embedded in indicators are produced, accepted, and supported tend to involve derivation of power from scientific knowledge. As the awareness or the significance of indicators as standards rises, indicator design and production are likely to become increasingly subject to demands made of other standard-setting processes, including demands for transparency, participation, explanation, justification, and review (Reference Bogdandy and GoldmannBogdandy & Goldmann 2008; Reference KingsburyKingsbury et al. 2005).

Because indicators are by stipulation capable of being used in evaluation, they frequently blend standard setting with evaluation by conveying information such as a ranking of one state's performance relative to that of other states and a direction of change in the state's relative or absolute performance by comparison to previous iterations of the indicator. This has the potential to intensify demands for “due process,” especially within intergovernmental bodies, as each specific ranked entity has a direct focused interest, eclipsing the general interest in good standards, which it may regard as conferring “standing” to raise a challenge.

Decisionmaking

In the practice of global governance, many decisions by governing entities are in some way influenced by indicators, although few rely on indicators entirely and mechanically. In the most straightforward case, an indicator promulgated by an extranational entity is then used by that entity in generating or allocating resources or in influencing behavior. This is, for example, what the World Bank does in promulgating “good governance” indicators that the World Bank itself uses in deciding how to allocate aid. A modest extension of this practice occurs when one entity's indicators are used for governance purposes by other entities in the same sector, as when the MCC uses World Bank indicators. A more subtle case arises when the promulgation of the indicator by an extranational entity spurs demands and governance-related action by diffuse but nonetheless influential groups of other actors. For instance, as we will discuss at greater length below, the World Bank claims that it has prompted many countries to reform their legal systems simply by promulgating and promoting its country-level indicators on the ease of doing business. The U.S. State Department's Trafficking in Persons Report claims that it has fostered national antitrafficking legislation. The regulatory influence of these indicators does not stem exclusively from the ways in which the World Bank or other development agencies use them, but also stems from the ways in which they are expected to be used in lobbying and decisionmaking by local political constituencies or prospective foreign investors. This shades into a further scenario, in which the indicators have regulatory effects primarily because they have been embraced as guides to appropriate conduct by actors within the state who shape national governmental decisions regarding national governance. The majority of prominent indicators appears to operate in global governance in even more diffuse ways than this, by influencing professional, public, and political opinion to craft new approaches or to take different policy orientations.

Indicators are attractive to decision makers and designers of decisionmaking processes because decisionmaking processes that rely on indicators can be presented as efficient, consistent, transparent, scientific, and impartial. Reference PorterPorter (1995) refers to these virtues compendiously as “objectivity.” It is difficult to say which of these factors is most important in any given context. Efficiency and consistency may be factors of special importance in high-volume decisionmaking; transparency, scientific authority, and impartiality are considerations relevant to the use of indicators in both standard setting and decisionmaking, although special issues arise in decisionmaking.

Contestation

A great deal remains to be learned about when, how, and why the governed (or rival governors) contest the use of indicators, but we expect it to take both general and long-established forms such as lobbying and litigation as well as distinctive forms that are especially suited to changing or resisting governance through indicators, such as refusal to participate in data collection, challenges to scientific validity, and creation of alternative indicators. Contestation can take the form of debates about the data used or not used in indicators, the criteria for weighting the indicators, or the embedded social and political theory of the indicator. Contestation strategies can include the creation of new indicators and resistance to or discrediting of existing indicators and their producers or users. This may in turn prompt modifications to the indicator or counterstrategies by producers and users.

Because indicators obscure the sociopolitical theoretical claims embedded in their construction, the use of indicators can make it relatively difficult to contest the use of those theories in global governance. Indicators may mask large areas of missing or incomplete data, inability to draw significant distinctions among entities that are nonetheless hierarchically ranked, much higher levels of underlying uncertainty than the indicators depict, and choices about weighting of different components of composite indicators (which in some notable privately produced indicators are not disclosed at all). On the other hand, those with special expertise in the construction or analysis of indicators can overcome these impediments to technical contestation and exercise greater influence than they could in purely political settings. Limitations in the ability to contest the exercise of power by global decision makers tend to shift the balance of power toward “technical” experts—that is to say, people with expertise in the construction or analysis of indicators.

The rapid growth in prominent indicators in global governance is a time-compressed phenomenon that was initially sufficiently circumscribed for case studies about early patterns of debate, acceptance, and challenge to be used as a basis for some cautious generalizations. Several of the most prominent indicators in global governance began as efforts led and shaped by social science communities. Eventual “scientific” acceptance of these indicators can be traced to a time before “public” knowledge about the issue was settled through various controversies and challenges. The development of these indicators instantiates, to some extent, the process of developing scientific knowledge described by Reference LatourLatour (1987). Like other forms of scientific knowledge production, an indicator builds on existing concepts, techniques, and categories of understanding that are taken for granted as correct, as well as on networks of experts. Indicators are shaped both by technical factors, such as the statistical properties of an indicator when compared to other indicators, and by social factors, such as social networks, perceived expertise, relational interactions, institutions, and allies (Reference LatourLatour 1987: 29). These processes are collective, and they take place over time. Once the indicator is established with wide scientific support (even while continuous scientific debate and refinement remains part of the indicator's further life), a process of wider public acceptance occurs, as networks of actors and institutions adopt the indicator and consequently increase its credibility and legitimacy, perhaps even converting it into a standard against which other indicators are evaluated.

The results of a survey we conducted of reporting in three major U.S. and UK opinion-shaping newspapers and magazines about four major indicators—UNDP's HDI, Transparency International's Corruption Perception Index (Transparency International 2010), Freedom House's Freedom in the World indicator (Freedom House 2010), and the World Bank's Doing Business indicators—are consistent with this model.Footnote ² In the first year or two after an indicator is released, there are discussion and debate about the indicator itself, but after a few years, the indicator is presented in these news media largely as a fact that describes a country's situation, with virtually no discussion about the source of the data or the nature of the indicator itself. The case studies below of the Doing Business indicators and the HDI show that in certain international political forums, however, efforts at contestation can be intense. As these indicators have become more and more significant as technologies of global governance, the stakes of contestation have risen. Producers of indicators who may have viewed themselves as scientists or technicians working outside political and legal arenas have been drawn into highly political conflicts.

Regulation

One outcome of contestation can be demands for regulation of indicator-related processes and activities. Some of these demands instantiate general patterns of demands for increased transparency, reason giving, participation, review, and accountability in global governance institutions and processes, particularly where public authority is being exercised but also in relation to some private governance actors (Reference BogdandyBogdandy et al. 2009; Reference KingsburyKingsbury 2009; Reference KingsburyKingsbury et al. 2005). Given the distinctive features of indicators as a technology of global governance, we expect growth in specifically adapted proposals or efforts to regulate indicators. These may take a variety of forms, several of which have been manifested in debates about regulation relating to sovereign debt ratings by credit rating agencies (Reference SySy 2009). For instance, producers of indicators could be subject to scrutiny (although not necessarily legal obligations) with reference to human rights standards, domestic constitutional norms, and principles of global administrative law. Others may be regulated in the same ways as private actors such as multinational corporations or networks of firms linked by transnational supply chains. These analogies suggest also the possible relevance of regulatory mechanisms such as competition law, transnational tort claims, and self-regulation (cf Reference FoucaultFoucault 2008). Procedural obligations on producers might require those producers to be transparent about the methods used to produce indicators and their limitations, to allow interested parties to participate in the design process, or to accept some accountability for effects on external actors in problematic cases. Alternatively, producers might find their indicators held to externally administered standards of reliability and validity. Finally, structural interventions might be designed to foster healthy competition among producers. So, for example, public bodies might support or subsidize the production of competing indicators, or certain organizations already exercising other substantial powers as governors might be encouraged to refrain from promulgating indicators.

Other regulatory interventions might target the users of indicators. For example, use of indicators in global governance may spawn systematic efforts to educate users of indicators—and the members of the public who confer authority upon them—about both the costs and the benefits associated with using indicators. Alternatively, regulation could focus on empowering actors who are governed by indicators—for example, by giving them access to the scientific expertise they need to contest decisions based upon indicators.

Genealogy, Design, and Production

The Doing Business indicators measure the quality of business laws and related legal institutions across 183 countries (World Bank 2011). The Doing Business team, with a large group of partners, compiles the raw data by asking lawyers in each country to report on the steps that a hypothetical firm would have to undertake in order to perform various tasks, including starting a business, hiring and firing workers, and enforcing a contract. The indicators generally reflect the time, cost, and number of procedures associated with each task. The creators of the Doing Business indicators are very explicit about their theoretical presumptions:

These presumptions about the relationship between law and economic development inform every aspect of the construction of the Doing Business indicators. To begin with, the very existence of the indicators—which are not very costly to create by the standards of a large global organization—reflects a presumption that rules and regulations are important.Footnote ⁴ Second, the fact that the indicators focus exclusively on rules embodied in the formal legal system reflects a presumption that it is those rules, as opposed to rules reflected in informal practices, that influence economic activity. Third, the idea that regulations that make transactions such as starting a business or firing a worker fast, cheap, and simple are automatically desirable clearly informs the choice of time, cost, and simplicity as metrics. The Doing Business project's empirical methodology implicitly presumes that elite lawyers are reliable sources of information about how small and medium-size enterprises navigate the formal legal system. The project's proponents tend to gloss over the fact that all of these claims are in fact contestable (Reference ArruñadaArruñada 2007; Reference Davis and KruseDavis & Kruse 2007; Reference SantosSantos 2009).

The specific theoretical claims embodied in the Doing Business indicators reflect ideas disseminated through networks linking elite academic economists to the World Bank. For instance, the authors of the 2010 Doing Business report claim to have been inspired by the work of Peruvian economist Hernando de Soto. In the late 1980s de Soto ran simulations in which firms created by his research team struggled to comply with the voluminous formal requirements associated with entering various economic activities in Lima, Peru.Footnote ⁵ De Soto's work in turn inspired a team of economists led by Andrei Shleifer and Robert Vishny, together with their former students, Rafael La Porta and Florencio Lopez-de-Silanes, to collect similar data from a large sample of countries for the purpose of testing claims about the relationship between legal institutions and various economic outcomes. All four of the economists were affiliated with top-ranked U.S. Ivy League economics departments or business schools, and the academic papers they produced are among the most widely cited in the entire discipline of economics.Footnote ⁶ The so-called gang of four also collaborated on academic projects with economists at the World Bank, including most notably Simeon Djankov, who eventually became the leader of the Doing Business project before entering government as Bulgaria's finance minister in 2009.

Governance Effects and Other Effects

The Doing Business indicators are tremendously influential and exemplify the range of mechanisms through which power can be exercised beyond the state (Reference SchuethSchueth 2011). To begin with, the Doing Business indicators are used, in combination with other indicators, to guide the allocation of foreign aid by multilateral development banks, as well as the MCC and USAID in the United States. For example, at the World Bank, five of the ten Doing Business indicators are used as “guideposts” (together with other sources) to assist country teams in determining country scores on “Business Regulatory Environment,” one of the 16 criteria of the Country Policy and Institutional Assessment (CPIA), the primary determinant of World Bank aid allocations (World Bank Operations Policy and Country Services 2009). Footnote ⁷ At the MCC, Doing Business indicators are used in two of the six indicators of whether countries are “Encouraging Economic Freedom”; countries must score above the median on at least three of the six indicators in this category to be eligible for MCC funding (Millennium Challenge Corporation 2009).Footnote ⁸ Finally, USAID officials have informally expressed commitments to support countries that are willing to reform in areas measured by the Doing Business reports (Reference SantosSantos 2009: 60).

The Doing Business indicators also appear to be successful in attracting the attention of senior policy makers, government officials, and business leaders in many of the World Bank's client countries, as well as potential foreign investors in those countries, thus prompting significant amounts of benchmarking, dialogue, and reform (Reference SchuethSchueth 2011; World Bank Independent Evaluation Group 2008). Even critics of the Doing Business indicators seem to agree that their promulgation has prompted many countries to reform their legal systems (Reference Benjamin, Theron and CorderBenjamin & Theron 2009). These impacts undoubtedly reflect some combination of the ease of use, transparency, and scientific authority of the indicators; the overall influence of the World Bank; and the substantial amount of effort that the Doing Business team and others have made to disseminate the indicators and their associated annual reports to the general public. The Doing Business team's communications strategy includes a Web site, press conferences, road shows, and workshops around the world. The Doing Business indicators are featured in news reports that appear in major publications such as the New York Times, the Washington Post, and the Economist.Footnote ⁹ In at least one case, Georgia—a country that has scored high on the Doing Business indicators—has bought advertisements in major news publications to tout its “success” (Reference SchuethSchueth 2011: 68–70). We suspect that when the Doing Business indicators are summarized or endorsed by influential journalists, politicians, and economists, the effect is to enhance the weight of the indicators with governing elites in the countries being assessed and with prospective foreign investors.

Contestation and Regulation

The Doing Business indicators also shed light on how governance through indicators can be contested and regulated. A case study of Georgia by Reference SchuethSchueth (2011) shows that contestation sometimes involves resistance at the local level. Schueth reports that sometime around 2006 USAID and the Saakashvili government made a priority of raising Georgia's rankings in the Doing Business indicators. Overall they enjoyed considerable success—in 2006 the Doing Business team named Georgia Top Reformer—but in some areas they were relatively unsuccessful. According to Schueth, some of the failures could be attributed to opposition from competing political interests. For example, at one point USAID proposed the adoption of a computerized system that would target only the most risky shipments and customs declarations for inspection by Georgian officials. Schueth claims that large importers with histories of customs violations and midlevel bureaucrats campaigned against, and initially blocked, the reforms (66–67).

The Doing Business indicators have also provoked contestation at the global level. A transnational group of workers' representatives (spearheaded by the International Trade Union Confederation), together with the International Labour Organization (ILO), several key figures in the U.S. Congress, and a range of academics and NGOs, campaigned to achieve significant change in the use of the World Bank's Employing Workers indicator (EWI) (Reference BakvisBakvis 2009; ILO 2007; Reference ParksParks 2008). The activists complained that the indicator was biased in favor of labor market deregulation and so was being used by international financial institutions to pressure developing countries to dismantle protections for workers. Their efforts involved both direct communications with the international financial institutions and actions at the national level. In the United States, the labor lobby even succeeded in securing passage of legislation directing the secretary of the treasury to use his influence over the World Bank to effect change (Supplemental Appropriations Act 2009, s. 1626). In April 2009 the World Bank agreed that it would stop using the controversial indicator in its CPIA measures, which affect decisions about the allocation of funds; that it would begin revising the EWI to give more favorable scores to certain worker-protection policies aligned with ILO conventions; and that it would establish a consultative group to formulate a new worker-protection indicator (World Bank 2009). In the 2011 Doing Business report, the International Finance Corporation (the entity in the World Bank group that produces the indicators) ceased to give any weight to the EWI, while preparations were made for a replacement indicator much more closely aligned with ILO conventions.

The campaign against the labor indicators was aided by the release in 2008 of a report by the World Bank's Independent Evaluation Group (IEG), which endorses complaints that the indicators were inconsistent with the spirit of key ILO conventions. The IEG's evaluation represents the kind of accountability mechanism that might serve as a model for future efforts to regulate the production of indicators. The IEG not only reviews the substance of the indicators in terms of reliability and compliance with ILO standards, but also takes the Doing Business project to task for failing to be sufficiently transparent about certain aspects of the process of constructing the indicators and failing to include a systematic process for validating the information they contained. At the same time, the IEG's assessment reveals that decision makers who used the Doing Business indicators typically used them in combination with a number of other indicators. This suggests that there is an upper bound on the potential impact of any effort to regulate the production as opposed to the use of indicators.

The IEG's evaluation of the Doing Business indicators includes recommendations on how the World Bank and other institutions ought to use the indicators. It also offers a few general principles to guide the use of other indicators in the bank's operations. Thus the evaluation represents an effort to control the use as well as the production of indicators. Much of the IEG's analysis is consistent with our analysis of the advantages and disadvantages of using indicators as a technology of governance. For instance, the IEG concludes that the simplicity of the Doing Business indicators (and the language in the associated reports), combined with the fact that they were used to produce rankings, were crucial components of their influence. The IEG also acknowledges the tensions between the benefits and costs of simplification and offers a mild criticism of the balance struck by the Doing Business project in its conclusion: “DB's [Doing Business's] simple and bold communication is integral to the product, but at times simplicity comes at the expense of rigor” (World Bank Independent Evaluation Group 2008: 39). Finally, the IEG recognizes that the Doing Business indicators are implicitly premised on claims that certain regulatory reforms bear a linear relationship to better development outcomes. The evaluation report notes that while they may be “credible,” those claims are not necessarily universally valid. Thus, the IEG recommends “caution” in using the Doing Business indicators and suggests that they typically be used in conjunction with other country-specific information. Accordingly, the IEG (2008: 51, 53) expresses concern about how the Doing Business indicators were being used by the MCC. The IEG report does not, however, consider or propose mechanisms that might be used to monitor and control future uses of indicators such as the Doing Business ones.

While no special-purpose control mechanism exists, concerns about the Doing Business indicators have prompted some World Bank personnel and outside commentators to express private or public skepticism about relying (or overly relying) on the indicators to make policy. More formally, reports suggest that at the October 2010 meeting of the World Bank's executive board, executive directors representing Brazil and China were among those expressing opposition to the bank's continued use of rankings in this area (Reference BoscoBosco 2010).

Genealogy, Design, and Production

Where the Doing Business indicators address the experiences of businesses, the HDI is an indicator of the quality of a society for its human inhabitants. The HDI combines proxies for three human capabilities: health (measured by life expectancy at birth), education (measured wholly or partly by literacy rates until 2010, when it was decided to measure education solely by mean and expected years of schooling), and income (measured from 2010 in gross national income per capita adjusted for purchasing power parity to eliminate differences in national price levels) (Reference KlugmanKlugman et al. 2011: 4). While the ways of calculating and indexing these three variables have been modified several times since 1990, the three measures have always been given equal weight and have always been averaged together. The HDI is produced together with the annual Human Development Report (HDR) by the Human Development Report Office (HDRO), a largely independent group within UNDP (United Nations Development Program 2010: v).

The launch of the HDI in 1990 grew out of almost 30 years of work and thought in the field of development economics and represented a significant shift from a focus on utility to a focus on welfare. The impact of global events, the rise of the human rights movement and new concerns about gender inequality all contributed to the change in theoretical orientation. Efforts to produce welfare-focused indicators began in the 1960s, along with a critique of the dominant focus on growth in GDP, since this measure neglected issues of employment, income distribution, and justice (Reference Streeten, Fukuda-Parr and KumarStreeten 2003: 94). By the 1980s the previously influential “basic needs” approach had come to seem too narrow for new concerns about women and children, the physical environment, human rights, political freedom and governance, and the role of culture. Reference SenAmartya Sen (1999) proposed an approach that expanded the basic-needs idea by emphasizing the importance of freedom to choose as the basis for well-being. According to Reference Streeten, Fukuda-Parr and KumarStreeten (2003: 94–100), Sen argues that a standard of living should be judged by a person's “capability” to lead the life that he or she values, from being well fed and healthy to achieving self-respect and participating in the life of the community.

The creators of the HDI came from prominent academic institutions as well as the World Bank and the UN. Like those who formed the Doing Business indicators, they were supported by powerful international organizations. The principal architect of the concept of an HDI, Mahbub ul Haq, had experience in the World Bank, while his advisors held academic positions at Oxford, Cambridge, London School of Economics, Yale, and Boston University (Reference Fukuda-Parr, Kumar, Fukuda-Parr and KumarFukuda-Parr & Kumar 2003: 85–91, 393–95).

The creators and consultants behind the HDI fully appreciated that the index was a simplification intended to represent only certain features and designed to persuade. For example, Sen calls the HDI a “deliberately constructed crude measure” but notes that its creator, Haq, “… did succeed in getting the ear of the world through the high publicity associated with the transparent simplicity of the HDI as an index. But it is extremely important not to read more into the HDI than is there” (2003: x). Sen was one of the principal consultants on the Human Development Report of 1990, which first presented the HDI, and he at first objected to a crude composite index like the HDI, since there was so much other information in the report that was not included in the index (see also Ravallion Reference Ravallion2010a, Reference Ravallionb). Haq replies, “We need a measure of the same level of vulgarity as GNP—just one number—but a measure that is not as blind to social aspects of human lives as GNP is” (Reference StantonStanton 2007: 14; see also Reference Haq, Fukuda-Parr and KumarHaq 2003). Haq has also claimed that although the index is a useful measure for policy purposes it should be supplemented by other, more detailed, socioeconomic indicators. The HDRO has consistently maintained this position (Reference KlugmanKlugman et al. 2011).

Sen also thought that using constant weights for the three constituent elements was an oversimplification. As Haq comments, however, the reason for equal weights was that “all these choices were very important and that there was no a priori rationale for giving a higher weight to one choice than to another” (Reference HaqHaq 2005: 47). Sen says that Haq was impatient with theory and that he created a broad vehicle that accommodated many theoretical approaches but did not necessarily resolve their differences. Haq wanted a practical accord, not conceptual agreement, and was always ready to revise (Reference Sen, Fukuda-Parr and KumarSen 2005: ix).

Governance Effects and Other Effects

The HDI was originally designed, and remains, primarily a means to reshape experts' thinking about development and to attract public and political support for development policies directed to enhancing human “capabilities.” We have found only modest examples of the HDI's being used as a formal element of global governance decisionmaking. The UNDP does not tie its aid decisions systematically to HDI rankings. Only one major multilateral organization—the European Union (EU), under the auspices of the European Development Fund—appears to have explicitly taken components of countries' HDI scores into account for the purposes of allocating aid (IFAD 2008). The UNDP does not even urge governments to adopt policies that will maximize their states' performance on the HDI. It does, however, use the HDI to advocate capabilities-enhancing policies that promote education and health as well as income.

Thus, the HDI is an example of an indicator with broad public recognition yet largely indirect impact on policy formation and decisionmaking. It embodies a particular ideology of development and, as a product of the type of scientific processes that Reference LatourLatour (1987) analyzes, it is influential partly because of the networks of prominent scientists and organizations that have worked together on it over many years. An analysis of media focusing on English-language publications' reports about the HDI indicates that references to the HDI increased during the 2000s and that these references typically cast the HDI as a factual description of a country. In contrast, media reports from the early 1990s focus on the nature of the index itself and its credibility. It appears that over two decades, journalists came to see the HDI as a convenient shorthand for describing a country. The settled indicator nonetheless continued to attract substantial media coverage and academic engagement.Footnote ¹⁰ Moreover, its approach to development and to measurement have had mobilizing effects, including production by national entities of national Human Development Reports and related media and political attention (Reference McNeillMcNeill 2007: 10), and other initiatives such as calculation of HDI values for indigenous peoples in Canada and slotting of these values into the HDI rankings.

Contestation and Regulation

The HDI quickly attained sufficient political acceptability among government representatives that they did not seek its abolition or radical rethinking. At times, however, the HDI has provoked contestation analogous to the contestation surrounding the Doing Business project's EWI.

Governmental objections began in 1991, the second year of the HDR, when the producers of the HDI included a ranking of countries on the Human Freedom Index, based on data in 40 categories developed by U.S. academic Charles Humana. The Group of 77 developing countries objected to a UN organization's embracing one particular person's or society's concept of freedom. Highlighting especially the inclusion of freedom of homosexual activity, the group voiced its opposition strongly in both the UNDP Governing Council and the UN General Assembly (Reference BarshBarsh 1993). The UNDP staff developed a separate system of categorization and measurement not framed in terms of Humana's work and, in the 2002 Human Development Report, launched the new system as the Political Freedom Index (PFI). The PFI consisted of five measures—personal security, rule of law, freedom of expression, political participation, and equality of opportunity—and included data on 102 countries. It did not rank countries but provided aggregates for “high,” “medium,” and “low” countries by HDI, income, and industrialization as compared to developing countries. However, it was dropped the following year, having “generated a huge political backlash during which the continuity of the Report was perceived to be in jeopardy” (Reference KlugmanKlugman et al. 2011: 17; see also Reference Fukuda-Parr, Kumar, Fukuda-Parr and KumarFukuda-Parr & Kumar 2003: xxvii). The HDRO continued, however, to include measures relating to political freedom in HDRs. This led to political debate in the UN General Assembly in 2002, as well as a related (although more technical) series of discussions in the UN Statistical Commission from 2000 to 2002. In 2010 Cuba and the League of Arab States complained to the UN about the use of indicators of political freedom, civil liberties, accountability, and agency in the 2010 HDR (Reference KlugmanKlugman et al. 2011: 17).

Within the UN Statistical Commission, a body consisting of national representatives usually from national government statistics offices, contestation of the HDRO's work was renewed from 2008 onward. It was the subject of an extensive series of comments, many of which challenged the 2010 HDI report, made by over 40 countries at the annual meeting of the UN Statistical Commission in 2011, observed by one of the authors. Objections were voiced to the addition in 2010 of a “multi-dimensional poverty index” produced by a team at Oxford University (Brazil, Morocco, and South Africa 2010). With regard to this poverty index and to the HDI, many national statistical agencies objected to a lack of consultation or transparency in decisions by the HDRO not to use data supplied by governmental or intergovernmental agencies. Some were concerned about disparagement of the quality or veracity of their own data and undermining of genuine national assessments of development, but many spoke more in the name of an “international statistical community” that ought to have been consulted (Brazil, Morocco, and South Africa 2010).

The various challenges to the 2010 HDI and other HDR indicators were considered by the UNDP Executive Board in 2011. It welcomed the HDRO's further efforts “to engage with the international statistical community on statistical matters” and its consultations with governments. It supported efforts to improve the quality and accuracy of the HDRs, “while also preserving the Report's credibility and impartiality, and without compromising its editorial independence” (UNDP Executive Board 2011). The HDRO was thus protected from serious political interference, and left instead to take what guidance it wished from expert statisticians, who themselves were understood as persuasive through expertise and through embodying epistemic and statistical-institutional interests more than national political interests.