Hostname: page-component-7c8c6479df-8mjnm Total loading time: 0 Render date: 2024-03-29T06:18:57.780Z Has data issue: false hasContentIssue false

The available evidence suggests the percent measure should not be used to study inequality: Reply to Norton and Ariely

Published online by Cambridge University Press:  01 January 2023

Kimmo Eriksson*
Affiliation:
Mälardalen University Kimmo Eriksson, Centre for the Study of Cultural Evolution, Stockholm University, Wallenberglab, SE–10691, Stockholm, Sweden
Brent Simpson
Affiliation:
University of South Carolina
*
Email: kimmoe@gmail.com.
Rights & Permissions [Opens in a new window]

Abstract

In this reply, we reiterate the main point of our 2012 paper, which was that the measure of inequality used by Norton and Ariely (2011) was too difficult for it to yield meaningful results. We describe additional evidence for this conclusion, and we also challenge the conclusion that political differences in perceived and desired inequality are small.

Type
Reply
Creative Commons
Creative Common License - CCCreative Common License - BY
The authors license this article under the terms of the Creative Commons Attribution 4.0 License.
Copyright
Copyright © The Authors [2013] This is an Open Access article, distributed under the terms of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

We are pleased that Norton and Ariely have written a comment on our paper (Reference Eriksson and SimpsonEriksson and Simpson, 2012). Their comment makes two related points: that there is more agreement than disagreement between our papers, and that a fruitful direction for future research is to understand why the different measures we used generate different results. We believe that both points need qualification in the light of our paper’s main arguments and findings (including some additional findings that we did not report).

To summarize, we argued that Reference Norton and ArielyNorton and Ariely’s (2011) findings are limited because they are derived from a method—measuring perceptions of wealth inequality in terms of percentages of quintiles—that is so conceptually and computationally demanding that it does not tap into people’s actual perceptions of inequality. To support our claims, we introduced a logically equivalent but less demanding measure that asked respondents about the average wealth in each quintile. What we found:

  • The two measures gave radically different results: whereas respondents’ estimates of wealth inequality using the percent measure were highly inaccurate, the same respondents made far more accurate estimates of inequality using the average measure.

To further illustrate that the percent measure does not track respondents’ actual perceptions, we assessed perceptions of inequality in other domains as well. One domain was popularity of web pages, where common sense says that the most popular web pages, such as Facebook and Google, receive many orders of magnitude more clicks per day than the least popular web pages (e.g., our personal web pages). For another domain, school teachers’ salaries, common sense says that inequality is much more modest, because not even the highest paid teachers are very highly paid. What we found:

  • The average measure yielded inequality perceptions that qualitatively corresponded to the high or low levels that actually exist in a given domain while the percent measure yielded low levels of perceived inequality regardless of domain (see Table 1, Reference Eriksson and SimpsonEriksson and Simpson, 2012).

Why does the percent measure fail to track perceptions of inequality? In order to transform perceptions into a response to the percent measure, a respondent must first estimate averages in each quintile and then calculate percentages. (We encourage readers to try to answer the measure for themselves.) The fact that most of of our respondents gave completely inconsistent responses to the two measures suggested that they did not attempt to calculate the percentages or could not do so.

In light of the above, we do not believe the two points of Norton and Ariely’s comment are justified. First, consider the extent to which the measures agree. As we noted repeatedly in our paper (see pp. 742, 744), both measures showed that respondents want less inequality than they believe exists. But it is worth reiterating that the average measure did not lead to anything near the egalitarian preferences elicited by the percent measure.Footnote 1 To underscore how the measures lead to very different conclusions, consider findings on political differences from our second study, not included in our paper. For the percent measure, the median ratio of wealth that respondents thought should be owned by the poorest versus the richest was 1:3 among liberals and 1:5 among conservatives. These results suggest not only that respondents are incredibly egalitarian, but also that there is a high level of consensus between liberals and conservatives in ideal inequality. But we argue that these findings reflect less a consensus across the political spectrum than a common inability to answer the percent question. The average measure showed much less egalitarianism and much more dissensus between liberals and conservatives, who gave median ratios of 1:50 and 1:200, respectively.Footnote 2 Further, the two measures generated an even larger difference in perceptions of inequality, the domain on which our paper was mainly focused. While the percent measure yielded estimates that the wealthiest Americans are only about 20 times richer than the poorest, the average measure yielded a ratio of 1:1,500, as noted earlier.Footnote 3

The second issue is how future research should be guided by what we have learned thus far. Norton and Ariely suggest some reasons that the two measures yield different results and that future work might address. However, their suggestions do not fit the data we presented in our paper.Footnote 4 Norton and Ariely do not mention the explanation given in our paper: the percent question is simply too demanding for respondents to answer. Because our original paper provided only indirect evidence that participants did not or could not do these computations we conducted a new study to provide more direct evidence for this claim. Specifically, we gave 100 users of Amazon’s Mechanical Turk the following information about the distribution of wealth in a country:

The richest 20% of households have an average wealth of 1,000,000 dollars per household. The 2nd richest 20%: 200,000 dollars per household. The middle 20%: 50,000 dollars per household. The 2nd poorest 20%: 10,000 dollars per household. The poorest 20%: 1,000 dollars per household.

We then asked respondents whether, given this information, it is possible to tell what percent of the country’s total wealth each quintile controls. We asked those who stated that they could do the calculations to do so. We found:

  • Of the 100 respondents, 37 thought the percentages could not be derived from the information given; 48 thought the percentages could be derived but said they didn’t know how to do it; 15 thought they knew how to do it, but 6 of them did it incorrectly, typically saying that the percentages were 20, 20, 20, 20, 20.

These results strongly suggest people are generally unable to transform perceptions of typical households’ wealth into percentages, as we suggested.

We sum up by noting that the original Reference Norton and ArielyNorton and Ariely (2011) paper has received a very high level of mostly uncritical attention, probably because of the importance of the topic, the surprising findings, and the fact that both authors are well-known for doing excellent research. In this case, however, the available evidence suggests that their original conclusions are artifacts of an invalid measure. Our original paper was motivated by our concerns that policy recommendations might be founded on unsupported conclusions and that the paper might set a precedent for future research to employ similar invalid measures of perceived and desired inequality.

Footnotes

Work supported by the Swedish Research Council [2009-2390] and the National Science Foundation [SES-0551895, SES-0647169].

1 The average measure yielded over a 12-fold increase in the level of ideal inequality, compared to the percent measure. This difference is substantially larger than the difference in wealth inequality that exists in the U.S. vs. Sweden.

2 We also did not observe, in unreported results, so much consensus between liberals and conservatives about how to achieve greater equality. Most liberal and moderate respondents felt that policy makers should work to achieve more equitable distributions. Conservatives, on the other hand, tended to think that a more equal distribution should come about by the poor working harder. These findings echo prior research showing that conservatives are much more likely than liberals to view existing inequality as fair and good.

3 In their comment, Norton and Ariely downplay the similarity, noting that “estimating a 1:1000 ratio as 1:1500 remains far from accurate; students generating that answer to some arithmetic problem on an exam would fail just as readily as those who came up with 1:20.” But 1:1500 might be considered an acceptable answer to an estimation problem. Note also that even 1:1000 is a rough estimate. And, even if an exact ratio could be defined, it could be anywhere between 1:557 and 1:1690 just from rounding errors in the estimates (0.1% of the wealth controlled by the poor, 84% by the rich) reported by Reference Norton and ArielyNorton and Ariely (2011).

4 One suggestion was a potential priming effect of how wealth was defined to participants. But our respondents gave very different responses following the same definition of wealth, so this cannot explain the difference. The other suggestion was that, compared to the average measure, the percent measure is zero-sum and thus may prompt more concerns about equality. But such concerns would not seem to apply to perceptions of inequality, and are irrelevant to domains such as web-page visits. Thus, this suggestion cannot explain the systematic difference in responses to the two measures.

References

Eriksson, K., & Simpson, B. (2012). What do Americans know about inequality? It depends on how you ask them. Judgment and Decision Making, 7, 741745.CrossRefGoogle Scholar
Norton, M., & Ariely, D. (2011). Consensus on building a better America—one wealth quintile at a time. Perspectives on Psychological Science, 6, 912.CrossRefGoogle ScholarPubMed