Hostname: page-component-848d4c4894-v5vhk Total loading time: 0 Render date: 2024-06-15T17:27:26.516Z Has data issue: false hasContentIssue false

Evaluating Policy Mood Measures in the American States

Published online by Cambridge University Press:  21 November 2022

Jeffrey J. Harden*
Affiliation:
Department of Political Science, University of Notre Dame, Notre Dame, IN, USA
Julianna Pacheco
Affiliation:
Department of Political Science, University of Iowa, Iowa City, IA, USA
Gerald C. Wright
Affiliation:
Department of Political Science, Indiana University, Bloomington, IN, USA
*
Corresponding author: Jeffrey J. Harden; Email: jeff.harden@nd.edu
Rights & Permissions [Opens in a new window]

Abstract

The scholarly exchange over approaches to measuring public preferences in the American states dates back several years. This introduction to the debate attempts to provide broad perspective on how scholars have conceptualized and measured policy mood among state mass publics in the past and the implications of those choices for theoretical and empirical questions in state politics. We frame the discussion around two questions: (1) what is the concept to be measured? and (2) how does the concept fit with the research question? We conclude with some insight on the future of the debate and its implications for state politics research.

Type
Original Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2022. Published by Cambridge University Press and State Politics & Policy Quarterly

State politics scholars have long been interested in the measurement and analysis of indicators of the mass public’s preferences in the states. Such measures hold scholarly interest in their own right but are also useful to researchers studying policymaking, responsiveness, and other outcomes related to democratic accountability at the state level. A central concern for these studies is the strategic choices for measuring mass preferences. Scholars have devoted substantial attention to this question for decades and the available options range widely: simulation-based approaches (Weber et al. Reference Weber, Hopkins, Mezey and Munger1972), disaggregation of survey data (e.g., Carsey and Harden Reference Carsey and Harden2010; Erikson, Wright, and McIver Reference Erikson, Wright and McIver1993), proxy measures (Berry et al. Reference Berry, Ringquist, Fording and Hanson1998), and modeling and/or poststratification of national surveys (e.g., Caughey and Warshaw Reference Caughey and Warshaw2018; Enns and Koch Reference Enns and Koch2013; Pacheco Reference Pacheco2011). While such an array of choices is generally useful to the state politics community, the various measures and the discussions over them that have taken place in the last several years may ultimately create confusion, especially for new scholars in the field.

The debate over measuring one such indicator of public preferences – state policy mood – continues in this issue, with contributions from Berry, Fording, Hanson, and Crofoot (BFHC) and Lagodny, Jones, Koch, and Enns (LJKE). Their articles provide detailed comparisons of the measurement strategies for state mood proposed by Berry et al. (Reference Berry, Ringquist, Fording and Hanson1998) and Enns and Koch (Reference Enns and Koch2013). Our aim in this introduction to the debate is to step back, assess the context, and evaluate some important issues for researchers choosing between these measures or any other measures of state preferences. In particular, we focus on two related questions:

  1. 1. What is the concept to be measured?

  2. 2. How does the concept fit with the research question?

We hope that our answers to these questions can help frame the discussion moving forward and provide useful insight to scholars studying the role of the mass public in American state politics.

What is the concept to be measured?

Our first task is to reflect on the concept these indices purport to measure and to suggest some implications of the very different operationalizations these researchers use. The concept of interest is state policy mood, which is the predisposition of state electorates for more or less government activity on an array of economic policy areas. Both the BRFH and the LJKE measures seek to replicate Stimson’s (Reference Stimson1999) measure of national mood at the state level. BFHC’s effort in this issue offers correlations of their measure with a variety of combinations of items drawn from the General Social Survey (GSS) to make the case for its validity. In LJKE’s update of the Enns and Koch (Reference Enns and Koch2013) work on state policy mood, the primary data used to construct the measure are the same GSS items Stimson uses to measure national policy mood. These items are restricted to a limited set of economic and related policies, including health, education, and welfare. They explicitly exclude cultural issues such as race, abortion, and guns, which means that these mood measures do not capture that underlying dimension. This measurement strategy explicitly excludes state opinions toward cultural issues. If that is the case, then state policy mood as a primarily economic indicator may not be appropriate for analyses involving morality policies.

Another potential problem with the validity of both the BRFH and LJKE measures is that they have, at best, an indirect connection to factors that scholars often want to explain, such as patterns of state policy or mass attitudes toward state policy. Stimson’s measure clearly assumes that survey respondents are thinking about the policy efforts of the federal government when they answer GSS questions. So if either index is able to replicate Stimson’s measure at the state level, we still have the problem of using a state-level measure of national policy mood to explain changes in state policies and election outcomes.

One way to theoretically address this problem is to assume that citizens do not differentiate between levels of government in their policy preferences. If so, a national policy mood would be roughly the same as a state mood measure and the problem is mitigated. However, just a bit of reflection suggests that this is not a defensible way around the problem. It is only too easy to imagine sets of citizens who would have quite different preferences for policy changes at the state versus the national level due to variations in the status quo at each level. Take, for example, the group of respondents who might have moderate (“spending enough”) responses to national welfare or education expenditures. Those living in Alabama or Mississippi might see the stinginess of their state governments’ efforts as “too little,” whereas their counterparts living in higher-spending states like New York or California could feel that “too much” is being spent by their state governments. We can imagine parallel scenarios in other policy areas.

If either of the measures seems not to “work” in an analysis, we would have to question whether state governments do indeed ignore the preferences of their electorates, or whether we actually have measures of something else (national policy mood) that is not systematically related to the key concept of state policy mood. In fact, it would be surprising if public reactions to national economic policies – a constant at any time point – could capture the variety of citizen reactions to the increasingly divergent patterns of a subset of state policies. If the patterns of “leapfrog representation” identified by Bafumi and Herron (Reference Bafumi and Herron2010) hold for state governments, then we might expect wildly divergent reactions of state electorates as the blue states race toward more liberal policy regimes and the red states adopt increasingly conservative policies. Our simple, but for state politics researchers quite important, point is that we have lots of reasons to suspect that measures of national policy mood, even measured at the state level, do not consistently measure citizens’ evaluations of the policies in their states.

How does the concept fit with the research question?

Decisions on how to measure state public preferences depend on how central the concept is to answer the research question. For some lines of inquiry, state preferences may be a control variable included to capture a state’s tendency toward a particular policy or political outcome. Take the vast literature on policy diffusion as an example. Here, scholars are primarily interested in understanding why a state adopts a policy at a particular time and how policies spread across the states. While public opinion may matter for these processes (Pacheco Reference Pacheco2012), it is often secondary to the role of elites (Mintrom Reference Mintrom1997), pressures within and across states (e.g., Berry and Berry Reference Berry and Berry1990; Desmarais, Harden, and Boehmke Reference Desmarais, Harden and Boehmke2015), and influence from local governments and the federal government (e.g., Karch and Rose Reference Karch and Rose2019; Shipan and Volden Reference Shipan and Volden2006).

For research questions where public opinion is a secondary concern, researchers might do well to follow the lead of early state opinion scholars (e.g., Brace et al. Reference Brace, Butler, Arceneaux and Johnson2002; Erikson, Wright, and McIver Reference Erikson, Wright and McIver1993) and include direct aggregate measures of partisanship, ideology, or vote choice. These measures may be drawn from single national surveys with large sample sizes, such as the Cooperative Election Study (CES) or the National Annenberg Election Surveys (NAES), pooled cross-sectional surveys, such as the American National Election Survey (ANES), or state exit polls. All of these options are simplified ways to categorize the broad political leanings of state residents. Regional measures of geography can sufficiently capture meaningful variations in state culture. Of course, the indicators discussed here could also be used, but researchers should be careful to accurately describe what and why they include particular control variables over others.

Scholars must be more thoughtful when state opinion is the dependent variable. A scholar might ask whether state opinion is changing – is state opinion stable or dynamic? If state opinion changes over time, does it change gradually or quickly? Related, are states generally trending in similar or divergent ways? Are there parallel publics across the states (e.g., Page and Shapiro Reference Page and Shapiro1992)? These are important questions to ask because the combination of the rate of change (stable, gradual, or abrupt) and the patterns of change (homogeneous or heterogeneous) provide insight into the determinants of aggregate state opinion. Thus, these types of descriptive analyses are useful for understanding theories in political science. They are also informative to journalists, electoral strategists, and policymakers, all of whom are likely especially interested in the practical implications of what determines general state opinions and, specifically, state policy mood.

A guiding principle for these research questions is whether we are interested in describing broad political patterns or narrowly defined issue attitudes. For the former, scholars essentially have three choices: partisanship, ideology, or policy mood. State macropartisanship and macroideology move slowly over time where the relative rankings of the states are fairly consistent (Enns and Koch Reference Enns and Koch2013; Pacheco Reference Pacheco2012; Wright and Birkhead Reference Wright and Birkhead2014). State policy mood, however, is more dynamic, regardless of whether one uses the measure from LJKE or BFHC’s indicator. The implication is that shifts in the partisan or ideological leanings of states are likely caused by gradual population changes such as generational replacement, migration, immigration, and differential birth and death rates (Brace et al. Reference Brace, Arceneaux, Johnson and Ulbig2004; Carmines and Stimson Reference Carmines and Stimson1989). Abrupt movements in opinion, such as is the case with state policy mood, are likely caused by reactions to current events, the economy, or other transient outcomes (Erikson, MacKuen, and Stimson Reference Erikson, MacKuen and Stimson2002; Stimson Reference Stimson2004).

The rate and patterns of opinion change across states on specific issues vary depending on the issue. In the most comprehensive analysis of the dynamic properties of specific state opinions, Pacheco (Reference Pacheco2014) finds that state attitudes on consumer sentiment and presidential approval are highly dynamic and exhibit homogenous trends. State attitudes toward education spending, welfare spending, and the death penalty exhibit more change over time with heterogeneous trends (Brace et al. Reference Brace, Butler, Arceneaux and Johnson2002; Pacheco Reference Pacheco2014). State attitudes toward abortion are essentially stable. More recent studies find that state attitudes toward the Affordable Care Act (ACA) exhibit various patterns, depending on whether residents responded to a general question about ACA favorability, the impact of the ACA, or the future impact of the ACA (Pacheco and Maltby Reference Pacheco and Maltby2019; see also Brodie, Deane, and Cho Reference Brodie, Deane and Cho2011). Finally, there is evidence that state racial resentment changes slowly over time with heterogenous trends (Smith, Kreitzer, and Suo Reference Smith, Kreitzer and Suo2020). Continuing this type of research to understand the descriptive characteristics of state public opinion is a fruitful avenue for the future.

Thoughtfully considering how to measure state preferences is equally important when it is the main independent variable of interest, which is the case for studies asking how closely state policies align with the mass public’s policy attitudes. For these studies, a scholar must be purposeful in making sure that the measure of state preferences reasonably matches the dependent variable. If the dependent variable is capturing broad policy tendencies of the states, for instance, state policy liberalism (e.g., Caughey and Warshaw Reference Caughey and Warshaw2018; Erikson, Wright, and McIver Reference Erikson, Wright and McIver1993), then it is reasonable to measure state opinion along a similar ideological dimension (see Caughey and Warshaw Reference Caughey and Warshaw2016). When the dependent variable is a specific policy outcome; for instance, on gay rights policies (Lax and Phillips Reference Lax and Phillips2009), the ACA (Pacheco and Maltby Reference Pacheco and Maltby2019), or antismoking legislation (Pacheco Reference Pacheco2012), it makes more sense for scholars to measure specific state attitudes. Linking concepts to data is a crucial aspect of any research design, especially in cases such as this one where there are multiple options from which scholars can choose.

Conclusions

The study of American public preferences carries a great deal of theoretical and empirical nuance to which scholars must be attentive. We commend the authors in this years-long debate on state policy mood for their careful consideration of these details and encourage researchers employing their data to do the same. The measurement of public preferences in the states is relevant to many research agendas, each with their own unique challenges. Thus, readers should not necessarily look to this debate for guidance on which measure is the “best,” because that distinction is heavily dependent on the context. Rather, we encourage scholars to consider the concept to be measured based on their research question, select an indicator, then assess the sensitivity of their results to that choice. Indeed, none of these measures is perfect and we will never learn “true” state preferences of any kind. It is important for researchers to acknowledge that the available options are proxy measures that contain an error.

Of course, we certainly do not intend to persuade researchers to shy away from using the indicators discussed in this issue. They are the best measures we have (to date) and the state policy–public opinion relationship is too central to the state politics and policy subfield and political science more generally to ignore. But it is equally important to be aware of the shortcomings of even our best measures and to temper our conclusions, especially if they do not reinforce one’s democratic images of state government responsiveness. As in a great deal of state politics research, the continued expansion and increasing availability of new data will no doubt provide additional advances in this area in years to come.

Funding Statement

The authors declare no funding support for this article.

Conflict of Interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Author Biographies

Jeffrey J. Harden is an Andrew J. McKenna Family Associate Professor of Political Science at the University of Notre Dame, Notre Dame, IN 46556. Julianna Pacheco is an Associate Professor of Political Science at the University of Iowa, Iowa City, IA 52242. Gerald C. Wright is an Emeritus Professor of Political Science at Indiana University, Bloomington, IN 47405.

References

Bafumi, Joseph, and Herron, Michael C.. 2010. “Leapfrog Representation and Extremism: A Study of American Voters and Their Members in Congress.” American Political Science Review 104 (3): 519–42.Google Scholar
Berry, Frances Stokes, and Berry, William D.. 1990. “State Lottery Adoptions as Policy Innovations: An Event History Analysis.” American Political Science Review 84 (2): 395415.Google Scholar
Berry, William D., Ringquist, Evan, Fording, Richard C., and Hanson, Russell L.. 1998. “Measuring Citizen and Government Ideology in the American States, 1960-93.” American Journal of Political Science 41 (1): 327348.Google Scholar
Brace, Paul, Arceneaux, Kevin, Johnson, Martin, and Ulbig, Stacy G.. 2004. “Does State Political Ideology Change over Time?Political Research Quarterly 57 (4): 529540.Google Scholar
Brace, Paul, Butler, Kellie Sims, Arceneaux, Kevin, and Johnson, Martin. 2002. “Public Opinion in the American States: New Perspectives Using National Survey Data.” American Journal of Political Science 46 (1): 173189.Google Scholar
Brodie, Mollyann, Deane, Claudia, and Cho, Sarah. 2011. “Regional Variations in Public Opinion on the Affordable Care Act.” Journal of Health Politics, Policy and Law 36 (6): 10971103.Google Scholar
Carmines, Edward G. and Stimson, James A.. 1989. Issue Evolution: Race and the Transformation of American Politics. Princeton, NJ: Princeton University Press.Google Scholar
Carsey, Thomas M., and Harden, Jeffrey J.. 2010. “New Measures of Partisanship, Ideology, and Policy Mood in the American States.” State Politics & Policy Quarterly. 10 (2): 136156.Google Scholar
Caughey, Devin, and Warshaw, Christopher. 2016. “The Dynamics of State Policy Liberalism, 1936-2014.” American Journal of Political Science 60 (4): 899913.Google Scholar
Caughey, Devin, and Warshaw, Christopher. 2018. “Policy Preferences and Policy Change: Dynamic Responsiveness in the American States, 1936-2014.” American Political Science Review 112 (2): 249266.Google Scholar
Desmarais, Bruce A., Harden, Jeffrey J., and Boehmke, Frederick J.. 2015. “Persistent Policy Pathways: Inferring Diffusion Networks in the American States.” American Political Science Review 109 (2): 392406.Google Scholar
Enns, Peter K., and Koch, Julianna. 2013. “Public Opinion in the U.S. States: 1956-2010.” State Politics & Policy Quarterly 13 (3): 349372.Google Scholar
Erikson, Robert S., MacKuen, Michael B., and Stimson, James A.. 2002. The Macro Polity. New York: Cambridge University Press.Google Scholar
Erikson, Robert S., Wright, Gerald C., and McIver, John P.. 1993. Statehouse Democracy: Public Opinion and Policy in the American States. New York: Cambridge University Press.Google Scholar
Karch, Andrew, and Rose, Shanna. 2019. Responsive States: Federalism and American Public Policy. New York: Cambridge University Press.Google Scholar
Lax, Jeffrey R., and Phillips, Justin H.. 2009. “How Should We Estimate Public Opinion in the States?American Journal of Political Science 53 (1): 107121.Google Scholar
Mintrom, Michael. 1997. “Policy Entrepreneurs and the Diffusion of Innovation.” American Journal of Political Science 41 (3): 738770.Google Scholar
Pacheco, Julianna. 2011. “Using National Surveys to Measure Dynamic U.S. State Public Opinion: A Guideline for Scholars and an Application.” State Politics & Policy Quarterly 11 (4): 415539.Google Scholar
Pacheco, Julianna. 2012. “The Social Contagion Model: Exploring the Role of Public Opinion on the Diffusion of Anti-Smoking Legislation across the American States.” Journal of Politics 74 (1): 187202.Google Scholar
Pacheco, Julianna. 2014. “Measuring and Evaluating Changes in State Opinion Across Eight Issues.” American Politics Research 42 (6): 9861009.Google Scholar
Pacheco, Julianna, and Maltby, Elizabeth. 2019. “Trends in State Level Opinions toward the Affordable Care Act.” Journal of Health Politics, Policy and Law 44 (5): 737764.Google Scholar
Page, Benjamin I., and Shapiro, Robert Y.. 1992. The Rational Public: Fifty Years of Trends in Americans’ Policy Preferences. Chicago: University of Chicago Press.Google Scholar
Shipan, Charles R., and Volden, Craig. 2006. “Bottom-up Federalism: The Diffusion of Antismoking Policies from U.S. Cities to States.” American Journal of Political Science 50 (4): 825843.Google Scholar
Smith, Candis Watts, Kreitzer, Rebecca J., and Suo, Feiya. 2020. “The Dynamics of Racial Resentment across the 50 U.S. States.” Perspectives on Politics 18 (2): 527538.Google Scholar
Stimson, James A. 1999. Public Opinion in America: Moods, Cycles, and Swings, 2nd edition. Boulder, CO: Westview Press.Google Scholar
Stimson, James A. 2004. Tides of Consent: How Public Opinion Shapes American Politics. New York: Cambridge University Press.Google Scholar
Weber, Ronald E., Hopkins, Anne H., Mezey, Michael L., and Munger, Frank J.. 1972. “Computer Simulation of State Electorates.” Public Opinion Quarterly 36 (4): 549565.Google Scholar
Wright, Gerald C., and Birkhead, Nathaniel A.. 2014. “The Macro Sort of State Electorates.” Political Research Quarterly 67 (2): 426439.Google Scholar