Skip to main content Accessibility help
×
×
Home

Explaining Systematic Bias and Nontransparency in U.S. Social Security Administration Forecasts

  • Konstantin Kashin (a1), Gary King (a1) and Samir Soneji (a2)

Abstract

The accuracy of U.S. Social Security Administration (SSA) demographic and financial forecasts is crucial for the solvency of its Trust Funds, other government programs, industry decision-making, and the evidence base of many scholarly articles. Because SSA makes public insufficient replication information and uses antiquated statistical forecasting methods, no external group has ever been able to produce fully independent forecasts or evaluations of policy proposals to change the system. Yet, no systematic evaluation of SSA forecasts has ever been published by SSA or anyone else—until a companion paper to this one. We show that SSA's forecasting errors were approximately unbiased until about 2000, but then began to grow quickly, with increasingly overconfident uncertainty intervals. Moreover, the errors are largely in the same direction, making the Trust Funds look healthier than they are. We extend and then explain these findings with evidence from a large number of interviews with participants at every level of the forecasting and policy processes. We show that SSA's forecasting procedures meet all the conditions the modern social-psychology and statistical literatures demonstrate make bias likely. When those conditions mixed with potent new political forces trying to change Social Security, SSA's actuaries hunkered down, trying hard to insulate their forecasts from strong political pressures. Unfortunately, this led the actuaries into not incorporating the fact that retirees began living longer lives and drawing benefits longer than predicted. We show that fewer than 10% of their scorings of major policy proposals were statistically different from random noise as estimated from their policy forecasting error. We also show that the solution to this problem involves SSA or Congress implementing in government two of the central projects of political science over the last quarter century: (1) transparency in data and methods and (2) replacing with formal statistical models large numbers of ad hoc qualitative decisions too complex for unaided humans to make optimally.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Explaining Systematic Bias and Nontransparency in U.S. Social Security Administration Forecasts
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Explaining Systematic Bias and Nontransparency in U.S. Social Security Administration Forecasts
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Explaining Systematic Bias and Nontransparency in U.S. Social Security Administration Forecasts
      Available formats
      ×

Copyright

This is an Open-Access article, distributed under the terms of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

Corresponding author

e-mail: king@harvard.edu (corresponding author)

Footnotes

Hide All

Authors' note: For helpful advice or comments, we are grateful to Bill Alpert, Jim Alt, Steve Ansolabehere, Neal Beck, Nicholas Christakis, Mo Fiorina, Dan Gilbert, Alexander Hertel-Fernandez, Martin Holmer, David Langer, and Theda Skocpol. Thanks also to the many participants in the forecasting and policy process for information and advice. Replication data are available on the Political Analysis Dataverse at http://dx.doi.org/10.7910/DVN/28323. Supplementary materials for this article are available on the Political Analysis Web site.

Footnotes

References

Hide All
Altman, Nancy J. 2005. The battle for Social Security: From FDR's vision to Bush's gamble. Hoboken, NJ: John Wiley & Sons.
Autor, David H., and Duggan, Mark G. 2006. The growth in the Social Security disability rolls: A fiscal crisis unfolding. Journal of Economic Perspectives 20(3): 7196.
Banaji, Mahzarin R., and Greenwald, Anthony G. 2013. Blindspot: Hidden biases of good people. New York, NY: Delacorte Press.
Beland, Daniel. 2005. Social Security: History and politics from the New Deal to the privatization debate. Lawrence, KS: University Press of Kansas.
Beland, Daniel, and Waddan, Alex. 2012. The politics of policy change: Welfare, Medicare, and Social Security reform in the United States. Washington, DC: Georgetown University Press.
Blahous, Charles P. III 2007. Have the Social Security Trustees been too conservative? Paper presented at the American Enterprise Institute.
Blahous, Charles P. III 2010. Social Security: The unfinished work. Stanford, CA: Hoover Institution Press.
Edwards, George C III. 2007. Governing by campaigning: The politics of the Bush presidency. New York, NY: Longman Publishing.
Gilbert, Daniel T. 1998. Ordinary psychology. In The handbook of social psychology, eds. Gilbert, Daniel T., Fiske, Susan T., and Lindzey, G., Vol. 2, 89150. New York: McGraw Hill.
Girosi, Federico, and King, Gary 2008. Demographic forecasting. Princeton, NJ: Princeton University Press. http://gking.harvard.edu/files/smooth/(accessed April 29, 2015).
Human Mortality Database. 2015. University of California, Berkeley (USA), and Max Planck Institute for Demographic Research (Germany). or (data downloaded on 31 January 2015) http://www.mortality.org or http://www.humanmortality.de (accessed April 29, 2015).
Kahneman, Daniel. 2011. Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux.
Kashin, Konstantin, King, Gary, and Soneji, Samir 2015a. Replication data for: Explaining systematic bias and nontransparency in U.S. Social Security Administration forecasts. Harvard Dataverse [Distributor] V1 [Version]. UNF:6:967llFHgiywsHWWp1cVg9A== http://dx.doi.org/10.7910/DVN/28323 (accessed April 29, 2015).
Kashin, Konstantin, King, Gary, and Soneji, Samir 2015b. Replication data for: Systematic bias and nontransparency in U.S. Social Security Administration forecasts. Harvard Dataverse [Distributor] V1 [Version]. UNF:5:1oerGFXQ0Bu9bcMFU5/t2A== http://dx.doi.org/10.7910/DVN/28122 (accessed April 29, 2015).
Kashin, Konstantin, King, Gary, and Soneji, Samir 2015c. Systematic bias and nontransparency in U.S. Social Security Administration forecasts. Journal of Economic Perspectives. In press.
King, Gary. 1995. Replication, replication. PS: Political Science and Politics 28(3): 443–99. http://j.mp/jCyfF1 (accessed April 29, 2015).
King, Gary, and Zeng, Langche 2004. Inference in case-control studies. In Encyclopedia of biopharmaceutical statistics, ed. Chow, Shein-Chung, 2nd ed. New York: Marcel Dekker. http://gking.harvard.edu/files/abs/1s-enc-abs.shtml (accessed April 29, 2015).
King, Gary, and Soneji, Samir. 2011. The future of death in America. Demographic Research 25(1): 138. http://j.mp/iXUpBv (accessed April 29, 2015).
Laursen, Eric. 2012. The people's pension: The struggle to defend Social Security since Reagan. Edinburgh, Scotland: AK Press.
Lee, Ronald D., and Carter, Lawrence R. 1992. Modeling and forecasting U.S. mortality. Journal of the American Statistical Association 87(419): 659–75.
Office of the Chief Actuary. 2012. The long-range demographic assumptions for the 2012 Trustees Report. Technical report, Social Security Administration. http://j.mp/OCACT12 (accessed April 29, 2015).
Office of the Chief Actuary. 2013. The long-range demographic assumptions for the 2013 Trustees Report. Technical report, Social Security Administration. http://j.mp/OCACT13 (accessed April 29, 2015).
Office of the Chief Actuary. 2014. The long-range demographic assumptions for the 2014 Trustees Report. Technical report, Social Security Administration. http://j.mp/OCACT14 (accessed April 29, 2015).
Rosenblatt, Robert, and DeWitt, Larry 2005. The role of Social Security's chief actuary. Contingencies (July/August):40–5. http://j.mp/ChiefA (accessed April 29, 2015).
Social Security Advisory Board Technical Panel. 2003. Technical Panel on assumptions and methods. Technical report, Social Security Advisory Board. http://j.mp/SSATech03 (accessed April 29, 2015).
Social Security Advisory Board Technical Panel. 2007. Technical Panel on assumptions and methods. Technical report, Social Security Advisory Board. http://j.mp/SSATech07.
Social Security Advisory Board Technical Panel. 2011. Technical Panel on assumptions and methods. Technical report, Social Security Advisory Board http://j.mp/SSATech11.
Soneji, Samir, and King, Gary. 2012. Statistical security for Social Security. Demography 49(3): 1037–60. http://j.mp/Qvla7N (accessed April 29, 2015).
Tetlock, Philip E. 2005. Expert political judgment: How good is it? How can we know? Princeton, NJ: Princeton University Press.
The Board of Trustees, Federal OASDI Trust Funds. 1982–2010. Annual report of the Board of Trustees of the federal old-age and survivors insurance and federal disability insurance Trust Funds. Technical report, Social Security Advisory Board.
Weaver, R. Kent. 1988. Automatic government: The politics of indexation. Brookings Institution Press.
Wilson, Timothy D., and Brekke, Nancy. 1994. Mental contamination and mental correction: Unwanted influences on judgments and evaluations. Psychological Bulletin 116(1):117.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed