Hostname: page-component-76fb5796d-vvkck Total loading time: 0 Render date: 2024-04-26T14:54:59.642Z Has data issue: false hasContentIssue false
  • English
  • Français

The Cost-Benefit Fallacy: Why Cost-Benefit Analysis Is Broken and How to Fix It

Published online by Cambridge University Press:  11 October 2021

Bent Flyvbjerg Open the ORCID record for Bent Flyvbjerg [Opens in a new window]  and
Dirk W. Bester
Bent Flyvbjerg*
Affiliation:
Saïd Business School, University of Oxford, Oxford, UK and Computer Science Department, IT University of Copenhagen, Copenhagen, Denmark
Dirk W. Bester
Affiliation:
Department of Statistics, University of Oxford, Oxford, UK
*
*e-mail: flyvbjerg@mac.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Most cost-benefit analyses assume that the estimates of costs and benefits are more or less accurate and unbiased. But what if, in reality, estimates are highly inaccurate and biased? Then the assumption that cost-benefit analysis is a rational way to improve resource allocation would be a fallacy. Based on the largest dataset of its kind, we test the assumption that cost and benefit estimates of public investments are accurate and unbiased. We find this is not the case with overwhelming statistical significance. We document the extent of cost overruns, benefit shortfalls, and forecasting bias in public investments. We further assess whether such inaccuracies seriously distort effective resource allocation, which is found to be the case. We explain our findings in behavioral terms and explore their policy implications. Finally, we conclude that cost-benefit analysis of public investments stands in need of reform and we outline four steps to such reform.

Keywords

behavioral economicsbehavioral sciencecost-benefit analysiscost-benefit fallacyforecastingpublic investment planningresource allocationwelfare economicsD60D61H00H54H80
Type
Article
Information
Journal of Benefit-Cost Analysis , Volume 12 , Issue 3 , Fall 2021 , pp. 395 - 419
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of the Society for Benefit-Cost Analysis

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akerlof, George A., and Shiller, Robert J.. 2009. Animal Spirits: How Human Psychology Drives the Economy, and Why It Matters for Global Capitalism. Princeton: Princeton University Press.Google Scholar
Albalate, Daniel, and Bel, Germa. 2014. The Economics and Politics of High-Speed Rail. New York: Lexington Books.Google Scholar
Altshuler, Alan, and Luberoff, David. 2003. Mega-Projects: The Changing Politics of Urban Public Investment. Washington, DC: Brookings Institution Press.Google Scholar
Ansar, Atif, Flyvbjerg, Bent, Budzier, Alexander, and Lunn, Daniel. 2014. “Should We Build More Large Dams? The Actual Costs of Hydropower Megaproject Development.” Energy Policy, 69: 43–56.CrossRefGoogle Scholar
Ariely, Dan. 2010. Predictably Irrational: The Hidden Forces That Shape Our Decisions. New York: Harper Collins.Google Scholar
Awojobi, Omotola, and Jenkins, Glenn P.. 2016. “Managing the Cost Overrun Risks of Hydroelectric Dams: An Application of Reference Class Forecasting Techniques.” Renewable and Sustainable Energy Reviews, 63: 19–32.CrossRefGoogle Scholar
Bain, Robert. 2009. Toll Road Traffic and Revenue Forecasts: An Interpreter’s Guide (Published by author).Google Scholar
Bain, Robert, and Wilkins, Michael. 2002. Traffic Risk in Start-up Toll Facilities. London: Standard and Poor’s.Google Scholar
Batselier, Jordy. 2016. Empirical Evaluation of Existing and Novel Approaches for Project Forecasting and Control, doctoral dissertation. Ghent: University of Ghent.Google Scholar
Batselier, Jordy, and Vanhoucke, Mario. 2017. “Improving Project Forecast Accuracy by Integrating Earned Value Management with Exponential Smoothing and Reference Class Forecasting.” International Journal of Project Management, 35(1): 8–43.CrossRefGoogle Scholar
Bordley, Robert F. 2014. “Reference Class Forecasting: Resolving Its Challenge to Statistical Modeling.” The American Statistician, 68(4): 221–229.CrossRefGoogle Scholar
Cantarelli, Chantal C., Flyvbjerg, Bent, and Buhl, Søren L.. 2012. “Geographical Variation in Project Cost Performance: The Netherlands versus Worldwide.” Journal of Transport Geography, 24: 324–331.CrossRefGoogle Scholar
Chang, Welton, Chen, Eva, Mellera, Barbara, and Tetlock, Philip. 2016. “Developing Expert Political Judgment: The Impact of Training and Practice on Judgmental Accuracy in Geopolitical Forecasting Tournaments.” Judgment and Decision Making, 11(5): 509–526.Google Scholar
Collins, Jeffrey. 2020. “Former Executive Faces Prison Time in SC Nuclear Debacle.” Associated Press, November 25, 2020.Google Scholar
Danish Ministry for Transport and Energy, Transport- og Energiministeriet. 2006. Aktstykke om nye budgetteringsprincipper (Act on New Principles for Budgeting), Aktstykke nr. 16, Finansudvalget, Folketinget, Copenhagen, October 24.Google Scholar
Danish Ministry for Transport and Energy, Transport- og Energiministeriet. 2008. “Ny anlægsbudgettering på Transportministeriets område, herunder om økonomistyrings–model og risikohåndtering for anlægsprojekter.” Copenhagen, November 18.Google Scholar
Dantata, Nasiru A., Touran, Ali, and Schneck, Donald C.. 2006. “Trends in US Rail Transit Project Cost Overrun.” In Transportation Research Board Annual Meeting. Washington, DC: National Academies.Google Scholar
Dezember, Ryan and Glazer, Emily. 2013. “Drop in Traffic Takes Toll on Investors in Private Roads.” Wall Street Journal, November 20, 2013.Google Scholar
Evans, Anthony G. 2010. “Declaration of Anthony G. Evans, Chief Financial Officer of South Bay Expressway, L.P., in Support of the Debtors’ Chapter 11 Petitions and First Day Motions,” case 10-04516-LA11, filed March 22, 2010, United States Bankruptcy Court Southern District of California.Google Scholar
Federal Transit Administration. 2003. Predicted and Actual Impacts of New Starts Projects: Capital Cost, Operating Cost and Ridership Data, Office of Planning and Environment with support from SG Associates, Inc. Washington, DC: Author.Google Scholar
Federal Transit Administration. 2008. Before and After Studies of New Starts Projects, report to Congress. Washington, DC: Author.Google Scholar
Federal Transit Administration. 2013. Before and After Studies of New Starts Projects, report to Congress. Washington, DC: Author.Google Scholar
Flyvbjerg, Bent. 2005. “Measuring Inaccuracy in Travel Demand Forecasting: Methodological Considerations Regarding Ramp Up and Sampling.” Transportation Research A, 39(6): 522–530.Google Scholar
Flyvbjerg, Bent. 2006. “From Nobel Prize to Project Management: Getting Risks Right.” Project Management Journal, 37(3): 5–15.CrossRefGoogle Scholar
Flyvbjerg, Bent. 2007. “How Optimism Bias and Strategic Misrepresentation in Early Project Development Undermine Implementation.” In Sunnevåg, Kjell J. (Eds.) Beslutninger på svakt informasjonsgrunnlag: Tilnærminger og utfordringer i projekters tidlige fase (Decisions Based on Weak Information: Approaches and Challenges in the Early Phase of Projects): 41–55. Trondheim, Norway: Concept Program, The Norwegian University of Science and Technology.Google Scholar
Flyvbjerg, Bent. 2008. “Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice.” European Planning Studies, 16(1): 3–21.CrossRefGoogle Scholar
Flyvbjerg, Bent. 2013. “Quality Control and Due Diligence in Project Management: Getting Decisions Right by Taking the Outside View.” International Journal of Project Management, 31(5): 760–774.CrossRefGoogle Scholar
Flyvbjerg, Bent. 2016. “The Fallacy of Beneficial Ignorance: A Test of Hirschman’s Hiding Hand.” World Development, 84: 176–189.CrossRefGoogle Scholar
Flyvbjerg, Bent, Ansar, Atif, Budzier, Alexander, Buhl, Søren, Cantarelli, Chantal, Garbuio, Massimo, Glenting, Carsten, Holm, Mette Skamris, Lovallo, Dan, Lunn, Daniel, Molin, Eric, Rønnest, Arne, Stewart, Allison, and van Wee, Bert. 2018. “Five Things You Should Know about Cost Overrun.” Transportation Research Part A: Policy and Practice, 118: 174–190.Google Scholar
Flyvbjerg, Bent, Ansar, Atif, Budzier, Alexander, Buhl, Søren, Cantarelli, Chantal, Garbuio, Massimo, Glenting, Carsten, Holm, Mette Skamris, Lovallo, Dan, Molin, Eric, Rønnest, Arne, Stewart, Allison, and van Wee, Bert. 2019. “On De-bunking ‘Fake News’ in the Post-Truth Era: How to Reduce Statistical Error in Research.” Transportation Research Part A: Policy and Practice, 126: 409–411.Google Scholar
Flyvbjerg, Bent and Budzier, Alexander. 2011. “Why Your IT Project May Be Riskier than You Think.” Harvard Business Review, September, pp. 601–603.Google Scholar
Flyvbjerg, Bent, Glenting, Carsten, and Rønnest, Arne. 2004. Procedures for Dealing with Optimism Bias in Transport Planning: Guidance Document. London: UK Department for Transport.Google Scholar
Flyvbjerg, Bent, Holm, Mette K. Skamris, and Buhl, Søren L.. 2002. “Underestimating Costs in Public Works Projects: Error or Lie?” Journal of the American Planning Association, 68(3): 279–295.CrossRefGoogle Scholar
Flyvbjerg, Bent, Holm, Mette K. Skamris, and Buhl, Søren L.. 2005. “How (In)accurate Are Demand Forecasts in Public Works Projects? The Case of Transportation.” Journal of the American Planning Association, 71(2): 131–146.CrossRefGoogle Scholar
Flyvbjerg, Bent, Hon, Chi-keung, and Fok, Wing Huen. 2016. “Reference Class Forecasting for Hong Kong’s Major Roadworks Projects.” Proceedings of the Institution of Civil Engineers, 169: 17–24.Google Scholar
Flyvbjerg, Bent and Stewart, Allison. 2012. “Olympic Proportions: Cost and Cost Overrun at the Olympics 1960–2012.” Working Paper, Saïd Business School, University of Oxford.CrossRefGoogle Scholar
Fouracre, P. R., Allport, R. J., and Thomson, J. M.. 1990. The Performance and Impact of Rail Mass Transit in Developing Countries. Research Report no. 278. Crowthorne: Transport and Road Research Laboratory.Google Scholar
Gao, Nan, and Touran, Ali. 2020. “Cost Overruns and Formal Risk Assessment Program in US Rail Transit Projects.” Journal of Construction Engineering and Management, 146(5): 1–11.CrossRefGoogle Scholar
Hals, Tom. 2013. “Detroit Windsor Tunnel Operator Files for Bankruptcy.” Reuters, July 25, 2013.Google Scholar
Hamer, Andrew Marshall. 1976. The Selling of Rail Rapid Transit: A Critical Look at Urban Transportation Planning. Lexington, MA: Lexington Books.Google Scholar
Harrington, Winston. 2006. “Grading Estimates of the Benefits and Costs of Federal Regulation.” Discussion Paper RFF DP 06-39, Resources for the Future, Washington, DC.CrossRefGoogle Scholar
Harrington, Winston, Morgenstern, Richard D., and Nelson, Peter. 2000. “On the Accuracy of Regulatory Cost Estimates.” Journal of Policy Analysis and Management, 19(2): 297–322.3.0.CO;2-X>CrossRefGoogle Scholar
Hirschman, Albert O.. 2014. “The Principle of the Hiding Hand,” originally published in The Public Interest, Winter 1967, pp. 10–23. In Flyvbjerg, Bent (Ed.) Megaproject Planning and Management: Essential Readings (Vol. I): 149–162. Cheltenham, UK: Edward Elgar.Google Scholar
HM Treasury. 2003. The Green Book: Appraisal and Evaluation in Central Government. Treasury Guidance. London: TSO.Google Scholar
Huo, Tengfei, Ren, Hong, Cai, Weiguang, Shen, Geoffrey Qiping, Liu, Bingsheng, Zhu, Minglei, and Wu, Hengqin. 2018. “Measurement and Dependence Analysis of Cost Overruns in Megatransport Infrastructure Projects: Case Study in Hong Kong.” Journal of Construction Engineering and Management, 144(3): 1–10.CrossRefGoogle Scholar
Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.Google Scholar
Kahneman, Daniel, and Tversky, Amos. 1979. “Intuitive Prediction: Biases and Corrective Procedures.” In Makridakis, S. and Wheelwright, S. C. (Eds.) Studies in the Management Sciences: Forecasting (Vol. 12): 313–327. Amsterdam: North Holland.Google Scholar
Kim, Byung-Cheol, and Reinschmidt, Kenneth F.. 2011. “Combination of Project Cost Forecasts in Earned Value Management.” Journal of Construction Engineering and Management, 137(11): 958–966.CrossRefGoogle Scholar
Leavitt, Dan, Ennis, Sean, and McGovern, Pat. 1993. “The Cost Escalation of Rail Projects: Using Previous Experience to Re-evaluate the CalSpeed Estimates.” Working Paper 567. Berkeley, CA: Institute of Urban and Regional Development, University of California.Google Scholar
LeBlanc, Richard D. 2020. Muskrat Falls: A Misguided Project (Vol. 1–6). St. John’s, Canada: Commission of Inquiry Respecting the Muskrat Falls Project, Province of Newfoundland and Labrador.Google Scholar
Lee, Jin-Kyung. 2008. “Cost Overrun and Cause in Korean Social Overhead Capital Projects: Roads, Rails, Airports, and Ports.” Journal of Urban Planning and Development, 134(2): 59–62.CrossRefGoogle Scholar
Liu, Li, and Napier, Zigrid. 2010. “The Accuracy of Risk‐Based Cost Estimation for Water Infrastructure Projects: Preliminary Evidence from Australian Projects.” Construction Management and Economics, 28(1): 89–100.CrossRefGoogle Scholar
Liu, Li, Wehbe, George, and Sisovic, Jonathan. 2010. “The Accuracy of Hybrid Estimating Approaches: A Case Study of an Australian State Road and Traffic Authority.” The Engineering Economist, 55: 225–245.CrossRefGoogle Scholar
Miller, Eric. 2013. “American Roads LLC Files for Bankruptcy.” Transport Topics, August 5, 2013.Google Scholar
National Audit Office, NAO. 1992. Department of Transport: Contracting for Roads. London: National Audit Office.Google Scholar
Nijkamp, Peter and Ubbels, Barry. 1999. “How Reliable are Estimates of Infrastructure Costs? A Comparative Analysis.” International Journal of Transport Economics, 26(1): 23–53.Google Scholar
Pickrell, Don. 1990. Urban Rail Transit Projects: Forecast Versus Actual Ridership and Cost. Washington, DC: US Department of Transportation.Google Scholar
Pickrell, Don. 1992. “A Desire Named Streetcar: Fantasy and Fact in Rail Transit Planning.” Journal of the American Planning Association, 58(2): 158–176.CrossRefGoogle Scholar
Plummer, M.. 2003. “JAGS: A Program for Analysis of Bayesian Graphical Models Using Gibbs Sampling.” Proceedings of the 3rd International Workshop on Distributed Statistical Computing. Google Scholar
Plummer, M. 2012. "rjags: Bayesian Graphical Models Using MCMC.” R package version 3-7.Google Scholar
R Core Team. 2012. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing.Google Scholar
Riksrevisionen. 2011. Kostnadskontroll i stora järnvägsinvesteringar? Swedish National Audit Bureau Audit, Report, RiR 2011:6. Stockholm: Riksrevisionen.Google Scholar
Riksrevisionsverket. 1994. Infrastrukturinvesteringar: En kostnadsjämförelse mellan plan och utfall i 15 större projekt inom Vägverket och Banverket. RRV 1994:23. Stockholm: Riksrevisionsverket.Google Scholar
Roberts, Terry. 2020. “Scathing Muskrat Falls Inquiry Report Lays Blame on Fired Executives.” CBC News, March 10. Available at https://www.cbc.ca/news/canada/newfoundland-labrador/muskrat-falls-inquiry-misguided-project-1.5492169?__vfz=medium%3Dsharebar (accessed March 11, 2020).Google Scholar
Rubin, Debra K. 2017. “Suit over Failed Brisbane P3 Focuses on Traffic Projections.” Engineering News-Record, October 18.Google Scholar
Saulwick, Jacob. 2014. “Trial to Start on $144 Million Lane Cove Tunnel Debacle.” Sydney Morning Herald, August 10.Google Scholar
Singh, Anil C. 2017. “Syncora Guar. Inc. v. Alinda Capital Partners LLC.” 2017 NY Slip Op 30288(U), Supreme Court, New York Country: Docket Number 651258/2012.Google Scholar
Slovic, Paul. 2000. The Perception of Risk. Sterling, VA: EarthScan.Google Scholar
Swiss Association of Road and Transportation Experts. 2006. Kosten-Nutzen-Analysen im Strassenverkehr, Grundnorm 641820, valid from August 1. Zürich: Swiss Association of Road and Transportation Experts.Google Scholar
UK Department for Transport. 2006. The Estimation and Treatment of Scheme Costs: Transport Analysis Guidance, TAG Unit 3.5.9. London: Department for Transport.Google Scholar
UK Department for Transport and Oxford Global Projects, 2020, Updating the Evidence Behind the Optimism Bias Uplifts for Transport Appraisals: 2020 Data Update to the 2004 Guidance Document “Procedures for Dealing with Optimism Bias in Transport Planning” (London: UK Department for Transport).Google Scholar
Vickerman, Roger. 2017. “Wider Impacts of Megaprojects: Curse or Cure?” In Flyvbjerg, Bent (Ed.) The Oxford Handbook of Megaproject Management: 389–405. Oxford: Oxford University Press.Google Scholar
Wachs, Martin. 1986. “Technique vs. Advocacy in Forecasting: A Study of Rail Rapid Transit.” Urban Resources, 4(1): 23–30.Google Scholar
Wachs, Martin. 1989. “When Planners Lie with Numbers.” Journal of the American Planning Association, 55(4): 476–479.Google Scholar
Wachs, Martin. 1990. “Ethics and Advocacy in Forecasting for Public Policy.” Business and Professional Ethics Journal, 9(1 and 2): 141–157.CrossRefGoogle Scholar
Wachs, Martin. 2013. “The Past, Present, and Future of Professional Ethics in Planning.” In Carmon, Naomi, and Fainstein, Susan S. (Eds.) Policy, Planning, and People: Promoting Justice in Urban Development: 101–119. Philadelphia, PA: University of Pennsylvania Press.CrossRefGoogle Scholar
Walmsley, D. A. and Pickett, M. W.. 1992. The Cost and Patronage of Rapid Transit Systems Compared With Forecasts. Research Report 352. Crowthorne: Transport Research Laboratory.Google Scholar
Ward, William A. 2019. “Cost-Benefit Analysis: Theory versus Practice at the World Bank 1960 to 2015.” Journal of Benefit-Cost Analysis, 10(1): 124–144.CrossRefGoogle Scholar
World Bank. 1994. World Development Report 1994: Infrastructure for Development. Oxford: Oxford University Press.Google Scholar
World Bank. 2010. Cost-Benefit Analysis in World Bank Projects. Washington, DC: The World Bank.Google Scholar
Worthington, Elise. 2012. “Clem7 Tunnel Investors Launch $150m Class Action.” ABC News, June 1.Google Scholar
Wright, Robert. 2014. “US Infrastructure Crisis Drives Toll Road Bankruptcy.” Financial Times, September 21, 2014.Google Scholar