Skip to main content Accessibility help

Differential and Distributional Effects of Energy Efficiency Surveys: Evidence from Electricity Consumption

  • Thomas J. Kniesner (a1) (a2) (a3) and Galib Rustamov (a4)


Our research investigates the effects of residential energy efficiency audit programs on subsequent household electricity consumption. Here there is a one-time interaction between households, which participate voluntarily, and the surveyors. Our research objective is to determine whether and to what extent the surveys lead to behavioral changes. We then examine how persistent the intervention is over time and whether the effects decay or intensify. The main evaluation problem here is survey participants’ self-selection, which we address econometrically via several non-parametric estimators involving kernel-based propensity-score matching. In the first method we use difference-in-differences (DID) estimation. Our second estimator is quantile DID, which produces estimates on distributions. The comparison group consists of households who were not yet participating in the survey but participated later. Our evidence is that the customers who participated in the survey reduced their electricity consumption by about 7%, on average compared to customers who had not yet participated in the survey. Considering the total number of high-usage households participating in the survey in 2009, we estimate that electricity consumption was reduced by an aggregate of 2 million kWh per year, which is approximately equal to the monthly consumption of 3500 typical households in California with an estimated 1527 metric tons less of carbon dioxide emissions. Because the energy audit program is inexpensive ($10–$20 per household) a key issue is that while the program is cost-effective, is it regressive? We find that as the quantiles of the outcome distribution increase, high-use households save proportionally less electricity than do low-use customers. Overall, our results imply that program designers can better target low-use and low-income households, because they are more likely to benefit from the programs through energy savings.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Differential and Distributional Effects of Energy Efficiency Surveys: Evidence from Electricity Consumption
      Available formats

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Differential and Distributional Effects of Energy Efficiency Surveys: Evidence from Electricity Consumption
      Available formats

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Differential and Distributional Effects of Energy Efficiency Surveys: Evidence from Electricity Consumption
      Available formats


This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (, which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.

Corresponding author


Hide All

We thank Hal Nelson, C. Monica Capra, Joshua Tasoff, Quinn Keefer, Shahana Samiullah, W. Kip Viscusi, Richard Zeckhauser, V. Kerry Smith and participants of the 2017 Annual Meeting of the Society for Benefit Cost Analysis for their helpful comments.



Hide All
Abrahamse, Wokje, Steg, Linda, Vlek, Charles & Rothengatter, Talib (2005). A Review of Intervention Studies Aimed at Household Energy Conservation. Journal of Environmental Psychology, 25(3), 273291.
Accenture(2012). Actionable Insights for the New Energy Consumer. Accenture End-Consumer Observatory 2012.
Akerlof, George A. (1978). The Economics of ‘Tagging’ As Applied to the Optimal Income Tax, Welfare Programs, and Manpower Planning. American Economic Review, 68(1), 819.
Allcott, Hunt (2011). Social Norms and Energy Conservation. Journal of Public Economics, 95(9–10), 10821095.
Allcott, Hunt (2016). Paternalism and Energy Efficiency: An Overview. Annual Review of Economics, 8, 145176.
Allcott, Hunt & Kessler, Judd (2018). The Welfare Effects of Nudges: A Case Study of Energy Use Social Comparisons. American Economic Journal: Applied Economics, forthcoming.
Allcott, Hunt, Knittel, Christopher & Taubinsky, Dmitry (2015). Tagging and Targeting of Energy Efficiency Subsidies. American Economic Review: Papers and Proceedings, 105(5), 187191.
Allcott, Hunt & Mullainathan, Sendhil (2010). Behavioral Science and Energy Policy. Science, 327(5970), 12041205.
Allcott, Hunt & Rogers, Todd (2014). The Short-Run and Long-Run Effects of Behavioral Interventions: Experimental Evidence from Energy Conservation. American Economic Review, 104(10), 30033037.
Allcott, Hunt & Taubinsky, Dmitry (2015). Evaluating Behaviorally-Motivated Policies: Experimental Evidence from the Light Bulb Market. American Economic Review, 105(8), 25012538.
Angrist, Joshua D. & Pischke, Jörn-Steffen (2009). Mostly Harmless Econometrics. Princeton, NJ: Princeton University Press.
Athey, Susan & Imbens, Guido W. (2006). Identification and Inference in Nonlinear Difference-in-Difference Models. Econometrica, 74(2), 431497.
Austin, Peter C. (2009). Balance Diagnostics for Comparing the Distribution of Baseline Covariates Between Treatment Groups in Propensity-Score Matched Samples. Statistics in Medicine, 28, 30833107.
Baser, Onur (2006). Too Much Ado about Propensity Score Models? Comparing Methods of Propensity Score Matching. Value in Health, 9(6), 377385.
Black, Nick (1996). Why We Need Observational Studies to Evaluate the Effectiveness of Health Care. BMJ, 312(7040), 12151218. May 11.
Black, Dan A. & Smith, Jeffrey A. (2004). How Robust is the Evidence on the Effects of College Quality. Journal of Econometrics, 121, 99124.
Blundell, Richard & Costa Dias, Monica (2009). Alternative Approaches to Evaluation in Empirical Microeconomics. Journal of Human Resources, 44(3), 565640.
Borenstein, Severin (2017). The Private Net Benefits of Residential Solar PV: The Role of Electricity Tariffs, Tax Incentives and Rebates. Journal of the Association of Environmental and Resource Economists, 4(S1, Part 2), S85S122.
Brennan, Timothy J. (2010). Decoupling in Electric Utilities. Journal of Regulatory Economics, 38(1), 4969.
Caliendo, Marco & Kopeinig, Sabine(2005). Some Practical Guidance for the Implementation of Propensity Score Matching. IZA DP No. 1588.
Cameron, Colin A. & Trivedi, Pravin K. (2010). Microeconometrics Using Stat. Rev. ed. College Station, TX: Stata Press.
Charness, Gary & Gneezy, Uri (2009). Incentives to Exercise. Econometrica, 77(3), 909931.
Chen, Caroline, Rustamov, Galib, Hirsch, Kelly, Lau, Kenneth, Buendia, Jose & Ayuyao, Eugene (2015). 10 (Electricity)-10(Gas)-10(Water) Plus Multi-family Competition and Energy Star Portfolio Manager Benchmarking Pilot Program Design. In ACEEE Summer Study on Energy Efficiency in Industry.
Considine, Timothy & Manderson, Edward (2014). The Role of Energy Conservation and Natural Gas Prices in the Costs of Achieving California’s Renewable Energy Goals. Energy Economics, 44, 291301.
Davis, Lucas W., Fuchs, Alan & Gertler, Paul (2014). Cash for Coolers: Evaluating a Large-Scale Appliance Replacement Program in Mexico. American Economic Journal: Economic Policy, 6(4), 207238.
Deaton, Angus (2000). In The Analysis of Household Surveys: A Microeconomics Approach to Development Policy. (3rd ed.). Baltimore: The John Hopkins University Press; Published for the World Bank.
Dehejia, Rajeev H. & Wahba, Sadek (2002). Propensity Score-Matching Methods for Nonexperimental Causal Studies. The Review of Economics and Statistics, 84(1), 151161.
Delmas, Magali A., Fischlein, Miriam & Asensio, Omar I. (2013). Information Strategies and Energy Conservation Behavior: A Meta-Analysis of Experimental Studies from 1975 to 2012. Energy Policy, 61, 729739.
Du, Yingjuan, Hanna, Dave, Shelton, Jean & Buege, Amy (2014). What Behaviors Do Behavior Programs Change. In ACEEE Summer Study on Energy Efficiency in Buildings.
Dubin, Jeffrey A., Miedema, Allen K. & Chandran, Ram V. (1986). Price Effects of Energy-Efficient Technologies: A Study of Residential Demand for Heating and Cooling. RAND Journal of Economics, 17(3), 310325.
ECONorthwest(2009). Process Evaluation of the SCE 2006-08 Home Energy Efficiency Survey (HEES) Program: Final Report. Study ID: SCE0275.01.
Fowlie, Meredith, Greenstone, Michael & Wolfram, Catherine (2015). Are the Non-Monetary Costs of Energy Efficiency Investments Large? Understanding Low Take-up of a Free Energy Efficiency Program. American Economic Review: Papers and Proceedings, 105(5), 201204.
Frölich, Markus & Melly, Blaise (2010). Estimation of Quantile Treatment effects with Stata. The Stata Journal, 10(3), 423457.
Fu, Alex Z., Dow, William H. & Liu, Gordon G. (2007). Propensity Score and Difference-in-Difference Methods: A Study of Second-Generation Antidepressant Use in Patients with Bipolar Disorder. Health Service and Outcomes Research Methodology, 7, 2338.
Fullerton, Don(2008). Distributional Effects of Environmental and Energy Policy: An Introduction. NBER Working Paper 14241.
Gayer, Ted & Viscusi, W. Kip (2013). Overriding Consumer Preferences with Energy Regulations. Journal of Regulatory Economics, 43(3), 248264.
Gerarden, Todd D., Newell, Richard G. & Stavins, Robert N. (2017). Assessing the Energy-Efficiency Gap. Journal of Economic Literature, 55(4), 14861525.
Gillingham, Kenneth, Newell, Richard & Palmer, Karen (2006). Energy Efficiency Policies: A Retrospective Examination. Annual Review of Environment and Resources, 31, 161192.
Gillingham, Kenneth, Newell, Richard & Palmer, Karen (2009). Energy Efficiency Economics and Policy. Annual Review of Resource Economics, 1, 597620.
Greenstone, Michael, Kopits, Elizabeth & Wolverton, Ann (2013). Developing a Social Cost of Carbon for U.S. Regulatory Analysis: A Methodology and Interpretation. Review of Environmental Economics and Policy, 7(1), 2346.
Hamermesh, Daniel S. & Trejo, Stephen S. (2000). The Demand for Hours of Labor: Direct Evidence from California. The Review of Economics and Statistics, 82(1), 3847.
Hansen, Ben B. (2004). Full Matching in an Observational Study of Coaching for the SAT. Journal of the American Statistical Association, 99(467), 609618.
Harding, M. & Hsiaw, A. (2014). Goal Setting and Energy Conservation. Journal of Economic Behavior and Organization, 107, 209227.
Heckman, James, Ichimura, Hidehiko, Smith, Jeffrey & Todd, Petra (1998). Characterizing Selection Bias Using Experimental Data. Econometrica, 66(5), 10171098.
Heckman, James, LaLonde, Robert & Smith, Jeffrey (1999). The Economics and Econometrics of Active Labor Market Programs. In Ashenfelter, Orley & Card, David (Eds.), Handbook of Labor Economics Vol. III (pp. 18652097). Amsterdam: Elsevier.
Imbens, Guido W. & Rubin, Donald B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. New York, NY: Cambridge University Press.
Ingle, Aaron, Moezzi, Mithra M., Lutzenhiser, Loren, Hathaway, Zac, Lutzenhiser, Susan, Joe, Van Clock, Peters, Jane S., Smith, Rebecca, Heslam, David & Diamond, Richard(2012). “Behavioral Perspectives on Home Energy Audits: The Role of Auditors, Labels, Reports, and Audit Tools on Homeowner Decision-Making”. Lawrence Berkeley National Laboratory, LBNL-5715E.
Itron, Inc.(2013). 2010–2012 CPUC HEES Impact Evaluation: Final Report. Prepared for California Public Utilities Commission.
Keefer, Quinn & Rustamov, Galib (2017). Limited Attention in Residential Energy Markets: A Regression Discontinuity Approach. Empirical Economics,
Kennedy, Peter (2003). A Guide to Econometrics. (5th ed.). Cambridge, MA: The MIT Press.
Lechner, Michael (2002). Some Practical Issues in the evaluation of heterogeneous Labour Market Programmes by Matching Methods. Journal of the Royal Statistical Society, 165, 5982.
Levinson, Arik(2016). Energy Efficiency Standards Are More Regressive Than Energy Taxes: Theory and Evidence. NBER Working Paper 22956.
List, John A., Millimet, Daniel L., Fredriksson, Per G. & McHone, Warren W. (2003). Effects of Environmental Regulations on Manufacturing Plant Births: Evidence from a Propensity Score Matching Estimator. The Review of Economics and Statistics, 85(4), 944952.
Meyer, Bruce D. (1995). Natural and Quasi-Experiments in Economics. Journal of Business and Economic Statistics, 13(2), 151161.
Meyer, Bruce D., Viscusi, W. Kip & Durbin, David L. (1995). ‘Workers’ Compensation and Injury Duration: Evidence from a Natural Experiment. The American Economic Review, 85(3), 322340.
Nadel, Steve & Keating, Kenneth (1991). Engineering Estimates vs. Impact Evaluation Results: How Do They Compare and Why? Research Report U915. Washington, D.C.: American Council for an Energy-Efficient Economy.
Newell, Richard & Siikamäki, Juah (2014). Nudging Energy Efficiency Behavior: The Role of Information Labels. Journal of the Association of Environmental and Resource Economists, 1(4), 3338.
Poi, Brian P. (2004). From the Help Desk: Some Bootstrapping Techniques. Stata Journal, 4, 321328.
Rosenbaum, Paul R. & Rubin, Donald B. (1983). The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrica, 70(1), 4155.
Rosenbaum, P. & Rubin, D. (1985). Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score. American Statistician, 39, 3338.
Rubin, Donald B. & Thomas, Neal (1992). Affinely Invariant Matching Methods with Ellipsoidal Distributions. The Annals of Statistics, 20(2), 10791093.
Sallee, James M. (2014). Rational Inattention and Energy Efficiency. Journal of Law and Economics, 57(3), 781820.
Shin, Jeong-Shik (1985). Perception of Price When Price Information Is Costly: Evidence from Residential Electricity Demand. The Review of Economics and Statistics, 67(4), 591598.
Sianesi, Barbara (2004). An Evaluation of the Swedish System of Active Labour Market Programs in the 1990s. The Review of Economics and Statistics, 86(1), 133155.
Sianesi, Barbara (2008). Differential Affects of Active Labour Market Programs for the Unemployed. Labour Economics, 15, 370399.
Smith, Herbert L. (1997). Matching with Multiple Controls to Estimate Treatment Effects in Observational Studies. Sociological Methodology, 27, 325353.
Smith, J. A. & Todd, P. E. (2001). Reconciling Conflicting Evidence on the Performance of Propensity Score Matching Methods. American Economic Review, Papers and Proceedings, 91, 112118.
The Cadmus Group, Inc.(2017). Evaluation of Southern California Edison 2015 Home Energy Efficiency Survey Programs. Draft Report. California Public Utilities Commission.
Titus, Marvin A. (2007). Detecting Selection Bias, Using Propensity Score Matching, and Estimating Treatment Effects: An Application to the Private Returns to a Master’s Degree. Research in Higher Education, 48, 487521.
U.S. Energy Information Administration(2011). Electric Power Monthly, January 2011.
U.S. Environmental Protection Agency(2015). Greenhouse Gas Equivalencies Calculator. Retrieved from on January 15 from, last updated on October 23.
Wirl, Franz & Orasch, Wolfgang (1998). Analysis of United States’ Utility Conservation Programs. Review of Industrial Organization, 13(4), 467486.



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed