Skip to main content Accessibility help
×
Home

Information:

  • Access

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        How Effective Are Federally Mandated Information Disclosures?
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        How Effective Are Federally Mandated Information Disclosures?
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        How Effective Are Federally Mandated Information Disclosures?
        Available formats
        ×
Export citation

Abstract

Government mandates to disclose information are a standard response to problems of asymmetric information. We examine recent major U.S. regulations issued between 2008 and 2013 to identify disclosure mandates and look for quantitative assessments of their effectiveness in improving comprehension. We also reviewed the retrospective regulatory review reports of four federal agencies – prepared pursuant to Executive Orders 13563 and 13579 – for analyses of existing mandates. We find that although mandated disclosures underpin a number of major federal regulatory initiatives, agencies infrequently issue such mandates based on scientifically valid, controlled studies of the improvements in comprehension from such disclosure. We also conduct four case studies drawn from the relatively few instances where federal agencies have evaluated improvements in comprehension, to identify current best practices. Finally, we recommend reforms to improve federally mandated information disclosure.

1 Introduction

Mandates to disclose information have become a standard government response to problems of asymmetric information in most industrialized countries. U.S. federal agencies, like those in other industrialized democracies, use this regulatory strategy in areas ranging from automobile fuel efficiency labeling to food labeling, mortgage lending, and consumer finance. Requirements that private parties divulge certain information prior to specific transactions are intended to ensure that potentially uninformed parties, such as consumers or individual investors, are adequately informed. 1

Executive Order 13563 issued by President Obama emphasized the importance of considering information disclosure strategies as an alternative or a complement to prescriptive regulatory approaches. The U.S. Office of Information and Regulatory Affairs (OIRA), has also highlighted information disclosure as a federal regulatory strategy, placing special emphasis on testing such approaches before they are disseminated, as well as ongoing monitoring of their effectiveness. 2 In doing so, it has called for scientifically valid, randomized experiments to test effects of alternative methods of disclosure before imposing a specific design (OIRA, 2010a , 2012).

In this paper, we explore the extent to which federal regulators are adopting analytic methods to evaluate the effectiveness in increasing comprehension of mandated information disclosure, in response to this Administration’s emphasis on information disclosure. 3 Such methods can be seen as consistent with the benefit-cost analyses required by E.O. 13563 and its predecessors such as E.O. 12866. 4 These E.O.s and the related A-4 guidance (Office of Management and Budget, 2003) from the federal Office of Management and Budget focus on regulatory action to address market failures and, in particular, they recommend the use of mandated information disclosure as an appropriate response to information asymmetries. Thus, we accept that the purpose of mandated information disclosure is to correct an information deficiency – and not to alter consumer behavior. 5

We focus here on comprehension as a measure of effectiveness because it would seem to be a necessary first step for claims that consumers are better off from the information disclosure. In general, if the market failure is asymmetric information and the purported solution involves information disclosure then improved comprehension of that information should be seen as necessary for a proposed disclosure to be an effective remedy to the problem. Put differently, claims of gains from mandated information disclosure without improvements in comprehension should be seen as questionable. Of course, assessment of comprehension entails additional costs that should be considered in decisions associated with the evaluation of mandated information disclosures.

Although federal mandates to disclose information underpin a number of flagship regulatory initiatives, we have found only a very few exceptional cases where there is any evidence that the responsible regulatory agencies conducted a quantitative evaluation of the effects on comprehension or comparable measures of effectiveness. The absence of information on effectiveness includes the failure to conduct both prospective analyses of new requirements and retrospective analyses of information disclosures that have already been mandated. As a result whether federally mandated information disclosures are generally effective at ameliorating the problem of asymmetric information is, at this point based on a very limited set of anecdotes.

Our finding that very little is known about the effectiveness of federally mandated information disclosures on improving comprehension may be surprising. While it is costly and time consuming to acquire data appropriate to evaluate the costs and benefits of most federal regulations, data sufficient to make inferences about the effects of information disclosures on comprehension can generally be obtained relatively quickly and at relatively low cost. Further, studies of comprehension have been conducted by federal agencies – albeit in a limited number of cases – and should be viewed as a practical first step for agencies’ mandatory information disclosures given current budget constraints.

Our analysis leads us to several recommendations to foster additional quantitative analysis of the effectiveness of federally mandated information disclosures. We suggest that major federal rulemakings mandating information disclosure be accompanied by a rigorous evaluation of its effectiveness at improving comprehension. We recommend that OIRA revisit and consolidate directives on information disclosure, including clarifying best practices and streamlining OIRA review, under the Paperwork Reduction Act, of scientific studies of effectiveness. We also recommend that Congress increase appropriations for FTC’s consumer research, given its leadership in this area.

The rest of this paper presents background information on the estimation of the effects of federally mandated information disclosure, methods and data, a systematic analysis of economically significant federal rules issued over six years, selected case studies, and a discussion and conclusions.

2 Background

Economists’ recognition that mandatory information disclosure may be ineffective in changing beliefs or understanding is nearly 35 years old, e.g., Beales, Craswell and Salop (1981), even if this notion is not prominent in the recent literature. 6 More broadly, difficulties in understanding information, especially information pertaining to risk, may be seen as manifestations of the heuristics and biases in judgments under uncertainty identified by Tversky and Kahneman (1974). They documented systematic errors in thinking of normal people and traced those errors to the machinery of the cognition process (Kahneman, 2011).

The rise of mandatory information disclosure as government policy has been accompanied by substantial scholarship about such disclosure and its effects. Weil, Fung, Graham and Fagotto (2006) and Fung, Graham and Weil (2007) provide a qualitative appraisal of circumstances where mandatory disclosure of information is effective, assuming a broad definition of effectiveness. They argue that disclosure is effective only if the disclosed information becomes “embedded” in the everyday decision-making routines of both information users and disclosers. Weil et al. (2006) note “Even if valuable and compatible with users’ routines, [newly disclosed] information is unlikely to become embedded in everyday choices unless it can be readily understood.”

In a review of Fung et al. (2007), Winston (2008) reviews existing empirical studies to determine whether the information policies identified by Fung et al. have significantly reduced the costs to consumers created by imperfect information. On the basis of this review, Winston concludes that government mandated information policies have “amounted to weak solutions in search of a problem.” He suggests that more empirical evidence about the role of laws and regulations mandating disclosure is needed. More recently, Ben-Shahar and Schneider (2014) argue that mandatory information disclosure has become ineffective and represents a failed approach to solve a wide variety of policy problems.

Several recent papers have focused on specific disclosure policies, including, for example, building energy performance (Hsu, 2014), carbon emissions (Matisoff, 2013), chemical releases (Finger & Shanti, 2013), lead-based paint (Bae, 2012), and electric power emissions (Delmas, Montes-Sancho & Shimshack, 2010). These papers do not, however, assess the effectiveness of federal disclosure policies in remedying information asymmetries relevant to buyers or sellers engaging in specific transactions.

There is little systematic research on the reasons why information disclosures fail. We examine whether regulators adequately evaluate the extent to which mandatory disclosures successfully convey the required information to consumers. We offer a focused assessment of mandated information disclosures as the standard government response to information asymmetries. We concentrate on comprehension or understanding – a specific measure of effectiveness – because it seems to underpin the action cycle of Weil et al. (2006), it is uniquely under regulators’ control, and recent OMB documents highlight its policy importance. Our approach may be seen as complementary to that of other researchers who have questioned the effectiveness of current disclosure policies (e.g., Ben-Shahar & Schneider, 2014; Guthrie, 2006). It goes beyond researchers who have focused in one area, (e.g., Willis (2006), regarding mortgage lending), by suggesting a remedy that is fairly broadly applicable: agencies should conduct prospective and retrospective testing of mandated disclosure instruments to determine their effectiveness.

Assessments of how well mandatory information disclosures improve consumers’ comprehension of the context and consequences of their decisions can provide information that helps inform regulatory policy decisions. Regulators often can choose between either a ban on products that have adverse consequences, or a mandate for the disclosure of relevant information. Product bans may reduce risk but at the cost of consumer choice. Mandatory information disclosure may promote informed choice, but at the risk of potential misunderstanding of this information by consumers. 7 In addition, mandatory disclosure will change behavior to address public policy goals in the desired way only to the extent that market participants respond to the disclosure as desired. Where regulators have adopted mandatory disclosures, assessments of the effectiveness of alternative forms of disclosure can help to improve comprehension of key product characteristics.

OIRA has also made recommendations regarding the potential role of information disclosure as a regulatory tool in recent guidance and in its reports to Congress. These recommendations recognize that unduly complex disclosure requirements might fail to inform consumers, a notion consistent with a longstanding awareness of potential difficulties in effectively communicating information (e.g., Beales et al., 1981). As a result, OIRA (2009) urges that agencies study in advance – to the extent possible – the actual effects of alternative disclosure designs.

OMB’s recent guidance outlined some of the elements appropriate to an evaluation of “significant” information disclosures (OIRA, 2010b ). 8 In these cases, OIRA stated that agencies should test several methods of disclosure before imposing one, use scientifically valid and randomized experiments, and elicit information about actual choices and behavior.

The OIRA (2010a ) recommendation to use “scientifically valid” and “randomized” experiments is more specific than earlier OMB advice. But the recommendation does not identify a preferred methodological approach or identify a specific example of best practices.

An FTC study demonstrates the importance of careful evaluation in determining the effectiveness of mandatory information disclosures (Lacko & Pappalardo, 2007, 2010). Consistent with the OIRA guidance (OIRA, 2010c , 2012) and following Lacko and Pappalardo (2004), Lacko and Pappalardo (2010) focused on quantitative testing that is scientifically valid because it uses random assignment and controls (although it is not blinded). Lacko and Pappalardo conducted a qualitative study prior to the quantitative testing. In addition, Lacko and Pappalardo (2010) focused on understanding or comprehension by the relevant population (i.e., recent home mortgage borrowers) – an approach suitable for measuring “actual effects,” following Executive Order (E.O.) 13563. The study surveyed more than 800 recent mortgage borrowers to examine how well borrowers in 2007 understood the terms and conditions of their mortgages, as described on mandatory disclosure forms. It evaluated consumers’ comprehension not only in absolute terms but also relative to comprehension assessed using an alternative information disclosure form that the authors had designed themselves. Lacko and Pappalardo (2007) reported that their prototype form significantly improved consumers’ understanding:

“The prototype disclosures provided improvements across a wide range of loan terms and for substantial proportions of respondents. The improvements provided by the prototype form included:

  • 66% point increase in the proportion of respondents correctly identifying the total amount of up-front charges in the loan.

  • 43% point increase in the proportion of respondents recognizing that the loan contained charges for optional credit insurance.

  • 37% point increase in the proportion correctly identifying the amount borrowed.

  • 24% point increase in the proportion recognizing that a prepayment penalty would be assessed if the loan was refinanced in two years.

  • 21% point increase in the proportion correctly identifying why the APR and interest rate may differ in a loan.

  • 16% point increase in the proportion correctly identifying the APR amount.

  • 15% point increase in the proportion correctly identifying the amount of settlement charges.” (Lacko & Pappalardo, 2007, p. ES-8)

Lacko and Pappalardo (2010) demonstrated avoidable, major, and statistically significant inadequacies in consumer comprehension of pertinent mortgage information resulting from the use of federally mandated disclosure forms. These inadequacies existed even in a case like residential mortgages where the individual and aggregate stakes are high and federal involvement is both extensive and well established. Lacko and Pappalardo concluded that in this case the mandatory disclosures confused consumers, leading many to choose loans that were more expensive than the available alternatives. They showed that careful quantitative assessment of comprehension of disclosed information may be necessary for revising information disclosure forms to achieve significantly better performance. Implicitly, they viewed the time and paperwork costs of disclosing the information and documenting the disclosure as small relative to the gains to borrowers from being much better informed about the terms of their mortgage. More recently, Perry and Blumenthal (2012) also argue that better-designed research could lead to significant improvements in public policy and consumer protection and result in a more efficient allocation of resources. Thus, federally mandated information disclosure, if not tested in rigorous randomized experiments, may not work as intended and may in fact leave many people uninformed, if not misled.

3 Methods and data

We adopt a three-pronged approach, consisting of an automated text search of the universe of major rules issued over six years, an analysis of reports of retrospective regulatory reviews conducted by four major regulatory agencies, and selected case studies. Our systematic review tries to identify all analyses by federal regulatory agencies of the effectiveness of mandatory information disclosures in major final regulations issued from fiscal year 2008 through fiscal year 2013 (FY13). In total, we identified 357 major final rules, including interim final rules. For all of these rules, we conducted an automated search using Adobe Acrobat to identify the use of selected key words related to assessments of the effectiveness of mandatory information disclosure. We believe that this search procedure provides a sound basis for making rigorous statements about the content of the final regulations in question.

Second, we evaluated the retrospective regulatory review reports that agencies are required to prepare and publish to comply with E.O. 13610, Identifying and Reducing Regulatory Burdens, issued in July 2012. We focus on diverse U.S. regulatory agencies with large regulatory programs: the Environmental Protection Agency (EPA), the Department of Energy’s Office of Energy Efficiency and Renewable Energy, and the Food and Drug Administration and the Centers for Medicare and Medicaid Services, both of the Department of Health and Human Services.

In addition, we considered a few prominent regulatory initiatives for which information disclosure was a key regulatory strategy. For these initiatives we reviewed the relevant final rules and assessed the analysis the agency conducted to evaluate the effectiveness of mandatory information disclosure. This in-depth appraisal of a few recent regulations allows us to show how information disclosure regulation might be made more effective at relatively low incremental cost.

We turn below to each prong of our analysis.

4 Systematic analysis

To carry out our systematic review, we collected electronic copies of all Federal Register Notices for all final and interim final rules that OIRA identified as major in its annual reports to Congress for FY08 to FY12, e.g., OIRA (2010c ). 9 We therefore instead compiled a list of final and interim final regulations that are economically significant by using the search tool on the “Historical Reports” page of OIRA’s website (http://www.reginfo.gov/public/do/eoHistoricReport), making adjustments for the differences between the date that OIRA concluded its review and the date of publication in the Federal Register. We have not included major or economically significant final rules issued by independent regulatory agencies in FY13, as we are unaware of a centralized listing of such rules. For the 357 major rules identified through this search, executive branch agencies issued 278, and independent regulatory agencies, such as the Securities and Exchange Commission and the Consumer Financial Protection Bureau (CFPB), issued 79 (22%).

We then searched using Adobe Acrobat for keywords that might permit us to characterize the subset of these major rules that mandate information disclosure. We found 217 with the word “disclosure,” 141 with the word “label” and 106 with the words or word fragments “disclos,” which could be disclosure or disclose, “label,” or “information collection,” a term of art in implementing the Paperwork Reduction Act. Fifty seven of the 357 major final rules contained the phrase “must disclose.” We note, however, that these occurrences of key words may not indicate a mandated information disclosure. For example, a search for “label” may turn up rules that do not mandate information disclosure, if “label” appears in the preamble to describe a rule other than the new rule being issued. Or the search results may exclude some rules that mandate disclosure by using synonymous language such as “regulated firms shall be required to add the following information to their stickers” instead of “label.” The search results might therefore overestimate or underestimate the number of new rules on labeling.

We then extended our search to look for rules that included the keywords “assess” and “comprehension.” 10 We identified 16 final rules that appear to be the most likely to have been prepared using quantitative assessments of the effects of mandatory information disclosure on the target audience. [Appendix A provides a list of the 16 rules.] A more detailed review indicated that of these 16 rules, five mandated disclosure of product information that would be helpful to consumers or investors. In addition, one rule issued by the Occupational Health and Safety Administration (OSHA) mandated disclosure of information pertinent to workers’ safety. A second OSHA rule addressed communication of health hazards in the workplace and pointed to a pre-2000 literature that examined the effectiveness of hazard communication. But for this rule OSHA did not conduct any new studies to assess comprehension by workers or consumers of the mandated disclosure relative to alternative forms of disclosure. Only one of these rules, issued by the Board of Governors of the Federal Reserve to regulate consumer finance, contained a quantitative assessment of comprehension.

5 Analysis of the retrospective regulatory review reports of selected federal agencies

President Obama’s E.O. 13,563 directs executive branch agencies to conduct retrospective review and analysis of extant federal regulations. We focus on the most recent retrospective regulatory review reports of four federal agencies: EPA, Department of Energy’s Office of Energy Efficiency and Renewable Energy (DOE/EE), and Department of Health and Human Services’ Centers for Medicare and Medicaid Services (HHS/CMS) and Food and Drug Administration (HHS/FDA). 11 We selected these agencies because their regulatory programs are quite different, they are widely known to have extensive regulatory programs, and they are all executive branch agencies subject to OMB oversight.

In studying the retrospective reports of these four executive agencies – EPA (2013), DOE/EE (2013), HHS/CMS (2013), and HHS/FDA (2013) – we looked for rules with information disclosure requirements for which retrospective review was either ongoing or completed. We then reviewed both the “brief description” included in the reports to describe the nature of the disclosure, the nature of the retrospective review that the agency had undertaken, and whether the agency had undertaken a quantitative assessment of third-party comprehension for any required information disclosure. The retrospective regulatory review reports do not indicate that the agencies conducted any quantitative assessments of comprehension or comparable measures of effectiveness as part of their retrospective regulatory reviews. Further, none of these agencies included a review of an existing information disclosure requirement based on a quantitative analysis of the effectiveness of such a requirement.

6 Case studies

We consider several case studies, which we selected (1) to illustrate the promise of informational remedies to classic problems of informational asymmetry in the markets for goods and services; and (2) to demonstrate the importance of retrospective analyses of the effectiveness of information disclosure. All but one of these case studies fall outside the scope of our systematic search (discussed above) because they either were not designated as major rules or involve rulemakings outside the time period of our systematic search.

The four case studies are (1) EPA and the National Highway Traffic Safety Administration’s (NHTSA) fuel economy labeling rule for cars and light-duty trucks; (2) FTC’s revision of its consumer product labeling rules; (3) the Federal Reserve’s major final rule regarding disclosure of terms and conditions for use of revolving credit unsecured by a home, mostly consumer credit cards (Truth in Lending, 2009); and (4) the Consumer Finance Protection Bureau (CFPB, 2013c ) final rule governing residential mortgage disclosures.

1. EPA/NHTSA fuel economy labeling for cars and light-duty trucks

Recent reports to Congress from OIRA specifically cited a revision by EPA/NHTSA to fuel economy labels as an example of an effort to study the effectiveness of alternative fuel label designs (OIRA, 2009, p. 37, OIRA, 2011, p. 53).

EPA and NHTSA issued the joint 2011 final rule to implement the Energy Independence and Security Act of 2007. They identified it as the most significant overhaul of the federal government’s fuel economy label since its inception 30 years ago (Revisions and Additions to Motor Vehicle Fuel Economy Label, 2011). 12 The agencies sought to improve the label to give consumers more information about vehicle performance. They also believed the time was right to develop new labels for advanced-technology vehicles (Revisions and Additions to Motor Vehicle Fuel Economy Label, 2010).

The rule adopted a structured program to inform the development of the revised fuel labels, including the use of three focus groups, a daylong session with an expert panel (drawn from advertising and product development experts), and an Internet survey to evaluate reactions to three alternative label designs. The focus groups and the expert panel session were not designed to provide quantitative results; instead, they were intended to help the agencies understand consumers’ comprehension and preferences in making car-buying decisions (Revisions and Additions to Motor Vehicle Fuel Economy Label, 2011).

The online survey (conducted following the release of the proposed rule) was designed to determine whether any of the proposed label designs had serious flaws that would undermine its effectiveness, particularly with respect to consumers’ abilities to choose fuel-efficient vehicles (Revisions and Additions to Motor Vehicle Fuel Economy Label, 2011). Respondents (self-identified new-car buyers) were shown only one of the three proposed label designs for two different vehicles. 13 The respondents were asked a series of questions to assess their understanding of information on the label and to detect any differences in vehicle selection. Overall, the results showed only small differences in respondents’ understanding and in vehicle selection with each of the alternative designs and did not find any “fatal flaws” (Revisions and Additions to Motor Vehicle Fuel Economy Label, 2011, p. 39483).

There are some important issues with the EPA/NHTSA evaluation. First, the Internet survey was based on a convenience sample – it was not designed to be representative of any larger group of new-vehicle buyers, and the actual response rate was low, so the responses reflect the experiences only of those who completed the survey. 14

Second, respondents were asked to respond to one of three new alternative label designs. No comparison was made with the existing labels (adopted in 2006), and the Internet study did not have a well-designed “control” for comparison purposes. 15 As a result, it is not possible to determine whether the revised label for gasoline-fueled vehicles improves or degrades consumers’ comprehension relative to the existing 2006 label. This issue is of some importance because most consumers will continue to buy gasoline-fueled vehicles.

Finally, the study focused on consumers’ selection of electric and other advanced-technology vehicles. Most of the label pairs used in the survey involved a comparison of electric vehicles with various attributes; only a few pairs involved a comparison between a gasoline vehicle and an electric vehicle. There were no pairs comparing labels for two gasoline vehicles with different attributes. 16 In particular, the study did not address one of the key issues – the frequent misunderstanding by consumers of the miles-per-gallon (mpg) measure of fuel economy (OIRA, 2009).

This particular example highlights the tension between a general concern with consumers’ comprehension of information provided by a label and the agency’s interest in achieving a specific policy objective – in this case, promoting the sale of advanced-technology vehicles. 17

The agencies acknowledged the central importance of using the best available research to inform judgments about disclosure requirements, stating that they “will continue to consider such research in the future (including, where feasible and appropriate, randomized controlled trials)” (Revisions and Additions to Motor Vehicle Fuel Economy Label, 2011, p. 39483). However, there was no mention in the final rule of the retrospective review requirements of E.O. 13563 or of E.O. 13610, and the final rule did not discuss any plans for a prospective review (or retrospective review) of the effectiveness of the final fuel labels.

2. FTC revision of energy efficiency labels

The Energy Policy Act of 2005 required FTC to consider changes to its consumer product labeling programs to improve their effectiveness for assisting consumers in making purchasing decisions and improving energy efficiency of consumer appliances. In response to this mandate, FTC launched a rulemaking to review and amend its Appliance Labeling Rule. (The FTC completed this rulemaking in 2007 prior to the period covered by our systematic review.) To support the rulemaking, FTC held a workshop and conducted consumer research to evaluate the effectiveness of alternative label designs.

The consumer research study was based on a sample of roughly 4000 respondents drawn from an Internet panel of more than 4 million individuals. The respondents in the sample were recent or likely buyers of major appliances. The preamble acknowledges that the sample for this research was not nationally representative in the “classic sense,” but argued that the contractor had taken steps to minimize the differences between samples from its Internet panel and true probability samples. The FTC staff, therefore, believed that the sample fairly represented the population of appliance buyers (Appliance Labeling Rule Notice of Proposed Rulemaking (NPR) 2007).

The study focused on four label designs: the then-current Energy Guide label presenting information on energy use, a revised version of the then-current label, a label featuring operating costs, and a categorical “five-star” label reflecting the energy efficiency of the appliance relative to DOE minimum standards. The study provided a well-defined baseline by evaluating the three alternative label designs with the existing Energy Guide label. The study randomly assigned respondents to different label design groups. Each respondent was shown only a single label design and asked questions designed to determine its effectiveness in conveying information on operating costs and energy use and efficiency. Respondents were also asked questions about product quality to determine whether there were differences across the label designs in information conveyed on product quality.

The final rule requires that labels for major appliances provide operating cost as the primary disclosure; energy use data – formerly the primary disclosure – would appear as secondary information. The FTC reported,

The research suggests that the operating cost disclosure provides a clear, understandable tool to allow consumers to compare the energy performance of different models. The operating cost design not only performed well on objective tasks in the research but research participants identified the design as the most useful method for communicating energy information …” (Appliance Labeling Final Rule, 2007b , p. 49953)

Although FTC recognized the potential value of a categorical label design, the consumer research study found that buyers might misinterpret the “categorical” five-star label. Some respondents interpreted the stars as representing a broader quality rating, and some respondents also believed that it indicated that the product was a participant in EPA’s Energy Star program; in both cases the difference in the misinterpretation by respondents was statistically significant compared with the alternative label designs (Appliance Labeling Rule NPR, 2007). As a result, FTC did not adopt the categorical five-star label, even though it received strong support from several important consumer and constituent groups.

3. Consumer finance

In January 2009, the Federal Reserve Board of Governors (“the Board”) issued a Truth in Lending final rule, which is a major final rule requiring disclosure of terms and conditions for use of revolving credit unsecured by a home, mostly consumer credit cards. This final rule used both qualitative and quantitative consumer testing. We identified this rule as part of our systematic review. The rule stated, “The goal of quantitative testing was to measure consumers’ comprehension and the usability of the newly-developed disclosures relative to existing disclosures and formats” (Truth in Lending, 2009, p. 5248).

The quantitative consumer testing consisted of mall-intercept interviews of a total of 1022 participants in seven cities (Truth in Lending, 2009). In each interview the questioner showed the participant models of the summary table provided in direct-mail credit card solicitations and applications and the periodic statement (Truth in Lending, 2009). The questioner then asked a series of questions designed to assess the effectiveness of certain formatting and content requirements proposed by the Board or suggested by commenters (Truth in Lending, 2009). These questions led to 13 quantitative findings about the relative superiority of different types of disclosure (Truth in Lending, 2009). For example, the fourth finding was that study participants who saw forms that “listed the over-the-credit limit fee on the back of the page [as opposed to the front page] were statistically less likely to be able to identify this fee (40.6% vs. 68.7%, $p<0.01$ )” (MacroInternational, 2008, p. iii).

The Board then used the quantitative testing results in a variety of ways in its rulemaking. One example pertaining to payment allocation illustrates that quantitative testing can play a critical, even decisive, role. “Payment allocation” refers to the methods that banks use to allocate consumers’ payments among different balances associated with different interest rates all within the same consumer account. The testing associated with the final rule examined whether consumers’ understanding of payment allocation practices could be improved through disclosure (Truth in Lending, 2009). The results showed that disclosure, even of the relatively simple payment allocation practice of applying payments to lower interest balances before higher interest balances, improved understanding for only a very few consumers. The disclosure also confused some consumers who had understood payment allocation based on prior knowledge before reviewing the disclosure. Based on this result, quantitative testing led the Board to reject this specific part of its information disclosure, because of inadequate consumer comprehension, in favor of a regulatory approach.

A second example involved the effective annual percent rate (APR), a summary measure of the average cost of borrowing that reflects both interest and other finance charges, such as cash advance fees or balance transfer fees imposed for the billing cycle. Testing under the final rule overwhelmingly showed that few consumers understood the disclosure and that some consumers were less able to locate the interest rate applicable to cash advances when the effective APR also was disclosed on the periodic statement. Accordingly, the final rule also eliminated the requirement to disclose an effective APR for open-end (not home-secured) credit.

Because it was based on a convenience sample in shopping malls in seven cities, this analysis is not necessarily representative of borrowers and has unknown sampling error.

4. Residential mortgage lending

Decisions to buy and finance residential real estate are some of the most important transactions that American families make – a fact illustrated vividly by the human toll from the 2008–2009 crash in real estate prices. Thus, it is not surprising that regulatory agencies have conducted consumer testing of some of the federally mandated information disclosures. The CFPB, created by the Dodd–Frank Wall Street Reform and Consumer Protection Act of 2010 (“Dodd–Frank”), is well positioned to be a leader in the use of quantitative studies of consumer comprehension. Dodd–Frank’s primary charge to CFPB is “[e]nsuring that consumers have timely and understandable information to make responsible decisions about financial transactions” (CFPB, 2013a , p. 7).

CFPB took a significant step toward data-driven consumer protection in developing the final rule governing residential mortgage disclosures (CFPB, 2013c ). The CFPB completed this rulemaking in late 2013 after the period covered by our systematic review. In developing this rule, CFPB used extensive qualitative and quantitative testing of its new Loan Estimate Form and the Closing Disclosure Form, relative to alternative forms, including those already in use. Importantly, in its quantitative testing, the CFPB randomized the disclosure type and the loan type, and the “initial” loans that the respondents saw (CFPB, 2013b and Kleimann Communication Group, 2013). In addition, CFPB studied subjects recruited from nationally representative random samples. Further, the study design compared comprehension of the new and old forms.

CFPB reported significant gains in comprehension with the new forms. For example, the new forms outperformed the current forms, with a 49% improvement in understanding the amount borrowed, and a 37% improvement in understanding the total monthly payment (CFPB, 2013b , Table 8.1).

In the rare instances where agencies have conducted high-quality assessments of comprehension of mandated information disclosures, these analyses have also provided useful information. For example, Lacko and Pappalardo pointed the way to revising confusing disclosure forms that lenders were required to present to mortgage borrowers. The CFPB study in 2013 showed that the revised forms offered significantly better comprehension.

7 Discussion

OIRA guidance points to several elements that might comprise “best practice” in evaluating the effectiveness of an information disclosure, including whether the respondents “…understand the disclosure, [and] whether they remember the relevant information when they need it …” (OIRA, 2010a , p. 101). In addition, the guidelines state that “Scientifically valid experiments are generally preferable to focus group testing, and randomized experiments can be especially valuable” (OIRA, 2010a , p. 101).

Earlier OIRA (2006) guidance also offered direction on the structure of a scientifically valid experiment. First, a valid study would need a representative sample of program participants to accurately describe the population and generalize the results. For example, comprehension may differ significantly across different groups in the population, such as borrowers willing and able to secure a conventional mortgage and those who consider balloon payments and variable interest rate mortgages. Thus, refined quantitative testing of consumer comprehension may need to consider effects of alternative disclosures on different populations to best address problems with imperfect information. In addition, OIRA (2006) also cautioned about the use of nonprobability samples, such as convenience samples and Internet panels for federal surveys. 18 For these nonprobability samples, the results cannot be generalized to any target population using traditional statistical criteria. Finally, OIRA (2006) also pointed to the response rate as a key indicator of the quality of the survey and the extent to which the survey results can be generalized to the target population. 19

Critics have argued that OIRA’s implementation of the Paperwork Reduction Act represents a barrier to a variety of scientific studies, including studies evaluating the effectiveness of mandated information disclosures. 20 In response to this criticism, OIRA (2010a , 2010b ) has moved to streamline its review process by establishing a generic clearance process for specific types of information collections. In a memo titled “Facilitating Scientific Research by Streamlining the Paperwork Reduction Act Process,” OIRA (2010d ) has outlined options and strategies for agencies to use to streamline the process of getting OMB approval of research-related information collections. OIRA could establish a similar streamlined review process for agency studies evaluating mandated information disclosures that adopted best practices along the lines identified below.

Based on this OMB guidance, Lacko and Pappalardo (2010), and our review of some recent agency studies about information disclosures, we have identified the following as most appropriate practices for the evaluation of agency-mandated information disclosures with a focus on consumers’ comprehension. Such evaluations should

  • elicit information on comprehension;

  • be based on a representative sample (not a convenience sample) and yield an acceptably high response rate;

  • use random assignment;

  • adopt a well-defined baseline or appropriate controls or comparisons;

  • assess reasonable alternative disclosures to evaluate their effectiveness; and

  • use appropriate statistical tests in assessing the effectiveness of alternative information disclosures.

Lacko and Pappalardo (2010), the FTC study associated with its Appliance Labeling Final Rule (2007b ), and the CFPB residential mortgage study are largely consistent with this guidance. They provide randomized controlled experiments to evaluate respondents’ understanding of several alternative forms of information disclosure.

Our review, however, finds only a very few cases where agencies have undertaken a quantitative assessment that largely conforms with the best practices identified above, notwithstanding presidential and OMB directives that they conduct high-quality assessments. 21 Given the emphasis that this administration has placed on such assessments, it is surprising that they are so scarce because benefit-cost analysis typically requires more time and money than the rigorous quantitative assessments of comprehension discussed here. From the standpoint of senior executives at regulatory agencies, conducting such assessments offers little advantage. First, it requires real resources – perhaps $100,000 or more in contractor support, and substantial time from specialized senior analysts and managers with the skills to oversee such work. These costs are relatively small compared with the potential social benefits, but they can discourage managers from undertaking what are essentially discretionary projects during times of tight budgets. Second, these studies may require six months or more to complete and thus may delay the regulatory decisions that political appointees wish to make during the two to three years they expect to lead their agencies. Third, a rigorous, quantitative assessment of comprehension does not necessarily point to more or less stringent regulation and thus is not naturally embraced by political appointees with incentives to pursue either the former or the latter.

It is worth noting that the lack of high-quality analysis of the effectiveness of mandated information disclosures is distinguishable from a broader problem of flaws in federal regulatory analysis highlighted e.g., by Hahn, and Tetlock (2008), Ellig and Morrall (2010), Ellig, McLaughlin, and Morrall III (2013), and Fraas and Lutter (2011).

8 Conclusion

We offer three types of recommendations to remedy the lack of credible evidence regarding the effectiveness of federally mandated information disclosures: (1) additional analyses of consumers’ comprehension in rulemakings where more analysis would be consistent with recent executive orders and OMB directives; (2) improved management and reporting by OMB of analyses of consumers’ comprehension of mandated disclosures; and (3) legislative action to address the apparent torpor of some regulators regarding such analyses. As a practical matter, however, reform need not require legislative action. The OMB already retains authority under various E.O.s (e.g., E.O. 12866 and E.O. 13563), as well as the Paperwork Reduction Act, to require that regulatory agencies conduct the analysis that OIRA has recommended. 22

  1. (1) As articulated in executive orders and OMB directives, major ongoing federal rulemakings mandating information disclosure or prohibiting certain high-risk products merit a full and fair evaluation of the effectiveness of mandatory disclosure, either as a policy or as an alternative to a product ban.

  2. (2) We recommend that OMB consolidate and formalize its directives on information disclosure by issuing specific guidelines on best practices, along the lines outlined above, to evaluate the effectiveness of mandated information disclosures. It should include in this directive a streamlined process of Paperwork Reduction Act review for studies that incorporate such best practices. In addition, OIRA in its review of agency rules under EO 13563 should ensure that agencies complete evaluations of the effectiveness of mandated information disclosures as a precondition for issuing regulatory proposals. Perhaps most importantly, OIRA, in its annual report to Congress on the benefits and costs of federal regulations, should provide additional information about significant federally mandated information disclosures. Specifically, it should identify rules mandating disclosures that are supported by analyses that comply with the standards articulated above, as well as rules where the available analyses fall short of such standards.

  3. (3) Finally, given FTC’s leadership in this area, we recommend that Congress increase appropriations for FTC’s consumer research, with a specific call for additional work from FTC about the effectiveness of a designated list of extant and new federally mandated information disclosures.

References

Bae, Hyunhoe (2012). Reducing Environmental Risks by Information Disclosure: Evidence in Residential Lead Paint Disclosure Rule. Journal of Policy Analysis and Management, 31(2), 404431.
Beales, Howard, Craswell, Richard & Salop, Steven (1981). The Efficient Regulation of Consumer Information. Journal of Law and Economics, 24(3), 491539.
Ben-Shahar, Omri & Schneider, Carl E. (2014). More than You Wanted to Know: The Failure of Mandated Disclosure. Princeton, NJ: Princeton University Press.
Consumer Financial Protection Bureau (2013a). Semi-Annual Report of the Consumer Financial Protection Bureau. Retrieved from http://files.consumerfinance.gov/f/201303_CFPB_SemiAnnualReport_March2013.pdf.
Consumer Financial Protection Bureau (2013b). Testing “Know Before You Owe” Mortgage Forms: Qualitative and Quantitative Tests Show New Forms Improve Consumer Understanding and Aid Comparison. Retrieved from http://files.consumerfinance.gov/f/201311_cfpb_factsheet_kbyo_testing.pdf.
Consumer Financial Protection Bureau2013c. Integrated Mortgage Disclosures under the Real Estate Settlement Procedures Act (Regulation X) and the Truth In Lending Act (Regulation Z): Final Rule; Official Interpretation. Federal Register, 78, 251, 79730–80365. Retrieved from https://www.federalregister.gov/articles/2013/12/31/2013-28210/integrated-mortgage-disclosures-under-the-real-estate-settlement-procedures-act-regulation-x-and-the.
Delmas, Magali, Montes-Sancho, Maria & Shimshack, Jay P. (2010). Information Disclosure Policies: Evidence from the Electricity Industry. Economic Inquiry, 48, 483498, doi:10.1111/j.1465-7295.2009.00227.x.
Department of Energy, Office of Energy Efficiency and Renewable Energy (2013). DOE Retrospective Review Plan and Burden Reduction Report. Retrieved from http://energy.gov/sites/prod/files/2013/07/f2/DOE%2013563%20and%20ICR%20Report%20-%20Final.pdf.
Department of Health and Human Services, Centers for Medicare and Medicaid Services (2013). July 2013 Retrospective Review Update and Two-Year Forecast. Retrieved from http://www.hhs.gov/open/execorders/13563/retrospectivereviewchart2013-07.pdf.
Department of Health and Human Services, Food and Drug Administration (2013). July 2013 Retrospective Review Update and Two-Year Forecast. Retrieved from http://www.hhs.gov/open/execorders/13563/retrospectivereviewchart2013-07.pdf.
Ellig, Jerry & Morrall, John(2010). Assessing the Quality of Regulatory Analysis: A New Evaluation and Data Set for Policy Research (Mercatus Center Working Paper). Arlington, VA: George Mason University. Retrieved from George Mason University Mercatus Center website, http://mercatus.org/publication/assessing-quality-regulatory-analysis.
Ellig, Jerry, McLaughlin, Patrick & Morrall, John III (2013). Continuity, Change, and Priorities: The Quality and Use of Regulatory Analysis across US Administrations. Regulation & Governance, 7, 153173, doi:10.1111/j.1748-5991.2012.01149.x.
Environmental Protection Agency (2013). Final Plan for Periodic Retrospective Reviews of Existing Regulations, E.O. 13563 Progress Report, July 2013. Retrieved from http://www.epa.gov/retrospective/documents/eparetroreviewprogressrpt-july2013.pdf.
Exec. Order No. 12,866, 58 Fed. Reg. 51,735 (October 4, 1993).
Exec. Order No. 13,563, 76 Fed. Reg. 3,821 (Jan. 18, 2011).
Exec. Order No. 13,579, 76 Fed. Reg. 41,857 (July 14, 2011).
Finger, S. & Shanti, G. R. (2013). Mandatory Disclosure of Plant Emissions into the Environment and Worker Chemical Exposure inside Plants’. Ecological Economics, 87, 124.
Fischhoff, Baruch, Brewer, Noel & Downs., Julie(Eds.) (2011). In Communicating Risks and Benefits: An Evidence-Based User’s Guide. Washington, DC: Food and Drug Administration. Retrieved from http://www.fda.gov/AboutFDA/ReportsManualsForms/Reports/ucm268078.htm.
Fraas, Arthur & Lutter, Randall (2011). The Challenges of Improving the Economic Analysis of Pending Regulations: The Experience of OMB Circular A-4. Annual Review of Resource Economics, 3, 7185, doi:10.1146/annurev-resource-083110-120042, http://www.annualreviews.org/doi/abs/10.1146/annurev-resource-083110-120042.
Fung, Archon, Graham, Mary & Weil, David (2007). Full Disclosure: The Perils and Promise of Transparency. Cambridge: Cambridge University Press.
Guthrie, Chris(2006). Law, Information, and Choice: Capitalizing on Heuristic Habits of Thought, In Engle and Gigerenzer (Eds). Heuristics and the Law. Cambridge, MA: MIT Press; Vanderbilt Law and Economics Research Paper No. 06-11. Retrieved from http://ssrn.com/abstract=900133 Accessed August 5, 2016.
Hahn, Robert & Tetlock, Paul (2008). Has Economic Analysis Improved Regulatory Decisions? Journal of Economic Perspectives, 22(1), 6784.
Hsu, David (2014). How Much Information Disclosure of Building Energy Performance Is Necessary? Energy Policy, 64, 263.
Kahneman, Daniel (2011). Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.
Kleimann Communication Group (2013). Know Before You Owe: Quantitative Study of the Current and Integrated TILA-RESPA Disclosures (presented to the CFPB). Rockville, MD: Kleimann Communication Group, Inc. Retrieved from http://files.consumerfinance.gov/f/201311_cfpb_study_tila-respa_disclosure-comparison.pdf.
Lacko, James & Pappalardo, Janis(2004). The Effect of Mortgage Broker Compensation Disclosures on Consumers and Competition: A Controlled Experiment. Bureau of Economics Staff Report. Washington, DC: Federal Trade Commission.
Lacko, James & Pappalardo, Janis(2007). Improving Consumer Mortgage Disclosure: Empirical Assessment of Current and Prototype Disclosure Forms. Bureau of Economics Staff Report, Federal Trade Commission, Bureau of Economics staff report. Washington, DC: Federal Trade Commission.
Lacko, James & Pappalardo, Janis (2010). The Failure and Promise of Mandated Consumer Mortgage Disclosures: Evidence from Qualitative Interviews and a Controlled Experiment with Mortgage Borrowers. American Economic Review, 100(2), 516521.
Macro International (2008). Design and Testing of Effective Truth in Lending Disclosures: Findings from Experimental Study. Submitted to the Board of Governors of the Federal Reserve. Calverton, MD: Macro International. Retrieved from http://www.federalreserve.gov/newsevents/press/bcreg/bcreg20081218a8.pdf.
Matisoff, Daniel (2013). Different Rays of Sunlight: Understanding Information Disclosure and Carbon Transparency. Energy Policy, 55, 579.
Miller, Sophie (2015). Learning from Experience: Retrospective Review of Regulations in 2014. Washington, D.C.: The George Washington Regulatory Studies Center. Retrieved from https://regulatorystudies.columbian.gwu.edu/learning-experience-retrospective-review-regulations-2014.
National Center for Environmental Ethics (NCEE) (2010). Guidelines for Preparing Economic Analyses. Washington, DC: Environmental Protection Agency. Retrieved fromhttp://yosemite.epa.gov/ee/epa/eed.nsf/pages/guidelines.html.
National Research Council, Committee on Risk Perception and Communication (1989). Improving Risk Communication. Washington, DC: National Academy Press.
Office of Information and Regulatory Affairs (OIRA) (2006). Guidance on Agency Survey and Statistical Information Collections. Memorandum for the President’s Management Council. Washington, DC: Office of Management and Budget. Retrieved from http://www.whitehouse.gov/sites/default/files/omb/inforeg/pmc_survey_guidance_2006.pdf.
Office of Management and Budget (OMB) (2003). Circular A-4: Regulatory Analysis. Washington, DC. Retrieved from http://www.whitehouse.gov/sites/default/files/omb/assets/omb/circulars/a004/a-4.pdf.
OIRA (2009). 2009 Report to Congress on the Benefits and Costs of Federal Regulations and Unfunded Mandates on State, Local, and Tribal Entities. Washington, DC: Office of Management and Budget. Retrieved from: http://www.whitehouse.gov/omb/inforeg_regpol_reports_congress.
OIRA (2010a). Paperwork Reduction Act–Generic Clearances. Memorandum for the Heads of Executive Departments and Agencies. Office of Management and Budget. May. Retrieved from http://www.whitehouse.gov/omb/inforeg_infocoll.
OIRA (2010b). Disclosure and Simplification as Regulatory Tools. Memorandum for the Heads of Executive Departments and Agencies. Office of Management and Budget. June. Retrieved from http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/disclosure_principles.pdf.
OIRA (2010c). 2010 Report to Congress on the Benefits and Costs of Federal Regulations and Unfunded Mandates on State, Local, and Tribal Entities. Washington, DC: Office of Management and Budget. Retrieved from: http://www.whitehouse.gov/omb/inforeg_regpol_reports_congress.
OIRA (2010d). Facilitating Scientific Research by Streamlining the Paperwork Reduction Act Process. Memorandum for the Heads of Executive Departments and Agencies. Washington, DC: Office of Management and Budget. December. Retrieved from http://www.whitehouse.gov/omb/inforeg_infocoll.
OIRA (2011). Informing Consumers through Smart Disclosure. Memorandum for the Heads of Executive Departments and Agencies. Washington, DC: Office of Management and Budget. Retrieved from http://www.whitehouse.gov/sites/default/files/omb/inforeg/for-agencies/informing-consumers-through-smart-disclosure.pdf.
OIRA (2012). Testing and Simplifying Federal Forms. Memorandum for the Heads of Executive Departments and Agencies. Washington, DC: Office of Management and Budget. Retrieved from http://www.whitehouse.gov/sites/default/files/omb/inforeg/memos/testing-and-simplifying-federal-forms.pdf.
Perry, Vanessa G. & Blumenthal, Pamela M. (2012). Understanding the Fine Print: The Need for Effective Testing of Mandatory Mortgage Loan Disclosures. Journal of Public Policy & Marketing, 31(2), 305312, Fall.
Shapiro, Stuart(2012). The Paperwork Reduction Act: Research on Current Practices and Recommendations for Reform. Administrative Conference of the United States (ACUS). Retrieved from http://www.acus.gov/sites/default/files/documents/Revised-PRA-Report-2-9-12.pdf.
Truth in Lending (2009). 74 Fed. Reg. 5,244 (Jan. 29, 2009 (to be codified at 12 C.F.R. pt. 226).
Tversky, Amos & Kahneman, Daniel (1974). Judgment Under Uncertainty: Heuristics and Biases. Science, 185(4157), 11241131.
Weil, David, Fung, Archon, Graham, Mary & Fagotto, Elena (2006). The Effectiveness of Regulatory Disclosure Policies. Journal of Policy Analysis and Management, 25, 155181, doi:10.1002/pam.20160.
Willis, Lauren(2006). Decisionmaking and the Limits of Disclosure: The Problem of Predatory Lending: Price, 65 Maryland Law Review, 707, Accessed at http://digitalcommons.law.umaryland.edu/cgi/viewcontent.cgi?article=3262&context=mlr, March 29, 2015.
Winston, Clifford (2008). The Efficacy of Information Policy: A Review of Archon Fung, Mary Graham, and David Weil’s “Full Disclosure: The Perils and Promise of Transparency. Journal of Economic Literature, 46(3), 704717.

Regulations cited

Appliance Labeling Rule (“Appliance Labeling Rule NPR”), 72 Fed. Reg. 6838 (Feb. 13, 2007) (to be codified in 16 CFR Part 305) (Notice of proposed rulemaking and request for comment).
Business Conduct Standards for Swap Dealers and Major Swap Participants with Counterparties, 77 Fed. Reg. 9,734 (Feb. 17, 2012) (to be codified at 17 C.F.R. pts. 4 and 23).
Cranes and Derricks in Construction, 75 Fed. Reg. 47,906 (Aug. 9, 2010) (to be codified at 29 C.F.R. pt. 1926).
Electronic Fund Transfers, 74 Fed. Reg. 59,033 (Nov. 17, 2000) (to be codified at 12 C.F.R. pt. 205).
Electronic Fund Transfers, 75 Fed. Reg. 16,580 (Apr. 1, 2010) (to be codified at 12 C.F.R. pt. 205).
Food Labeling: Revision of the Nutrition and Supplement Facts Labels, 79 Fed. Reg. 11880. (March 3, 2014) (to be codified at 21 CFR pt. 101) (Proposed rule).
Food Labeling: Serving Sizes of Foods That Can Reasonably Be Consumed at One-Eating Occasion; Dual-Column Labeling; Updating, Modifying, and Establishing Certain Reference Amounts Customarily Consumed; Serving Size for Breath Mints; and Technical Amendments; Proposed Rule, 79 Fed. Reg. 11990 (March 3, 2014), (21 C.F.R. pt. 101).
Hazard Communication, 77 Fed. Reg. 17,574 (Mar. 26, 2012) (to be codified at 29 C.F.R. 1910, 1915, and 1926).
Medicare and Medicaid Programs: Hospital Outpatient Prospective Payment; Ambulatory Surgical Center Payment; Hospital Value-Based Purchasing Program; Physician Self- Referral; and Patient Notification Requirements in Provider Agreements (Nov. 30, 2011) (to be codified at 42 C.F.R. pts. 410, 411, 416, 419, 489, and 495).
Medicare Program: Payment Policies under the Physician Fee Schedule and Other Revisions to Part B for CY 2010, 74 Fed. Reg. 61,738 (Nov. 25, 2009) (to be codified at 42 C.F.R. pts. 410, 411, 414, 415, 485, and 498).
Medicare Program: End-Stage Renal Disease Prospective Payment System, Quality Incentive Program, and Bad Debt Reductions for All Medicare Providers, 77 Fed. Reg. 67,450 (Nov. 9, 2012) (to be codified at 42 C.F.R. pts. 413 and 417).
Medicare Program: Revisions to Payment Policies under the Physician Fee Schedule, DME Face-to-Face Encounters, Elimination of the Requirement for Termination of Non-Random Prepayment Complex Medical Review and Other Revisions for CY 2013, 77 Fed. Reg. 68,892 (Nov. 16, 2012) (to be codified C.F.R. pts. 410, 414, 415, 421, 423, 425, 486, 495).
National Registry of Certified Medical Examiners, 77 Fed. Reg. 24,104 (Apr. 20, 2012) (to be codified at 49 C.F.R. pts. 350, 383, 390, and 391).
Nondiscrimination on the Basis of Disability in State and Local Government Services, 75 Fed. Reg. 56,164 (Sept. 15, 2010) (to be codified at 28 C.F.R. pt. 35).
Positive Train Control Systems, 75 Fed. Reg. 2,598 (Jan. 15, 2010) (to be codified at 49 C.F.R. pts. 229, 234, 235, and 236).
Program Integrity Issues, 75 Fed. Reg. 66,832, (Oct. 29, 2010) (to be codified at 34 C.F.R. pts. 600, 602, 603, 668, 682, 685, 686, 690, and 691).
Real Estate Settlement Procedures Act (RESPA): Rule to Simplify and Improve the Process of Obtaining Mortgages and Reduce Consumer Settlement Costs, 73 Fed. Reg. 68,204 (Nov. 17, 2008) (to be codified at 24 C.F.R. pts. 203 and 3500).
Revisions and Additions to Motor Vehicle Fuel Economy Label, 75 Fed. Reg. 58,078 (proposed Sept. 23, 2010) (to be codified at 40 C.F.R. pts. 600, 85, 86 and 29 C.F.R. pt. 575).
Revisions and Additions to Motor Vehicle Fuel Economy Label, 76 Fed. Reg. 39,481 (July 6, 2011) (to be codified at C.F.R. pts. 600, 85, 86 and 29 C.F.R. pt. 575).
Rule Concerning Disclosures Regarding Energy Consumption and Water Use of Certain Home Appliances and Other Products Required under the Energy Policy and Conservation Act (“Appliance Labeling Final Rule”), 72 Fed. Reg. 49,948 (Aug. 29, 2007b) (to be codified at 16 C.F.R. pt. 305).
Standards for Living Organisms in Ships’ Ballast Water Discharged in U.S. Waters, 77 Fed. Reg. 17,254 (Mar. 23, 2012) (to be codified at 33 C.F.R. pt. 151 and 46 C.F.R. pt. 162).
Truth in Lending, 74 Fed. Reg. 5,244 (Jan. 29, 2009) (to be codified at 12 C.F.R. pt. 226).

1 We use the term “consumers” throughout the paper to reflect the broad range of consumers of information who are the recipients of federally mandated information disclosures.

2 The OIRA, part of the U.S. Office of Management and Budget, has responsibility for federal regulatory oversight.

3 Our focus is on information disclosure for a broad category of goods (excluding only those that are addictive or habit forming), such as labeling of food, financial services, and energy efficiency.

4 The OMB’s Circular A-4, which provides guidance on regulatory analysis, states that mandated information measures should be evaluated in terms of their benefits and costs (specifically noting that the costs may include the possibility that information may be misinterpreted) (Office of Management and Budget, 2003).

5 Evaluating the effect of information disclosure on behavior and on the effects of these changes in behavior on outcomes that matter for policy, such as energy consumption and energy-related emissions, dietary intake and the associated health risks, or financial risk, especially in support of a federal rulemaking, may be quite challenging. Estimating the effects of information on behavior is as time consuming and expensive as other empirical studies of behavior. In addition, the resulting estimates are valid only for the specific context where they were derived.

6 The National Research Council (1989) noted that “Risk communication is successful only if it adequately informs the decision-maker,” acknowledging that this may not always occur.

7 We believe that most mandates to disclose information entail relatively small direct costs because the information to be disclosed is known to the party required to disclose it. In addition, the costs of formulating, copying and disseminating a compliant message are typically small. Further, the costs to the recipients of the information, i.e., the time to read or listen to the information and the attention and focus necessary to understand it or at least to decide not to bother with understanding it are also small. Of course, small here does not imply that these costs if summed among all affected parties are not large. It does mean that these costs are expected to be less than the potential benefits to the recipients of receiving and comprehending the information.

8 Other U.S. federal agencies have also mentioned information disclosure in guidelines for preparing economic analyses. For example, the Environmental Protection Agency’s National Center for Environmental Ethics (NCEE, 2010) has issued its Guidelines for Preparing Economic Analysis. FDA has also had an active program to improve and evaluate its risk communication activities For example, Chapter 3 of its 2011 Evidence-Based User’s Guide (Fischhoff, Brewer & Downs, 2011) is dedicated to evaluation of risk communication messages.

9 The definitions for economically significant and major are quite similar.

10 We also looked for phrases that mentioned an evaluation or assessment of comprehension, understanding or knowledge of mandated information disclosures. We found no final rules with the phrases “evaluate comprehension,” “assess comprehension,” “evaluate understand” “evaluate knowledge,” “assess knowledge” or “assess understand,” where “understand” might refer to either understanding or understandability.

11 We also considered the independent agencies. E.O. 13579 urged the independent agencies to comply with the basic provisions of EO 13563 – including the consideration of flexible approaches (like information disclosure) and the development of a plan for retrospective review. Our review found that most of the independent financial agencies offered generic assurance that they embraced the principles of EO 13563, and pointed to the substantial rulemaking agenda (and regulatory overhaul) mandated by Dodd–Frank as an initial priority. However, they have not developed a targeted list of rules slated for a retrospective review.

12 This revision of the fuel economy label was not classified as a major or economically significant rule and therefore was not identified through our systematic search described above.

13 The study randomly assigned to each respondent one of six vehicle pairs involving various combinations of conventional, hybrid, and advanced-technology vehicles.

14 EPA and NHTSA recognized that the survey would not be representative of any broader group of new-vehicle buyers. Two sources of new-vehicle buyers were used in developing the sample: those who requested a price quote from a dealer through Autobytel and who indicated that they had purchased a new vehicle (120,000 contacted; response rate ${<}$ 1%) and those drawn from the e-Rewards™ panel of new-vehicle buyers (12,025 contacted; response rate about 25%) (EPA, Internet Survey Results on the Effects of Fuel Economy Labels on Understanding and Selection November 2010, prepared by PRR Inc.)

15 The final rule provided an estimate of the cost to manufacturers of revising the label, but it offered only a qualitative discussion of the benefits. The rule preamble noted “It is difficult to develop a good baseline for the fleet using the existing label, partly because the existing label is not designed to incorporate advanced-technology vehicles. It is even more difficult to develop a comparison for the fleet with the new labels, because the effects of label designs on vehicle purchases are not known” (p. 39517).

16 Ibid, Appendix 1.

17 The agencies reported that they undertook the Internet survey “…to test the understandability of the labels as to whether the label designs affected consumers’ abilities to select efficient and environmentally friendly vehicles …” (Revisions and Additions to Motor Vehicle Fuel Economy Label, 2011, p. 39483).

18 “Convenience samples are mostly drawn from units of the population of interest that are close at hand or willing to participate. …Examples of convenience samples include shoppers at a mall, truck drivers visiting a weigh station, attendees at a conference, or visitors at a web site” (p. 29–30). With respect to Internet surveys, OIRA (2006, p. 29) noted that “…these panels [often] are not recruited using probability methods …, and the panels are typically simply convenience samples of persons interested in taking part in surveys on the Internet …”

19 OIRA (2006) stated, “…a low response rate can be an indicator of potential nonresponse bias, which would be detrimental to the accuracy of the results of a study in a variety of ways …”

20 The Paperwork Reduction Act requires OMB/OIRA approval of all federal collections of information. For a discussion of the direct and indirect costs associated with the act, see Shapiro (2012).

21 Miller (2015) reports a similar finding in a review of the 20 proposed major rules issued by Executive branch agencies in calendar year 2014 – including 4 rules containing an information disclosure mandate – that is, not a single one included a plan for a retrospective review.

22 The OMB’s Circular A-4 (Office of Management and Budget, 2003), which provides guidance on regulatory analysis, states that mandated information measures should be evaluated in terms of their benefits and costs (specifically noting that the costs may include the possibility that information may be misinterpreted) (Office of Management and Budget, 2003). The OIRA (2010a ) provides a summary of the basic requirements of the Paperwork Reduction Act.