V ^£Dsr% sSy *L prqI^ T> 2 UJ O U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL Catalyst for Improving the Environment Evaluation Report ENERGY STAR Label Needs to Assure Superior Energy Conservation Performance A Summary Report Report No. 11-P-0010 October 28, 2010 ------- Report Contributors: Laurie Adams Daniel Carroll Jerri Dorsey Jill Ferguson Jeffrey Harris Abbreviations DOE U.S. Department of Energy EPA U.S. Environmental Protection Agency GAO U.S. Government Accountability Office GHG Greenhouse gas MMTCE Million metric tons of carbon equivalent MOU Memorandum of Understanding OIG Office of Inspector General Cover photos: Examples of ENERGY STAR product types. (Photos courtesy U.S. Department of Energy) ------- vitD srAr. * J- U.S. Environmental Protection Agency n-p-0010 October 28, 2010 Iwi U O ^1 I V11 vl 111ICI I LCI I I I UlUl/ll \ Office of Inspector General u! At a Glance Catalyst for Improving the Environment Why We Did This Review Since 2006, the U.S. Environmental Protection Agency (EPA) Office of Inspector General has conducted three evaluations of selected aspects of the ENERGY STAR program. This report summarizes past findings that remain relevant and identifies design and management challenges that present risks to the program's integrity as a means of greenhouse gas avoidance and as a credible tool to promote energy efficiency and consumer savings. Background ENERGY STAR is a voluntary program designed to help businesses and individuals enhance their energy efficiency. In 1996, EPA partnered with the U.S. Department of Energy (DOE) to promote the ENERGY STAR label and broaden the product coverage. For further information, contact our Office of Congressional, Public Affairs and Management at (202) 566-2391. To view the full report, click on the following link: www.epa.qov/oiq/reports/2011/ 20101028-11-P-0010.pdf ENERGY STAR Label Needs to Assure Superior Energy Conservation Performance What We Found EPA's implementation of the ENERGY STAR program has become inconsistent with the program's authorized purpose: to achieve environmental benefits by identifying and promoting energy-efficient products and practices that meet the highest energy conservation standards. We believe the ENERGY STAR program has sought to maximize the number of qualified products available at the expense of identifying products and practices that maximize energy efficiency. We previously found that EPA could not assure that the purchase of ENERGY STAR products and adoption of ENERGY STAR practices actually deliver the energy or greenhouse gas emission savings that EPA reports annually, or that consumers are purchasing the most energy-efficient products on the market. We found that the design and execution of the ENERGY STAR program ensured neither the integrity of the label nor the achievement of greenhouse gas emission savings. Products historically qualified for the ENERGY STAR label based on manufacturer self-certification rather than EPA testing. In 2009, EPA and DOE signed a new memorandum of understanding to enhance and expand federal programs that advance energy efficiency. These enhancements include adding new product categories to the program, instituting new measures to ensure that ENERGY STAR specifications are tightened as necessary to consistently represent top performing products, and enhancing the qualification and verification testing of ENERGY STAR products. Because these changes have not yet occurred, their effectiveness remains to be determined. What We Recommend We recommend that the Assistant Administrator for Air and Radiation develop a strategic vision and program design that assures that the ENERGY STAR label represents superior energy conservation performance. We also recommend that the Assistant Administrator for Air and Radiation develop a set of goals and valid and reliable measures that can accurately inform shareholders and the public of the benefits of the program. EPA disagreed with many of our conclusions, but concurred with the proposed recommendations. Based on the Agency's comments to our draft report, we changed our first recommendation to assure that the Agency's strategic vision and design complies with the intent of the Energy Policy Act of 2005. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 THE INSPECTOR GENERAL October 28, 2010 MEMORANDUM SUBJECT: ENERGY STAR Label Needs to Assure Superior Energy Conservation Performance Report No. ll-P-0010 FROM: Arthur A. Elkins, Jr. Mwifi 35'udek fac Inspector General TO: Gina McCarthy Assistant Administrator for Air and Radiation This is our report on the subject evaluation conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe the problems the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. Final determinations on matters in this report will be made by EPA managers in accordance with established resolution procedures. The estimated cost of this report - calculated by multiplying the project's staff days by the applicable daily full cost billing rates in effect at the time - is $194,733. Action Required In accordance with EPA Manual 2750, you are required to provide a written response to this report within 90 calendar days. Your response will be posted on the OIG's public Website, along with our comments on your response. Your response should be provided in an Adobe PDF file that complies with the accessibility requirements of section 508 of the Rehabilitation Act of 1973, as amended. If your response contains data that you do not want to be released to the public, you should identify the data for redaction. You should include a corrective actions plan for agreed-upon actions, including milestone dates. We have no objections to the further release of this report to the public. This report will be available at http://www.epa.gov/oig. * o % USSJ % PRO^° If you or your staff have any questions, please contact Wade Najjum at 202-566-0832 or naiiurn. wade@epa.gov, or Jeffrey Harris at 202-566-0831 or harris.ieffrev@epa.gov. ------- ENERGY STAR Label Needs to Assure Superior Energy Conservation Performance 11-P-0010 Table of C Purpose 1 ENERGY STAR'S Role in Promoting Energy Efficiency 2 EPA Plans to Improve Management Controls to Ensure ENERGY STAR Results .... 5 Management Controls for Setting Product Performance Specifications Have Been Lacking 5 Testing of ENERGY STAR Products by EPA Has Been Limited 6 EPA Lacks Assurance that Reported GHG Emissions Avoided Are Valid 7 ENERGY STAR Does Not Indentify and Promote Products and Practices that Meet the Highest Energy Conservation Standards 10 ENERGY STAR Specifications Have Not Necessarily Identified Only Top Performers 10 Non-ENERGY STAR Product Performance Similar to ENERGY STAR Product Performance 11 Conclusions 12 Recommendations 12 Agency Comments and OIG Evaluation 12 Status of Recommendations and Potential Monetary Benefits 14 Appendices A Summaries of Prior EPA OIG, DOE OIG, and U.S. Government Accountability Office Reports 15 B Status of Corrective Actions for Prior EPA OIG Reports 18 C Scope and Methodology 22 D Agency Comments (9/13/10) and OIG Evaluation 23 E Agency Comments (10/4/10) and OIG Evaluation 28 F Distribution 32 ------- 11-P-0010 Purpose Since 2006, the U.S. Environmental Protection Agency (EPA) Office of Inspector General (OIG) has conducted three evaluations of selected aspects of the ENERGY STAR program. These reports are: ENERGY STAR Program Can Strengthen Controls Protecting the Integrity of the Label, Report No. 2007-P-00028, August 1, 2007. http://www.epa.gov/oig/reports/2007/20070801-2007-P-00Q28.pdf Improvements Needed to Validate Reported ENERGY STAR Benefits, Report No. 09-P-0061, December 17, 2008. http://www.epa.gov/oig/reports/2009/20081217-09-P-0Q61.pdf ENERGY STAR Program Integrity Can Be Enhanced Through Expanded Product Testing, Report No. 10-P-0040, November 30, 2009. http://www.epa.gov/oig/reports/2010/20091130-10-P-004Q.pdf These evaluations assessed elements of the program's design and management that are essential for assuring that the program delivers what its label promises. Appendix A provides summaries of these three reports. Beginning with the first report, EPA has taken steps to address the OIG's recommendations. Appendix B notes the status of corrective actions taken by the Agency in response to the prior three OIG reports. This report will summarize past findings that remain relevant, as well as identify design and management challenges that present risks to the program's integrity as a source of greenhouse gas (GHG)1 avoidance and as a credible tool to promote energy efficiency and consumer savings. Appendix C provides details on the scope and methodology for our current review. 1 Gases that trap heat in the atmosphere are often called greenhouse gases. Some GHGs, such as carbon dioxide, occur naturally and are emitted into the atmosphere through natural processes and human activities. Other GHGs are created and emitted solely through human activities. Many, but not all, human-activity sources of GHG emissions are expected to rise in the future. The increase in GHG emissions may be mitigated through the use of newer, cleaner technologies and other measures. Additionally, everyday choices about such tilings as commuting, housing, electricity use, and recycling can influence the amount of GHGs being emitted. 1 ------- 11-P-0010 ENERGY STAR'S Role in Promoting Energy Efficiency In 1992, EPA's Office of Air and Radiation established the ENERGY STAR Product Labeling Program as an innovative approach to environmental protection. Congress formally authorized the ENERGY STAR program in the Energy Policy Act of 2005.2 ENERGY STAR is a voluntary program designed to help businesses and individuals protect the environment through superior energy efficiency. By identifying energy- efficient products and practices, the ENERGY STAR label is intended to help the public save money and protect the environment. According to EPA, the ENERGY STAR program has helped drive investment in energy-efficient products, technologies, and practices that surpass existing standards and building codes. The program uses an assortment of strategies to "catalyze market transformation." The Agency further asserts that consumers, homeowners, and businesses rely on the ENERGY STAR program as a trusted source of unbiased information to help lower their energy bills while fighting global climate change. The ENERGY STAR program was first introduced to recognize and promote energy- efficient computers. It has since grown to cover many consumer products and services. In 1996, EPA partnered with the U.S. Department of Energy (DOE) to promote the ENERGY STAR label and broaden the product coverage.3 By 2008, the program included more than 40,000 ENERGY STAR-qualified product models across 60 product categories, which were produced by more than 2,400 manufacturers. The ENERGY STAR program has four primary program sectors: Products Commercial Industrial Residential Table 1 provides descriptions for each sector and also notes the GHG emissions that were avoided for each sector in 2008. 2 Public Law 109-58, August 8, 2005; Subtitle CEnergy Efficient Products, Sec. 131. 3 A memorandum of cooperation was signed jointly on May 29, 1996. The memorandum described each agency's responsibilities as they relate to using and overseeing the ENERGY STAR logo. On September 29, 2009, DOE and EPA signed a memorandum of understanding to better define the roles of each agency and to outline new provisions to advance energy efficiency. 2 ------- 11-P-0010 Table 1: ENERGY STAR Program Sector Descriptions ENERGY STAR Program Sector Program Sector Description 2008 GHG Emissions Avoided (MMTCE)* Product Designed to promote energy-efficient products for purchase by consumers. Features the ENERGY STAR label. 19.4 Commercial Designed to promote superior corporate energy management approaches and provide partners with guidance on assessing current energy use and developing plans that will lead to energy reductions. 18.5 Industrial Designed to help industrial companies develop robust energy programs that create the necessary infrastructure for cost-effective GHG management. 6.6 Residential Designed to help make residential homes more energy efficient through existing home improvements and use of ENERGY STAR- qualified products in existing homes. 0.5 Source: OIG review of program materials. * Million metric tons of carbon equivalent avoided as reported in the ENERGY STAR and Other Climate Protection Partnerships 2008 Annual Report. In 2008, the ENERGY STAR program reported avoiding a total of 45 MMTCE of GHG emissions. According to EPA, the program helped prevent GHG emissions equivalent to those from 29 million vehicles, while saving Americans nearly $18 billion on their energy bills. These reported benefits are more than double those claimed in 2000. The ENERGY STAR program is one of an array of federal partnerships and programs designed to promote opportunities to reduce GHG emissions. In 2002, President Bush announced a goal of reducing America's GHG intensity4 18 percent by 2012. Subsequently, in 2010, President Obama updated the goal to reduce the Federal Government's GHG emissions by 28 percent by 2020. Several departments and agencies, including the U.S. Department of Transportation, DOE, and EPA, are working to reduce GHG emissions by improving energy efficiency, conserving fossil fuels, recovering methane, and sequestering carbon. EPA plays a significant role in the Federal Government's agenda to reduce GHG emissions. In September 2009, the EPA Administrator finalized the mandatory regulation requiring the nation's largest sources of GHGs to report their GHG emissions. This new rule is expected to allow EPA to track approximately 85 percent of total U.S. emissions while only requiring a small percentage of facilities to report 4 GHG intensity is the ratio of greenhouse gas emissions to economic output. 3 ------- 11-P-0010 data. In December 2009, the Administrator finalized the "endangerment finding," which determined that GHGs contribute to the endangerment of public health and welfare. According to its strategic plan, EPA plans to reduce GHG intensity by enhancing partnerships with businesses and other sectors. EPA has several programs to address GHG reductions. The total contribution of all EPA programs is expected to satisfy 70 percent of the President's GHG intensity reduction goal. While regulatory efforts are underway, the core of EPA's climate change efforts are voluntary government/industry programs such as the ENERGY STAR program.5 According to the savings reported by the Agency, the ENERGY STAR program is EPA's most significant GHG avoidance program and the largest single contributor to the U.S. national strategy. As illustrated in Figure 1, EPA's annual reported GHG reductions have increased steadily since 2002, with the ENERGY STAR program consistently accounting for over 50 percent of EPA's reported reductions. Figure 1: EPA GHG Emission Reductions 100 90 SO o 1- 3= 70 2 C 60 i/i b 50 -s j T! ¦n 40 O 30 I (D 20 10 0 2002 2003 2004 2005 Year 2006 2007 2008 ~ ENERGY STAR contributions ~ Utfier EPA program contributions Source: OIG analysis of ENERGY STAR program annual reports. - In 1992, the United States signed and Congress ratified the United Nations Framework Convention on Climate Change Treaty in Rio de Janeiro. The Rio Treaty requires the United States to implement programs to reduce GHG emissions. The United States decided to achieve this goal through voluntary programs. 4 ------- 11-P-0010 EPA Plans to Improve Management Controls to Ensure ENERGY STAR Results We previously found that EPA could not assure that the purchase of ENERGY STAR products and adoption of ENERGY STAR practices actually deliver the energy or GHG emission savings EPA reports annually,6 or that consumers are purchasing the most energy-efficient products on the market. We found that the design and execution of the ENERGY STAR program ensures neither the integrity of the label nor the achievement of GHG emission savings. Products historically qualified for the ENERGY STAR label based on manufacturer self-certification rather than EPA testing.7 On September 30, 2009, EPA and DOE signed a new memorandum of understanding (MOU) to enhance and expand federal programs that advance energy efficiency. These planned enhancements include adding new product categories to the program, instituting new measures to ensure that ENERGY STAR specifications are tightened as necessary to consistently represent top performing products, and enhancing the qualification and verification testing of ENERGY STAR products. Because these changes have not yet occurred, their effectiveness remains to be determined. Management Controls for Setting Product Performance Specifications Have Been Lacking Traditionally, EPA set an initial performance specification for a product category and then monitored the product in the marketplace to determine when it was appropriate to begin revising each specification. The Agency has followed six key principles when establishing consumer product energy efficiency specifications: 1. Significant energy savings can be realized on a national basis. 2. Product performance can be maintained or enhanced with increased energy efficiency. 3. Purchasers will recover their investment in increased energy efficiency within a reasonable time. 4. Energy efficiency can be achieved with several technology options, at least one of which is nonproprietary. 5. Product energy consumption and performance can be measured and verified with testing. 6. Labeling would effectively differentiate products and be visible for purchasers. 6 EPA annually reports net savings on energy bills and GHG emissions avoided as reportable benefits of the program. 7 Companies self-certify that their products meet ENERGY STAR specifications. This process involves the manufacturer testing its product models per ENERGY STAR specifications and reporting the results to EPA. 5 ------- 11-P-0010 Before EPA would set or revise a product specification, it would request input from manufacturers and other stakeholders. This input was used to determine the availability of new or advanced technologies and the level of interest in producing products under a new revised specification level, and to notify manufacturers of a pending revision. According to ENERGY STAR staff, this voluntary collaboration was an essential part of the process. We found that the criteria for initiating and revising specifications were unclear and were not documented. Because the Agency did not document the results of the specification-setting process, it is unclear how individual decisions regarding specifications revisions are made. According to EPA, several actions resulting from the MOU will address these historic shortcomings.8 The MOU also allows for more frequent revisions to existing ENERGY STAR specifications so that the label will continue to highlight the most energy-efficient products among manufacturers that volunteer to participate. EPA will conduct revisions on a preset timeline,9 and specifications will be set so that the ENERGY STAR logo is applied consistently with established program principles. Testing of ENERGY STAR Products by EPA Has Been Limited Throughout the history of ENERGY STAR, qualified products have largely been untested by EPA. EPA relied on manufacturer self-certifications. EPA conducted only minimal verification testing and assumed that in a competitive market, manufacturers would test each other's products and report failures to EPA. However, the Agency could not provide any examples as evidence that self-policing occurred.10 We found that EPA had not conducted any verification testing for the first 10 years of the program. When verification testing began, it accounted for only a small component of the program's activities and budget. For example, in Fiscal Year 2006, the ENERGY STAR Product Labeling Program had an estimated $18.2 million budget, of which $70,000 was allocated for verification testing. In its first 5 years of verification testing, EPA averaged only two sets of product verification tests per year. When our 2007 report was issued, 44,000 qualified product models existed within the qualified ENERGY STAR product categories. At the end of 2006, EPA had only conducted verification testing on 160 product models in 9 out of 48 product categories managed by EPA. 8 According to the May 7, 2010, DOE/EPA Partnership Work Plan, 21 actions are underway, 7 of which will be completed in 2010. 9 For product categories with longer-lived product model lifecycles (e.g., heating and cooling equipment and home appliances), specifications will be reviewed at least every 3 years or when the market share for ENERGY STAR- qualified products reaches about 35 percent. For product categories with rapidly evolving product models (e.g., consumer electronics and office equipment), specifications will be updated about every 2 years. 111 In response to a March 2010 U.S. Government Accountability Office report critical of ENERGY STAR's lack of third-party testing, EPA continued to reiterate this position. EPA stated that for 18 years, manufacturers' market incentive to test competitors' products and report violations to EPA has been an effective quality control measure to ensure that consumers are getting products that cut energy costs and GHG emissions. 6 ------- 11-P-0010 The recent MOU between DOE and EPA includes new testing and verification procedures. According to the MOU, verification of ENERGY STAR-qualifying products will be enhanced by the following: All products will be required to be tested in an accredited laboratory and qualifying product information will be submitted to the government before the product can be qualified as ENERGY STAR. Enhanced off-the-shelf product testing across all ENERGY STAR-covered product categories will be conducted by a combination of EPA/DOE testing, manufacturer funded EPA/DOE administered testing, or other third-party testing. EPA and DOE also plan to pursue product qualification prior to labeling, laboratory qualification, and comprehensive verification testing to ensure that ENERGY STAR remains a trusted symbol for environmental protection. According to EPA, the transition to independent testing for products not subject to a third-party certification program will be complete by November 30, 2010. Furthermore, all product categories will be subject to independent qualification testing by the end of 2010.11 EPA Lacks Assurance that Reported GHG Emissions Avoided Are Valid Based on control weaknesses identified in our previous evaluations, we conclude that EPA cannot be certain that its reported savings claims are valid or supportable, and that large amounts of GHG emissions are in fact being avoided. In October 2009, the DOE OIG reported similar findings, stating that these deficiencies could reduce consumer confidence in the integrity of the ENERGY STAR label.12 A key assumption of the ENERGY STAR program is that the purchase and use of ENERGY STAR products and practices will save energy and lower GHG emissions. In 2008, EPA reported that ENERGY STAR benefits resulted in Americans saving about $18 billion on their utility bills, avoiding the need for about 190 billion kilowatt hours of electricity, and avoiding 45 million metric tons of GHG emissions. Reported ENERGY STAR benefits represented over 50 percent of EPA's total GHG emissions avoided in 2006 and 2007; ENERGY STAR benefits are a major metric in EPA's efforts to reduce these emissions. Therefore, the accuracy of the program's reported ENERGY STAR savings is important in assessing EPA's overall efforts to reduce GHG emissions. EPA computes the energy savings for each qualified ENERGY STAR product with a computer model. EPA used the formula shown in Figure 2 to calculate the annual 11 EPA will be requiring products to be tested for qualification in an EPA-recognized, accredited laboratory, either through participation in a third-party certification program or through manufacturer-arranged testing in an accredited laboratory, and the resulting data must be shared with the ENERGY STAR program. 12 DOE OIG, The Department's Management of the ENERGY STAR Program, DOE/IG-0827, October 14, 2009. 7 ------- 11-P-0010 energy savings benefits and GHG reductions resulting from the ENERGY STAR program. Figure 2: ENERGY STAR Energy Savings Calculation Non-ENERGY STAR product energy consumption minus ENERGY STAR product minimum energy consumption multiplied by ENERGY STAR product sales equals ENERGY STAR Products' Energy Savings Source: OIG analysis of ENERGY STAR program material. A key number in the computation is the annual ENERGY STAR-qualified product shipment total by product category.13 In an earlier review, we reported that the ENERGY STAR program's reported savings claims for products were inaccurate and that the reported annual savings were unreliable. We identified several deficiencies with the shipment data used in calculating benefits. Deficiencies included the lack of a data quality review, reliance on estimates, forecasting and unverified third-party reporting, and the potential inclusion of exported items. EPA also included savings from products that are no longer ENERGY STAR qualified in the benefits calculation formula.14 Secondly, we determined through product testing that there may not be a distinction between the minimum energy consumption of ENERGY STAR products and the actual product energy consumption of non-ENERGY STAR products. The ENERGY STAR label is advertised as representing superior energy efficiency. However, our test of ENERGY STAR and non-ENERGY STAR products disclosed that some non- ENERGY STAR products were more efficient. The ENERGY STAR specification was a minimum standard that 98 percent of qualified tested products met and many products exceeded.15 Additionally, many nonqualified products we tested also met or exceeded the specification. The performance results of the tested ENERGY STAR and non-ENERGY STAR products call into question the assumptions used to calculate energy savings and GHG reductions attributed to the program. If non-ENERGY STAR products are found to be at least as energy efficient as qualified ENERGY STAR products, then the energy and GHG savings reported by the Agency cannot be valid. 13 The ENERGY STAR program uses the term "shipments" interchangeably with "sales." When a qualified ENERGY STAR manufacturer ships its ENERGY STAR-qualified product, the shipment is considered sold for annual savings calculation purposes. 14 This calculation is known as the market transformation benefit calculation. 15 We tested 120 ENERGY STAR-qualified products and 118, or 98 percent, met program requirements for compliance. 8 ------- 11-P-0010 Furthermore, the methodology used to compute the ENERGY STAR savings for the commercial sector was based on unverified assumptions and used formulas rather than actual program data. Based on reported 2006 program savings, the commercial component of the program was the second-largest contributor to carbon emissions avoided and nearly equaled the total benefits of the products component.16 EPA used a formula created, operated, and maintained by its contractor to compute the annual commercial sector savings. In this formula, the contractor (1) calculates all commercial sector gains in energy efficiency, (2) subtracts amounts from utility and State programs, and (3) concludes that the remaining energy savings can be attributed to the ENERGY STAR program. According to the Agency's response to the draft report, the Agency is enhancing the way in which it accounts for program savings, including removing the market transformation effect, improving consistency, and reassessing the baseline of core products. Furthermore, according to EPA, a peer review of the program allocation methodology that EPA uses to estimate the program benefits in the commercial buildings market is currently underway. 16 We excluded the residential and industrial program sectors from our review. The commercial and products sectors combined encompass 81.6 percent of the 2006 reported carbon emissions avoided. 9 ------- 11-P-0010 ENERGY STAR Does Not Identify and Promote Products and Practices that Meet the Highest Energy Conservation Standards The Energy Policy Act of 2005 established ENERGY STAR as a voluntary program to identify and promote energy-efficient products and buildings. The goals of the program are to reduce energy consumption, improve energy security, and reduce pollution. These goals were to be achieved through voluntary labeling of, or other forms of communication about, products and buildings that meet the highest energy conservation standards. In contrast to the expectations set out in this Act, the ENERGY STAR program does not seek to achieve benefits by promoting products that meet the highest energy conservation standards. EPA's emphasis on achieving the broadest number of ENERGY STAR transactions rather than identifying and promoting products with the highest efficiencies has led to results inconsistent with the intent of the Act. We have found historical instances in which the vast majority of products in the marketplace are ENERGY STAR qualified and instances in which ENERGY STAR qualified products were outperformed by some non-ENERGY STAR qualified products. ENERGY STAR Specifications Have Not Necessarily Identified Only Top Performers In the past, the ENERGY STAR label was intended to identify the top performers in energy efficiency. Agency documents state that specifications were designed to typically capture the top 25 percent of energy performing models on the market. However, we found that historically, specifications in some product categories were not revised in a timely manner, resulting in the majority of products in the marketplace being ENERGY STAR qualified. For example, prior to specification revisions, 98 percent of all computers, 95 percent of all monitors, 90 percent of all fax machines, and 99 percent of all mailing machines were ENERGY STAR qualified. In those product categories, the ENERGY STAR label did not identify superior energy efficiency over other products in the marketplace. Rather, it represented the lowest common standard of energy efficiency. If the majority of products in a certain category bear the ENERGY STAR label, ENERGY STAR becomes an inclusive program. In such instances, the integrity of the label is diminished and credibility with consumers may decline. The inclusion of the majority of products in a category also raises questions about whether energy is actually saved if most products meet the ENERGY STAR specification. The media have reported such concerns. In October 2008, Consumer Reports opined that the qualifying standards for ENERGY STAR products were too lax. It found that 92 percent of all dishwashers qualified as ENERGY STAR products. A February 23, 2010, Washington Post article questioned both the reliability of the program label and 10 ------- 11-P-0010 whether less-than-efficient appliances are making the cut. As reported in the Washington Post, in the past the ENERGY STAR label was "rather exclusive," but now it is difficult to find appliances in some categories that are not ENERGY STAR certified. As stated in the Consumer Reports article, "It makes the EnergyStar worth a little less to the consumer if it's something everybody's got." EPA has stated that, "... the ENERGY STAR program achieves significant results by delivering high efficiency to a large segment of consumers rather than targeting the most efficient products which would appeal to a much more limited set of purchasers." The inclusive design and implementation of the program was reinforced when EPA stated in its response to the draft report on September 13, 2010, "Based on expert advice and experience in terms of what makes a consumer label effective, the ENERGY STAR products program has not and does not seek to achieve environmental benefits by promoting products that meet the highest energy conservation standards. Rather, the program operates under the principle that the greatest environmental impact can be achieved by affecting the broadest number of transactions, namely the purchases of the average consumer, who will choose products that are good for the environment as long as they don't cost more or involve a sacrifice in performance." We see this as a departure from the intent of the Energy Policy Act of 2005, which authorized EPA to ". . . identify and promote energy-efficient products and buildings in order to reduce energy consumption, improve energy security, and reduce pollution through voluntary labeling of, or other forms of communication about, products and buildings that meet the highest energy conservation standards " (emphasis added). Competing visions of the ENERGY STAR program within EPA were evident in its October 4, 2010, comments that stated, "The goal of the ENERGY STAR products program is to identify and promote energy-efficient products in order to reduce energy consumption, improve energy security, and reduce pollution through voluntary labeling of products that meet the highest energy conservation standards as authorized under the Energy Policy Act of 2005 and the Clean Air Act Section 103(g)." The OIG sees these conflicting statements as evidence of a need for strategic vision and program design for an exclusive program that assures that the ENERGY STAR label represents superior energy conservation performance to the consumer. Non-ENERGY STAR Product Performance Similar to ENERGY STAR Product Performance According to EPA, the ENERGY STAR label enables consumers to easily identify energy-efficient products in the market. However, manufacturer participation in the program is voluntary, and the ENERGY STAR specification is a minimum standard. In 2009, we tested ENERGY STAR and non-ENERGY STAR products and found that products not labeled as ENERGY STAR met or exceeded the ENERGY STAR performance level. These results call into question the ability of the program to identify products with superior energy efficiency. If the program cannot identify 11 ------- 11-P-0010 products with superior energy efficiency, it cannot assure purchasers that they will recover their investment within a reasonable period of time in increased energy efficiency through utility bill savings. Consequently, the ENERGY STAR designation only identifies products from voluntary partners that self-report that the products meet the minimum standard; it does not necessarily identify the best- performing products in the marketplace. The level of product performance affects the ENERGY STAR label's image as a trusted national symbol for environmental protection through superior energy efficiency. If non-ENERGY STAR-labeled products consume the same or less energy than ENERGY STAR-qualified products, the value of the label is diminished. Therefore, EPA cannot currently be certain ENERGY STAR products are a good choice for the consumers looking for energy efficiency or cost effectiveness. Moreover, the basis for energy savings or GHG reductions claims is questionable. Conclusions We believe the ENERGY STAR program has sought to maximize the number of transactions of qualified products at the expense of identifying products and practices that maximize energy efficiency. We believe this is because the ENERGY STAR program's design has become inconsistent with the intended outcomes. Although the program delivers the message to the consumer that the ENERGY STAR label designates cost-effective, energy-efficient products, this may not be the case. The program is currently taking steps to enhance program management. However, the integrity of the label remains at risk because it does not necessarily identify and promote the most energy-efficient products on the market. Agency management should assure that the program complies with the intent of Energy Policy Act of 2005 and redesign the program accordingly. Recommendations We recommend that the Assistant Administrator for Air and Radiation: 1. Develop a strategic vision and program design that assures that the ENERGY STAR label represents superior energy conservation performance. 2. Develop a set of goals and valid and reliable measures that can accurately inform shareholders and the public of the benefits of the program. Agency Comments and OIG Evaluation The Agency agreed with our draft report's recommendations, but did not concur with the summary findings. While the Agency concurred with our recommendations, it is not clear how EPA's ENERGY STAR program as described meets the intent of the Energy Policy Act of 2005. The Act sets out that 12 ------- 11-P-0010 products in the ENERGY STAR program should meet "the highest energy conservation standards," which should result in a more exclusive program. Based on the Agency's comments to the draft report, we changed our first recommendation to assure that the Agency's strategic vision and design complies with the intent of the Act. Subsequent to the exit meeting, wherein we discussed the OIG's conclusion that ENERGY STAR's vision and design should be more exclusive, the Agency provided a revised second response. In its initial response, the Agency stated, ... the ENERGY STAR products program has not and does not seek to achieve environmental benefits by promoting products that meet the highest energy conservation standards. Rather, the program operates under the principle that the greatest environmental impact can be achieved by affecting the broadest number of transactions. . . . However, the Agency's second response claimed that the program had an exclusive goal: ... to identify and promote energy-efficient products in order to reduce energy consumption, improve energy security, and reduce pollution through voluntary labeling of products that meet the highest energy conservation standards. . . . These divergent responses, along with the recent program shortcomings summarized in the report, reinforce the OIG's concern that the program direction is inconsistent. The planned programmatic improvements described in the MOU should be implemented in conjunction with an exclusive strategic vision and design. Both are needed to assure consumers that labeled products have superior energy conservation performance. Moreover, the program's contribution to GHG emissions avoidance continues to require a defined set of goals and measures to accurately report results to the public. The Agency's two formal written responses, as well as our evaluation of both sets of the Agency's comments, are presented in Appendices D and E. 13 ------- 11-P-0010 Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Planned Rec. Page Completion Claimed Ag reed-To No. No. Subject Status1 Action Official Date Amount Amount 12 Develop a strategic vision and program design that assures that the ENERGY STAR label represents superior energy conservation performance. 12 Develop a set of goals and valid and reliable measures that can accurately inform shareholders and the public of the benefits of the program. Assistant Administrator for Air and Radiation Assistant Administrator for Air and Radiation 1 0 = recommendation is open with agreed-to corrective actions pending C = recommendation is closed with all agreed-to actions completed U = recommendation is undecided with resolution efforts in progress 14 ------- 11-P-0010 Appendix A Summaries of Prior EPA OIG, DOE OIG, and U.S. Government Accountability Office Reports Below are summaries of the three EPA OIG reports we published as part of a series of evaluations of EPA's ENERGY STAR program. Appendix B lists all the recommendations and the status of each. Also included here is a summary of recent reports by the DOE OIG and the U.S. Government Accountability Office (GAO). EPA OIG, ENERGY STAR Program Can Strengthen Controls Protecting the Integrity of the Label, Report No. 2007-P-00028, August 1, 2007. http://www.epa.gov/oig/reports/2007/20070801-2007-P-00Q28.pdf To ensure the efficiency and effectiveness of the ENERGY STAR program and the integrity of its label, EPA established several processes. These processes include product specification setting and revision, product self-certification, product verification testing, and label utilization monitoring. We reviewed these processes and found improvements could be made that could better assure the integrity of the ENERGY STAR label for the consumer of home and office products. We found the criteria for revising specifications were unclear and not documented. It was not evident when or what factors would trigger a specification revision. Furthermore, EPA did not have reasonable assurance that the self-certification process is effective. EPA relied on some alternative verification mechanisms, but lacked any quality assurance or review of these reported results. The Agency's verification testing also lacked a clear documented methodology governing products selected for verification tests and does not test for statistically valid results. Consequently, product efficiency and energy savings reported by manufacturers were, for the most part, unverified by EPA review. We found little oversight in using the ENERGY STAR label in retail stores, which is commonly the purchase point for the consumer. EPA could not provide documentation related to follow-up actions taken, final results for all retail store assessments, or the resolution status of label inconsistencies. We also found that manufacturers could label and sell products as ENERGY STAR qualified prior to submitting test results to the Agency. Using the label on products that do not meet ENERGY STAR requirements may weaken the value of the label and negatively impact the ENERGY STAR program. 15 ------- 11-P-0010 EPA OIG, Improvements Needed to Validate Reported ENERGY STAR Benefits, Report No. 09-P-0061, December 17, 2008. http://www.epa.gov/oig/reports/2009/20081217-09-P-0Q61.pdf EPA reported that ENERGY STAR benefits represented one-half of the Agency's total GHG emissions avoided in 2006. ENERGY STAR benefits are a major component of efforts reducing such emissions. The accuracy of the program's reported energy savings is important in monitoring United States efforts to reduce GHG emissions. We found the ENERGY STAR program's reported savings claims were inaccurate and the reported annual savings were unreliable. We identified deficiencies with the shipment data and the process used in calculating benefits. Deficiencies included the lack of a quality review of the data collected; reliance on estimates, forecasting, and unverified third-party reporting; and the potential inclusion of exported items. Also, EPA included savings for one DOE product that DOE also claimed. Additionally, sales of formerly qualified products are used to determine ENERGY STAR's market transformation benefits, but we found this benefit was computed inconsistently. Also, the methodology used to compute the ENERGY STAR commercial sector benefits uses unverified assumptions. EPA OIG, ENERGY STAR Program Integrity Can Be Enhanced Through Expanded Product Testing, Report No. 10-P-0040, November 30, 2009. http://www.epa.gov/oig/reports/2010/20091130-10-P-004Q.pdf This evaluation was initiated to independently test ENERGY STAR products to determine whether their energy-efficient performance complied with the ENERGY STAR program's required specifications. Almost all of the ENERGY STAR products in our test sample met, and in most cases exceeded, the program's performance standards. However, selected non-ENERGY STAR products performed comparably to, and in some cases better than, ENERGY STAR products. That level of product performance affects the ENERGY STAR label's image as a trusted national symbol for environmental protection through superior energy efficiency. In addition, the performance results of ENERGY STAR and non-ENERGY STAR products call into question the assumptions used to calculate energy savings and GHG reductions attributed to the program. Without an enhanced testing program, including the testing of non-ENERGY STAR products, EPA cannot be certain ENERGY STAR products are the more energy-efficient and cost-effective choice for consumers. 16 ------- 11-P-0010 DOE OIG, The Department's Management of the ENERGY STAR Program, DOE/IG-0827, October 2009. http://www.ig.energy.gov/documents/IG-0827-508.pdf The October 2009 DOE OIG report identified many of the same findings relative to DOE's administration of the program that prior EPA OIG reports identified. The DOE OIG reported that DOE officials had not: developed a formal quality assurance program to help ensure that product specifications were adhered to, effectively monitored the use of the ENERGY STAR label to ensure that only qualifying products were labeled as compliant, and formalized procedures for establishing and revising product specifications and for documenting decisions regarding those specifications. The DOE OIG reported that these deficiencies could reduce consumer confidence in the integrity of the ENERGY STAR label. Such loss could also reduce reported energy savings, increase consumer risk, and diminish the value of the recent infusion of $300 million for ENERGY STAR rebates under the American Recovery and Reinvestment Act of 2009. GAO, Covert Testing Shows the Energy Star Program Certification Process Is Vulnerable to Fraud and Abuse, GAO-10-470, March 5, 2010. http://www.gao.gov/new.items/dl0470.pdf GAO's investigation shows that ENERGY STAR is for the most part a self-certification program vulnerable to fraud and abuse. GAO was able to obtain ENERGY STAR certifications for 15 bogus products, including a gas-powered alarm clock. GAO reported that certification controls were ineffective primarily because ENERGY STAR does not verify energy-savings data reported by manufacturers, as was reported in the previous EPA OIG reports. In addition, two of the bogus ENERGY STAR firms developed by GAO received requests from real companies to purchase products because these bogus firms and products were listed as ENERGY STAR partners. GAO reported that this clearly shows the heavy reliance of American consumers on the ENERGY STAR brand and the program. Companies use the ENERGY STAR certification to market their products, and consumers buy products relying on the certification by the government of reduced energy consumption and costs. Furthermore, the program is promoted through tax credits and appliance rebates, and federal agencies are required to purchase certain ENERGY STAR-certified products. 17 ------- 11-P-0010 Appendix B Status of Corrective Actions for Prior EPA OIG Reports Report Recommendation Status of Corrective Action17 2007-P-00028 3-1: Clarify the decision criteria and document the process for revising an ENERGY STAR specification including identifying circumstances when a specification revision would not be revised, despite a high market share of qualified products. EPA agreed to provide interim direction to program administration staff that product selection should take specification setting and revising into account and that selected products should be tested within 6-12 months of the specification effective date by October 2007. Further, EPA agreed to document a broader Compliance Audit Program, including criteria for establishing testing priorities and a protocol for addressing new products and products with specification revisions by March 2008. The September 2009 MOU signed addresses the intent of this recommendation. 4-1: Establish a formal Quality Assurance Plan for product and verification testing to provide a reasonable assurance results represent the products available and the certification of others may be relied upon. According to EPA, it agreed to review its product testing and verification efforts with the intent to establish a formal, comprehensive Compliance Audit Program. The recent MOU between DOE and EPA included new enhancements to the testing and verification procedures. According to the MOU, verification of ENERGY STAR-qualifying products will be enhanced by the following: All products will be required to be tested in an accredited laboratory and qualifying product information be submitted to the government before the product can be qualified as ENERGY STAR. Enhanced "off-the-shelf' product testing across all ENERGY STAR covered product categories will be conducted by a combination of EPA/DOE testing, manufacturer funded EPA/DOE administrated testing, or other third party testing. EPA and DOE also plan to pursue qualification prior to labeling, laboratory qualification and comprehensive verification testing to ensure that ENERGY STAR remains a trusted symbol for enviromnental protection. According to EPA, the transition to independent testing for products not subject to a third-party certification program will be complete by November 30, 2010. All product categories will be subject to independent qualification testing by the end of 2010. 17 As part of this review, we did not verify the status of the corrective actions for all previous OIG reports. 18 ------- 11-P-0010 Report Recommendation Status of Corrective Action17 2007-P-00028 (continued) 4-2: Coordinate verification testing with product specification setting and revision processes to ensure products are selected in a timely and relevant basis. EPA agreed to provide interim direction to program administration staff that product selection should take specification setting and revising into account and that selected products should be tested within 6-12 months of the specification effective date by October 2007. Further, EPA agreed to document a broader Compliance Audit Program, including criteria for establishing testing priorities and a protocol for addressing new products and products with specification revisions by March 2008. The recent DOE and EPA MOU included new provisions for the product specification process. Specifications will be set so that the ENERGY STAR label is applied consistently with established program principles. According to the MOU, specifications will be set to identify the 25 percent most efficient models within a product class. The MOU also allows for more frequent revisions to existing ENERGY STAR specifications so that the label will highlight top energy-efficient products among manufacturers that volunteer to participate. 5-1: Establish standards to ensure label use inconsistencies found during the retail store level assessments are systematically recorded, appropriate actions are taken and infractions are tracked until resolved or otherwise completed. According to EPA, it instituted a formal process for documenting and addressing possible label infractions found during retail-store-level assessments, to include regular management status reviews and final reports. 5-2: Establish standing operating procedures for contract oversight to assure that all contractually required work is complete and meets the contract requirements. On September 28, 2007, the ENERGY STAR Product Labeling Branch Chief issued a memorandum to staff reiterating standard operating procedures for contractor oversight. 19 ------- 11-P-0010 Report Recommendation Status of Corrective Action17 09-P-0061 2-1: To improve the validity of reported annual savings for the ENERGY STAR program, establish and perform quality controls to ensure that: Data in benefits calculations, whether from partners or third parties, are timely, complete, valid and documented. The contractor and third party associations receiving the manufacturer data submittal forms reconcile submittals in a a manner that ensures the total annual shipments reported by product category are accurate and reflect actual numbers (not estimates) and are for domestic shipments only. Agency officials improve contractor oversight by obtaining actual support for annual savings in a manner that demonstrates that the numbers are valid and can be reconciled. Data in benefits calculations attributable to DOE products should be clearly identified and developed in consultation with DOE to avoid redundancy. According to the Agency, as of March 26, 2009, EPA has implemented the following: Instituted additional checks on submitted data to ensure completeness and validity. Instituted systematic quality assurance check on all data entered into the unit shipment database to ensure accuracy of entered data and resulting sums. Instituted documentation of all communications with individual partners to clarify submissions. Developed and made available through Website a set of frequently asked questions designed to reinforce data submittal requirements, including "U.S. only" and "no estimates" to improve quality of data submitted. Met with the relevant trade associations prior to the beginning of 2008 shipping data collection effort to reinforce the purpose of the collection and the importance of submitting high-quality data. 3-1: Develop and consistently apply a data-driven methodology to compute the market transformation effect of all product categories and report the benefits separately from ENERGY STAR-qualified products. EPA agreed to seek the advice of outside experts through a formal peer review process on its overall approach to evaluating benefits from the labeling program, including the methodology for assessing the market transformation effect. EPA agreed to make changes to the methodology based on the recommendations received, applying them consistently where the market transformation effect applies, and update the model by July 31, 2010. 4-1: Validate the formula (methodology) used for calculating the benefits of the ENERGY STAR commercial program in accordance with £714 Quality Manual for Environmental Programs CIO 2105- P-Ol-O, to ensure that it accurately reflects the impacts of EPA actions. EPA agreed to secure additional outside expert review of the methodology being used to estimate the benefits of the ENERGY STAR program in the commercial sector. The review will assure that assumptions, data sources, and methods used to estimate the benefits are reasonable and supported. A peer review is currently being conducted (according to the EPA guidance about how to conduct peer reviews) of the program allocation methodology that EPA uses to estimate the program benefits in the commercial buildings market. The peer reviewed is estimated to be completed by summer 2010. 20 ------- 11-P-0010 Report Recommendation Status of Corrective Action17 10-P-0040 2-1: Verily estimated energy savings and greenhouse gas reduction calculations using a market-based performance testing program that includes testing non-ENERGY STAR products. EPA presented to stakeholders plans for and a proposed framework for a market-based performance program on December 2, 2009, that leverages and expands the infrastructure of certification programs that test both qualified and nonqualified products. According to the Agency's response to the draft report, the Agency is in the process of enhancing its approach to accounting for program savings including removing the market transformation effect, improving consistency and reassessing the baseline of core products. 2-2: Revise the ENERGY STAR Website to include the established standard alongside qualifying product performance data and to provide a summary listing of the highest performers. According to EPA, beginning in December 2009, EPA began amending the ENERGY STAR Qualified Product lists and the Find-a-Product search tool to address the three main parts of the OIG's request: adding key energy-efficiency performance data, summarizing the ENERGY STAR energy-efficiency criteria, and sorting qualified product information based on performance. This was expected to be completed by spring 2010. 21 ------- 11-P-0010 Appendix C Scope and Methodology We conducted this performance evaluation in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the evaluation to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based upon our objectives. We conducted our field work for all three previous evaluations and this current summary work from September 2006 through August 2010. Our review included an examination of applicable laws and regulations as well as Agency guidance. We reviewed those internal controls that were relevant to our objectives. We reviewed ENERGY STAR annual reports, Agency guidance documents, and the EPA-DOE MOU. We reviewed planning documents, including logic models. We also reviewed relevant reports by GAO, Lawrence Berkley National Laboratory, the Consortium for Energy Efficiency, and current media reports on the program. We met with Agency staff and contractors. We reviewed the methodologies governing the savings-benefit calculations for the commercial and products program sectors.18 Additionally, for the products sector, we reviewed documentation in support of the 2006 reported savings benefits. We also reviewed the processes in place to ensure the efficiency and effectiveness of the ENERGY STAR program and the integrity of its label. These processes include product specification setting and revision, product self-certification, product verification testing, and label utilization monitoring. Additionally, we tested a sample of both ENERGY STAR- and non-ENERGY STAR-qualified products. From December 2008 through February 2009, we selected 20 different ENERGY STAR-qualified products for testing from each of 3 product categories. Two identical models of each product category were purchased for a total of 40 ENERGY STAR-qualified products per category, 120 products in all. We also tested the performance of some non-ENERGY STAR products. We purchased 10 non-ENERGY STAR products (2 each of 5 models) from the same 3 product categories for a total of 30 non-ENERGY STAR products. These products underwent the same testing as our sample of ENERGY STAR products, and the results were compared with ENERGY STAR specifications. We integrated the results of all our prior ENERGY STAR reports to provide this overall assessment of the effectiveness of EPA's oversight and management of the ENERGY STAR program and the controls in place to ensure the overall integrity of the ENERGY STAR label. 18 The residential and industrial program sectors were excluded from our second review. 22 ------- 11-P-0010 Appendix D Agency Comments (9/13/10) and 01G Evaluation The initial response from the Assistant Administrator was received on September 13, 2010. MEMORANDUM SUBJECT: Response to Evaluation Report: ENERGY STAR Needs to Define Its Primary Goal and Consumer Expectations - A Summary Report FROM: Gina McCarthy Assistant Administrator TO: Jeffrey Harris, Director Cross Media Issues, Office of Program Evaluation Thank you for the opportunity to respond to the Evaluation Report: ENERGY STAR Needs to Define Its Primary Goal and Consumer Expectations - A Summary Report (Summary Report). We appreciate and share your interest in making the ENERGY STAR program as effective as possible. While we are not in a position to concur with the report's findings, as outlined below, we concur with the proposed recommendations in the spirit of our ongoing commitment to greater transparency and clarity around how the ENERGY STAR program operates. The ENERGY STAR Program Has a Clearly Defined Goal and Set of Operating Principles The Summary Report demonstrates a misunderstanding of the purpose of the ENERGY STAR program. Based on expert advice and experience in terms of what makes a consumer label effective, the ENERGY STAR products program has not and does not seek to achieve environmental benefits by promoting products that meet the highest energy conservation standards. Rather, the program operates under the principle that the greatest environmental impact can be achieved by affecting the broadest number of transactions, namely the purchases of the average consumer, who will choose products that are good for the environment as long as they don't cost more or involve a sacrifice in performance. Consistent with this principle, the ENERGY STAR program achieves significant results by delivering higher efficiency products to a large segment of consumers rather than targeting only (?) the most efficient products, which would appeal to a much more limited set of purchasers and likely result in less aggregate improvement in energy efficiency. OIG Response: The ENERGY STAR program, as described above in the Agency's response, does not meet the intent of the Energy Policy Act of 2005. The Act sets out that products in the ENERGY STAR program should meet "the highest energy conservation standards" - an exclusive program. As noted above, the program operates by emphasis on achieving the broadest number of ENERGY STAR transactions rather than identifying and promoting products with the highest efficiencies. We believe this emphasis by the program has led to results that are inconsistent with the intent of the Act. 23 ------- 11-P-0010 This goal, to achieve significant greenhouse gas savings by balancing energy efficiency with other consumer expectations, drives decisions about where to set energy efficiency requirements. Taking into account considerations related to consumer expectations, ENERGY STAR requirements are generally established to reflect the performance of the top 25 percent of models in a product category when the requirements go into effect. A growing market share of products meeting those requirements simply means that more consumers, as well as the environment, have benefitted from the use of more efficient products. OIG Response: Another potential consequence of "a growing market share meeting those requirements" is the corresponding growth in program metrics - greenhouse gas emissions avoided. We noted in a previous report that the integrity of these benefits become questionable when the market share of qualified products range is at a high percentage. Maximizing the number of voluntary business partners and qualified products is not a consideration in our decision making process. OIG Response: We believe this statement is inconsistent with the Agency's previous statement that the greatest environmental impact can be achieved by affecting the broadest number of transactions, namely the largest number of purchases by the average consumer, who will choose products that are good for the environment as long as they do not cost more or sacrifice performance. Demonstrated Commitment by Senior Officials at EPA and the Department of Energy to Continued ENERGY STAR Program Effectiveness On September 30, 2009, EPA and the Department of Energy (DOE) signed a new Memorandum of Understanding (MOU) to enhance and expand federal programs that advance energy efficiency. This new EPA/DOE partnership defines roles and responsibilities in a way that capitalizes on the strengths of each agency and outlines a set of key improvements that will build the value of the ENERGY STAR Program. These improvements include adding new product categories to the program, instituting new measures to ensure that ENERGY STAR specifications are tightened as necessary to consistently represent top performing products, and enhancing the qualification and verification testing of ENERGY STAR products. Through a DOE pilot program, government testing of ENERGY STAR qualified products has also stepped up. In addition, we are developing a new program to promote the top tier of products within certain categories. OIG Response: Over the series of OIG evaluations of the ENERGY STAR program, the OIG has issued several recommendations intended to strengthen management controls to protect the integrity of the ENERGY STAR label, as well as improve and enhance the ENERGY STAR program. Beginning with the first report, EPA has taken steps to address the OIG's recommendations. As highlighted in the body of the report and in Appendix B, the plans outlined in the MOU address many of our previously issued recommendations. However, we cannot make any assessments regarding the value or effectiveness of the planned enhancements because they have not been fully implemented. 24 ------- 11-P-0010 Significant Enhancements to the ENERGY STAR Product Qualification and Verification Process Have Been Finalized Despite recent investigations, the Inspector General's Offices at EPA and DOE and the Government Accountability Office have found no evidence of consumer fraud relating to the quality or performance of ENERGY STAR qualified products. EPA and the Department of Energy have extensive procedures in place today to prevent and uncover fraud and abuse, including a broad infrastructure of controls, audits and other measures to ensure that the ENERGY STAR name and logo are applied properly and consistently in the marketplace. In 2010, EPA has made dramatic changes to the way products will be qualified as part of the ENERGY STAR Program going forward. In March, EPA instituted a policy across all 60 product categories that products may no longer be labeled by manufacturers until qualifying product information, including lab test reports, is submitted and approved by EPA. In addition, the Agency suspended the automated qualification process previously in place for certain product categories and restricted access to the ENERGY STAR certification mark to partners until after EPA approves a product for qualification. On June 30, EPA finalized the requirements accreditation bodies and laboratories must meet in order to receive EPA recognition for purposes of ENERGY STAR product testing. On August 23, EPA finalized the requirements certification bodies must meet to be recognized by the Agency as a third-party certifier of ENERGY STAR qualified product performance. EPA is currently refining the eligibility criteria and partner commitments across more than 60 categories to officially impose third-party certification for all products effective December 30. OIG Response: The bulleted actions are responsive to some of the OIG's findings; however, the Agency did not address other key aspects of our report. The Agency did not provide adequate information on how EPA plans to enhance its approach to accounting for program savings, including the calculation of GHG emissions saved. Based on control weaknesses identified in a previous evaluation, we concluded that EPA cannot be certain that it's reported savings claims are valid or supportable, and that large amounts of GHG emissions are in fact being avoided. This uncertainty remains a potential program deficiency. The Agency has not provided specifics as to how these new proposed enhancements will correct weaknesses previously noted. Recommendations and Corrective Actions Recommendation 1: Develop a strategic vision that articulates whether the ENERGY STAR label represents superior energy conservation performance or a balance of voluntary partner interests and non-energy efficiency performance features. 25 ------- 11-P-0010 Corrective Action Plan The September 30, 2009 MOU between EPA and DOE reflects a mutually agreed upon strategic vision for the ENERGY STAR Products Program. Namely: "Program Design. The ENERGY STAR label will identify energy efficient products that offer meaningful energy savings (at an individual and/or national level) over those products typically purchased. Specifications will be established which overlay the consumer perspective and the need to consistently identify top performing products. Specifications will be set to recognize products that are cost-effective from the purchaser standpoint; offer at least equivalent functionality and features as standard products; and are proven and broadly available. Cost-effectiveness in terms of payback periods will be defined on a case-by-case basis, taking into account both the expected useful life of the product and the general desirability of shorter payback periods, but will in general be 3-5 years. To identify top performing products, ENERGY STAR specifications will be set to identify approximately the top 25% most efficient of models within a product class under the ENERGY STAR specification at the time that specification becomes effective, with consideration of expected improvements in product efficiency and market penetration trends of those products that will take place between establishing a specification and the specification becoming effective." This vision was subsequently integrated into the Enhanced Program Plan for ENERGY STAR Products early December, 2009 as the first important step in advancing the partnership between EPA/DOE and program stakeholders and in engaging with interested parties in a discussion of the key program enhancements outlined in the MOU. As noted above, one of the ENERGY STAR program enhancements referenced in the new EPA/DOE MOU is the addition of a top tier program nested within ENERGY STAR. The launch of this program, which is currently under development, presents an opportunity for EPA to clarify its strategic vision for the ENERGY STAR product label and how that relates to the new program. Milestone Date Program Development Phase 1: Market research and data collection Completed Program Development Phase 2: Program Design Option Development Completed Program Development Phase 3: Market Testing of Design Options September 2010 Program Development Phase 4: Final Report and Recommendations November 2010 Program Description/Materials Development (including refined articulation of strategic vision for ENERGY STAR label) December 2010 26 ------- 11-P-0010 Recommendation 2: Develop a set of goals and valid and reliable measures that can accurately inform shareholders and the public of the benefits of the program. Corrective Action Plan Consistent with recommendations made through a recent peer review of the model used to document ENERGY STAR products benefits, EPA is in the process of enhancing its approach to accounting for program savings. An important aspect of this is better articulation of program goals and documentation of a market model for key product areas. (A market model is similar to program logic but adapted to the unique aspects of market transformation programs.) Milestone Date External Peer Review on ENERGY STAR Products Benefits Calculations Completed Refined Benefits Model Removing Market Transformation Effects and Improving Consistency August 2010 Enhanced Documentation of Market Model January 2011 Phase I Migration of Benefits Model to Nationally Accepted Platform March 2011 Phase II Reassessment of Baselines for Core Products June 2011 27 ------- 11-P-0010 Appendix E Agency Comments (10/4/10) and 01G Evaluation The second response from the Assistant Administrator was received on October 4, 2010 MEMORANDUM SUBJECT: Response to Evaluation Report: ENERGY STAR Needs to Define Its Primary Goal and Consumer Expectations - A Summary Report FROM: Gina McCarthy Assistant Administrator TO: Jeffrey Harris, Director Cross Media Issues, Office of Program Evaluation Thank you for the opportunity to respond to the Evaluation Report: ENERGY STAR Needs to Define Its Primary Goal and Consumer Expectations - A Summary Report (Summary Report). We appreciate and share your interest in making the ENERGY STAR program as effective as possible. While we are not in a position to concur with the report's findings, as outlined below, we concur with the proposed recommendations in the spirit of our ongoing commitment to greater transparency and clarity around how the ENERGY STAR program operates. The ENERGY STAR Program Has a Clearly Defined Goal and Set of Operating Principles The goal of the ENERGY STAR products program is to identify and promote energy-efficient products in order to reduce energy consumption, improve energy security, and reduce pollution through voluntary labeling of products that meet the highest energy conservation standards as authorized under the Energy Policy Act of 2005 and the Clean Air Act Section 103(g). OIG Response: We disagree that the Agency has a clearly defined goal and set of operating principles. The divergence between the initial and subsequent responses highlights the competing visions of the ENERGY STAR program within EPA. The Agency has yet to decide whether the program is intended to lead the marketplace as an exclusive program (as implied in the October 4, 2010, response) or follow the marketplace as an inclusive program (as stated in the September 13, 2010, response). More than restating the language of the Energy Policy Act of 2005, EPA should design, implement, and establish internal controls that assure consumers that they are purchasing energy-efficient products that meet the highest standards. EPA applies a set of principles when implementing the program. These principles are consistent with this goal by virtue of the fact that they narrow the pool of eligible, highest-conserving products to those likely to enhance consumer acceptance and confidence in the program, thereby increasing overall reductions in energy consumption. They include: 28 ------- 11-P-0010 1) Significant energy savings can be realized on a national basis 2) Product performance can be maintained or enhanced with increased energy efficiency 3) Purchasers will recover their investment in increased energy efficiency within a reasonable time 4) Energy efficiency can be achieved with several technology options, at least one of which is non-proprietary. 5) Product energy consumption and energy performance can be measured and verified with testing 6) Labeling would effectively differentiate products and be visible for purchasers Each time an ENERGY STAR performance standard is established, whether for the first time or as part of a revision, these principles are balanced to ensure that the specified level will deliver significant aggregate energy savings while at the same time ensuring that products are cost- effective to the consumer and do not compromise functionality or performance. Considering these principles, identification and promotion of products with the highest energy conservation standards may result in setting an ENERGY STAR efficiency level that the highest 25% of models in terms of efficiency can meet because this level offers the desired amount of selection and availability while also promising significant energy savings, cost-effective options and no compromise in performance. OIG Response: The OIG is aware of the above criteria and we reported on this process at length in our initial 2007 report to the Agency and again in this report. We previously identified the specification setting and revisions process as being unclear and lacking documentation. For example, although we found several instances of products with high market shares (some in the high 90 percentile), there was no documentation to show why the specifications for these products had not been revised to a more efficient standard in a timely manner. A key provision of the program is to ensure that consumers can identify the most energy-efficient products. Inconsistent application of the criteria means that the label may not be identifying for consumers the most energy-efficient products. Furthermore, the integrity of ENERGY STAR program savings or benefits becomes questionable when qualified products account for a high overall percentage of the market. According to EPA, several actions resulting from the MOU will address these historic shortcomings. However, until these provisions are fully implemented, their effectiveness remains unknown. An increase in qualified product market share is an important measure of program success. As the market is transformed, consumer confidence in the program is maintained by updating the standards to capture additional savings. New procedures, consistent with the September 30, 2009 Memorandum of Understanding between EPA and DOE, are now in place to ensure that ENERGY STAR standards across all product categories are reviewed and updated in a timely manner so that they continue to represent the highest energy conservation standards. 29 ------- 11-P-0010 OIG Response: The integrity of these savings or benefits becomes questionable when the market share of qualified products is at a high percentage. We believe that such cases illustrate how the ENERGY STAR program has sought to maximize the number of partners and qualified products at the expense of identifying products and practices that maximize energy efficiency. Although the program delivers to consumers the message that the ENERGY STAR label designates cost-effective, energy-efficient products, this may not be the case. We recognized both within the body of this report and in Appendix B that EPA has taken some steps to address the OIG's previously reported recommendations. Included in this analysis are the plans outlined in the MOU, which appear to be, in many cases, a direct response to previous OIG recommendations. However, we cannot make any assessments regarding the value or effectiveness of these enhancements because they have neither been fully implemented nor reviewed by the OIG. Demonstrated Commitment by Senior Officials at EPA and the Department of Energy to Continued ENERGY STAR Program Effectiveness On September 30, 2009, EPA and the Department of Energy (DOE) signed a new Memorandum of Understanding (MOU) to enhance and expand federal programs that advance energy efficiency. This new EPA/DOE partnership defines roles and responsibilities in a way that capitalizes on the strengths of each agency and outlines a set of key improvements that will build the value of the ENERGY STAR Program. These improvements include adding new product categories to the program, instituting new measures to ensure that ENERGY STAR specifications are tightened as necessary to consistently represent the highest performing products, and enhancing the qualification and verification testing of ENERGY STAR products. Through a DOE pilot program, government testing of ENERGY STAR qualified products has also stepped up. In addition, we are developing a new program to promote the top tier of products within certain categories. Significant Enhancements to the ENERGY STAR Product Qualification and Verification Process Have Been Finalized Despite recent investigations, the Inspector General's Offices at EPA and DOE and the Government Accountability Office have found no evidence of consumer fraud relating to the quality or performance of ENERGY STAR qualified products. EPA and the Department of Energy have extensive procedures in place today to prevent and uncover fraud and abuse, including a broad infrastructure of controls, audits and other measures to ensure that the ENERGY STAR name and logo are applied properly and consistently in the marketplace. In 2010, EPA has made dramatic changes to the way products will be qualified as part of the ENERGY STAR Program going forward. In March, EPA instituted a policy across all 60 product categories that products may no longer be labeled by manufacturers until qualifying product information, including lab test reports, is submitted and approved by EPA. In addition, the Agency suspended the automated qualification process previously in place for certain product 30 ------- 11-P-0010 categories and restricted access to the ENERGY STAR certification mark to partners until after EPA approves a product for qualification. On June 30, EPA finalized the requirements accreditation bodies and laboratories must meet in order to receive EPA recognition for purposes of ENERGY STAR product testing. On August 23, EPA finalized the requirements certification bodies must meet to be recognized by the Agency as a third-party certifier of ENERGY STAR qualified product performance. EPA is currently refining the eligibility criteria and partner commitments across more than 60 categories to officially impose third-party certification for all products effective December 30. EPA is in the process of enhancing its approach to accounting for program savings. This effort will include better documentation of the market model (i.e. the market response assumptions), re-evaluation of baseline assumptions and migration to a nationally recognized, commercially available efficiency program evaluation model. If you have any questions, please contact Beth Craig at 202 343 9312. cc: WadeNajjum Brian Mclean Ann Bailey Jerri Dorsey 31 ------- 11-P-0010 Appendix F Distribution Office of the Administrator Assistant Administrator for Air and Radiation Agency Follow-up Official (the CFO) Agency Follow-up Coordinator General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for External Affairs and Environmental Education Audit Follow-up Coordinator, Office of Air and Radiation Inspector General 32 ------- |