U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL Improving air quality Collecting Additional Performance Data from States Would Help EPA Better Assess the Effectiveness of Vehicle Inspection and Maintenance Programs Report No. 18-P-0283 September 25, 2018 ------- Report Contributors: James Hatfield Gabrielle Fekete Dan Howard Nyquana Manning Abbreviations CAA Clean Air Act CFR Code of Federal Regulations CO Carbon Monoxide EPA U.S. Environmental Protection Agency FY Fiscal Year MOVES Motor Vehicle Emission Simulator NOx Nitrogen Oxides OIG Office of Inspector General OTAQ Office of Transportation and Air Quality VOC Volatile Organic Compound Cover Photo: Dense vehicular traffic in smog. (EPA photo) Are you aware of fraud, waste or abuse in an EPA program? EPA Inspector General Hotline 1200 Pennsylvania Avenue, NW (2431T) Washington, DC 20460 (888) 546-8740 (202) 566-2599 (fax) OIG Hotline@epa.gov Learn more about our OIG Hotline. EPA Office of Inspector General 1200 Pennsylvania Avenue, NW (2410T) Washington, DC 20460 (202) 566-2391 www.epa.gov/oiq Subscribe to our Email Updates Follow us on Twitter @EPAoig Send us your Project Suggestions ------- ^£DS7X • JL v I®/ U.S. Environmental Protection Agency Office of Inspector General At a Glance 18-P-0283 September 25, 2018 Why We Did This Project We conducted this audit to determine whether the U.S. Environmental Protection Agency's (EPA's) oversight has ensured that vehicle inspection and maintenance programs are effective and efficient in reducing vehicle emissions in enhanced inspection and maintenance areas. Cars and light-duty trucks are significant sources of pollutant emissions to the nation's air. Some of these pollutants chemically react in sunlight to form harmful ground-level ozone. Vehicles that are poorly maintained or have malfunctioning emission controls significantly contribute to these pollutants. To address this problem, the Clean Air Act (CAA) made vehicle inspection and maintenance programs mandatory for areas where air quality does not meet health- based standards set by the EPA. The CAA requires areas with the most severe air quality problems to implement more stringent programs. As of June 2018, 23 states run these enhanced programs. This report addresses the following: • Improving air quality. Send all inquiries to our public affairs office at (202) 566-2391 or visit www.epa.gov/oiq. Listing of OIG reports. Collecting Additional Performance Data from States Would Help EPA Better Assess the Effectiveness of Vehicle Inspection and Maintenance Programs What We Found The EPA should collect additional program Inspection and maintenance performance data to better assess the programs help improve air effectiveness of enhanced inspection and quality and protect human maintenance programs for reducing vehicle health by identifying emissions. For example, nine states operating vehicles in need of repair, enhanced programs did not conduct the required biennial program evaluations to assess the effectiveness of their programs in reducing vehicle emissions. Another four state programs did not conduct the CAA-required on-road testing to obtain information about the performance of in-use vehicles. For three states, a lack of clarity in the EPA's guidance on program evaluation and on-road testing methodologies contributed to states' failure to conduct required reviews and tests. As a result of states failing to meet the program evaluation and/or on-road testing requirements, the EPA lacks data to determine the effectiveness of their enhanced vehicle inspection and maintenance programs. The EPA lowered the National Ambient Air Quality Standards for ozone in 2015, which will require states to further reduce emissions of ozone precursors. Knowing the effectiveness of these programs in reducing emissions will be critical for states as they plan how to comply with the new standard. In addition, states are required to submit annual reports about the performance of their vehicle inspection and maintenance programs to the EPA. The agency strengthened its oversight of these annual reports since we issued our report on the vehicle inspection and maintenance program in 2006. However, further improvements should be made. For example, in our current audit, we found that the EPA did not consistently communicate errors in reports back to states. Also, three states continued to report a significant number of repair waivers for vehicles that failed emission tests. Recommendations and Planned Agency Corrective Actions We recommend that the Assistant Administrator for Air and Radiation take actions to assure consistent and effective implementation of enhanced inspection and maintenance programs; determine why states did not conduct mandated on-road testing and assist these states, as necessary, to comply with statutory requirements; and conduct outreach to states with deficiencies in program implementation and issue guidance to address any common problems identified. The EPA agreed with our recommendations and provided acceptable corrective actions and completion dates. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 THE INSPECTOR GENERAL September 25, 2018 MEMORANDUM SUBJECT: Collecting Additional Performance Data from States Would Help EPA Better Assess the Effectiveness of Vehicle Inspection and Maintenance Programs Report No. 18-P-0283 FROM: Arthur A. Elkins Jr. TO: William Wehrum, Assistant Administrator Office of Air and Radiation This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). The project number for this audit was OPE-FY17-0018. This report contains findings that describe the problems the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. The EPA office responsible for responding to issues in this report is the Office of Air and Radiation's Office of Transportation and Air Quality. The Office of Transportation and Air Quality collaborates with the Air Divisions in EPA regional offices to oversee state implementation of inspection and maintenance requirements. In accordance with EPA Manual 2750, your office provided acceptable corrective actions and estimated completion dates in response to OIG recommendations. All recommendations are resolved and no final response to this report is required. However, if you submit a response, it will be posted on the OIG's website, along with our memorandum commenting on your response. Your response should be provided as an Adobe PDF file that complies with the accessibility requirements of Section 508 of the Rehabilitation Act of 1973, as amended. The final response should not contain data that you do not want to be released to the public; if your response contains such data, you should identify the data for redaction or removal along with corresponding justification. We will post this report to our website at www.epa.gov/oig. ------- Collecting Additional Performance Data from States Would Help EPA Better Assess the Effectiveness of Vehicle Inspection and Maintenance Programs 18-P-0283 Table of C Chapters 1 Introduction 1 Purpose 1 Background 1 Responsible Offices 5 Scope and Methodology 6 Prior OIG Reports 7 2 Collection and Review of Required Performance Data Would Enhance EPA's and States' Ability to Assess the Impact of Inspection and Maintenance Programs 9 EPA Regulations and Guidance Require Biennial Program Evaluations and On-Road Testing 9 Biennial Program Evaluations Were Not Conducted by Many Enhanced Inspection and Maintenance States 11 Mandated On-Road Testing Was Not Fully Executed 13 Conclusion 14 Recommendations 15 Agency Response and OIG Evaluation 15 3 Additional EPA Oversight Would Improve Quality and Consistency of State Reports 16 EPA Regulations Require Annual Reporting of State Vehicle Inspection and Maintenance Data 16 Regions Did Not Always Discuss Errors in Reports with States 17 Four States Continued to Experience Deficiencies in Program Performance 18 Conclusion 19 Recommendations 20 Agency Response and OIG Evaluation 20 Status of Recommendations and Potential Monetary Benefits 21 Appendices A Agency's Response to Draft Report 22 B Distribution 27 ------- Chapter 1 Introduction Purpose The Office of Inspector General (OIG) for the U.S. Environmental Protection Agency (EPA) conducted this audit to determine whether EPA oversight has ensured that vehicle inspection and maintenance programs are effective and efficient in reducing vehicle emissions in enhanced inspection and maintenance areas. Background Mobile sources are a major contributor to air pollution in the United States. Mobile sources generate pollution through internal combustion engines that burn gasoline, diesel and other types of fuels. Emissions from on-road mobile sources, such as cars and light-duty trucks, account for a significant amount of three key pollutants in our nation's air: carbon monoxide (CO), volatile organic compounds (VOCs) and nitrogen oxides (NOx). NOx and VOCs react in sunlight to form harmful ground-level ozone. An abundance of ozone near the earth's surface is harmful to humans. Vehicles that are poorly maintained or have malfunctioning emission controls are significant contributors to these key pollutants, especially in major urban areas. Increased NOx emissions result in increased levels of smog and acid rain, as well as increased cases of lung disease, reduced lung capacity, asthma attacks, bronchitis and premature death. Increased exposure to CO emissions can lead to headaches, dizziness, impaired vision, fatigue and chest pain. The health effects of VOCs can include irritation to the eyes, nose and throat, as well as damage to the kidney, liver and central nervous system. Breathing ozone can trigger a variety of health problems particularly for children, the elderly, and people of all ages who have lung diseases such as asthma. Based on the 2014 National Emissions Inventory data, which are the most recent available and can be found on the EPA's website, mobile sources account for 60 percent of the total NOx emissions and 57 percent of the total CO emissions. The majority of these mobile source emissions are from on-road sources. Lowering emission limits and strengthening compliance and enforcement practices are essential for achieving low, real-world emissions. Vehicles idled in dense traffic. (EPA photo) 18-P-0283 1 ------- Enhanced Inspection and Maintenance Programs Implemented in 23 States Across the Nation To address the health effects associated with emissions exposure, the Clean Air Act (CAA), as amended by the CAA Amendments of 1990, require inspection and maintenance programs in several areas across the nation. Inspection and maintenance programs help to improve air quality by identifying cars and trucks that have high emissions and/or need repairs. Owners and operators of vehicles with high emissions are notified to make repairs so that their vehicles can be repaired and retested to verify that emissions are within legal limits. For on-road mobile sources, the inspection and maintenance program is one of the main methods used to address mobile source pollution in urban areas where air pollution exceeds safe levels. Depending on an area's air quality status and location, a state may be required to implement a "basic" or a more stringent "enhanced" inspection and maintenance program in that area or across the entire state. Enhanced inspection and maintenance programs are required in the following areas: • All serious, severe or worse ozone nonattainment areas with a 1980 urban population of 200,000 or more. A nonattainment area is an area considered to have air quality worse than the National Ambient Air Quality Standards of the CAA. • Metropolitan statistical areas with a 1990 population of 100,000 or more in the Ozone Transport Region. The Ozone Transport Region includes a group of northeast states that must reduce emissions that contribute to ozone pollution, irrespective of their air quality classification. • All moderate or higher CO nonattainment areas with a 1980 urban population of 200,000 or more. Enhanced programs include a requirement to also test light-duty trucks, while basic programs are only required to test light-duty cars. The 23 states1 that implement CAA enhanced inspection and maintenance programs must meet a performance standard that includes on-board diagnostic testing,2 tailpipe 1 The term "states," as used in this report, includes the District of Columbia. States with enhanced programs are Arizona, California, Colorado, Connecticut, Delaware, District of Columbia, Georgia, Illinois, Indiana, Louisiana, Maryland, Massachusetts, Nevada, New Hampshire, New Jersey, New York, Ohio, Oregon, Pennsylvania, Rhode Island, Texas, Virginia and Wisconsin. Oregon self-identifies as an enhanced inspection and maintenance program area while not meeting the statutory or regulatory criteria that would mandate an enhanced inspection and maintenance program. 2 An on-board diagnostic inspection and maintenance check involves two types of examinations: (1) a visual check of the dashboard display function and status (also known as the Malfunction Indicator Light and/or bulb check); and (2) an electronic examination of the on-board diagnostics computer. Most 1996 and newer vehicles are tested using on-board diagnostics. Tailpipe tests are used primarily on older vehicles. 18-P-0283 2 ------- emissions testing, on-road testing and an inspection to detect tampering. Figure 1 shows the states with enhanced inspection and maintenance programs. Figure 1: States with enhanced inspection and maintenance programs Source: EPA OIG. States are required to submit annual report data obtained from four main areas— test data, quality assurance, quality control and enforcement—to EPA regions. States must also issue biennial reports, which should include the weaknesses or problems identified in the program within the 2-year reporting period, the steps already taken to correct those problems, the results of those steps, and any future efforts planned. In addition, states with enhanced inspection and maintenance programs are also required to submit program evaluations on a biennial basis. These program evaluations are required to assess the emission reductions achieved by the program and use EPA-approved evaluation methods. In 2015, the EPA lowered the National Ambient Air Quality Standards for ozone, which in turn will require states to further reduce emissions of ozone precursors. Based on its review of the air quality criteria for ozone, the EPA lowered primary and secondary ozone standard levels from 0.075 to 0.070 parts per million. This tightened standard highlights the importance of having states demonstrate the effectiveness of their programs in achieving planned emission reductions. Requirements for Implementing Enhanced Inspection and Maintenance Programs The CAA Amendments of 1990 made enhanced inspection and maintenance programs mandatory in serious ozone nonattainment areas and urbanized areas States implementing enhanced inspection and maintenance programs 18-P-0283 3 ------- within the Ozone Transport Region across the country. The EPA implements enhanced inspection and maintenance programs under the CAA through a series of regulations and guidance documents designed to address key programmatic requirements. States are required to incorporate their vehicle inspection and maintenance programs into their CAA State Implementation Plans, which describe the various air pollution control measures and activities that the states will implement to meet national air quality standards. CAA Amendments For serious ozone nonattainment areas and urbanized areas within the Ozone Transport Region, the CAA requires that states revise their applicable implementation plans to outline how they plan to reduce VOC and NOx emissions from in-use motor vehicles registered within each urbanized area in the nonattainment area or Ozone Transport Region. The CAA also provides a list of minimum required elements for each state program. These elements include the following: • Computerized emission analyzers, such as on-road testing devices. • Inspection of emission control diagnostic systems. • Program operation on a centralized or decentralized basis. • Annual emissions testing. The CAA also requires that each state with an enhanced inspection and maintenance program evaluate program effectiveness biennially. Specifically, the CAA requires that each state submit a biennial report to the EPA. The report is required to assess emission reductions achieved by the program based on data collected during the inspection and repair of vehicles. Implementing Regulations The vehicle inspection and maintenance rule is found in 40 CFR Part 51 Subpart S. Specific requirements for the enforcement, data collection, analysis and reporting of inspection and maintenance programs are found in 40 CFR § 51.366, which states the following: Data analysis and reporting are required to allow for monitoring and evaluation of the program by program management and EPA, and shall provide information regarding the types of program activities performed and their final outcomes, including summary statistics and effectiveness evaluations of the enforcement mechanism, the quality assurance system, the quality control program, and the testing element. 18-P-0283 ------- Per 40 CFR § 51.366, on an annual basis, states with inspection and maintenance programs are required to submit information related to test data (basic program statistics), quality assurance, quality control and enforcement to the EPA. In addition to the annual state reports, 40 CFR § 51.366(e) requires submission of biennial reports that address any changes made in program design, as well as any weaknesses or problems identified within the 2-year reporting period. Furthermore, 40 CFR § 51.353 codifies the CAA requirement for enhanced inspection and maintenance programs to conduct a program evaluation and report the results on a biennial basis. Also, 40 CFR § 51.371 requires that 0.5 percent of the vehicles in the inspection and maintenance program area—or 20,000 vehicles, whichever is less—be tested using either remote-sensing devices or roadside pullovers that include tailpipe or on-board diagnostic tests. EPA Guidance To address requirements set by the CAA and 40 CFR Part 51, the EPA provided guidance in 1998, 2001 and 2004 related to the methods that states could use to address program evaluation requirements for the inspection and maintenance program.3 The EPA issued its most recent guidance in July 2004 in Guidance on Use of Remote Sensing for Evaluation ofI/M [Inspection and Maintenance] Program Performance (EPA420-B-04-010). According to the EPA, the focus of the 2004 guidance document was to address the agency's concerns related to program evaluation methods based on remote-sensing devices. The July 2004 guidance document addresses equipment specifications, site selection and data collection, and it outlines and explains the advantages and limitations of each remote-sensing device analysis methodology. To incorporate the most up-to-date practices into evaluations, the guidance strongly recommends that states considering the use of remote-sensing devices work closely with their respective EPA regional office and the EPA's Office of Transportation and Air Quality (OTAQ). The guidance also states that an evaluation based on both remote-sensing devices and data collected from the inspection and maintenance emission test will provide a more accurate estimate of overall program performance than simply relying on one method alone. Responsible Offices Located within the Office of Air and Radiation, OTAQ is responsible for the national implementation of the inspection and maintenance program. OTAQ develops national policies on mobile source emission control, determines the 3 Inspection and Maintenance (I/M) Program Effectiveness Methodologies, EPA, Office of Air and Radiation, EPA-420-S-98-015, October 1998; and Draft Guidance on Use ofIn-Program Data for Evaluation of I/M Program Performance, EPA-420-P-01-003, August 2001. 18-P-0283 5 ------- contribution of mobile sources to pollutant emission inventories, assesses the effectiveness of various inspection and maintenance tests used to identify vehicles in need of repair, and provides technical assistance as needed. EPA regional offices have oversight responsibility for state inspection and maintenance programs. EPA regional offices must verify that the programs meet their respective State Implementation Plan commitments and include required reporting. Scope and Methodology We conducted our audit from June 2017 through July 2018. We conducted this audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our objective. To understand requirements for the inspection and maintenance program, we reviewed the following statutes, policies and guidance: • The CAA, as amended. • 40 CFR Part 51 Subpart S, Inspection/Maintenance Program Requirements. • EPA guidance from 1998, 2001 and 2004 related to evaluating inspection and maintenance program performance. • Motor Vehicle Emission Simulator (MOVES2014) and MOVES2014a Technical Guidance: Using MOVES to Prepare Emission Inventories for State Implementation Plans and Transportation Conformity, EPA-420-B- 15-093, November 2015. • Performance Standard Modeling for New and Existing Vehicle Inspection and Maintenance (I/M) Programs Using the MOVES Mobile Source Emissions Model, EPA-420-B-14-006, January 2014. • The EPA's Office of Air and Radiation National Program Manager Guidance, Fiscal Year (FY) 2016-2017, as well as the Final FY 2017 Office of Air and Radiation National Program Manager Guidance Addendum. To determine how the EPA oversees the inspection and maintenance program, we interviewed staff and managers from OTAQ's Ann Arbor, Michigan, and Washington, D.C., offices. We conducted the interviews to identify what policies, procedures and guidance documents the EPA has issued for the program; what controls the EPA has established over the program; how the EPA oversees the 18-P-0283 6 ------- program and its policies and procedures; and what information the EPA collects and reviews for state programs. We emailed written questionnaires to nine of the 10 EPA regions. These nine regions have states with enhanced inspection and maintenance programs, and we contacted them to obtain regional perspectives. The only EPA region that did not receive the questionnaire is Region 7, which has no states with enhanced programs. Responses to the questionnaires provided information about the background and oversight of the inspection and maintenance program, the determination of any issues or concerns, and the quantification of inspection and maintenance program air benefits. In addition, we contacted four of the regions to determine whether they provided feedback to states relative to OTAQ summary reports and whether they discussed specific inspection and maintenance problem areas with the states. We reviewed state annual and biennial reports for the 23 states with enhanced inspection and maintenance programs. We wanted to determine whether the state reports identified significant problems and met EPA reporting requirements under 40 CFR §§ 51.366 and 51.353(c). We also reviewed data provided in the reports and information related to emission benefits of the program. We interviewed representatives from the environmental departments of five states with enhanced inspection and maintenance programs to determine the states' perspectives on enhanced inspection and maintenance programs, including EPA oversight and how the states estimate emission reductions from the program. We selected states for follow-up interviews with the goal of obtaining a sample that covered different program types and qualities, and that included centralized and decentralized programs as well as annual and biennial programs. Our sample also included a variety of regions and pollution profiles. We also emailed two specific questions to 14 states that were not initially interviewed to determine whether the states were (1) conducting biennial program evaluations that address the 40 CFR § 51.353(c) program evaluations and (2) using remote sensing and/or roadside testing.4 We asked each region for the most recent biennial program evaluation report from each state. Not all states had completed a biennial report for the most recent year; thus, we received and reviewed reports from 2013 through 2017. Prior OIG Reports EPA OIG Report No. 2007-P-00001. EPA's Oversight of the Vehicle Inspection and Maintenance Program Needs Improvement, issued October 5, 2006, addressed EPA oversight of inspection and maintenance programs. We reported 4 We did not send the questions to four of the 18 states not initially interviewed because those states provided information on program evaluations and on-road testing in their annual or biennial reports. 18-P-0283 7 ------- that the EPA had not obtained sufficient information to verify that states were meeting their inspection and maintenance program commitments. From 1999 through 2004, 11 of the 34 inspection and maintenance programs submitted timely reports, 14 programs never submitted the required reports or the regions were unsure whether these states submitted the reports, and four programs submitted reports that were not timely. Our 2006 report included nine recommendations to the Assistant Administrator for Air and Radiation. The EPA agreed with the recommendations after one revision, and the agency completed all corrective actions on September 5, 2008. OTAQ and EPA regions made several improvements related to reporting required test data for state inspection and maintenance programs because of the 2006 report. The EPA implemented corrective actions that included the development of a checklist for 40 CFR § 51.366 reporting requirements, which states can use to prepare their annual submissions. The EPA also established an annual workshop to review annual inspection and maintenance state reports, as well as discuss any identified deficiencies, best practices or anomalies. EPA OIG Report No. 18-P-0181. EPA Did Not Identify Volkswagen Emissions Cheating; Enhanced Controls Now Provide Reasonable Assurance of Fraud Detection, issued May 15, 2018, addressed the EPA's internal controls over the on-road light-duty vehicle program. We conducted this audit to determine whether, in light of the 2015 Volkswagen emissions fraud case, the EPA has internal controls that can effectively detect and prevent on-road light-duty vehicle emissions fraud. Effective internal controls provide reasonable, though not absolute, assurance that the potential for fraud is minimized. After uncovering Volkswagen's emissions fraud, the EPA's light-duty vehicle compliance program added controls to effectively detect and prevent noncompliance—a precursor to potential fraud. Our 2018 report made seven recommendations to the Assistant Administrator for Air and Radiation, which involve defining performance measures; conducting a formal risk assessment; formalizing the role of special testing; tracking compliance issues; better using remote sensing and other data sources; updating email inboxes maintained for feedback; and developing protocols for sharing information with the EPA's regulatory partner, the California Air Resources Board. The EPA agreed with all of our recommendations and provided acceptable corrective actions, two of which are completed. 18-P-0283 8 ------- Chapter 2 Collection and Review of Required Performance Data Would Enhance EPA's and States' Ability to Assess the Impact of Inspection and Maintenance Programs The EPA lacked the required performance data to assess the effectiveness of many state inspection and maintenance programs. Nine states with enhanced inspection and maintenance programs did not conduct required biennial program evaluations to assess the effectiveness of their programs in reducing vehicle emissions, and five states conducted limited evaluations that did not fully comply with regulatory requirements. Further, four programs had not conducted required on-road testing of vehicles, which provides information on the effectiveness of a vehicle's emission controls during real-world driving conditions. The availability of resources and other factors can impact a state's ability to conduct program evaluations and on-road testing. Three states said that EPA guidance was not clear on what assessment methodologies states should use, since the inspection and maintenance program has evolved from tailpipe testing to analyzing on-board diagnostic systems. As a result of these states not meeting the program evaluation and/or on-road testing requirements, the EPA lacks important program performance data to determine the overall effectiveness of many enhanced inspection and maintenance programs. Since the EPA increased the stringency of the National Ambient Air Quality Standards for ozone in 2015, the effectiveness of enhanced inspection and maintenance programs in reducing emissions of ozone precursors will become more critical to attaining the national air quality standard for ozone. EPA Regulations and Guidance Require Biennial Program Evaluations and On-Road Testing The program evaluation requirement in 40 CFR § 51.353(c) states that enhanced inspection and maintenance programs shall include an ongoing evaluation to quantify the emission-reduction benefits of the program.5 One goal of the program evaluation requirement was to determine the extent to which emission reductions projected for the program in the State Implementation Plan were being achieved. According to 40 CFR § 51.353(c)(2), the evaluation shall be considered in establishing actual emission reductions achieved from inspection and maintenance for the purpose of satisfying the requirements of Sections 182(g)(1) and 182(g)(2) 5 Per 40 CFR § 51.353(c)(5), there is one exception: Areas that qualify for and choose to implement an Ozone Transport Region low-enhanced inspection and maintenance program, as established in 40 CFR § 51.351(h), and that claim in their State Implementation Plan less emission-reduction credit than the basic performance standard for one or more pollutants are exempt from the requirements of 40 CFR § 51.353 (c)(1) through (c)(4). 18-P-0283 9 ------- of the CAA, which relate to reductions in emissions and compliance demonstration. Further, according to the EPA's 2001 amendments to the vehicle inspection and maintenance program requirements,6 the agency does not believe that switching to an on-board, diagnostic-based inspection prevents a state from evaluating the inspection and maintenance program's overall effectiveness. The EPA also provided guidance in 1998, 2001 and 2004 related to the methods that states could use to address the requirement for program evaluati ons for the inspection and maintenance program. Although the EPA provided guidance on approved methodologies for evaluating enhanced inspection and maintenance programs, those methodologies were not formally approved through a Federal Register Notice or other agency action. According to the EPA, it provided guidance to implement this requirement to provide states with options and flexibility regarding the program evaluation methods they adopt as part of their State Implementation Plans. The CAA requires that enhanced inspection and maintenance programs include on-road testing to supplement periodic testing requirements. Specifically, 40 CFR § 51.371 requires that 0.5 percent of the fleet—or 20,000 vehicles, whichever is less—be tested using either remote-sensing devices (Figure 2) or roadside pullovers. Further, according to 40 CFR § 51.371(a)(3): The on-road testing program shall provide information about the performance of in-use vehicles, by measuring on-road emissions through the use of remote sensing devices or by assessing vehicle emission performance through roadside pullovers including tailpipe or evaporative emission testing or a check of the on-board diagnostic (OBD) system for vehicles so equipped. Figure 2: Remote-sensing device emissions testing illustration The machines circled in red show the use of infrared scanning technology to measure emissions, speed and acceleration. A camera records the license plate number, which can be matched to state vehicle registries. Source: EPA OIG image. 6 Amendments to Vehicle Inspection Maintenance Program Requirements Incorporating the Onboard Diagnostic Check, 66 Fed. Reg. 18,156 (April 5, 2001). 18-P-0283 10 ------- The use of remote-sensing devices is one option available for meeting the requirement for on-road testing. According to OTAQ, the ability to gather a large amount of data relatively quickly is what makes remote sensing a useful technology for fleet characterization and program evaluation. However, individual vehicle emission measurements can be imprecise due to the remote-sensing device capturing emissions from other vehicle or other factors. Biennial Program Evaluations Were Not Conducted by Many Enhanced Inspection and Maintenance States Of the 22 states that implement enhanced inspection and maintenance programs and that are also required to perform biennial program evaluations,7 14 did not conduct biennial program evaluations that included estimates of the emission reduction benefits of the program, as prescribed by regulation. Of these 14 states, nine did not conduct any evaluation, while five conducted evaluations solely based on modeling instead of actual emission measurements (Figure 3).8 While modeling provides information related to emissions, it does not establish actual emission reductions achieved from the inspection and maintenance program. Modeling as an evaluation tool cannot be used as the sole method for addressing the requirements of Sections 182(g)(1) and 182(g)(2) of the CAA, which relate to reductions in emissions and compliance demonstration, as required by 40 CFR § 51.353(c)(2). The implementing regulation for inspection and maintenance program evaluations caused confusion for three states. Paragraph 40 CFR § 51.353(c)(3) describes the program evaluation requirement and includes a cross reference to another paragraph for a description of the minimum program items. However, the referenced paragraph was marked "reserved" and provided no additional information.9 OTAQ stated that the reference should be removed, 7 According to the EPA, although 23 states implement enhanced inspection and maintenance programs. New Hampshire is exempt from the program evaluation requirement based on a 2001 final rule approving the CAA State Implementation Plan revision. 8 The EPA and states use computer modeling to estimate air pollution emissions from mobile sources. The EPA created MOVES as a model for estimating emissions from all on-road vehicles including cars, trucks, motorcycles and buses. The MOVES model is based on the analysis of millions of emission test results and considerable advances in the EPA's understanding of vehicle emissions. 9 In 40 CFR § 51.353(c)(3), the requirements for an evaluation program are described. Paragraph (c)(3) references paragraph (b)(1) of the same section for a list of minimum program items. However, paragraph (b)(1) is marked "reserved" and provides no information on minimum program items. Figure 3: State performance of biennial program evaluations a 8 states conducted evaluations in accordance with the CFR A 9 states did not conduct evaluations 5 states conducted limited evaluation Source: OIG analysis of state data. a Of the 23 states implementing enhanced inspection and maintenance programs, 22 are required to perform biennial program evaluations. 18-P-0283 11 ------- and the office plans to correct the regulation the next time the EPA revises the inspection and maintenance rule. OTAQ also told us that the other requirements cited in the implementing regulation for program evaluations still apply. Three states with enhanced inspection and maintenance programs indicated confusion with this "reserved" paragraph and/or program guidance. One state noted the following: While 40 CFR 51.353(c)(3) indicates the program evaluation is to include items in 40 CFR 51.353(b)(1) and be based on a program evaluation methodology for OBD [on board diagnostic] emissions testing approved by EPA, paragraph 40 CFR 51.353(b)(1) has not been promulgated by EPA, and EPA has not approved an evaluation methodology. Another state said, "The minimum program evaluation requirements referenced at 40 CFR 51.353(c)(3) are no longer in federal code." Further, responses from states indicated confusion with the EPA's program evaluation guidance. According to OTAQ, the EPA has maintained since the release of its 2004 program evaluation guidance that the guidance was sufficient for on-board diagnostic-based programs to meet the biennial program evaluation requirement for enhanced inspection and maintenance programs. However, it does not appear that all states clearly understood this position. For example, the State Implementation Plan from one state said, "When EPA determines a testing methodology for programs without tailpipe testing, the program evaluation will meet the approved minimum requirements." In addition, the EPA has not established a specific metric in its National Program Manager Guidance for the submittal and review of program evaluations. The only National Program Manager Guidance metric directly related to inspection and maintenance programs refers to the speed with which regions review all submitted state reports.10 This metric does not address the content of the reports or the inspection and maintenance program's biennial program evaluation requirements under 40 CFR § 51.353(c). When states do not conduct program evaluations, the EPA and states do not have empirical evidence to determine whether the inspection and maintenance program is achieving its projected emission reductions. This lessens the EPA's assurance that the programs are achieving the anticipated emission reductions and air quality improvements projected for those programs. Further, in the absence of these reports, deficiencies in the program can go unidentified and uncorrected. The effectiveness of these programs will become even more critical as the EPA 10 According to the Final FY 2017 Office of Air and Radiation National Program Manager Guidance Addendum, Publication Number: 440B16001, May 6, 2016, Office of Air and Radiation, EPA regions are to review inspection and maintenance reports submitted by states for existing programs (including on-board diagnostics) within 4 months of submittal. The FY 2017 target was 100 percent. 18-P-0283 12 ------- implements the 2015 ozone standard, which is more stringent than the 2008 standard. Mandated On-Road Testing Was Not Fully Executed In addition to program evaluation requirements not being met, not all states conducted the required on-road testing. Four of the 23 states with enhanced inspection and maintenance programs did not conduct the statutorily mandated remote-sensing device or roadside pullover (on-road) testing. Since direct tailpipe measurements are not a part of on-board diagnostic emissions testing, if states do not conduct tailpipe tests via roadside pullovers or do not conduct on-road remote sensing, the states will lack on-road emissions data to assess the accuracy or effectiveness of the on-board testing results. According to the agency, it is important to have both on-board diagnostics and on-road testing. For example, an on-board diagnostics-equipped vehicle can have a valid "Check Engine" light illuminated for causes that traditional remote sensing cannot detect, such as an evaporative emission leak. On-road testing provides states with different types of information than these on-board diagnostics, including testing vehicles over a range of driving conditions (rather than only the conditions specified in the inspection and maintenance test) and evaluating the extent to which owners are repairing their vehicles prior to emissions testing. States are required to outline their vehicle inspection and maintenance program requirements, including on-road testing, in their State Implementation Plans and submit them to the EPA for approval. Although 40 CFR § 51.371 allows states to use a variety of testing methodologies, most states we surveyed opted to use remote-sensing technologies. The four states that did not conduct on-road testing either did not comply with the on-road testing program provisions in their State Implementation Plans or their plans were inadequate when originally approved by the EPA regions. The 2011 plan revision from one state reported that remote- sensing technology would be of low value to identify vehicles that do not meet emission standards or to exempt vehicles from inspection. Another state's 2009 plan narrative said that the use of remote sensing or road-side emissions measurement is not an accurate way to determine whether on-board diagnostic systems are operating correctly. In addition, two states with enhanced inspection and maintenance programs commented that remote sensing is an expensive and potentially inaccurate exercise. For example, one state concluded that such testing would be unlikely to improve vehicle emission estimates given the data and capabilities of the EPA's modeling software, and it therefore would not justify the additional costs needed to conduct such testing. With regard to remote-sensing devices, another state noted, "The RSD study measurements can be influenced by multiple factors 18-P-0283 13 ------- including wind, the amount of moisture in the air, and residual emissions from a prior vehicle that passed the sensors." There are states that use remote sensing to review the effectiveness of their inspection and maintenance programs. We identified six states that used remote- sensing data for program analysis, such as assessing the average emissions by model year, comparing emissions of vehicles that are in the inspection and maintenance area with emissions of vehicles outside the area, determining average emission rates, and estimating the participation rate of vehicles subject to inspection and maintenance testing. As demonstrated by a recent EPA enforcement action, standardized emissions testing may not accurately reflect a vehicle's emissions during real driving conditions.11 An on-road testing program could help identify these instances. For example, it was found upon retrospective analysis that emissions anomalies in Volkswagen vehicles were captured in remote-sensing data from Colorado prior to the third-party study that first alerted the EPA to the high levels of emissions. The data showed that standardized emission tests for those vehicles were not providing an accurate representation of on-road emissions. Without on-road testing, states are not complying with statutory requirements, and they are not able to link on-road testing to their program effectiveness evaluation, especially in on-board diagnostic-only program areas lacking other sources of vehicle emission data. Conclusion When states do not conduct program evaluations and on-road testing, they are failing to meet applicable statutory and regulatory inspection and maintenance requirements. This means that the EPA and states lack complete program performance data to determine whether the program is achieving projected emission reductions. They also do not have a complete picture of the overall performance of the programs in question. Specifically, without on-road testing data to provide real-world context for on- board diagnostic testing, the emissions inventory and vehicle inspection and maintenance program modeling are incomplete. Having an incomplete picture of how well the program is performing, in turn, inhibits the states' ability to plan effectively for meeting the applicable ozone standards, including the more stringent National Ambient Air Quality Standard for ozone. The effectiveness of 11 On September 18, 2015, the EPA alleged that Volkswagen had installed software in vehicles containing 2.0-liter diesel engines that sensed when the vehicle was being tested for compliance with EPA emission standards. This software, known as a "defeat device," allowed Volkswagen's vehicles to "emit up to 40 times more pollution than emission standards allow" during normal operation, while giving the appearance of passing regulatory testing. The enforcement actions against Volkswagen led to multiple settlements, including the largest fine ever imposed on an automaker. As of January 2017, the U.S. settlement against Volkswagen resulted in $4.3 billion in civil and criminal penalties and the indictment of six Volkswagen executives and employees for their roles in the case. 18-P-0283 14 ------- these programs in reducing ozone precursors will become more critical to obtaining the emission reductions needed to achieve air quality improvements. Recommendations We recommend that the Assistant Administrator for Air and Radiation: 1. Develop and distribute a supplement to the existing National Program Manager Guidance metrics to specifically identify state vehicle inspection and maintenance reports for regions to review. Include biennial program evaluation reports and verify that report quality complies with statutory requirements. 2. Confirm that biennial program evaluation reports are submitted by states with enhanced inspection and maintenance programs and that EPA regions review the reports, sharing best practices and providing additional clarification. 3. Revise the vehicle inspection and maintenance rule to remove the cross reference to Title 40 § 51.353(b)(1) of the Code of Federal Regulations, and provide defined evaluation methodology guidance to enable states to quantify emission reductions. 4. Verify whether states are performing mandatory on-road testing, or determine the reason why they are not and offer assistance to obtain compliance. Agency Response and OIG Evaluation The agency concurred with the recommendations and provided acceptable planned corrective actions and completion dates. All recommendations in this chapter are resolved. In addition to a response to our recommendations, the agency provided technical comments on the draft report. Based on the agency response and technical comments received, we made revisions to the report where appropriate. Appendix A contains the agency's response to the draft report. 18-P-0283 15 ------- Chapter 3 Additional EPA Oversight Would Improve Quality and Consistency of State Reports The EPA improved its oversight of the vehicle inspection and maintenance annual reporting process since our 2006 report was issued (see "Prior OIG Reports" section in Chapter 1 for more details). However, additional actions can be taken to further improve the effectiveness of the annual reporting oversight. We identified the following oversight concerns during this current audit: • The EPA did not always inform state programs of errors identified in state reports. • Not all program deficiencies in the report review process have been resolved. If these deficiencies in program implementation are not corrected, there is an increased risk that these programs will not achieve the projected ozone emission reductions included in their implementation plans. EPA Regulations Require Annual Reporting of State Vehicle Inspection and Maintenance Data All states with inspection and maintenance programs are required to submit annual reports on the implementation of their vehicle inspection and maintenance programs to the EPA. As described in 40 CFR § 51.366, the reports should include results for several key program performance indicators, such as the number of vehicles failing their inspection and maintenance test, the number of failed vehicles given a waiver, and the number of failed vehicles with no known final outcome. The reports are intended to help program managers and the EPA monitor and evaluate program performance. Our 2006 report noted that almost a quarter of the states with inspection and maintenance programs had not submitted required annual reports. In response to our 2006 report, the EPA developed and implemented a more rigorous oversight process that has resulted in all states submitting the required annual reports. EPA regional offices now review annual reports for completeness, red flags, inconsistencies or other indicators of possible problems using a checklist developed by OTAQ and regional inspection and maintenance contacts. After the regional reviews are completed, OTAQ conducts a national workshop to discuss the results. Once OTAQ has held the annual workshop and collected input from EPA regional offices about their findings, a summary report is drafted and circulated internally within the EPA. The workshop and subsequent report are intended to identify trends in inspection and maintenance program 18-P-0283 16 ------- implementation, with a focus on shared areas of deficiency that suggest a need for more national guidance. If applicable, the EPA can also use the workshop to identify and highlight best practices to share with other state inspection and maintenance programs. Figure 4 illustrates this process. Figure 4: Annual reporting process State submits annual report to EPA region EPA regions review annual reports OTAQ prepares summary report EPA regions and OTAQ discuss reports and results Source: OIG analysis. As part of this current audit, we compared the states' annual reporting results with the final state program results reported by OTAQ in its annual summary report. We also contacted applicable EPA regions to determine whether they discussed with their states any problems or reporting errors during the EPA's review of annual reports. Regions Did Not Always Discuss Errors in Reports with States 12 of 23 state reports differed from OTAQ reports We compared state reports to the OTAQ summary report and identified 12 state reports that contained discrepancies from the summary report (Figure 5).12 The differences occurred in a variety of program performance indicators, including failure rates. The differences ranged from very minor numbers to millions of vehicles. Figure 5: Graphic depicting the ratio of state reports that differed from OTAQ reports Two of the four EPA regions we contacted told us that they discussed the results of the summary review process with their states. However, due to a lack of documentation, we were unable to determine the extent to which the differences were identified by the EPA or communicated to the appropriate states. Since state program managers use the reports to assess their programs' performance, it is important for state managers to be made aware of any errors in their reports. Source: EPA OIG image One reason for differences in inspection and maintenance performance indicators between the states' annual reports and the EPA's summary were miscalculations from the states or in the regional analysis. Miscalculations can occur when regions or states 12 Even though New Hampshire is exempt from conducting program evaluations, the state is required to submit annual reports, along with all other mandatory inspection and maintenance elements. Thus, the total number of enhanced programs required to submit annual reports is 23. 18-P-0283 17 ------- report their data. For example, a region noted that the testing rate discrepancy for a state in its region is due to an issue with the way the state reports its "number of vehicles tested by model year and vehicle type." The region explained that the state reports 170,815 initial tests for 135,956 unique vehicles, giving the appearance of more than one initial test per vehicle and making it difficult to relate other statistics that rely on the number of initial tests. This makes it difficult to accurately report the results of tests compared to number of vehicles. While EPA guidance specifies which program data states should include in their annual reports, the reports are not provided in a consistent format. Different states have adopted different means for calculating the annual reporting numbers. According to OTAQ, even a seemingly straightforward statistic such as the "number of initially failed vehicles" is interpreted differently by different states. For example, not all states count vehicles that fail the "gas cap" test as failures. Other states count a vehicle that fails both the "gas cap" test and the on-board diagnostic test as two initially failed tests instead of one failed vehicle. One state counts commercial vehicles that are tested quarterly along with the vehicles that are only tested once a year, so that the number of initially tested vehicles reported is larger than the total number of vehicles registered in the program. According to OTAQ, there is no requirement that states revise their original reports. In addition, the lack of consistency combined with the differences in interpretation can cause OTAQ to incorrectly summarize a state's data in its summary report. For example, one region explained that the number of vehicles listed in the "Total Tested" column of OTAQ's summary report for one state was incorrect. The number reported was actually the total number of vehicles that passed the initial test. Four States Continued to Experience Deficiencies in Program Performance Programs continue to experience waiver rates or "no known final outcome" rates well above national averages, as shown in OTAQ summary reports. While three regions reported discussing these performance indicators with their respective states, two regions responded that they had not discussed these results with state program managers. Waiver Rates Remained High in Three States Under certain circumstances, a state program may grant waivers for vehicles that fail their inspection and maintenance tests. Waivers apply to vehicles that fail their initial tests and are not fully repaired because the repair cost limit is met.13 For enhanced programs, the performance standards include a default waiver rate 13 Per the regulations, states require motorists with vehicles that fail the inspection and maintenance test to make a minimum expenditure to qualify for a waiver. Different states require different minimum expenditures, however. 18-P-0283 18 ------- of 3 percent of initially failed vehicles. In practice, most programs report an average waiver rate between 1 to 3 percent. However, three programs continue to experience waiver rates significantly above the performance standard. Three states had waiver rates of 13 percent or more, including one state with a waiver rate reported at 15.2 percent. According to the EPA, double-digit waiver rates suggest that obtaining a waiver may be too easy. Furthermore, the EPA also noted that low repair cost limits can result in owners seeking waivers instead of repairing a failing vehicle, especially if the necessary repair includes a high-priced item like catalyst replacement. "No Known Final Outcome" Vehicle Rates in Three States Significantly Exceeded National Average Vehicles with "no known final outcome" status are vehicles that initially failed their inspection and maintenance tests, but the inspection and maintenance program has no record that one of the following actions has occurred: • Passing a subsequent inspection and maintenance test. • Receiving a waiver. • Being retired from service (scrapped). • Being sold outside of the nonattainment area. The national average for "no known final outcome" vehicles was 18 percent in 2014. However, two states had reported rates of vehicles with "no known final outcome" status that greatly exceeded the national average of 18 percent. For example, one state had a reported rate of 49 percent. According to the EPA, while it is likely that many of the "no known final outcome" vehicles have been scrapped, parked, moved or sold outside the program area, it is equally likely that at least some continue to operate in the program area without having received the needed repairs. Further, as noted in OTAQ's 2015 summary report, while a high "no-known final outcome" vehicle rate is not necessarily evidence of excessive levels of motorist fraud in any given program area, it should be treated as a possible red flag warranting investigation. Conclusion Increased communication and outreach to states about the importance of reporting will help the EPA assemble the necessary lessons learned and achieve consistency in the inspection and maintenance program. Focusing on and communicating with the states regarding waiver rates and "no known final outcome" vehicles would help decrease the incidence of high waiver rates in certain states. When waiver rates exceed model program limits, there is an increased likelihood the program is not achieving its projected emission reductions. 18-P-0283 19 ------- While it is likely that many of the "no known final outcome" vehicles are no longer driven or included in the universe of inspection and maintenance testing, it may also be likely that at least some vehicles continue to operate in the program area without receiving the needed repairs and, therefore, continue to create excess emissions. Recommendations We recommend that the Assistant Administrator for Air and Radiation: 5. Develop and implement guidance on the calculation of individual test statistics in state reports to provide consistency in state reports across regions. 6. Conduct outreach to states with deficiencies in program implementation to determine whether there are any methods whereby inspection and failure rates, waiver rates and "no known final outcome" vehicles can comply with the rates claimed for the program in the approved State Implementation Plan, as required by the inspection and maintenance rule and/or above national averages. 7. Issue guidance to address any trends or common problems identified by the outreach conducted to states with deficiencies in program implementation. Agency Response and OIG Evaluation The agency concurred with the recommendations and provided acceptable planned corrective actions and completion dates. All recommendations in this chapter are resolved. In addition to a response to our recommendations, the agency provided technical comments on the draft report. Based on the agency response and technical comments received, we made revisions to the report where appropriate. Appendix A contains the agency's response to the draft report. 18-P-0283 20 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS Rec. No. Page No. Subject Status1 Action Official Planned Completion Date Potential Monetary Benefits (in $000s) 1 15 Develop and distribute a supplement to the existing National Program Manager Guidance metrics to specifically identify state vehicle inspection and maintenance reports for regions to review. Include biennial program evaluation reports and verify that report quality complies with statutory requirements. R Assistant Administrator for Air and Radiation 1/31/20 2 15 Confirm that biennial program evaluation reports are submitted by states with enhanced inspection and maintenance programs and that EPA regions review the reports, sharing best practices and providing additional clarification. R Assistant Administrator for Air and Radiation 9/30/19 3 15 Revise the vehicle inspection and maintenance rule to remove the cross reference to Title 40 § 51,353(b)(1) of the Code of Federal Regulations, and provide defined evaluation methodology guidance to enable states to quantify emission reductions. R Assistant Administrator for Air and Radiation 6/30/19 4 15 Verify whether states are performing mandatory on-road testing, or determine the reason why they are not and offer assistance to obtain compliance. R Assistant Administrator for Air and Radiation 1/1/19 5 20 Develop and implement guidance on the calculation of individual test statistics in state reports to provide consistency in state reports across regions. R Assistant Administrator for Air and Radiation 6/30/19 6 20 Conduct outreach to states with deficiencies in program implementation to determine whether there are any methods whereby inspection and failure rates, waiver rates and "no known final outcome" vehicles can comply with the rates claimed for the program in the approved State Implementation Plan, as required by the inspection and maintenance rule and/or above national averages. R Assistant Administrator for Air and Radiation 9/30/19 7 20 Issue guidance to address any trends or common problems identified by the outreach conducted to states with deficiencies in R Assistant Administrator for Air and Radiation 3/31/20 1 C = Corrective action completed. R = Recommendation resolved with corrective action pending. U = Recommendation unresolved with resolution efforts in progress. 18-P-0283 21 ------- Appendix A Agency's Response to Draft Report (Dated August 10, 2018) MEMORANDUM SUBJECT: Agency's Response to the Office of Inspector General's report, Collecting Additional Performance Data from States Could Help EPA Better Assess the Effectiveness of Vehicle Inspection and Maintenance Programs (OPE-FY17- 0018) FROM: William L. Wehrum Assistant Administrator Office of Air and Radiation TO: Kevin Christensen Assistant Inspector General Office of Audit and Evaluation Office of Inspector General The U.S. Environmental Protection Agency's (EPA) Office of Air and Radiation (OAR) appreciates the opportunity to review and comment on the Office of Inspector General's (OIG) draft report entitled Collecting Additional Performance Data from States Could Help EPA Better Assess the Effectiveness of Vehicle Inspection and Maintenance Programs (OPE-FY17-0018). The Clean Air Act requires the EPA to establish and implement regulations to protect human health and the environment by ensuring, among other things, that the in-use fleet is well- maintained and that its mandated emission control systems are operating within approved specifications. EPA's Office of Transportation and Air Quality (OTAQ) within OAR fulfills this responsibility for EPA by promulgating vehicle emission inspection and maintenance (I/M) program regulations and guidance for individual state I/M programs to implement and follow. OTAQ collaborates with the Air Divisions of EPA's Regional Offices (ROs) to oversee the states' implementation of the I/M requirements and ensure national consistency within the framework of providing states the flexibility to design I/M programs that best meet local needs. OAR agrees with the observations and recommendations OIG has provided in its draft report. OAR's responses to OIG's specific recommendations have been provided as an attachment to this memo. If you have any questions regarding this response, please contact Karl Simon, Director, Transportation and Climate Division, Office of Transportation and Air Quality, at (202) 564- 7918. Attachment 18-P-0283 22 ------- Cc: Chris Grundler Betsy Shaw Mark Vincent Eleanor Marusiak Karl Simon 18-P-0283 23 ------- OAR's Responses to Recommendations from the Draft Evaluation Report: Collecting Additional Performance Data from States Could Help EPA Better Assess the Effectiveness of Vehicle Inspection and Maintenance Programs (OPE-FY17-0018) Recommendation 1: Develop and distribute a supplement to the existing National Program Manager metric to specifically identify state vehicle inspection and maintenance reports to review. Include biennial program evaluation reports and verify report quality complies with statutory requirements. Response 1: OAR agrees with this recommendation. As part of the next National Program Guidance negotiation between Headquarters and the EPA Regional Offices (ROs), OAR will develop and advocate for the distribution of a measure for applicable ROs to review these required reports within a fixed period after receipt and verify that such reports comply with statutory requirements. Planned Completion Date: OAR will implement this recommendation as part of the next National Program Guidance negotiation period during Q3 FY2019 with the aim of having the revised or new measure in place in time for the next biennial enhanced I/M program evaluation reporting period (i.e., January 2020). Recommendation 2: Confirm that biennial program evaluation reports are submitted by states with enhanced inspection and maintenance programs, and that EPA Regional Offices review the reports, sharing best practices and providing additional clarification. Response 2: OAR agrees with this recommendation. While OAR's response to Recommendation 1 will significantly address Recommendation 2, to ensure that the ROs remain on track to meet the revised or new National Program Guidance measure described above, OAR will direct OTAQ to solicit regular updates from the ROs as part of its monthly Regional I/M Contacts Calls. Planned Completion Date: OAR will initiate its response to Recommendation 2 during Q4 FY2019 and will continue thereafter, following the biennial enhanced I/M program evaluation reporting cycle. Recommendation 3: Revise the vehicle inspection and maintenance rule to remove the cross reference to Title 40, § 51.353(b)(1) of the Code of Federal Regulations, and provide defined evaluation methodology guidance to enable states to quantify emission reductions. Response 3: OAR agrees with this recommendation and - as noted by OIG in its draft report - intends to direct OTAQ to revise the I/M rule to remove the reference the next time the rule is revised for more substantial revisions. Additionally, and in the interim, OAR will direct OTAQ to issue guidance to clarify this provision as well as that enhanced I/M programs that are not 18-P-0283 24 ------- already using some other approved program evaluation methodology should be using the OTAQ guidance document issued in July 2004, Guidance on Use of Remote Sensing for Evaluation of I/MProgram Performance (EPA420-B-04-010). Planned Completion Date: OAR will direct OTAQ to issue this guidance during Q3 FY2019. Recommendation 4: Verify whether states are performing mandatory on-road testing, or determine the reason why they are not and offer assistance to obtain compliance. Response 4: OAR agrees with this recommendation and will respond by directing OTAQ to issue guidance that clarifies that on-road testing is required by the Clean Air Act (CAA) for all mandatory, enhanced I/M programs, that testing using Remote Sensing Devices (RSD) meets the definition of on-road testing, and that using RSD to perform program evaluation testing can be used to meet both the Act's on-road testing requirement for enhanced I/M areas as well as the biennial program evaluation requirement for enhanced I/M programs. OTAQ will also ask the EPA ROs to provide the status of applicable states performing mandatory on-road testing, to determine the reason(s) for any problems, and to identify technical assistance as needed to obtain compliance. Planned Completion Date: OAR will direct OTAQ to initiate the response to this recommendation by the beginning of Q2 FY2019. Recommendation 5: Develop and implement guidance on the calculation of individual test statistics in state reports, in order to provide consistency in state reports across regions. Response 5: OAR agrees with this recommendation and will respond by directing OTAQ to issue guidance clarifying how program statistics such as the rates of vehicle failures, waivers, and disappearing vehicles should be calculated. Planned Completion Date: OAR will direct OTAQ to issue the guidance described above in Q3 FY2019. Recommendation 6: Conduct outreach to states with deficiencies in program implementation to determine whether there are any methods whereby inspection and failure rates, waiver rates and no known final outcome vehicles can comply with the rates claimed for the program in the approved state implementation plan, as required by the inspection and maintenance rule and/or above national averages. Response 6: OAR agrees with this recommendation. In addition to existing and ongoing outreach efforts that are already occurring in the field, OTAQ and the ROs will meet on an annual basis to identify and discuss such deficiencies, including a discussion of the reasons why certain areas greatly exceed the national average and what corrective actions might be taken. 18-P-0283 25 ------- The results of these internal EPA meetings will help identify recommendations for additional outreach to specific areas, including how to address identified program deficiencies. Planned Completion Date: OAR will direct OTAQ to initiate the response to this recommendation by the end of Q4 FY2019 and to continue these meetings annually thereafter. Recommendation 7: Issue guidance to address any trends or common problems identified by the outreach conducted to states with deficiencies in program implementation. Response 7: OAR agrees with the recommendation and views the appropriate response to it as an extension of our response to Recommendation 6, with appropriate OTAQ guidance to be issued to address identified deficiencies. Planned Completion Date: OAR will direct OTAQ to provide appropriate guidance to address this recommendation by the end of Q2 FY2020 and to continue providing such guidance on a periodic basis thereafter as needed. 18-P-0283 26 ------- Appendix B Distribution The Administrator Deputy Administrator Special Advisor, Office of the Administrator Chief of Staff Chief of Operations Assistant Administrator for Air and Radiation Agency Follow-Up Official (the CFO) Agency Follow-Up Coordinator General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for Public Affairs Regional Administrators, Regions 1-10 Career Deputy Assistant Administrator for Air and Radiation Director, Office of Continuous Improvement, Office of the Administrator Director, Office of Transportation and Air Quality, Office of Air and Radiation Audit Follow-Up Coordinator, Office of the Administrator Audit Follow-Up Coordinator, Office of Air and Radiation Audit Follow-Up Coordinators, Regions 1-10 18-P-0283 27 ------- |