i o _ V ^ U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL Air Quality EPA Can Strengthen Its Reviews of Small Particle Monitoring in Region 6 to Better Ensure Effectiveness of Air Monitoring Network Report No. 16-P-0079 December 17, 2015 0 = Active fine particulate matter monitor locations in EPA Reaion 6 ------- This is one of our products associated with EPA oversight of states authorized to implement environmental programs. The Office of Inspector General cited absence of robust oversight of states as one of the EPA's key fiscal year 2015 management challenges. For details go to our report on management challenges. Report Contributors: Rick Beusse Jim Hatfield Andrew Lavenburg Geoff Pierce Wendy Wierzbicki Abbreviations AQI Air Quality Index CFR Code of Federal Regulations EPA U.S. Environmental Protection Agency FRM Federal Reference Method Louisiana DEQ Louisiana Department of Environmental Quality MSA Metropolitan Statistical Area NAAQS National Ambient Air Quality Standards OAQPS Office of Air Quality Planning and Standards OIG Office of Inspector General Oklahoma DEQ Oklahoma Department of Environmental Quality PM2.5 Fine Particulate Matter SLAMS State and Local Air Monitoring Stations Texas CEQ Texas Commission on Environmental Quality [j,g/m3 microgram per cubic meter Cover photo: Map of active PM2.5 monitors in Region 6 states (Arkansas, Louisiana, New Mexico, Oklahoma and Texas) from the EPA's Air Data interactive map website. Red diamonds indicate active PM2.5 monitors. Are you aware of fraud, waste or abuse in an EPA program? EPA Inspector General Hotline 1200 Pennsylvania Avenue, NW (2431T) Washington, DC 20460 (888) 546-8740 (202) 566-2599 (fax) OIG Hotline@epa.gov Learn more about our OIG Hotline. EPA Office of Inspector General 1200 Pennsylvania Avenue, NW (2410T) Washington, DC 20460 (202) 566-2391 www.epa.gov/oiq Subscribe to our Email Updates Follow us on Twitter @EPAoig Send us your Project Suggestions ------- ^tDsr-% * o \ U.S. Environmental Protection Agency Office of Inspector General At a Glance 16-P-0079 December 17, 2015 Why We Did This Review We conducted this evaluation to determine whether the U.S. Environmental Protection Agency (EPA) has used annual air monitoring network reviews and assessments to provide reasonable assurance that Region 6's fine particulate matter (PM2 5) air monitoring network is achieving its objectives. State, local and tribal agencies monitor the air to determine compliance with national air quality standards, including PM25standards. PIVhscan lodge in people's lungs, causing serious health effects such as heart and respiratory disease. Monitoring agencies must prepare annual network plans and conduct periodic network assessments to ensure monitoring requirements are met and resources are efficiently used. Our review focused on Region 6 because of its large population. This report addresses the following EPA goals or cross-agency strategies: Addressing climate change and improving air quality. Protecting human health and the environment by enforcing laws and assuring compliance. Send all inquiries to our public affairs office at (202) 566 2391 or visit www.epa.gov/oia. Listing of OIG reports. EPA Can Strengthen Its Reviews of Small Particle Monitoring in Region 6 to Better Ensure Effectiveness of Air Monitoring Network What We Found Generally, state and local annual monitoring network plans in Region 6 included most information required by the EPA for PM2 5 monitoring. Also, Region 6 identified several issues in its review of annual plans to help ensure monitoring networks were operated in accordance with requirements. However, annual plans did not include evidence to demonstrate monitoring sites were, in fact, in compliance with More thorough reviews of air monitoring networks in Region 6 will better ensure that PM2.5 monitoring is adequate to inform and protect the public. EPA's AirNow website provides the public with real-time air quality data. siting requirements. A lack of clarity in the EPA's requirements for demonstrating siting compliance in the annual plan could have contributed to this happening. The EPA needs to clarify this concept so that states can better address this annual plan requirement. It is important that monitoring organizations annually assess site conditions, as these can change overtime. In one instance, an annual plan did not identify changed site conditions, resulting in the EPA invalidating years' worth of data. Another annual plan did not include plans to establish near-road PM2 5 monitoring sites, instead EPA approved the near-road sites outside of the annual plan process, which limited the opportunity for public comment. Thorough annual plans provide reasonable assurance that monitors are located in the correct areas and will be operated to collect valid data. Properly located and operated monitors help protect public health by demonstrating whether air quality meets health standards and provide the public with information on current air quality. Opportunities exist in Region 6 to improve periodic network assessments, which are intended to identify opportunities to improve the efficiency of the network. The lack of an established review process resulted in Region 6 accepting incomplete periodic assessments. Recommendations and Planned Agency Corrective Actions We recommend that the Assistant Administrator for Air and Radiation clarify what constitutes sufficient evidence to demonstrate compliance with monitor siting and operational requirements when developing annual plans; develop a process to update analytic tools for future assessments; and emphasize the importance of network assessments. We recommend that Region 6 address state-specific deficiencies in monitoring plans and assessments and strengthen its network assessment review process. Based on the EPA's full response, it agreed with all six recommendations in our report and provided proposed corrective actions for each. All report recommendations are resolved. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 THE INSPECTOR GENERAL December 17, 2015 MEMORANDUM SUBJECT: EPA Can Strengthen Its Reviews of Small Particle Monitoring in Region 6 to Better Ensure Effectiveness of Air Monitoring Network Report No. 16-P-0079 FROM: Arthur A. Elkins Jr. TO: Janet McCabe, Acting Assistant Administrator Office of Air and Radiation Ron Curry, Regional Administrator EPA Region 6 This is our report on the subject evaluation conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe the problems the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. Final determinations on matters in this report will be made by EPA managers in accordance with established audit resolution procedures. The EPA offices responsible for implementing the recommendations in this report are the EPA Office of Air and Radiation's Office of Air Quality Planning and Standards and EPA Region 6. Action Required You are not required to provide a written response to this final report because you provided agreed-to corrective actions and planned completion dates for the report recommendations. Should you choose to provide a final response, we will post your response on the OIG's public website, along with our memorandum commenting on your response. Your response should be provided as an Adobe PDF file that complies with the accessibility requirements of Section 508 of the Rehabilitation Act of 1973, as amended. The final response should not contain data that you do not want to be released to the public; if your response contains such data, you should identify the data for redaction or removal along with corresponding justification. ^ฃDSX * A \ 1ฎ| VPR0^ฐ We will post this report to our website at www.epa.gov/oig. ------- EPA Can Strengthen Its Reviews of Small Particle Monitoring in Region 6 to Better Ensure Effectiveness of Air Monitoring Network 16-P-0079 Table of C Chapters 1 Introduction 1 Purpose 1 Background 1 Responsible Offices 5 Scope and Methodology 5 Prior Audit Coverage 6 2 Annual Monitoring Network Plans Could Better Assure That Air Quality Is Reliably Monitored 7 Annual Monitoring Network Plans Are a Key Oversight Tool 7 Region 6 State and Local Annual Plans Should Provide Better Evidence That Monitoring Sites Meet Applicable EPA Siting Criteria 8 Texas 2014 Annual Plan Did Not Include Required Near-Road Monitoring Plan 11 Some Annual Plans Contained Errors or Lacked Complete Information on Monitoring Network 12 Conclusion 13 Recommendations 13 Agency Comments and OIG Evaluation 14 3 Opportunities Exist for Network Assessments to Have More Impact in Region 6 15 Network Assessments Intended to Evaluate Monitoring Network Effectiveness and Efficiency 15 2010 Network Assessments in Region 6 Did Not Address All Required Elements 16 Continued Issuance of EPA Analytical Tools Could Aid State Assessments 22 EPA Could Better Emphasize the Significance of Region Reviews of Network Assessments 22 Conclusion 23 Recommendations 23 Agency Comments and OIG Evaluation 24 Status of Recommendations and Potential Monetary Benefits 25 - continued- ------- EPA Can Strengthen Its Reviews of Small Particle Monitoring in Region 6 to Better Ensure Effectiveness of Air Monitoring Network 16-P-0079 Appendices A Office of Air and Radiation and EPA Region 6 Comments on Draft Report 26 B Distribution 34 ------- Chapter 1 Introduction Purpose We conducted this evaluation to determine whether the U.S. Environmental Protection Agency (EPA) used annual network reviews and assessments to provide reasonable assurance that Region 6's air monitoring network for fine particulate matter (PM2.5) is achieving EPA monitoring objectives. Specifically, we reviewed the extent to which the annual network reviews and assessments were complete and met requirements, and how Region 6 was conducting its oversight of state and local agency network reviews and assessments. Background The EPA and its state, tribal and local agency partners manage and operate ambient air monitoring networks across the country. Ambient air monitoring networks must meet three basic EPA objectives: Provide air pollution data to the general public in a timely manner. Support compliance with air quality standards and emissions strategy development. Support air pollution research studies. The EPA requires each monitoring agency to establish a network of air monitoring stations, using siting and operational criteria set by the EPA Office of Air and Radiation's Office of Air Quality Planning and Standards (OAQPS). State and local air pollution control agencies operate state and local air monitoring stations (SLAMS) for the primary purpose of comparing state and local ambient air quality to the National Ambient Air Quality Standards (NAAQS). The EPA has set primary and secondary1 NAAQS for six common pollutants, including particulate matter.2 The primary standards are intended to protect the public health, including "sensitive" populations such as people with asthma, children and the elderly. Only sites that use specific methods approved by the EPA and meeting stringent quality assurance and siting criteria are eligible for comparing air quality to NAAQS. When the EPA determines that an area's air quality does not meet a standard, the state government must develop a plan, including enforceable measures for 1 Secondary standards provide public welfare protection including protection against decreased visibility and damage to animals, crops, vegetation and buildings. 2 The EPA lias established standards for two categories of "particulate matter"PM10 and PM2.5. PM10, or inlialable coarse particulate matter, are particles less than 10 micrometers in diameter. PM2.5, or fine particulate matter, are particles less than or equal to 2.5 micrometers in diameter. 16-P-0079 1 ------- reducing emissions, to improve air quality in that area. These measures can include more stringent and costly emission controls for industry and other sources within the nonattainment area than they would otherwise be required to implement. For example, construction of new sources or major modifications to existing sources are subject to the EPA's new source review (NSR) permit program. The NSR requirements are customized for the nonattainment area. However, all nonattainment NSR programs have to require the source to meet the lowest achievable emission rate, and to obtain emissions reductions from other sources to offset the emissions from the new source. Consequently, it is important that monitoring networks meet the location and operating criteria established by SLAMS are also used to report real-time data in certain large cities to calculate the air quality index (AQI). Other sites may be used to collect multi-pollutant data to characterize regional and urban patterns of air pollution. Agencies may designate some monitors as special purpose. Special purpose monitors may be used to calculate the AQI for particulate matter. The EPA provides Clean Air Act Section 103 funds to states and local agencies to operate and maintain its PM2.5 network and submit data to the EPA. In fiscal year 2014, Region 6 awarded almost $4 million in Section 103 grant funds to monitoring agencies. As of December 2014, the EPA data showed 82 active PM2.5 Federal Reference Method (FRM) or federal equivalent method monitors in Region 6. Both FRMs and federal equivalent methods can be used to assess compliance with NAAQS. Sources and Health Effects of Exposure to Fine Particulate Matter Particulate matter, or PM, is the EPA. Particulate matter includes "inhalable coarse particles," with diameters larger than 2.5 micrometers and smaller than 10 micrometers; and the term for a mixture of solid particles and liquid droplets found in the air. Some particlessuch as dust, dirt, soot or smoke are large enough to be seen with the naked eye. Others are so small they can only be detected using an electron microscope. Combustion particles, organic compounds, metals, etc. <2.5 fim (microns) in diameter HUMAN HAIR 50-70 (microns) in diameter WPM10 Dust, pollen, mold, etc. <10 Jim (microns) in diameter 90 fim (microns) in diameter FINE BEACH SAND Image showing relative size of fine particulate matter (PM 2.5). (EPA photo) 16-P-0079 2 ------- "fine particles," with diameters that are 2.5 micrometers and smaller. These particles can be made up of hundreds of different chemicals. Some particles, known as primary particles, are emitted directly from a source, such as construction sites, unpaved roads, fields, smokestacks or fires. Others form in complicated reactions in the atmosphere of chemicals such as sulfur dioxides and nitrogen oxides that are emitted from power plants, industries and automobiles. These particles, known as secondary particles, make up most of the fine particle pollution in the country. Numerous studies have linked exposure to particulate matter to a variety of health problems. These include premature death in people with heart or lung disease; nonfatal heart attacks; irregular heartbeat; aggravated asthma; decreased lung function; and increased respiratory symptoms such as irritation of the airways, coughing, or difficulty breathing. Children, older adults and people with heart or lung disease are most likely to be affected by particle pollution. Ambient Air Monitoring Program Quality Management System The EPA's Quality Assurance Handbook for Air Pollution Measurement Systems; Volume II, Ambient Air Quality Monitoring Program, provides guidance on implementing a quality management system for the ambient air monitoring program. The purpose of the system is to ensure that the ambient air monitoring surveillance program: (1) provides data of sufficient quality to meet the program's objectives, and (2) is implemented consistently across the nation. This guidance covers quality assurance activities such as: Project Management (e.g., program organization, training, documentation and records). Measurement Acquisition (e.g., network design, sampling methods, quality control, instrument testing and calibration). Assessment/Oversight (e.g., assessment and corrective action, reports to management). Data Validation and Usability (e.g., data verification, validation, reconciliation with data quality objectives). Combustion sources, such as automobiles, represent a significant source of emissions that result in particle pollution. (EPA photo) The size of particles is directly linked to their potential for causing health problems. Small particles pose the greatest problems, because they can get deep into your lungs, and some may even get into your bloodstream. 16-P-0079 3 ------- The assessment/oversight aspect of the quality management system includes network reviews, performance evaluations, technical systems audits and data quality assessments. All of these processes are used to measure the performance or effectiveness of the ambient air monitoring program. EPA regulation requires two network reviews: (1) annual monitoring network plans3 and (2) periodic network assessments. These two reviews are tools for ensuring that networks meet EPA requirements for monitoring air quality and that resources are used effectively. Annual Plans The EPA requires state or local agencies to prepare and submit an annual plan each year to provide the framework for establishment and maintenance of an air quality surveillance system. Annual plans are required to identify the purpose of each monitor in the network, and include evidence to demonstrate that each monitor is meeting applicable EPA requirements. Further, annual plans are the tool agencies use to document proposed changes to their networks. These changes can include adding new sites, removing sites or monitors, and re-classifying the purpose of a monitor or its suitability for comparing its data to NAAQS. Annual plans are subject to public review and comment, and the EPA's approval. As part of its 2006 rulemaking process to revise 40 Code of Federal Regulations (CFR) Part 58, the EPA stated that annual plans provide states, the EPA and the public an opportunity to review the status of active monitoring sites, and situations where monitors may be added, discontinued or relocated. Network Assessments The EPA requires state and local agencies to conduct an assessment of their network every 5 years. This assessment should identify changes and shifts to monitoring objectives over time and consider their impacts on monitoring networks. In its 2007 Ambient Air Monitoring Network Assessment Guidance, the EPA cited a variety of factors that can impact networks, such as: Changing air quality. Changing populations and behaviors. Establishment of new air quality objectives, including rules to reduce air toxics, PM2.5 and regional haze. Improvements in the understanding of air quality issues and monitoring capabilities. 3 Hereafter referred to as annual plans. 16-P-0079 4 ------- The above factors can result in air monitoring networks having unnecessary or redundant monitors, or ineffective and inefficient monitor locations for some pollutants. Further, these factors could create a need for monitors in areas that were not previously monitored. The monitoring network assessment requirement was created to encourage states to consider these factors in managing their networks. Network assessments are not subject to public review and comment, nor do they require EPA approval. Responsible Offices The EPA offices responsible for implementing the recommendations in this report are the EPA Office of Air and Radiation's OAQPS and EPA Region 6. Scope and Methodology We conducted our work from July 2014 to July 2015. We reviewed applicable federal regulations, as well as EPA guidance documents, to determine the required content of both the annual plans and the network assessments. We developed review checklists, based on EPA regulations and guidance, for both annual plans and network assessments. We used these checklists to determine whether Region 6 annual plans and network assessments met regulatory requirements and the extent to which they followed EPA guidance. We reviewed the 2010 and 2013 annual plans, and 2010 network assessments for all six state and local monitoring agencies in Region 6. This review included 12 out of 12 annual plans prepared for 2010 and 2013. We selected the 2010 annual plans since that was the same year the initial periodic network assessments were conducted. The annual plans for 2013 represented the most recent EPA-approved plans at the time we conducted our review. We reviewed all six periodic network assessments conducted by monitoring agencies in Region 6 since the requirement was implemented. Our review included annual plans and network assessments prepared by the: Arkansas Department of Environmental Quality. Louisiana Department of Environmental Quality (Louisiana DEQ). Oklahoma Department of Environmental Quality (Oklahoma DEQ). New Mexico Environment Department. Texas Commission on Environmental Quality (Texas CEQ). City of Albuquerque Environmental Health Department.4 We reviewed EPA criteria to determine if the PM2.5 monitoring networks described in the plans and assessments met applicable requirements. 4 The City of Albuquerque Environmental Health Department is the only local monitoring agency in Region 6 that operates a portion of the SLAMS network. 16-P-0079 5 ------- This included analysis of data from the U.S. Census Bureau and the EPA's Air Quality System database. We also interviewed staff and managers in OAQPS and Region 6 to obtain an understanding of the: (1) network plan and assessment requirements and guidance documents, and (2) processes used by the EPA to review the documents once submitted by the state and local agencies. We interviewed staff and managers at each agency in Region 6 to understand the processes used by each agency to develop these documents, and to discuss the results of our review of their network plans and network assessments. Our review was limited to annual plans and network assessments in Region 6. We also limited our review to the PM2.5 monitoring network. We selected Region 6 because of its large population relative to other EPA regions; Region 6 covers the fourth largest population of the EPA's 10 regions. Over 38 million people reside in states in Region 6. Population is a significant factor in determining PM2.5 monitoring requirements. At the time we initiated our evaluation, there were no areas in Region 6 that were in PM2.5 non-attainment. We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Prior Audit Coverage The EPA Office of Inspector General (OIG) has not conducted any prior evaluations or audits of EPA's air monitoring annual plan or network assessment process. However, our final report on the U.S. Virgin Islands' implementation of EPA environmental programs5 reported that the U.S. Virgin Islands was not meeting several EPA air monitoring requirements. The OIG recommended that Region 2 use the 2015 network assessment to determine whether the EPA should continue to provide grant funding to the U.S. Virgin Islands to operate their network as it was currently structured. Region 2 agreed with our recommendation and plans to complete its corrective action by December 31, 2015. In addition to our U.S. Virgin Islands report, the OIG conducted several evaluations in recent years that address the EPA's oversight of environmental programs and activities whose implementation has been delegated to state, local or tribal agencies. Based on that past work, our May 2015 report on the EPA's management challenges6 cited improving oversight of states authorized to accomplish environmental goals as a major EPA management challenge. 5 Conditions in the U.S. Virgin Islands Warrant EPA Withdrawing Approval and Taking Over Management of Some Environmental Programs and Improving Oversight of Others, Report No. 15-P-0137. April 17, 2015. 6 FY 2015 EPA Management Challenges, Report No. 15-N-0164. May 28, 2015. 16-P-0079 6 ------- Chapter 2 Annual Monitoring Network Plans Could Better Assure That Air Quality Is Reliably Monitored Generally, state and local air monitoring network plans in Region 6 included most information required by the EPA for PM2.5 monitoring. However: The annual plans did not provide evidence that each monitoring site met regulatory siting criteria. An annual plan prepared by Texas CEQ did not include a plan to establish required near-road PM2.5 monitoring sites by the regulatory deadline. Some annual plans contained errors or mischaracterizations in how they described the PM2.5 monitoring networks. Neither EPA regulation nor guidance defined what constitutes sufficient evidence in annual plans to demonstrate compliance with monitor siting requirements. The EPA needs to clarify this concept so that states can better address this annual plan requirement. Further, Region 6 could improve its review process to better ensure that annual plans are more complete and accurate, to provide reasonable assurance that monitors are located in representative areas and are operated in accordance with EPA requirements. Proper siting and operation of monitors is needed so that the EPA can make reliable determinations about an area's compliance with air quality standards, and so that the public can be informed of air quality risks. Annual Monitoring Network Plans Are a Key Oversight Tool Annual plans are primarily intended to document the status of existing networks and planned changes to meet monitoring objectives. For each existing and proposed site in the network, the annual plan must identify: any proposed changes, location, sampling and analysis methods, operating schedule, and spatial scale of representativeness.7 The annual plan should also include a statement of purpose for each monitor and evidence that each monitor's siting and operation meets EPA requirements. The EPA described the importance of preparing annual plans in responding to public comments on its 2006 proposed change to 40 CFR Part 58. The EPA noted that fulfilling the required elements of the annual plan helped ensure the optimal use of resources. In particular, the EPA noted it was important to review siting 7 Spatial scale of representativeness is described in terms of the physical dimensions of the air parcel nearest to a monitoring site throughout which actual pollutant concentrations are reasonably similar. These scales can be regional, urban or neighborhood. In general, most PM2.5 monitoring in urban areas should be conducted at a neighborhood scale. 16-P-0079 ------- criteria as conditions at once acceptable sites can change due to construction activity, tree growth or other factors. Region 6 State and Local Annual Plans Should Provide Better Evidence That Monitoring Sites Meet Applicable EPA Siting Criteria We reviewed the 2010 and 2013 annual plans for all six monitoring agencies in Region 6. These annual plans included most information required by regulation. However, none of the 12 plans included sufficient evidence to provide reasonable assurance that each monitor met EPA siting requirements. We attribute this condition to a lack of clear explanation in the regulation and EPA guidance as to how the plans should present sufficient evidence to demonstrate that siting requirements were met. Title 40 CFR Part 58.10 states that annual plans shall include a statement of purpose for each monitor and evidence that siting and operation of each monitor meets the requirements of Appendices A, C, D and E of Part 58. Neither the regulation nor EPA guidance explain the extent or type of evidence needed to be included in annual network plans. Six of 12 annual network plans contained generic statements regarding compliance with EPA requirements. However, the plans in both 2010 and 2013 lacked specific evidence showing that each site met all EPA requirements, particularly siting requirements in Appendix E. For example, the Oklahoma DEQ included the following statement in its annual plans: All DEQ/AQD sites and monitors conform to 40 CFR (Code of Federal Regulations), Subchapter C, Part 58 appendix A, Appendix C (see methods in column 6 of table 1), and appendices D & E (see photos located @ http://www.deq.state.ok.us/AQDnew/monitoring/cpdata.htm by clicking on desired location of the site map). The Oklahoma DEQ's annual plan was representative of how some other agencies in Region 6 presented evidence that the network met EPA requirements. The web link provided by Oklahoma DEQ in its annual plans provided access to site-specific data and pictures of each site. The photographs alone did not demonstrate that the required siting criteria were met. For example, the photographs we reviewed did not demonstrate whether sampling device probe heights were sufficient. Further, the photographs did not show whether the sampling devices were an appropriate distance from objects and roadways that could influence the validity of the PM2.5 data. PM2.5 sampler. The EPA's regulations establish criteria for probe/inlet heights and distance from roadways and other objects. (EPA photo) 16-P-0079 8 ------- The former Air Quality Analysis Section Chief in Region 6 told us that Region 6's technical systems audits are a more valuable tool for assessing compliance with siting and operational requirements. A technical systems audit is an on-site review and inspection of an organization's ambient air monitoring program to assess its compliance with regulations governing the collection, analysis, validation and reporting of ambient air quality data. However, these audits are required to be conducted every 3 years. If the annual plan does not verify siting criteria, changed conditions at a site could go unnoticed until the next technical systems audit. Such a change in conditions occurred for one Region 6 site as described in the following section. Monitoring Data Invalidated Due to Changed Site Conditions at New Mexico Site In June 2012, a Region 6 technical systems audit found indications that the ground cover vegetation surrounding a PM2.5 monitoring site in New Mexico had been removed. PM2.5 sites should not be located on an unpaved area unless there is vegetative ground cover year-round to minimize windblown dusts. A New Mexico Environment Department site analysis shows directions of prevailing winds. (Courtesy of New Mexico's confirmed that site Technical Analysis of Siting Criteria, 2013) conditions had significantly changed in the past 3 years. The New Mexico Environment Department stated that the changes in site conditions corresponded with an increase in the number of PM2.5 exceedances. Both the state and EPA concluded that the site no longer met EPA siting criteria. As a result, the New Mexico Environment Department and Region 6 agreed to invalidate PM2.5 data collected from this site. The EPA determined it still had sufficient data to make an attainment designation despite Region 6's invalidation of data from the New Mexico site. The EPA designated all of New Mexico as "unclassifiable/attainment" with respect to the PM2.5 annual standard in December 2014. An "unclassifiable/attainment" designation means that: (1) monitoring data shows the area meets the standard, or Google Earth aerial photo of a PM2.5 monitoring site from a New Mexico Environmental Department technical analysis of siting criteria for Sunland Park site. The small circle shows the monitor's location and the large circles show areas that were previously covered with vegetation but are now bare. The wind rose map 16-P-0079 9 ------- (2) the EPA has reviewed available data and determined the area is likely to be meeting the standard and not contributing to a nearby violation. However, without valid and complete air monitoring data, attainment decisions could be deferred. If the EPA designates an area as nonattainment with an air standard, the state must develop a plan with actions designed to reduce pollution levels in that area to bring the air quality into compliance with the standard. As the EPA noted in its 2006 response to public comments on its proposed revisions to 40 CFR Part 58: ... a review of existing sites including the objectives of each monitor and whether the site continues to meet siting requirements is warranted. Sites that once provided useful data may become less useful as emission patterns change in a particular vicinity, for example. Once-acceptable sites may become questionable due to construction activity, tree growth, or other factors that change over the many years that some sites operate. For these reasons, an annual evaluation of these issues together with the other required elements of the annual monitoring network plan provided a valuable and needed product to ensure the continued optimal use of monitoring resources.8 Thus, during the annual planning process, it is important to review site conditions. EPA Needs to Define Sufficient Evidence for Demonstrating Compliance With Monitor Siting Requirements Neither 40 CFR Part 58 nor the EPA guidance explain what is sufficient evidence to demonstrate compliance with siting requirements in an annual plan. Some Region 6 agencies provided statements in their annual plans stating compliance with all applicable EPA requirements. However, the extent of evidence in the annual plans to support these statements varied. Region 6 agencies stated that they typically evaluated compliance with siting requirements outside of the annual plan process, and kept the documentation in their agency site files. The EPA proposed revisions to 40 CFR Part 58 on September 11, 2014. An OAQPS group leader told us that OAQPS is considering options to clarify language in the final rule about what is "sufficient" evidence to demonstrate in the annual plan that all sites met EPA requirements. The EPA had not issued the final rule at the time we issued our final report in December 2015. 8 "Responses to Significant Comments on the 2006 Proposed Rule on the Revisions to the Ambient Air Monitoring Regulations (January 17, 2006, 71 FR 2710)." Docket Number EPA-HQ-OAR-2004-0018, U.S. Environmental Protection Agency, September 2006. 16-P-0079 10 ------- Texas 2014 Annual Plan Did Not Include Required Near-Road Monitoring Plan Texas CEQ's 2014 annual monitoring plan did not include plans to establish near-road PM2.5 monitoring sites in metropolitan statistical areas (MSAs) with populations over 2.5 million by the regulatory deadline of January 1, 2015. Subsequently, the EPA reviewed and approved these plans outside of the annual planning process, which limited public comment until after deployment. When EPA revised the PM2.5 annual standard in December 2012, it added a requirement to monitor near heavily traveled roads in large urban areas. The EPA added this requirement because particle pollution can be higher along these roads as a result of emissions from cars and heavy-duty diesel trucks and buses. In addition, the EPA required monitoring agencies to submit their plans for establishing near-road PM2.5 monitoring sites for MSAs with populations over 2.5 million as part of their annual monitoring plan to the regional administrator by July 1, 2014. The EPA regulation requires that any annual monitoring plan that proposes SLAMS network modifications is subject to approval by the regional administrator. The regional administrator is to provide an opportunity for public comment unless the monitoring agency has already provided an opportunity for public comment. Texas has two MSAs9 with populations over 2.5 million that were required to have near-road monitoring by January 1, 2015. Texas CEQ stated in its 2014 annual plan that it planned to deploy PM2.5 monitors at two near-road sites by the January 1, 2015, deadline, but did not identify the specific sites or other specific information regarding how the monitoring would meet EPA monitoring requirements. Due to problems in locating suitable sites, the near-road PM2.5 monitors were not deployed until March and April 2015. Region 6's review and approval of Texas CEQ's 2014 annual plan, dated January 14, 2015, did not mention that Texas CEQ had not met the deadline but noted that the EPA planned to work with Texas CEQ to establish sites. Since the near-road monitoring plans were submitted and approved outside of the annual plan process, the public was not provided with an opportunity to comment on the PM2.5 near- road plans prior to deployment. The EPA will require that near-road monitoring be operational in MSAs with populations of 1 million or more but less than 2.5 million by January 1, 2017. Agency plans for establishing these monitors should be submitted in the annual monitoring plans due by July 1, 2016. The region should ensure that Texas CEQ's 2016 annual monitoring plan addresses its plan to establish near-road monitoring in MSAs with populations over 1 million and that the near-road monitoring plan is made available for public comment prior to the sites beginning PM2.5 monitoring. 9 Dallas-Fort Worth-Arlington and Houston-The Woodlands-Sugar Land. 16-P-0079 11 ------- Some Annual Plans Contained Errors or Lacked Complete Information on Monitoring Network Region 6 reviewed and approved all annual plans. The region provided memoranda to each agency documenting the region's approval of the plans, and offering technical comments on the plans. Region 6 identified several issues in its technical comments to help ensure the networks were meeting EPA requirements. However, some annual plans contained factual errors such as mischaracterized sites, or were missing information related to PM2.5 sites that were not identified by Region 6's review. For example: Louisiana DEQ's 2013 annual plan included inaccurate information regarding the MSA for sites in Hammond and Houma. The plan also listed sites as representing MS As that were not characterized as MSAs by the Office of Management and Budget's definition. Louisiana DEQ's annual plans identified some of its continuous PM2.5 federal equivalent methods as not providing data that was sufficiently comparable to collocated FRM monitors. Generally, continuous monitors are collocated with FRM monitors as a quality control measure to ensure that the continuous monitor is recording reliable data. Louisiana DEQ's 2013 annual plan indicated that it was petitioning the EPA to exclude some PM2.5 data that was obtained from continuous monitors from being used to make comparisons to the PM2.5 air quality standard. Louisiana DEQ also stated in its 2014 annual plan that the EPA had approved excluding some PM2.5 monitors from being used to compare air quality to the NAAQS. However, the plans did not include an assessment of the data demonstrating this conclusion. Per 40 CFR Part 58, the monitoring agency's annual plan should include an assessment of how the data collected from continuous monitors compared to data collected from the collocated FRM monitors. Louisiana DEQ submitted its data assessment outside of the annual plan process. Oklahoma DEQ incorrectly stated the monitoring objective for one of the PM2.5 sites in its network. Also, Oklahoma DEQ's 2010 and 2013 network plans did not provide required information for a collocated monitor located at its Tulsa multi-pollutant site. The monitor was omitted from tables providing site descriptions. While we did not find any specific monitoring deficiencies related to the above errors, the examples noted above demonstrate instances in which the annual plan was either inaccurate or incomplete. Inaccurate or incomplete descriptions of the PM2.5 monitoring networks in annual plans create risk that the plans are unreliable and provide less assurance that the network is meeting EPA requirements, and thus providing reliable air quality data. The errors we identified also indicated that 16-P-0079 12 ------- the region can improve controls over its review process to provide better assurance that the plans are complete and accurate. During the OIG's evaluation, Region 6 staff informed us that, in 2014, they began using a new process for reviewing annual network plans. According to Region 6 staff, the review process includes a preliminary review of monitoring requirements and data for each MSA by Region 6 prior to the submittal of annual network plans by monitoring agencies. The region reviews the content of the plan against its preliminary review of the network. The region also reviews system modification requests in the annual plans, any issues involving compliance with siting criteria, and whether previous network changes were incorporated into the network plans. Conclusion It is important that ambient air monitoring networks meet EPA requirements because of the significant uses of the data generated from these networks. For example, the EPA uses air monitoring data to determine whether an area's air quality meets health-based standards set by the EPA. When an area's air quality does not meet standards, sources in those areas are subject to more stringent permitting requirements to limit emissions. Further, some monitoring data are used to develop daily air quality indices, which the public can use to make decisions about the risk involved with outdoor activities on days when air quality is bad. Clarification from the EPA describing the evidence needed to show that requirements for locating and operating monitors are being met, and improved oversight from Region 6, could strengthen the annual plan process. Strengthening this process can help ensure that the public is informed of air quality risks and that the EPA makes reliable assessments of air quality in Region 6 states. Recommendations We recommend that the Assistant Administrator for Air and Radiation: 1. Clarify how states can fulfill the CFR requirement to provide sufficient evidence that each monitoring location is meeting the requirements of 40 CFR Part 58 Appendices A, C, D and E. We recommend that the Regional Administrator, Region 6: 2. Ensure Texas CEQ's 2016 annual monitoring plan addresses its plan to establish near-road monitoring in MSAs with populations over 1 million. 16-P-0079 13 ------- Agency Comments and OIG Evaluation The acting Assistant Administrator for Air and Radiation and the Regional Administrator for Region 6 provided a response to our draft report on September 30, 2015. OAR and Region 6 agreed with Recommendations 1 and 2, and provided acceptable planned corrective actions and completion dates for both recommendations. We consider Recommendations 1 and 2 to be resolved and open pending completion of the corrective actions. Region 6 also provided technical comments in an attachment to its response to the draft report. We made revisions to the report to address Region 6's technical comments where appropriate. See Appendix A for the agency's complete response to our draft report and our specific response to each technical comment. 16-P-0079 14 ------- Chapter 3 Opportunities Exist for Network Assessments to Have More Impact in Region 6 The network assessments in Region 6 could be improved to better address the intent of the assessment process. The assessments we reviewed did not address several elements required by the EPA, which are intended to identify opportunities to improve the efficiency of the network. Increased oversight by Region 6 and the continued availability of EPA-provided analytical tools could improve the thoroughness and usefulness of future assessments. Such improvements in the process can identify opportunities to eliminate redundant monitors, or provide monitoring in areas lacking monitoring. These types of changes can result in more effective use of the resources available for PM2.5 monitoring. Network Assessments Intended to Evaluate Monitoring Network Effectiveness and Efficiency Beginning in July 2010, the EPA required state or local agencies to conduct a network assessment every 5 years. While the annual plan focuses on the existing network, the network assessment should focus on long-term needs. An important objective of the network assessment is to identify and eliminate redundancies or gaps in the network, thus optimizing the use of resources. The EPA requires state or local agency assessments to determine, at a minimum, whether: The network meets the monitoring objectives defined in the regulation. New sites are needed. Existing sites are no longer needed and can be terminated. New technologies are appropriate for incorporation into the ambient air monitoring network. In addition, the EPA stated in 40 CFR Part 58 that the assessment "must consider": The ability of existing and proposed sites to support air quality characterization for areas with relatively high populations of susceptible individuals (e.g., children). The effect on data users, other than the agency itself, for any sites that are being proposed for discontinuance, such as nearby states and tribes or health effects studies. The 2010 network assessments were the first assessments prepared by the state and local agencies. The EPA issued multiple guidance documents to aid in 16-P-0079 15 ------- preparing network assessments. The EPA guidance included different analytical techniques for assessing the technical aspects of ambient air monitoring networks.10 The EPA also identified data sources for organizations to use in conducting their assessments.11 The agency also developed and issued a set of data analysis tools12 to provide evidence for states and local agencies to address two key questions in their 2010 assessments: Which sites are redundant and could be removed or relocated? Where are new sites needed? We reviewed all 2010 assessments conducted by state and local agencies in Region 6 to determine whether the assessments addressed the required elements and the extent to which they used EPA guidance in addressing these elements. 2010 Network Assessments in Region 6 Did Not Address All Required Elements The EPA has cited multiple benefits that may arise from periodic network assessments. However, four of six state and local agencies in Region 6 described the 2010 assessment as having limited value in managing their PM2.5 monitoring networks. According to EPA guidance, monitoring agencies need to adjust networks to protect today's population and environment, while keeping the ability to understand air quality trends. Further, agencies can take advantage of the benefits of new technologies and improved scientific understanding of air quality issues. Reconfiguring air monitoring networks can enhance their value to stakeholders, scientists and the general public. To achieve the benefits of the network assessment process cited by the EPA, state and local agencies should address all required elements and implement EPA guidance. However, the 2010 network assessments in Region 6 did not address many of the required elements. Table 1 shows the result of our analysis of whether each network assessment addressed the required determinations and factors. 10 Ambient Air Monitoring Network Assessment Guidance: Analytical Techniques for Technical Assessments of Ambient Air Monitoring Networks (EPA 454/D-07-001, Feb. 2007). 11 Designing a Network Assessment for an Ambient Air Monitoring Program: Version 1.0- 2010 Assessment. 12 Network Assessment Analyses and Tools Documentation- Michael Rizzo, OAQPS/AQAD, AQAG, March 1, 2010. 16-P-0079 16 ------- Table 1: Required determinations/considerations addressed in 2010 network assessments Requirement Did the assessment address the required determination? Arkansas Department of Environmental Quality City of Albuquerque Louisiana DEQ New Mexico Environment Department Oklahoma DEQ Texas CEQ Does the network meet the monitoring objectives? No Yes No Yes No No Are new sites needed? Yes Yes No Yes Yes Yes Are any existing sites no longer needed? Yes Yes Yes Yes Yes Yes Are new technologies appropriate? No Yes No Yes Yes No Consider high populations of susceptible individuals. No Yes No Yes No No Consider the effect on data users for any proposed discontinued sites. NA Yes NA NA NA No Source: OIG analysis of 2010 network assessments in Region 6. The following sections discuss the results of our review for each required element. Network Meets Monitoring Objectives Four of six 2010 Region 6 network assessments did not make explicit determinations as to whether monitoring networks met EPA objectives to: Provide air pollution data to the general public in a timely manner. Support compliance with ambient air quality standards and emissions strategy development. Support air pollution research studies. All six network assessments contained information and analyses that were focused on how the networks met regulatory requirements and supported compliance with air quality standards. However, state and local agencies did not consistently address how their networks provided air pollution data to the general public in a timely manner or supported air pollution research studies. For example, the Louisiana DEQ and Texas CEQ 2010 assessments did not address how their networks reported data to the public or contributed to the EPA's air quality index. EPA guidance states that network assessments should describe the networks and the relative value of each monitor and station with consideration of the data users. However, none of the six monitoring agencies in Region 6 implemented this aspect of EPA guidance into their 2010 network assessments. Analyzing the 16-P-0079 17 ------- relative value of sites could have helped agencies in determining whether their network was meeting EPA objectives. For example, the EPA's guidance identified "critical sites and monitors" as: Design value site(s) for an area at or above the NAAQS. Long-term multi-pollutant site(s) used by multiple data users for trends and model evaluation (i.e., State Implementation Plan development and tracking). Dedicated site for health or atmospheric study, or to inform policy options for state or local agency. Each of the elements the EPA used to identify critical sites can be linked to the agency's overarching monitoring objectives. Analyzing sites according to such criteria could result in a better understanding of which sites contribute most to the monitoring objectives. Further, an assessment of the relative value of sites can aid in identifying gaps in the network and sites for removal. New Sites Needed Five out of six network assessments from 2010 included determinations regarding whether new PM2.5 monitoring sites were needed. Louisiana DEQ's assessment did not address this requirement. For those assessments that did make this determination, Texas CEQ was the only agency that concluded new PM2.5 sites were needed. Texas CEQ's assessment of whether new sites were needed could be more thorough. We identified areas in Texas where PM2.5 monitors might be warranted, either to assess compliance with the PM2.5 standard or to provide data to calculate an AQI. The AQI is an index for reporting daily air quality. The AQI informs users how clean or polluted the air is, and what the health effects might be based on the AQI13 for that day. According to 40 CFR Part 58 Appendix G, any MSA with a population over 350,000 is required to report an AQI. However, the EPA told us that it does not require states to establish a monitor solely for AQI purposes if a SLAMS monitor is not otherwise required by 40 CFR Part 58, Appendix D, to monitor compliance with a national standard. Table D-5 of Appendix D to Part 58 only requires PM2.5 monitoring in MS As with populations between 50,000 and 500,000 if past monitoring shows concentrations equal to or more than 85 percent of the standard. We noted that nine MSAs in Texas, with a total population of nearly 1.7 million people, did not have any PM2.5 monitors. The EPA's emissions data from our review of National Emissions Inventory in these nine MSAs showed that four of 13 The EPA calculates an AQI for five major air pollutants regulated by the Clean Air Act using NAAQS values for ground-level ozone, particle pollution (also known as particulate matter), carbon monoxide, sulfur dioxide and nitrogen dioxide. The data produced for AQI is not certified or tested for quality assurance purposes like Air Quality System data, so the data cannot be used to support regulation or any EPA decision or position. 16-P-0079 18 ------- these MS As (College Station-Bryan, Killeen Temple, Longview and San Angel o) had an increase of over 20 percent in PM2.5 emissions between 2008 and 2011. For example, emissions in the Killeen Temple MSA, with a population over 400,000, increased from 5,399 tons in 2008 to 7,337 tons in 2011. PM2.5 emissions in the Longview MSA, with a population over 214,000, increased from 6,337 to 9,034 tons during that time. Texas CEQ's 2010 network assessment did not include an analysis of emissions trends in these areas. None of the Texas annual plans or network assessments mentioned the lack of monitors in these MSAs. Further analyses would be needed to determine whether the lack of monitors in these MSAs limit the ability of Texas to properly characterize levels of pollution. Texas CEQ staff told us that they would like to have coverage of all areas, but that there is a limit on Texas CEQ's resources. They explained that they try to prioritize locations based on emission sources, and watch for changes in emission sources to determine whether they need to adjust their priorities. The absence of PM2.5 monitors in these areas does not violate EPA network design requirements. However, the network assessment process provides an opportunity for Texas CEQ to revisit whether the network is providing adequate coverage in these areas. Further, the assessment can provide positive assurance of the current network design or identify areas where new sites are needed. The EPA uses population and the most recent design value for an MSA to determine minimum monitoring requirements. Areas with design values less than 85 percent of the NAAQS require fewer monitors. Since the PM2.5 annual standard has changed since the 2010 network assessment, the 2015 and future assessments provide an opportunity for states to evaluate if previously terminated sites should be re-considered for monitoring. We noted several instances where PM2.5 monitors were terminated under an older, less stringent PM2.5 standard. The last valid design value for these monitors was generally below 85 percent of the old PM2.5 annual standard (15 jag /m3). However, some monitors' last complete design value was greater than 85 percent of the current PM2.5 annual standard (12 jag /m3), and in some cases would have exceeded the new standard. Some of these areas may be candidates for new monitors. Existing Site No Longer Needed All six of the 2010 network assessments in Region 6 included determinations regarding whether existing PM2.5 monitoring sites were needed or could be terminated. Four of the assessments determined that no PM2.5 sites should be terminated. One of these four assessments was the Arkansas assessment. However, Region 6 identified 11 monitors that Arkansas could consider for termination when providing technical comments to Arkansas's 2010 annual plan. Four of these monitors were eventually terminated. Two assessments identified specific PM2.5 monitors that could be terminated. However, some assessments did not fully support the determinations. For example: 16-P-0079 19 ------- Louisiana DEQ concluded that "the PM2.5 network is slightly redundant in the Baton Rouge and New Orleans areas..in its 2010 network assessment. However, information supporting that conclusion was not in the assessment document. Further, Louisiana DEQ did not identify which monitors or sites were redundant or could be terminated. Louisiana DEQ's 2010 annual plan, coinciding with the 2010 network assessment, did not propose any changes to the PM2.5 network. The Texas CEQ discussed discontinuing several PM2.5 monitors in its network assessment, but did not provide analysis supporting this conclusion. In its 2010 annual plan, Texas discussed these proposed terminations in more detail. The annual plan proposed terminating 13 PM2.5 monitors because they were recording low concentrations. Eight of these monitors were eventually terminated after discussions with Region 6.14 As noted in the previous section, there were nine MS As in Texas that do not have PM2.5 monitors but experienced increases in PM2.5 emissions between 2008 and 2011. An analysis in the 2010 network assessment of whether these areas needed PM2.5 monitors might have provided an opportunity to improve the effectiveness of Texas's PM2.5 network by deploying decommissioned PM2.5 monitors to these unmonitored areas. This required element of the network assessment process can identify cost savings by eliminating unneeded monitors. Further, when this analysis is considered in relation to the new sites analysis, it presents opportunities to optimize the efficiency of the network by moving monitors from a redundant location to an area that needs a monitor. However, OAQPS and Region 6 personnel told us that it can be difficult to eliminate an unneeded monitor because of public pressure to retain the monitor. New Technologies State and local agencies are required to include information in network assessments regarding whether new technologies are appropriate for incorporation into the network. Continuous PM2.5 monitoring instruments used for NAAQS comparisons are one example of a newer technology that provides near real time data, and do not require the periodic collection of weighing and filters to calculate ambient concentrations. Continuous monitors are less expensive to operate than FRMs15 because they eliminate the labor costs involved in collecting and weighing filters. The EPA has designated several continuous methods as federal 14 Region 6 did not approve the removal of three monitors since they were needed to meet minimum monitoring requirements. 15 The FRM for measuring PM2 5 is a filter-based method that uses sampling devices to pull ambient air through a filter. The filter is collected on an established scheduled (normally every 3 or 6 days) and sent to a laboratory for weighing and calculation of the PM2 5 ambient concentration Thus, the FRM monitoring data may not be available for days or weeks after the actual measurement was made. 16-P-0079 20 ------- equivalent methods, meaning that the data collected from these monitors can be used to determine whether an area meets air quality standards. Half of the 2010 network assessments stated intentions, or desires, to employ more PM2.5 continuous monitors in their networks. However, none of the assessments identified specific plans to do so, or addressed issues suggested by EPA guidance. Since the 2010 assessment, five of the state and local agencies in Region 6 told us that they have either began the transition to new PM2.5 continuous monitors or plan to replace unsupported and outdated monitors. Some Region 6 states have had difficulty in implementing PM2.5 continuous monitors that produce data that is reliable enough to compare to national standards. For example, Louisiana DEQ staff told us their continuous PM2.5 beta attenuation monitors did not correlate with FRM monitors. Louisiana DEQ purchased 20 continuous PM2.5 beta attenuation monitors at a cost of approximately $20,000 each, after successful field testing at one site. Louisiana DEQ employed seven of the 20 monitors at six sites; however, they have not been able to get many of these monitors to produce reliable data after they were employed in the network. Consequently, the Louisiana DEQ submitted a request to Region 6 to exclude data from all seven PM2.5 beta attenuation monitors. The EPA approved the request for five of these six sites. The EPA denied the request for the one site that was previously field tested, since its data quality was within the acceptable range for use in comparing its data to the national standard. A more thorough network assessment of Louisiana's plan to transition to continuous PM2.5 monitors could have mitigated some of the problems encountered. Consideration of Susceptible Populations Only two of the six network assessments discussed the ability of the existing and proposed monitoring sites to support air quality characterization for areas with relatively high populations of susceptible individuals. New Mexico was one of the two assessments that did cover this required topic. New Mexico included comparisons of subgroups affected by PM2.5, including children, the elderly, and groups with asthma or heart disease; and identified counties with the highest asthma and heart attack hospitalization rates. Also, New Mexico's assessment discussed grants and studies conducted to address air quality and environmental justice issues. Continuous PM2.5 monitor, the Beta Attenuation Mass Monitor 1020, in use at the Air Research Station at the EPA campus in Research Triangle Park, North Carolina. (OIG photo) 16-P-0079 21 ------- The EPA sets air quality standards to protect public health, including the health of "sensitive" populations such as people with asthma, children and the elderly. Thus, the susceptible population analysis is important to ensure that the state or local agency monitoring network is achieving that goal. The lack of information provided in this area results in less assurance that the networks are protecting the health of sensitive populations. Impact of Monitoring Network Changes on Data Users Only one of the 2010 network assessments provided information that considered the effect of discontinuing monitoring sites on data users. This requirement was not applicable for four assessments, as they did not propose to discontinue any monitors. The EPA noted, in response to public comments for its 2006 proposed revisions to 40 CFR Part 58, that states may not be aware that there are many users of air quality information. However, widely disseminating information about network changes in the annual plans can help identify data users. Such communication can lead to benefits, including protecting key monitors that are the basis for long-term trend analyses, and that support ongoing health studies used by stakeholders other than the operating agency. Continued Issuance of EPA Analytical Tools Could Aid State Assessments OAQPS developed a set of analytical tools for identifying potentially redundant sites or areas where new sites may be needed for states and local agencies to use in their 2010 network assessments. Five of the six Region 6 state and local agencies used these tools for their 2010 network assessments. Staff at two state agencies told us that they would be unable to conduct certain analyses for the 2015 assessment if the EPA did not provide the tools. However, OAQPS had not provided the tools for the 2015 assessment due to resource concerns. Because of the demand for these tools, Region 5, two states in Region 5 and a regional air group developed the 2015 assessment tools. Region 6's Air Quality Analysis section provided information on these tools to its state and local agencies in February 2015. EPA Could Better Emphasize the Significance of Region Reviews of Network Assessments Region 6 did not have written procedures for reviewing the network assessments and accepted incomplete assessments. An OAQPS group leader told us that, in general, because it was a new process, the EPA did not criticize agencies for not submitting complete network assessments in 2010. Further, the group leader told us that the assessments primarily benefit the state/local agency. Although state 16-P-0079 22 ------- and local agencies can benefit from the network assessment process, the assessments are also an oversight tool for the EPA in ensuring an effective ambient air monitoring network. In response to public comments on its 2006 proposed rule requiring the network assessments, the EPA stated the assessment was "a key tool to help ensure the right parameters are being measured in the right locations, and that monitoring resources are being used in the most effective and efficient manner...This statement indicates that the assessment was intended as an oversight function. Network assessments are also identified as an oversight activity in the EPA's Quality Assurance Handbook for Air Pollution Measurement Systems. Therefore, the EPA's sufficient review of the network assessments can ensure the assessments are providing value to state and local monitoring agencies, as well as serving an oversight function. Conclusion Missing information and limited use of EPA tools by Region 6 monitoring agencies limited the utility of the 2010 network assessments. As a result, these assessments can miss identifying changes that could improve the efficiency of the PM2.5 networks. Improved oversight of the network assessment processes can help produce more thorough assessments. More thorough assessments can identify unneeded monitors and new monitoring areas that could provide useful air data to the EPA and to the public. Such changes can help ensure that air monitoring resources are effectively used. Recommendations We recommend that the Assistant Administrator for Air and Radiation: 3. Develop a process for ensuring that state and local monitoring agencies are provided with updated data analysis tools for future network assessments. 4. Clarify the significance of the network assessments to EPA regions and emphasize that regional reviews should ensure that the assessments address the minimum elements required by regulation. We recommend that the Regional Administrator, Region 6: 5. Strengthen the network assessment review process to ensure the assessments meet minimum EPA requirements and implement EPA guidance. 6. Ensure Texas CEQ's network assessment evaluates the sufficiency of its PM2.5 monitoring network in metropolitan statistical areas that have never had, or currently do not have, PM2.5 monitors. 16-P-0079 23 ------- Agency Comments and OIG Evaluation In its response to our draft report, the agency agreed with Recommendations 3, 4, 5 and 6. In the response, the Office of Air and Radiation and Region 6 provided acceptable corrective actions and estimated completion dates for Recommendations 3 and 4. We met with the agency on October 28, 2015, to discuss its proposed corrective action plans to address Recommendations 5 and 6. Subsequently, the EPA provided supplemental information to clarify its corrective actions to address these recommendations. Based upon the official agency response to our draft report and the supplemental information provided, we consider Recommendations 3, 4, 5 and 6 resolved and open pending completion of corrective actions. Region 6 also provided technical comments on the draft report. Based on the agency response and technical comments received, we made revisions to the report where appropriate. See Appendix A for the agency's response to our draft report and our specific response to each technical comment. 16-P-0079 24 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. No. Subject Status1 Action Official Planned Completion Date Claimed Amount Ag reed-To Amount 13 Clarify how states can fulfill the CFR requirement to provide sufficient evidence that each monitoring location is meeting the requirements of 40 CFR Part 58 Appendices A, C, D and E. 13 Ensure Texas CEQ's 2016 annual monitoring plan addresses its plan to establish near-road monitoring in MSAs with populations over 1 million. 23 Develop a process for ensuring that state and local monitoring agencies are provided with updated data analysis tools for future network assessments. 23 Clarify the significance of the network assessments to EPA regions and emphasize that regional reviews should ensure that the assessments address the minimum elements required by regulation. 23 Strengthen the network assessment review process to ensure the assessments meet minimum EPA requirements and implement EPA guidance. 23 Ensure Texas CEQ's network assessment evaluates the sufficiency of its PM2.5 monitoring network in metropolitan statistical areas that have never had, or currently do not have, PM2.5 monitors. Assistant Administrator for 3/31/16 Air and Radiation Regional Administrator, 6/30/16 Region 6 Assistant Administrator for 3/31/18 Air and Radiation Assistant Administrator for 9/30/16 Air and Radiation Regional Administrator, 3/31/17 Region 6 Regional Administrator, 9/30/16 Region 6 O = Recommendation is open with agreed-to corrective actions pending. C = Recommendation is closed with all agreed-to actions completed. U = Recommendation is unresolved with resolution efforts in progress. 16-P-0079 25 ------- Appendix A Office of Air and Radiation and EPA Region 6 Comments on Draft Report kft) UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON. D.C. 20460 MEMORANDUM SEP 3 o 2OT5 SUBJECT: Response to Office of Inspector General (01G) Draft Report No. OPE-FY14-0011 "Region 6 Can Strengthen Reviews to Better Ensure That Air Monitoring for Small Particles Is Adequate to Protect the Public," dated August 31,2015 FROM: TO: Carolyn Copper, Assistant Inspector General Office of Inspector General The EPA's Office of Air and Radiation and Region 6 appreciate the opportunity to review and comment on the OIG's draft report titled "Region 6 Can Strengthen Reviews to Better Ensure That Air Monitoring for Small Particles Is Adequate to Protect the Public" (Draft Report). We agree with the OIG about the importance of the Annual Monitoring Network Plan and 5- Year Network Assessment. These requirements have played an important role in improving the public awareness of the air monitoring programs in communities and also encouraging state and local monitoring agenci es to better focus on periodic short-term and longer-term reviews of their programs. OAR has worked closely with regional offices since these requirements were established in 2006 and we have ongoing conversations about how to improve and standardize the review of monitoring agency submissions to ensure a consistent enforcement of key aspects of the requirements. Based on these factors, we propose the following title change to the Draft Report to better capture the subject at hand: EPA Can Strengthen Guidance for Review of Particulate Monitoring Regulatory Requirements. The Agency welcomes the OIG's review of Region 6's annual plans as a confirmation of the effectiveness of the regulation, with the report noting that "Generally, state and local air Assistant Administrator Janet G. McCabc, Acting Office of Air and Radiation Ron Curry, Regional * \ EPA Region 6 16-P-0079 26 ------- monitoring network plans in Region 6 included most information required by the EPA for PM2.5 monitoring." We acknowledge, however, that the wording and subsequent interpretation of certain aspects of the regulation has led to somewhat inconsistent levels of documentation in annual plans, and as mentioned in the OIG report, OAR is undertaking a rulemaking to clarify these sections (see proposal dated September 11, 2014, 79 FR 54356). This rulemaking is expected to be finalized in late 2015 or early 2016. With regard to the review of the network assessments, we note that the first round of assessments that were submitted in 2010 represented the initial phase of a learning curve for state and local monitoring agencies as well as regional offices. While EPA's specific requirements for network assessments were relatively modest and flexible, individual assessments were inconsistent in their reference to these requirements. As noted in the OIG report, OAR did develop a comprehensive set of analysis tools and a guidance document. However, the 2010 round represented the Agency's first attempt at requiring monitoring agencies to develop network assessments, and we expect improvement in the 2015 assessments that have been recently submitted to the regions. Both OAR and Region 6 do agree that the development of analytical tools is an important area for the Agency to lead, and as explained below, we believe the partnership efforts that resulted in the development of web-based tools for the 2015 network assessments represent an effective model for supporting future year analyses of monitoring networks. Below are the Agency's response to the OIG's specific recommendations. Additionally, the Agency requests the following detailed technical comments (attachment), including specific language suggestions for statements in the Draft Report, be included in the final report. Assistant Administrator for Air and Radiation (1, 3, 4) Recommendation 1: "Clarify how states can fulfill the CFR requirement to provide sufficient evidence that each monitoring location is meeting the requirements of 40 CFR Part 58 Appendices A, C, D, and E." OAR Response 1: The OAR is completing work on a final monitoring rule that includes revisions to this specific language in 40 CFR Part 58.10 (see 79 FR 54356). The intended revisions will simplify this language by requiring only that a statement of whether the operation of each monitor meets the applicable Part 58 requirements will be needed in annual plans, versus a more extensive requirement to provide evidence as required in the current language. We are making this change because we believe the current extensive list of required documentation elements in 40 CFR Part 58.10(b) is sufficient to provide the public with "evidence" of compliance with network design and operational requirements, and because we have other, more effective tools and processes to ensure adherence to more technical aspects of Part 58 such as quality assurance and site probe siting (i.e., Appendices A and E). These include the implementation of a new data certification and concurrence report16 as well as the development of a national Technical Systems Audit (TSA) workgroup to improve the procedures used by regional offices to conduct monitoring agency systems audits and network evaluations. We "'http://www.epa.gov/ttn/amtic/files/2014conference/monaa8papp.pdf 16-P-0079 27 ------- believe that the combination of the revised annual plan regulatory requirements together with improved training and OAR support for the conduct of TSAs will support a heightened level of network oversight as suggested by this recommendation. Planned Completion Date: Fiscal Year (FY) 2016, Quarter (Q) 2 Region 6 Administrator (2, 5, 6) Recommendation 2: "Ensure Texas CEQ 2016 annual monitoring plan addresses its plan to establish near-road monitoring in MSAs with populations over 1 million." Region 6 Response 2: The EPA Region 6 agrees with this recommendation. Although the Texas CEQ has established near-road monitoring in metropolitan statistical areas (MSAs) with populations over 2.5 million, we commit to ensuring that Texas complies with 40 CFR 58.10(a)(8)(ii) to submit a plan for establishing near-road PM2.5 monitoring sites in MSAs having 1 million or more persons by July 1, 2016. This effort may be completed independently or in conjunction with actions responsive to recommendations below. Planned Completion Date: FY 2016, Q 3 Assistant Administrator for Air and Radiation Recommendation 3: "Develop a process for ensuring that state and local monitoring agencies are provided with updated data analysis tools for future network assessments." OAR Response 3: The OAR agrees with this recommendation. As noted in the OIG report, an interagency workgroup developed data analysis tools to support the 2015 round of network assessments.17 This workgroup included representatives from Region 5 and LADCO, several states in the LADCO region, and OAQPS. National training was conducted in February 2015 providing some pre-deadline technical support for the assessment process. We agree that it would have been advantageous to have these tools available earlier and OAR will work with these partners to support the ongoing availability of the current work as well as supporting future enhancements. We expect this process to evolve over time given the rapid advancements in the availability of web-based analytical tools. Staff from the OAQPS ambient air monitoring and air quality analysis groups will work proactively with partners to ensure adequate preparation for the 2020 assessment. Planned Completion Date: FY 2018, Q 2 Assistant Administrator for Air and Radiation Recommendation 4: Clarify the significance of the network assessments to EPA regions and emphasize that regional reviews should ensure that the assessments address the minimum elements required by regulation. 17 http ://ladco. github. io/Net Assess App/ 16-P-0079 28 ------- OAR Response 4: The OAR agrees with the premise of this recommendation. We believe that the significance of this process has been made clear to regional offices through regularly scheduled national calls as well as training classes devoted to the network assessment process that were held at national monitoring conferences in the years prior to network due dates (e.g., 2009 and 2014). As noted earlier, the network assessment requirement is still relatively new and we believe there is value in exploring the latest round of assessments in more detail to determine how well submitting agencies addressed required elements. In partnership with regional offices, OAR will review this information as part of developing clear messaging on the significance and content of the network assessments. Planned Completion Date: FY 2016, Q 4 Region 6 Administrator Recommendation 5: Strengthen the network assessment review process to ensure the assessments meet minimum EPA requirements, and implement EPA guidance. Region 6 Response 5: The EPA Region 6 agrees with this recommendation. The Region agrees to strengthen the network review process. The Region plans to use the assessments as a discussion document for the years after it is submitted. It is beneficial to both the Region and the monitoring agencies in developing future plans. EPA OAR and various monitoring agencies discussed the guidance documents for network assessment reviews with our primary quality assurance organizations during the 2014 National Air Monitoring conference. The Region clarifies that EPA recommends but does not require that monitoring agencies implement EPA guidance. EPA offers the Monitoring Network Partners guidance and suggestions with respect to the network assessment review process. However, the regulations do not require explicit conformance to the guidance documents associated with the network assessment review process. This effort may be completed independently or in conjunction with actions responsive to recommendations below. Planned Completion Date: FY 2017, Q 4 16-P-0079 29 ------- OIG Comment: On November 6, 2015, the EPA provided an updated corrective action plan for Recommendation 5. The revised corrective action plan is as follows: Region 6 Response 5: The EPA Region 6 agrees with this recommendation. The Region agrees to strengthen the network review process. The Region plans to use the assessments as a discussion document for the years after it is submitted. It is beneficial to both the Region and the monitoring agencies in developing future plans. EPA OAR and various monitoring agencies discussed the guidance documents for network assessment reviews with our primary quality assurance organizations during the 2014 National Air Monitoring conference. EPA intends to review the network assessments to ensure that each element required by the regulations is included, and will provide comments back to our monitoring agency partners within one year of submittal. EPA intends to discuss the projected activities with our partners on an annual basis that coincides with the annual network plan reviews. The Region clarifies that EPA recommends but does not require that monitoring agencies implement EPA guidance. EPA offers the Monitoring Network Partners guidance and suggestions with respect to the network assessment review process. However, the regulations do not require explicit conformance to the guidance documents associated with the network assessment review process. This effort may be completed independently or in conjunction with actions responsive to recommendations below. Planned Completion Date: FY 2017, Q 2 Region 6 Administrator Recommendation 6: Ensure Texas CEQ's network assessment evaluates the sufficiency of its PM2.5 monitoring network in metropolitan statistical areas that have never had, or currently do not have, PM2.5 monitors. Region 6 Response 6: The EPA Region 6 agrees with this recommendation. The Texas CEQ evaluated the sufficiency of its PM2.5 monitoring network in its FY2015 annual monitoring network plan and 2015 network assessment. We commit to ensuring that Texas complies with 40 CFR 58 App. D to evaluate the PM2.5 monitoring network in subsequent annual monitoring network plans and network assessments. This effort may be completed independently or in conjunction with actions responsive to recommendations above. Planned Completion Date: FY 2016, Q 4 16-P-0079 30 ------- OIG Comment: On November 6, 2015, the EPA provided an updated corrective action plan for Recommendation 6. The revised corrective action plan is as follows: Region 6 Response 6: The EPA Region 6 agrees with this recommendation. The Texas CEQ evaluated the sufficiency of its PM2.5 monitoring network in its FY2015 annual monitoring network plan and 2015 network assessment. We commit to ensuring that Texas complies with 40 CFR 58 App. D to evaluate the PM2.5 monitoring network in subsequent annual monitoring network plans and five year network assessments. EPA intends to work with TCEQ during the 2016 fiscal year to review the existing data for small MS As to determine if there is a need to consider special purpose monitoring in these areas for the purposes of characterizing ambient concentrations relative to the PM2.5 NAAQS. The completion of such studies, if needed, would be subject to the availability of resources. This effort may be completed independently or in conjunction with actions responsive to recommendations above. Planned Completion Date: FY 2016, Q 4 If you have any questions regarding this response, please contact Richard (Chet) Wayland, Director, Air Quality Assessment Division in the Office of Air Quality Planning and Standards at (919) 541-4603 or Region 6's Multimedia Planning and Permitting Division Director, Wren Stenger at (214) 665-6583. Attachment cc: Elizabeth Shaw Deborah Jordan Steve Page Mike Koerber Richard A. Wayland Lewis Weinstock Maureen Hingeley Mike Jones Liz Naess Wren Stenger Frances Verhalen Josephine Hah 16-P-0079 31 ------- TECHNICAL COMMENTS ATTACHMENT EPA Region 6 offers these requests, corrections, disagreements, and comments for OIG review: Title: Request change As presented, the title appears to indicate that EPA Region 6 does not conduct, evaluate or review air monitoring for small particles to protect the public. EPA Region 6 requests that the title be changed to: "EPA Can Strengthen Guidance for Review of Particulate Monitoring Regulatory Requirements." OIG Response: As noted in the body of our report, Region 6 does review the PM2.5 monitoring networks in its states. We have revised the title of the report to reflect our general finding that the EPA can take steps to strengthen network reviews in Region 6. However, we believe these steps extend beyond just strengthening guidance, and thus used a different title than the one suggested by Region 6. Page 2: Incorrect statements: "SLAMS sites are also used to report real-time data in certain large cities to calculate the air quality index (AQI)." Should be SLAMS or SPM monitors and not SLAMS sites. "Agencies may also designate some sites as special purpose." Should be monitors and not sites. Please note that both SLAMS and/or SPM monitors can be used for AQI reporting, including some non-FRM or non-FEM methods. OIG Response: We agree that the statements should reflect that monitors, and not sites, are used to report data and are designated as SLAMS or special purpose. We have revised the report accordingly. Page 11: Disagreement with statement: "The monitoring agency's annual plan should include an assessment of how the data collected from continuous monitors compared to data collected from the collocated FRM monitors." LDEQ submitted the assessment report for continuous monitor deployment as part of a request for NAAQS exclusion. This assessment is not required by regulation to be part of the annual plan. EPA does not agree with OIG that this assessment should be a part of the annual plan. OIG Response: We disagree with Region 6's comment, since 40 CFR Part 58.11(e) states that ".... These assessments are required in the monitoring agency's annual monitoring network plan described in ง58.10(b) for cases where the FEM or ARM is identified as not of sufficient comparability to a collocated FRM." Page 17: Comment "Since the PM2.5 annual standard has changed since the 2010 network assessment, the 2015 and future assessments provide an opportunity for states to evaluate if previously terminated sites should be re-considered for monitoring.... However some monitors last complete design value is 16-P-0079 32 ------- greater than 85 percent of the current PM2.5Standard (12 ug/m3), and in some cases would have exceeded the new standard. Some of these areas may be candidates for new monitors." EPA Region 6 requests additional information about which locations OIG identified that may be candidates for additional monitors. OIG Response: In response to this comment, we provided the requested information to Region 6 on October 19, 2015. Page 20: Comments "Region 6 did not have written procedures for reviewing the network assessments and accepted incomplete assessments." Due to date of publication of the network assessment guidance documents (see page 14 of OIG report) for 2010, PQAOs had limited time to complete the assessments and submit by the July 1, 2010 deadline. OIG Response: Although network assessment guidance was published at various points leading up to the 2010 network assessment deadline, the requirement to conduct network assessments was issued by the EPA in 2006. No changes were made to the report based on this comment. "Although state and local agencies may benefit from the network assessment process, the assessments are a key oversight tool for the EPA in ensuring the effective ambient air monitoring network." EPA disagrees that the network assessments are key tools in ensuring effectiveness as these are planning tools that, every five years, prompt the States to make the minimum determinations specified at 40 CFR 58.10(d). The annual monitoring network plan is the key and ongoing tool for oversight. OIG Response: We deleted the word "key" from the report statement, as we recognize that other EPA oversight activities, such as the annual plan review and technical systems audits, are conducted more frequently. However, we note that in support of its 2006 proposed rule requiring network assessments, the EPA stated that the network assessment was "a key tool to help ensure the right parameters are being measured in the right locations, and that monitoring resources are being used in the most effective and efficient manner...." 16-P-0079 33 ------- Appendix B Distribution Office of the Administrator Assistant Administrator for Air and Radiation Regional Administrator, Region 6 Agency Follow-Up Official (the CFO) Agency Follow-Up Coordinator General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for Public Affairs Deputy Assistant Administrator for Air and Radiation Director, Office of Regional Operations Director, Office of Air Quality Planning and Standards, Office of Air and Radiation Deputy Regional Administrator, Region 6 Audit Follow-Up Coordinator, Office of Air and Radiation Audit Follow-Up Coordinator, Region 6 16-P-0079 34 ------- |