Monitoring Insights Relative accuracy in EPA CAMD's Power Sector Emissions Data May 10th, 2022 40 CFR part 75 requires regular quality assurance (QA) testing to ensure that continuous emission monitoring systems (CEMS) are providing accurate, consistent, and reliable data. There are several QA tests that power plant operators are required to perform on the CEMS. One of the required tests is a relative accuracy test audit, or RATA— a test used to determine the relative accuracy of sulfur dioxide (S02), nitrogen oxides (N0X), and carbon dioxide (C02) or oxygen (02) concentration measurements, and volumetric flow measurements of the flue gas (known as stack gas flow). This monitoring insights analysis reviews RATA results between 2017 and 2021 to assess the performance of the part 75 monitoring systems. Relative accuracy test audits are used to quality assure continuous emission monitoring systems What is relative accuracy? As defined in 40 CFR part 72.2, relative accuracy is "a statistic designed to provide a measure of the systematic and random errors associated with data from continuous emission monitoring systems". Relative accuracy represents the difference between the measurements of a CEMS installed at a facility to data collected concurrently with an EPA emission test method. The relative accuracy is expressed as a percent difference (% RA) between a facility's CEMS values and the applicable reference method test values. oEPA ------- Monitoring Insights Relative accuracy is based on independent measurements How is relative accuracy determined? Relative accuracy is determined by comparing emissions data recorded by a CEMS to data collected concurrently with an EPA reference method test.1 The EPA reference methods generally involve a qualified team of stack testers2 using a portable CEMS to measure emissions at a facility. The relative accuracy is calculated as follows: D _ | Mean ref. method value—Mean CEMS value\ + confidence coefficient 1 nn KA — -—- - X 1UU Mean ref. method value 1 Refer to section 6.5.10 Reference Methods of appendix A to part 75 for more information about reference method tests. 2 A qualified individual who meets certain competency standards (e.g., ASTM D7036), as certified by an Air Emission Testing Body (AETB), must be onsite during the RATA. What are acceptable RA values? To pass a RATA, the primary performance specification requires a relative accuracy of less than or equal to 10.0% for pollutant gas monitors (e.g., S02, NOx, C02, 02) and stack gas flow. However, if the relative accuracy is less than or equal to 7.5%, a facility may qualify to conduct RATAs less frequently. Facilities with low pollutant concentrations (or emission rate for NOx Ib/mmBtu) can pass a RATA based on the primary performance specifications expressed as a percentage, or an alternate performance specifications expressed as a pollutant concentration (or emission rate for NOx Ib/mmBtu). oEPA 2 ------- Monitoring Insights Relative accuracy s improving or stable for all parameters Distribution of relative accuracy by year and parameter C02 concentration NOx rate 02 concentration 10-- O -¦ 2017 2018 2019 2020 2021 The red line indicates the passing value of 10% and the blue line indicates 7.5%, the value to qualify for a reduced RATA frequency 2017 2018 2019 2020 2021 v>EPA 2017 2018 2019 2020 2021 Year For information about how to read these figures, refer to page 9 ------- Monitoring Insights Results and Conclusions Relative accuracy values are lower than required The relative accuracies for all parameters are concentrated below 7.5%, meaning that most units qualify for reduced RATA frequency. A lower RA indicates a smaller difference between the measured value and the reference method. For volumetric flow and C02, the median of the data is concentrated around 2%. ForS02, the median of the data is concentrated closer to 3%. 02data was concentrated around 3% in 2017 but has recently been closer to 2%. Note that 02 data is more variable because it has the fewest number of RATA tests. For NOx rate, the data is concentrated around 3.5%. The median RAfor NOx rate is likely higher because two monitors are required to measure the value (NOx concentration and diluent concentration). oEPA Relative accuracy values are constant or improving For parameters C02, S02, N0X rate, and stack gas flow, the median RA and spread of data is relatively consistent from 2015 through 2020. For 02, the median RA trends down, indicating an improvement, and the data spread (i.e., interquartile range) declines starting in 2017. The key finding is that relatively accuracy values are improving—getting smaller and more uniform. 4 ------- Monitoring Insights How does relative accuracy affect the data? If the RATA results indicate the CEMS is reading lower than the reference test method, a bias adjustment factor (BAF) may be applied to subsequent emission measurements until the next RATA. The BAF is designed to ensure there is not an underreporting of emissions. If a BAF is required, each subsequent emission measurement is multiplied by the BAF. The required frequency for performing a RATA is dependent on the results of the previous RATA. Section 2.3 in appendix B to part 75 defines the standard RATA frequency as once every two QA operating quarters or semi-annually; however, a unit may qualify for a reduced frequency of once every four QA operating quarters or annually if the relative accuracy is less than or equal to 7.5% for pollutant monitors and stack gas flow. What happens if a RATA is failed, performed incorrectly, or not completed? If a RATA is failed, performed incorrectly, or not completed by the required deadline, the facility must deem the CEMS measurements as invalid and use substitute data methods described in 40 CFR 75.30-37 to calculate emissions until the CEMS passes a subsequent RATA. More information about validity of CEMS data is available in the analysis of percent monitor data availability, and more information about substitute data is available in the analysis on substitute data. A failed or missing RATA can also lead to a field audit of the CEMS by state and/or federal regulators. oEPA 5 ------- Monitoring Insights For more information about the data or this analysis EPA's part 75 monitoring and reporting program ¦ 40 CFR part 75—Continuous Emission Monitoring ¦ Plain English Guide to Part 75 (PDF) ¦ EPA CAMP power sector programs—progress reports Power Sector Emissions Data ¦ CAMD's Power Sector Emission Data ¦ CAMD's Power Sector Emissions Data Guide (PDF) Contact information Stacey Zintgraff EPA's Clean Air Markets Division 202-564-2204 z i nteraff .sta cev@ e oa. gov oEPA 6 ------- Monitoring Insights Analytical methodology This analysis was completed in R. If you would like to review the code or source data, contact Stacev Zinteraff. The steps in this analysis include: 1. Compiled all RATA results from 2017 through 2021, including relative accuracy, parameter (e.g., pollutant), year, and test result 2. Filter out parameters not included in this analysis, tests that passed using alternate performance specifications (qualifying units only), and aborted tests Note that failed RATAs are included in this analysis. 3. Calculate quartiles and median for each year and parameter. 4. Create box and whisker plots. Outliers, including failed tests, were excluded from the plot on page 3, refer to appendix B on page 9 for a box and whisker plot including outliers. By the numbers Test counts in this analysis 2021 ¦ C02: 578 RAs (median 2.1% RA) ¦ S02: 324 RAs (median 3.5% RA) ¦ 02:19 RAs (median 2.3% RA) ¦ N0X rate: 1,901 RAs (median 3.1% RA) ¦ Flow: 1,456 RAs (median 2.2% RA) 2017 ¦ C02: 757 RAs (median 1.9% RA) ¦ S02: 458 RAs (median 3.0% RA) ¦ 02: 23 RAs (median 2.9% RA) ¦ N0X: 1,985 RAs (median 3.2% RA) ¦ Flow: 1,905 RAs (median 2.3% RA) v>EPA 7 ------- Monitoring Insights Appendix A: How to read a box plot Box plot key • Outliers/Maximum 1.5 times the upper IQR 75th percentile Median 25th percentile Minimum The RATA results for all reporters are displayed using a "box plot" also known as a "box-and-whisker plot". A box plot is a method to depict groups of numerical data in quartiles. It illustrates the distribution, central tendency, and variability. In the example box plot on the left: ¦ The blue box represents the middle half of all values—also known as the interquartile range (IQR)—those that fall between the 25th and 75th percentile. ¦ The grey horizontal line represents the median value (i.e., the 50th percentile value). ¦ The green vertical line, or top whisker, represents the values between the 75th percentile and 1.5 times the upper interquartile range. ¦ The red vertical line, or bottom whisker, represents the values between the minimum and the 25th percentile. ¦ The black dot represents outliers or values outside 1.5 times the IQR. The highest dot represents the maximum value. Each box plot provides visual representations of both the magnitude and variability of values for all reporters in a given year in a single chart. oEPA ------- Monitoring Insights Appendix B: Box and Whisker Plot including outliers Distribution of relative accuracy by year and parameter C02 concentration NOx rate 02 concentration 25- 20- 15- 10- „ 5- 1Z2 l f- > o n o =5 O o < jro a> CT o- i— —r T I I I I 2017 2018 2019 2020 2021 2017 2010 2019 2020 2021 Year v>EPA S E S02 concentration stack gas flow • t • J • : . • : ... 1 __T I 1 _ 2017 201S 2019 2020 2021 ------- |