STATE REVIEW FRAMEWORK Pennsylvania Philadelphia Air Management Services Clean Air Act Implementation in Federal Fiscal Year 2020 U.S. Environmental Protection Agency Region 3 Final Report March 22, 2022 ------- I. Introduction A. Overview of the State Review Framework The State Review Framework (SRF) is a key mechanism for EPA oversight, providing a nationally consistent process for reviewing the performance of state delegated compliance and enforcement programs under three core federal statutes: Clean Air Act, Clean Water Act, and Resource Conservation and Recovery Act. Through SRF, EPA periodically reviews such programs using a standardized set of metrics to evaluate their performance against performance standards laid out in federal statute, EPA regulations, policy, and guidance. When states do not achieve standards, the EPA will work with them to improve performance. Established in 2004, the review was developed jointly by EPA and Environmental Council of the States (ECOS) in response to calls both inside and outside the agency for improved, more consistent oversight of state delegated programs. The goals of the review that were agreed upon at its formation remain relevant and unchanged today: 1. Ensure delegated and EPA-run programs meet federal policy and baseline performance standards 2. Promote fair and consistent enforcement necessary to protect human health and the environment 3. Promote equitable treatment and level interstate playing field for business 4. Provide transparency with publicly available data and reports B. The Review Process The review is conducted on a rolling five-year cycle such that all programs are reviewed approximately once every five years. The EPA evaluates programs on a one-year period of performance, typically the one-year prior to review, using a standard set of metrics to make findings on performance in five areas (elements) around which the report is organized: data, inspections, violations, enforcement, and penalties. Wherever program performance is found to deviate significantly from federal policy or standards, the EPA will issue recommendations for corrective action which are monitored by EPA until completed and program performance improves. The SRF is currently in its 4th Round (FY2018-2022) of reviews, preceded by Round 3 (FY2012-2017), Round 2 (2008-2011), and Round 1 (FY2004-2007). Additional information and final reports can be found at the EPA website under State Review Framework. II. Navigating the Report The final report contains the results and relevant information from the review including EPA and program contact information, metric values, performance findings and explanations, program responses, and EPA recommendations for corrective action where any significant deficiencies in performance were found. 2 ------- A. Metrics There are two general types of metrics used to assess program performance. The first are data metrics, which reflect verified inspection and enforcement data from the national data systems of each media, or statute. The second, and generally more significant, are file metrics, which are derived from the review of individual facility files in order to determine if the program is performing their compliance and enforcement responsibilities adequately. Other information considered by EPA to make performance findings in addition to the metrics includes results from previous SRF reviews, data metrics from the years in-between reviews, multi-year metric trends. B. Performance Findings The EPA makes findings on performance in five program areas: • Data - completeness, accuracy, and timeliness of data entry into national data systems • Inspections - meeting inspection and coverage commitments, inspection report quality, and report timeliness • Violations - identification of violations, accuracy of compliance determinations, and determination of significant noncompliance (SNC) or high priority violators (HPV) • Enforcement - timeliness and appropriateness of enforcement, returning facilities to compliance • Penalties - calculation including gravity and economic benefit components, assessment, and collection Though performance generally varies across a spectrum, for the purposes of conducting a standardized review, SRF categorizes performance into three findings levels: Meets or Exceeds: No issues are found. Base standards of performance are met or exceeded. Area for Attention: Minor issues are found. One or more metrics indicates performance issues related to quality, process, or policy. The implementing agency is considered able to correct the issue without additional EPA oversight. Area for Improvement: Significant issues are found. One or more metrics indicates routine and/or widespread performance issues related to quality, process, or policy. A recommendation for corrective action is issued which contains specific actions and schedule for completion. The EPA monitors implementation until completion. C. Recommendations for Corrective Action Whenever the EPA makes a finding on performance of Area for Improvement, the EPA will include a recommendation for corrective action, or recommendation, in the report. The purpose of recommendations are to address significant performance issues and bring program performance back in line with federal policy and standards. All recommendations should include 3 ------- specific actions and a schedule for completion, and their implementation is monitored by the EPA until completion. III. Review Process Information Clean Air Act (CAA) Dates of Virtual File Review: August 23-26, 2021 Environmental Protection Agency (EPA) contacts include: Enforcement and Compliance Assurance Division Erin Malone, Air Inspector & State Liaison (Lead) Kurt Eisner, Senior Environmental Engineer Carly Joseph, Air Inspector Isabella Powers, Air Inspector Air and Radiation Division Riley Burger, Permit Specialist Air Management Services (AMS) contacts include: Thomas Barsley, Chief for Facility, Compliance & Enforcement Daniel Henkin, Engineering Supervisor ------- Executive Summary Areas of Strong Performance The following are aspects of the program that, according to the review, are being implemented at a high level: Clean Air Act (CAA) • The EPA Review Team found AMS's ICIS-Air data entry to be timely for compliance and enforcement minimum data requirements (MDRs) as well as stack tests and stack test results. • AMS's inspection program conducted all full compliance evaluations (FCEs) committed for major, mega, and synthetic minor 80% (SM-80) sources1. Compliance Monitoring Reports (CMRs) reviewed provided sufficient documentation to determine compliance and the EPA Review Team commented that the CMRs were well-written, organized, and thorough. Priority Issues to Address The following are aspects of the program that, according to the review, are not meeting federal standards and should be prioritized for management attention: Clean Air Act (CAA) • While data has been entered timely, as described above, the EPA Review Team found inaccurate data entered into ICIS-Air throughout the file review process. Inaccuracies were noted for Title V Annual Compliance Certifications, stack test results, penalty amounts, and formal enforcement actions. • AMS's enforcement actions did not consistently include corrective actions (e.g., injunctive relief or demonstration of compliance) to return the facility to compliance. A majority of the enforcement actions reviewed were penalty-only orders with nearly 20% of the actions not achieving or documenting compliance prior to close out of the action. • AMS's penalty matrix does not include a section for an economic benefit component. All penalty calculations reviewed included a gravity component, however, an economic benefit component was not included in the matrix. 1 AMS conducted virtual inspections in FY2020 per the Susan Bodine memo titled Recommended Processes for Adjusting Inspection Commitments Due to the COVID-19 Public Health Emergency dated July 22, 2020. 5 ------- Clean Air Act Findings CAA Element 1 - Data Finding 1-1 Meets or Exceeds Expectations Recurring Issue: No Summary: AMS entered all compliance monitoring minimum data requirements (MDRs), stack tests, and stack test results into ICIS-Air in a timely manner. Greater than 95% of FY2020 enforcement MDRs were entered timely into ICIS-Air. Furthermore, two of the three late enforcement MDRs were late by just two days. AMS did not have any HPVs in FY2020 or FY2019, therefore, the EPA Review Team looked at FY2016, FY2017, and FY2018 data to evaluate HPV timeliness. This supplemental review indicated that AMS entered all HPVs into ICIS-Air in a timely manner. Explanation: AMS demonstrated that their data is entered timely into ICIS-Air on a consistent basis. The data metric analysis found that 100% of the 62 MDRs were timely reported to ICIS-Air. Likewise, AMS also timely reported their stack tests and stack test results into ICIS-Air 100% of the time for FY2020. Lastly, AMS timely entered greater than 95% of FY2020 enforcement MDRs into ICIS-AIR. There were only three late entries for metric 3b3 for FY2020 and two of the three were merely two days late. AMS did not identify any HPVs in FY2020 so metric 3a2 could not be evaluated for FY2020. However, AMS had identified a total of three HPVs in FY2016, FY2017, and FY2018. All three of the HPVs identified were timely reported to ICIS-Air. Therefore, the complete evaluation of metric 3a2 is that AMS meets or exceeds expectations. The EPA Review Team evaluated 32 FRVs from FY 2020 to ensure that none of them should have been elevated to HPV status. The team determined that all 32 FRV case files reviewed were accurately determined not to be HPVs. Therefore, there were no HPVs to enter in relation to metric 3a2. 6 ------- Relevant metrics: Metric ID Number and Description Natl Goal Natl Avg State N State D State Total 3a2 Timely reporting of HPV determinations [GOAL] 100% 40.6% 0 0 N/A 3b 1 Timely reporting of compliance monitoring MDRs [GOAL] 100% 74.3% 62 62 100% 3b2 Timely reporting of stack test dates and results [GOAL] 100% 59.4% 33 33 100% 3b3 Timely reporting of enforcement MDRs [GOAL] 100% 76.3% 64 67 95.5% AMS Response: AMS agrees with EPA findings and will strive to maintain timely data entry. CAA Element 1 - Data Finding 1-2 Area for Improvement Recurring Issue: Recurring from Rounds 2 and 3 Summary: During the file review, the EPA Review Team found that only one third of files reviewed had completely accurate MDR data in ICIS-Air. Explanation: The EPA Review Team found that only 35% of the facility files had completely accurate MDR data entered into ICIS-Air. It should be noted that this was identified through the file review portion of the SRF where the EPA Review Team compared the actual file to what was entered in ICIS-Air. While the Data Metric Analysis found that the data was entered timely, as described in 7 ------- Finding 1-1, the file review identified the data entered was inaccurate. Although this is an improvement from Rounds 2 and 3, it is still a significant issue with AMS's reporting of compliance and enforcement activities to ICIS-AIR. Some of the issues that the EPA Review Team found in Round 4 include: • Title V Annual Compliance Certification received and reviewed dates in ICIS-Air were not aligned with the dates in the facility file; • Overdue stack test results showing as "pending" in ICIS-Air (the CMS Policy requires that the date and result of all stack tests are entered into ICIS-Air within 120 days of completion of the test. Timeliness was met, but data accuracy was not); • Assessed penalty amounts not entered into formal enforcement action case files in ICIS- Air; Issued date of formal enforcement actions incorrectly listed as date the penalty was paid in ICIS-Air; and • Some formal enforcement actions missing entirely from ICIS-Air. AMS has experienced a loss of institutional knowledge over the past few years due to staff turnover. Over the last few years since Round 3, AMS lost 14 experienced staff and managers that were familiar with MDRs and ICIS-Air entry. Currently, AMS does not have a dedicated staff person to enter MDRs into ICIS-Air and each inspector is responsible for entering their own data. The limitations of new and inexperienced staff with ICIS-Air is causing data deficiencies. EPA Region 3 generally recommends that one or two staff people act as gatekeepers to ensure ICIS-Air data is entered timely, accurately, and consistently. Entering accurate MDR data has been a historical issue for AMS. In Round 2, AMS was found not to have accurately entered data for stack test results into the data system. Additionally, only 14% of the files reviewed had accurate compliance monitoring and enforcement data in the data system. In Round 3, less than 20% of the facilities reviewed were found to have completely accurate MDR data in ICIS-Air when compared to the files. Relevant metrics: .. , . .. , . ,, Natl Natl State State State Metric ID Number and Description „ , ... _ < , , 1 Goal Avg N D Total 2b Files reviewed where data are accurately reflected in the national data system [GOAL] 100% I 11 f 31 35.5% AMS Response: AMS agrees with EPA findings. AMS will follow EPA recommendations to reach the target of minimum 85% accuracy. 8 ------- Recommendation: U"' Due Dale # Recommendation 1 | 05/01/2022 No later than 60 days from final report issuance, AMS to provide names of staff to EPA that will be dedicated to ICIS-Air entry and quality control. EPA to provide ICIS-Air training for selected AMS staff to be trained in entering data into ICIS-Air. 2 | 10/01/2022 After the first full quarter of implementation of the new data entry procedures, EPA will review a representative number of files to confirm that appropriate data is being accurately entered into ICIS-Air with a result of 85% for metric 2b. Files will be reviewed at 6 months, 9 months, and 12 months following the ICIS-Air training. CAA Element 2 - Inspections Finding 2-1 Meets or Exceeds Expectations Recurring Issue: No Summary: AMS met the negotiated frequency for compliance evaluations of the Compliance Monitoring Strategy (CMS). Additionally, all Compliance Monitoring Reports (CMRs) reviewed provided sufficient documentation to determine facility compliance and document the Full Compliance Evaluations (FCEs) elements. The EPA Review Team found that the CMRs were well-written, organized, and thorough. AMS also reviewed all of the Title V Annual Compliance Certifications (TVACCs) that were scheduled to be reviewed in FY2020. Explanation: Element 2 analyzes the file and data metrics regarding inspections to ensure that FCEs contain the required documentation, CMRs have sufficient documentation to determine compliance, all major and SM-80 sources on the CMS plan had an FCE in FY2020, and that all TVACCs due to be submitted and reviewed in FY2020 were in fact submitted and reviewed. In all of these metrics, AMS scored 100% for FY2020 achieving a level of meets or exceeds expectations for metrics under Element 2. 9 ------- Metrics 5a and 5b ensure that all of the committed major and SM-80 sources on the CMS Plan had completed FCEs during FY2020. AMS completed all CMS commitments for compliance evaluations for major and SM-80 sources in FY2020. Metric 5c is not applicable to AMS as they do not have an Alternative CMS plan. The final data metric for Element 2, Metric 5e, is to ensure that all TVACCs that are due are received and reviewed. AMS reviewed all 28 TVACCs due to be submitted and reviewed in FY2020. The file review metrics for Element 2 are to ensure that all of the FCE elements are documented (metric 6a) and that the CMRs and facility files provide sufficient documentation to determine compliance of the facility (metric 6b). The EPA Reviewer Team found that all of the FCE elements were well documented in the inspection reports and that there was sufficient documentation to determine compliance of the facilities reviewed. In FY2020, AMS conducted 13 FCEs at major sources, of which eight were conducted as off-site FCEs per the Susan Bodine memo2. AMS also conducted three FCEs at SM-80 sources, of which one was completed off-site per the Susan Bodine memo. Off-site activities included but are not limited to: • A thorough prescreening review of facility file and records, including reports, emission inventories, CEMs data, etc, • Extensive telephone interviews and discussions with facility representative, • Requesting and reviewing all records required to document facility compliance with operating permit, and • Photographic documentation where necessary for additional verification. 2 AMS conducted virtual inspections in FY2020 per the Susan Bodine memo titled Recommended Processes for Adjusting Inspection Commitments Due to the COVID-19 Public Health Emergency dated July 22, 2020 10 ------- Relevant metrics: Metric ID Number and Description Natl Goal Natl Avg State N State D State Total 5a FCE coverage: majors and mega-sites [GOAL] 100% 85.7% 13 13 100% 5b FCE coverage: SM-80s [GOAL] 100% 93.6% 3 3 100% 5e Reviews of Title V annual compliance certifications completed [GOAL] 100% 82.8% 28 28 100% 6a Documentation of FCE elements [GOAL] 100% 19 19 100% 6b Compliance monitoring reports (CMRs) or facility files reviewed that provide sufficient documentation to determine compliance of the facility [GOAL] 100% 19 19 100% AMS Response: AMS agrees with EPA finding. CAA Element 3 - Violations Finding 3-1 Meets or Exceeds Expectations Recurring Issue: No Summary: AMS made accurate compliance and HPV determinations. 11 ------- Explanation: Since FY2015, AMS has been consistently above the national average for data indicator metric 7al (FRV 'discovery rate' based on inspections at active CMS sources). Therefore, no supplemental files were pulled. Regarding accurate compliance determinations, 36 of the 37 files reviewed had accurate compliance determinations. Therefore, for metric 7a (accurate compliance determinations) AMS achieved a score of 97%. Metric 8a (HPV discovery rate at majors) is below the national average as AMS did not identify any HPVS in FY2020. Supplemental files were pulled from FY2019 because all facility files with activity to review were selected for FY2020 to evaluate the accuracy of HPV determinations. All 32 files reviewed had accurate HPV determinations. Therefore, the EPA Review Team concluded that AMS is accurately making HPV determinations (i.e., metric 8c). AMS did not identify any HPVs in FY2020 and therefore metric 13 could not be evaluated for FY2020. However, in FY2016, FY2017, and FY2018 AMS identified an HPV in each fiscal year and all three HPVs were timely identified (i.e., Day Zero was within 90 days of the discovery date). Relevant metrics: Metric ID Number and Description Natl Goal Natl Avg State N State D State Total 7a Accurate compliance determinations [GOAL] 100% 36 37 97.3% 7al FRV 'discovery rate' based on inspections at active CMS sources 6.8% 13 34 38.2% 8a HPV discovery rate at majors [SUPPORT] 2.4% 0 0 N/A 8c Accuracy of HPV determinations [GOAL] 100% 32 32 100% 13 Timeliness of HPV Identification [GOAL] 100% 83.8% 0 0 N/A AMS Response: AMS agrees with EPA finding. 12 ------- CAA Element 4 - Enforcement Finding 4-1 Meets or Exceeds Expectations Recurring Issue: No Summary: AMS took timely and appropriate enforcement actions consistent with the HPV policy3. Explanation: AMS did not identify any HPVs in FY2020, therefore, the EPA Review Team reviewed facility files from FY2017, FY2018, and FY2021 to assess AMS's enforcement responses to HPVs. Five HPVs were reviewed, and the EPA Review Team found that 100% of the reviewed HPVs had appropriate enforcement responses. Therefore, achieving "Meets or Exceeds Expectations" for metric 10b under the Enforcement Element. Metric 14 is in place to review CD&RTs to ensure that they meet the required HPV policy elements. Since AMS did not have any CD&RTs to evaluate the reviewed periods, metric 14 is not applicable. 3 Timely and Appropriate Enforcement Response to High Priority Violations- 2014 dated August 25, 2014 13 ------- Relevant metrics: Metric ID Number and Description Natl Goal Natl Avg State N State D State Total lOal Rate of Addressing HPVs within 180 days [SUPPORT] 44.2% 0 0 N/A 10b Percent of HPVs that have been addressed or removed consistent with the HPV Policy [GOAL] 100% 5 5 100% lObl Rate of managing HPVs without formal enforcement action [SUPPORT] 11.8% 0 0 N/A 14 HPV case development and resolution timeline in place when required that contains required policy elements [GOAL] 100% 0 0 N/A AMS Response: AMS agrees with EPA finding. CAA Element 4 - Enforcement Finding 4-2 Area for Attention Recurring Issue: No Summary: AMS's formal enforcement documents do not always include corrective actions (e.g. injunctive relief or demonstration of compliance) to return the facility to compliance or document compliance prior to close out. 14 ------- Explanation: Generally, the EPA Review Team found that enforcement actions included corrective actions in formal enforcement responses. However, five enforcement actions were closed out only by payment of a penalty with no documentation of injunctive relief or that compliance was achieved prior to remittance of the penalty. It was not clear to the EPA Review Team that compliance was achieved in the case file documents reviewed. After penalties were paid, the action was closed without reassurance that the violations had been resolved and the facility returned to compliance. While assessing a penalty for a violation is in line with the CAA, it is imperative for AMS to confirm that the facility has demonstrated a return to compliance for the specific violation(s) cited before the penalty has been remitted. Penalty-only orders are an important tool in enforcement and compliance activities but must be used in conjunction with assurance and evidence that the source has returned to compliance for the violation(s) cited. When appropriate, AMS should utilize consent orders in consultation with their legal department to ensure that corrective actions are/have been conducted, and enforcement actions are in line with the egregiousness of the violation(s). Relevant metrics: Metric ID Number and Description 9a Formal enforcement responses that include required corrective action that will return the facility to compliance in a specified time frame or the facility fixed the problem without a compliance schedule [GOAL] AMS Response: AMS agrees with EPA finding. AMS will update Assessed Penalty Letters to include language that the source returned to compliance or set a requirement to return to compliance by a specified time and add evidence of return to compliance in files. Natl Natl State State State Goal Avg N D Total 100% 24 29 82.8% 15 ------- CAA Element 4 - Enforcement Finding 4-3 Area for Improvement Recurring Issue: Recurring from Round 3 Summary: AMS addressed 50% of the HPVs timely or had a case development and resolution timeline (CD&RT) in place prior to day 225 in accordance with the HPV policy. Explanation: AMS did not have any HPVs to assess during FY2020. In FY2020, there was one HPV that was selected for off-ramping, meaning it is no longer subject to the HPV policy, but the state will still ensure the facility's return to compliance. Therefore, supplemental files were selected to assess AMS's performance of this element. HPVs from FY2017, FY2018, and FY2021 were selected. Of the reviewed HPVs, AMS only addressed 50% of them in a timely manner or had a CD&RT in place prior to day 225. However, there were extenuating circumstances surrounding the two HPVs that were not addressed timely or had a CD&RT in place. Both were for a refinery around the time of an explosion at the facility and its subsequent bankruptcy. In Round 3, AMS addressed two HPVs before day 180 so there were no unaddressed HPVs that required an HPV CD&RT during FY2015. In Round 2, metric 10a was an area for potential concern as AMS addressed 77% of their HPVs after 270 days after day zero or continue to be unaddressed and 270 days has passed. During the Round 2 review, 35 HPVs were reviewed for timeliness. 16 ------- Relevant metrics: Metric ID Number and Description Natl Natl State State State Goal Avg N D Total 10a Timeliness of addressing HPVs or alternatively having a case development and resolution timeline in place [GOAL] 100% 50% AMS Response: AMS agrees with EPA finding. AMS will institute an HPV tracking process to ensure that a case development and resolution timeline (CD&RT) is in place prior to day 225 in accordance with the HPV policy. Recommendation: Ucc Due Dale Recommendation 12/31/2022 AMS shall institute an HPV tracking process to ensure that unaddressed HPVs do not reach day 225 without a CD&RT in place. CAA Element 5 - Penalties Finding 5-1 Meets or Exceeds Expectations Recurring Issue: No Summary: Nearly all of the final penalties that were reduced from the initial assessed penalties had adequate justifications for those reductions. In addition, all penalties had documentation of penalty remittance in the file. 17 ------- Explanation: Metric 12a assesses the agency's ability to document the difference between initial penalty calculation and final penalty. The EPA Review Team found that 15 of the 17 penalties collected had documentation to explain the difference from initial penalty calculation to final penalty collected. The EPA Review Team noted that in previous SRF rounds, this finding was an "Area for Improvement" and the recommendation required an SOP. AMS has shown significant improvement in this metric with the implementation of the SOP that was developed. Metric 12b assesses the agency's ability to document that an assessed penalty was in fact collected. AMS successfully provided documentation for all 26 files reviewed with penalties. Relevant metrics: Metric ID Number and Description Natl Goal Natl Avg State N State D State Total 12a Documentation of rationale for difference between initial penalty calculation and final penalty [GOAL] 100% 15 17 88.2% | 12b Penalties collected [GOAL] 100% | 26 26 100% 1 ! | AMS Response: AMS agrees with EPA finding. CAA Element 5 - Penalties Finding 5-2 Area for Improvement Recurring Issue: No Summary: AMS did not assess an economic benefit component. 18 ------- Explanation: All penalty calculations reviewed included a gravity component, however economic benefit components are not consistently included in AMS penalty calculations. The majority of penalty matrices reviewed did not include a section for an economic benefit component. The 1991 Clean Air Act Stationary Source Civil Penalty Policy requires that an economic benefit be included in the penalty amount or a reason for mitigation to be documented in the case file. The AMS penalty calculations reviewed did not document a justification to explain why an economic benefit component was not assessed or applicable. Relevant metrics: Metric ID Number and Description Natl Natl State State State Goal Avg N D Total 1 la Penalty calculations reviewed that document gravity and economic benefit [GOAL] 100% 100% 28 7.1% AMS Response: AMS agrees with EPA finding. AMS will revise penalty calculation spreadsheet template to include a section for assessing economic benefit as well as a comments section to include notes in the event that the economic benefit component is mitigated. Recommendation: Ucc # Due Dale Recommendation 1 07/31/2022 AMS shall revise their penalty calculation spreadsheet template to include a section for assessing economic benefit as well as a comments section to include notes in the event that the economic benefit component is mitigated. EPA will review the revised spreadsheet prior to instituting the new version. 2 07/31/2023 EPA to review random penalty calculations on a quarterly basis to ensure that economic benefit is being considered and documented with 85%) accuracy percentage as the goal. 19 ------- |