$
<
33
\
mj
¦p.. ,\s
PRO"*4-
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Evaluation Report
EPA's Management of Interim
Status Permitting Needs
Improvement to Ensure
Continued Progress
Report No, 2007-P-00005
December 4, 2006
-------
Report Contributors: Carolyn Copper
Steve Hanna
Anne Emory
Anne Bavuso
Bao Chuong
Bryan Holtrop
Abbreviations
EPA U.S. Environmental Protection Agency
GPRA Government Performance Results Act
OIG Office of Inspector General
OSW Office of Solid Waste
PAA Permitting and Corrective Action Program Area Analysis
RCRA Resource Conservation and Recovery Act
RCRAInfo Resource Conservation and Recovery Act Information System
TSDs Treatment, Storage, and Disposal Facilities
Cover photo: US Filter Westates (or "Westates") is an interim status carbon regeneration
facility located on the Colorado River Indian Reservation near Parker, Arizona.
US Filter Westates recently changed its name to U.S. Filter Ionpure (EPA photo).
-------
$
<
73
\
(J
T
^ c>s°
pRQl^
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
2007-P-00005
December 4, 2006
Why We Did This Review
This review was conducted to
evaluate the effectiveness of
the U.S. Environmental
Protection Agency's (EPA's)
(1) efforts to address the
regulation of hazardous waste
units granted interim status
under the Resource
Conservation and Recovery Act
(RCRA), and (2) information
management system
(RCRAInfo) in tracking permit
information for interim status
units.
Background
Under Subtitle C of RCRA,
hazardous waste facility units
that perform treatment, storage,
and disposal activities must
receive permits prior to
operation. When new
hazardous waste statutes or
regulations are implemented,
units that already exist may
continue operating in a
condition known as interim
status by notifying EPA of their
intent and supplying basic
information. Units may remain
in interim status until issuance
or denial of a permit.
For further information,
contact our Office of
Congressional and Public
Liaison at (202) 566-2391.
To view the full report,
click on the following link:
www.epa.qov/oiq/reports/2007/
20061204-2007-P-00005.pdf
Catalyst for Improving the Environment
EPA's Management of Interim Status Permitting Needs
Improvement to Ensure Continued Progress
What We Found
Interim status is a temporary designation, but some units have existed for as many
as 25 years without formal issuance or denial of a permit, or other regulatory
controls. Under the Government Performance and Results Act (GPRA), EPA has
a RCRA National Permitting Goal to ensure that all units at hazardous waste
facilities have "controls in place." EPA includes interim status units in this goal,
and the Agency's data indicate that it has made progress in ensuring controls are
in place at interim status units. As of 2005, EPA had attained the "controls in
place" designation for 89 percent of RCRA hazardous waste facilities.
However, EPA's continued progress may be compromised because (1) the
Agency has not sufficiently documented some changes to the baseline it uses to
measure progress; (2) EPA does not prioritize its National Permitting Goal
activities according to the potential risks posed by hazardous waste facilities or
units, including the amount of time a unit may have been operating without
required controls; (3) EPA does not monitor the creation of "new" interim status
units in its reporting and tracking system (RCRAInfo); and (4) RCRAInfo lacks
other system controls to protect data integrity and data quality, which may lead to
the loss of historical information needed to track permit status. Despite data
quality problems, RCRAInfo data are available for public use without appropriate
disclaimers.
What We Recommend
In order to ensure valid progress in achieving "controls in place" at interim status
units, we recommend that the Assistant Administrator for Solid Waste and
Emergency Response:
• Implement a process to document changes to the GPRA National
Permitting Goal baseline.
• Review State GPRA National Permitting Goal projections for 2008 and
2011 to identify opportunities for prioritizing facilities based on risk,
including time in interim status.
• Oversee the designation of "new" interim status units in RCRAInfo.
• Implement RCRAInfo system controls to ensure data integrity and
improve data quality.
• Provide a disclaimer on data released publicly from RCRAInfo until data
quality controls are in place.
The Agency generally concurred with our recommendations.
-------
$ A \
\mli
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICE OF
INSPECTOR GENERAL
December 4, 2006
MEMORANDUM
SUBJECT:
EPA's Management of Interim Status Permitting Needs
Improvement to Ensure Continued Progress
Report No. 2007-P-00005
TO:
Susan Parker Bodine
Assistant Administrator for Solid Waste and Emergency Response
This is our report on the subject evaluation conducted by the Office of Inspector General (OIG)
of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe
the problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
Final determinations on matters in this report will be made by EPA managers in accordance with
the established resolutions procedures.
The estimated cost of this report - calculated by multiplying the project's staff days by the
applicable daily full cost billing rate s in effect at the time - is $480,000.
Action Required
In accordance with EPA Manual 2750, you are required to provide a written response to this
report within 90 calendar days. Your response should include a corrective action plan for agreed
upon actions, including milestone dates. We have no objections to the further release of this
report to the public. This report will be available at http://www.epa.gov/oig. If you or your staff
have any questions regarding this report, please contact Carolyn Copper at (202) 566-0829 or
copper.carolyn@epa.gov, or Steve Hanna at (415) 947-4527 or hanna.steve@epa.gov.
Sincerely,
BITTARoderi ck
Acting Inspector General
-------
EPA's Management of Interim Status Permitting Needs Improvement to Ensure Continued Progress
Table of C
Chapters
1 Introduction 1
Purpose 1
Background 1
Scope and Methodology 2
2 Implementation of the GPRA National Permitting Goal Needs
Improvement to Ensure Continued Progress 4
Development of the GRPA National Permitting Goal 4
EPA Reports Progress with the GPRA National Permitting Goal 4
Undocumented Changes to the GPRA Baseline May Affect EPA's
Reported GPRA Progress 5
Risk-Based Prioritization of Interim Status Units Would Be Consistent
With Other EPA Approaches 6
Creation of Interim Status Units Needs Additional Oversight 7
Recommendations 7
Agency Comments and OIG Evaluation 7
3 RCRAInfo Improvements Needed to Effectively Track and Validate
Permitting Progress 9
RCRAInfo Lacks System Controls to Prevent Data Loss and
Protect Data Integrity 9
Additional RCRAInfo Data Quality Concerns 10
RCRAInfo Data Released to the Public Wthout Needed Cautions on
Data Quality 11
Recommendations 11
Agency Comments and OIG Evaluation 11
Status of Recommendations and Potential Monetary Benefits 13
Appendices
A How is the Permitting Goal for GPRA Met? 14
B Legal and Operating Status Code Matrix 16
C Agency Response and OIG Comments 18
D Distribution 24
-------
Chapter 1
Introduction
Purpose
This report focuses on efforts of the U.S. Environmental Protection Agency
(EPA) to address the regulation of hazardous waste units granted interim status
under Subtitle C of the Resource Conservation and Recovery Act (RCRA). We
addressed the following questions:
• How effectively has EPA addressed the management of interim status units?
• How effective is EPA's information management system in tracking permit
information for interim status units?
Background
Subtitle C of RCRA regulates the identification, generation, transportation, and
disposal of hazardous waste. States implement RCRA Subtitle C programs with
EPA oversight, except for Alaska and Iowa, where EPA administers the program.
In 2003, over 1,700 treatment, storage, and disposal (TSD) facilities managed
over 40 million tons of hazardous waste. Hazardous waste includes metals such
as lead, cadmium, and chromium; and organic solvents such as xylene and
toluene.
Hazardous waste facilities are subject to permitting requirements. A hazardous
waste management facility contains a single or multiple TSD units, each of which
must have a permit for operation. Obtaining a RCRA permit requires public
participation and technical review by State agencies or EPA. With some
exceptions, RCRA permits are good for 10 years and then require renewal. The
RCRA permit process consists of two parts:
• Part A - This application requires basic facility information, proposed
activities, and the types of hazardous waste managed.
• Part B - This application consists of detailed site information that the State or
EPA extensively reviews before approving or denying. The review may take
several years to complete, and includes public notice and comment.
EPA tracks permit progress in the RCRAInfo database, which EPA maintains.
However, States have direct access and primary responsibility for inputting permit
data.
1
-------
If a hazardous waste unit is already in operation when new legislation or
regulation governing the unit comes into effect, the unit can operate in "interim
status." To obtain interim status, an owner or operator of a unit first files Part A
of the permit application, followed by Part B when requested by the State or EPA.
Under current regulations, interim status TSD units may continue operations
without a full permit as long as they comply with general facility and TSD unit-
specific standards. Land disposal units must also certify to the State or EPA that
they comply with groundwater monitoring and financial assurance requirements.
Although interim status is a temporary designation, units have remained in that
status for many years without issuance or denial of a permit, or the
implementation of other regulatory controls. The average time in interim status is
approximately 17 years, and the longest current time is 25 years.
Scope and Methodology
We performed this evaluation in accordance with the Government Auditing
Standards, issued by the Comptroller General of the United States. We conducted
fieldwork between May 2005 and April 2006. Our evaluation included analysis
of all units ever in interim status. This included over 14,000 hazardous waste
units and a smaller subset of these units, approximately 4,500 operating and post-
closure interim status units, which EPA has targeted through the Government
Performance Results Act (GPRA) National Permitting Goal. EPA's Office of
Solid Waste (OSW), within the Office of Solid Waste and Emergency Response,
supplied electronic data from RCRAInfo. The RCRAInfo data included the
facility identifier and name, unit, initial and current legal and operating status
codes, and details on the facility and unit GPRA National Permitting Goal status.
To address whether EPA's management of interim status units was effective, we
reviewed key regulations, documents, reports, and EPA Webpages relating to the
management and oversight of interim status units, including:
• 40 CFR 264 and 265
• EPA Annual and Strategic Plans
• EPA Regional Strategic Plans
• EPA Regional Beginning of Year Plans and Management Performance
Agreements
• RCRA Implementation Plans
• RCRA Orientation Manual
• RCRAInfo Comprehensive Permitting Reports
• RCRAInfo Reports about meeting the GPRA Goal
• Government Accountability Office Reports
• Prior EPA OIG Reports
We analyzed permitting data for interim status TSD units included in the GPRA
National Permitting Goal (approximately 4,500 units), including the year initially
2
-------
in interim status and length of time in interim status. We reviewed EPA's
reported progress under the National Permitting Goal and the composition of the
baseline of this goal.
To address whether RCRAInfo is effective at tracking permit information, we
performed extensive analyses of RCRAInfo data to evaluate internal system
controls and data quality, supplemented by comprehensive permitting reports
generated from RCRAInfo. We reviewed the recommendations in the Permitting
and Corrective Action Program Area Analysis (PAA)1 to identify the unsupported
legal and operating status code combinations. In evaluating internal system
controls, we analyzed the ability of RCRAInfo to (1) track historical changes to
data, and (2) ensure that data in the system have values within accepted ranges.
For both objectives, we supplemented the analyses of documentary evidence with
interviews of staff and managers from the following organizations:
• osw
• EPA's Office of Enforcement and Compliance Assurance
• EPA Regions 6, 8, 9, and 10
• Selected State environmental regulatory agencies (Colorado, Connecticut,
New Mexico, Texas, and Washington)
We selected States to interview that had made either more or less progress in
permitting or establishing controls for interim status units. By extension, we
interviewed regions where these States were located. During interviews, we
discussed EPA's efforts to address interim status units, the development of the
GPRA goal and the baseline, and RCRAInfo and its use as a management tool.
Data Quality
Our analyses were hampered by poor data quality. According to EPA, data for
units on the GPRA baseline (about 4,500) were more accurate than data for all the
units that had ever been in interim status (about 14,000). Interim status units on
the GPRA baseline are a subset of units ever having been in interim status.
Despite using the GPRA baseline data, we still encountered data quality
problems. These problems are described in our report.
Prior Audit/Evaluation Coverage
In a 2004 OIG report,2 we reported that the State of Idaho had neglected
permitting interim status units at Idaho's National Environmental and Engineering
Laboratory.
1 The PAA is a collaborative report developed by States and EPA to reassess and improve information management
within RCRAInfo. The PAA was released in July 2005.
2Review of EPA 's Response to Petition Seeking Withdrawal of Authorization for Idaho's Hazardous Waste Program,
Report No. 2004-P-00006, February 5, 2004.
3
-------
Chapter 2
Implementation of the GPRA National Permitting Goal
Needs Improvement to Ensure Continued Progress
We found challenges that potentially compromise the validity of EPA's reported
progress at achieving its GPRA National Permitting Goal. EPA established its
National Permitting Goal to achieve a "controls in place" designation at RCRA
facilities, including interim status facilities. However, EPA did not have a system
control in RCRAInfo to document the changes to the GPRA baseline. EPA does
not prioritize achieving "controls in place" for interim status units based on their
potential risks, including time in interim status. There is no documentation or
review to validate the creation of new interim status units.
Development of the GPRA National Permitting Goal
In 1997, EPA formalized efforts to prevent releases of hazardous waste from TSD
units, such as those in interim status, by establishing a National Permitting Goal
as part of GPRA. EPA's initial GPRA goal was to have 80 percent of all RCRA
baseline facilities achieve an approved "controls in place" designation by 2005.
Achieving this "controls in place" designation means that there are measures in
place to ensure the prevention of a hazardous waste release. A hazardous waste
TSD facility obtains the "controls in place" designation when all of its units have
a control in place. This designation is based on the combination of a unit's legal
and operational status (see Appendices A and B).
Roughly half (approximately 340) of the remaining interim status units without
"controls in place" are still operating. A unit may attain a control in place
designation by obtaining a permit or other regulatory controls recognized by EPA.
EPA Reports Progress with the GPRA National Permitting Goal
In 1999, EPA began formally tracking and reporting progress on the National
Permitting Goal in its annual performance report. As shown in Figure 2-1, EPA
reported consistently meeting or exceeding its annual GPRA National Permitting
Goal targets. EPA reported meeting the 2005 GPRA goal on schedule.
4
-------
Figure 2-1. RCRA National Permitting Goal Accomplishments (Source: OIG
analysis of EPA data)
GPRA Permitting Accomplishments
100
I 90
80
_c
» 70
o
"E 60
0
^ 50
1 40
w
| 30
£ 20
LL
5S 10
0
a ¦ ¦ I ¦
1999 2000 2001 2002 2003 2004 2005
^Actual • Planned
Furthermore, as shown in Figure 2-2, States and EPA attained "controls in place"
at 89 percent of RCRA hazardous waste facilities.
Figure 2-2. RCRA Permitting Progress (Source: EPA)
Regional Permitting Program Progress
Fiscal Year 2006, Mid-Year results
r5 P\ 1 1
1 \> VIII: 87%
X: 86%
IX: 85%
90% - 95%
85% - 89%
80% - 84%
D
VII: 90%
National
Results
Currently, EPA's GPRA goal is to have 95 percent of RCRA baseline facilities
achieve an approved "controls in place" designation by 2008.
Undocumented Changes to the GPRA Baseline May Affect EPA's
Reported GPRA Progress
EPA defined the GPRA baseline of RCRA facilities in 1997, and began formally
measuring progress against this baseline in 1999. The number of TSD facilities
on the original GPRA baseline decreased in 2000 and 2001, but then remained
constant through 2005. Table 2-1 shows reductions to the GPRA baseline and the
corresponding number of facilities with "controls in place" needed to meet the
GPRA goal for that year. Between 1999 and 2001, EPA reported an increase in
progress towards its goal, although the total number of GPRA facilities required
to achieve the goal decreased.
5
-------
Table 2-1. Baseline Reductions (Source: EPA)
YEAR
BASELINE
GOAL
FACILITIES REQUIRED TO
ACHIEVE GOAL
1999
3,380
61.0%
2062
2000
2,900
67.0%
1943
2001
2,750
68.0%
1870
2002
2,750
75.8%
2085
OSW did not have a system control in RCRAInfo to document why the changes
occurred. According to OSW staff, changes in the baseline resulted from a
nationwide effort to update data to determine if the facilities fit GPRA baseline
criteria. Because the baseline is used to measure progress towards the GPRA
goal, documentation is necessary to identify the reasons for adding or removing
units so that progress can be validated. Insufficient documentation of changes in
the GPRA baseline potentially compromises the validity of EPA's reported
progress.
In response to our review, EPA has modified its process and indicated it will now
document changes to the baseline in RCRAInfo. Beginning in 2006, EPA
instituted a system to track removals of facilities from the baseline. If a facility is
removed from the baseline, the facility will have an "R" (removed) designation
noted in its RCRAInfo entry, and a reason for the removal will be included in the
notes section/comment field.
Risk-Based Prioritization of Interim Status Units Would Be Consistent
With Other EPA Approaches
While EPA's RCRA corrective action goal3 prioritizes actions based on risk, EPA
does not prioritize achieving "controls in place" for interim status units based on
their potential risks. These potential risks include the amount of time units have
been operating without "controls in place." Consequently, progress in getting
"controls in place" at units that pose the greatest risk to human health and the
environment is unclear.
We found no indication that EPA prioritizes units based on their time in interim
status, or that EPA determines when too much time has elapsed for units in
interim status. For example, 88 percent of the existing interim status units on the
GPRA baseline have been in that status longer than the maximum permit length
of 10 years. EPA indicated that the length of time in interim status is not by itself
a meaningful measure of success for all units. However, because interim status is
3 The RCRA corrective action goal ensures that responsible parties who have had a release from their hazardous
waste facility timely investigate and clean up the release. The goal ensures that releases that present the greatest risk
to human health and the enviromnent are dealt with first. The corrective action goal prioritizes sites using the
National Corrective Action Prioritization System, which ranks sites as high, medium, or low priority based on the
potential for human exposure and groundwater contamination.
6
-------
intended to be a temporary state, failure to acknowledge, monitor, and apply
reasonable criteria to the amount of time units have been in interim status could
lead to perceptions that permitting and/or getting "controls in place" at interim
status units is not a priority for States or EPA.
EPA indicated that States have finalized their specific projected accomplishments
for the 2008 National Permitting Goal and are currently finalizing their
projections for 2011. As a result, EPA stated it could not feasibly implement a
risk prioritization mechanism at this time. In the past EPA has not reviewed State
GPRA permitting projections to determine if higher-risk facilities are being
prioritized, however EPA has plans to do so in the future.
Creation of Interim Status Units Needs Additional Oversight
Documentation for the reason a unit obtained interim status cannot be determined
from RCRAInfo. New interim status units have been designated every year since
1980. Given the emphasis placed on achieving controls at existing interim status
units, any new interim status designation in RCRAInfo should require oversight
by EPA. During interviews with States and regions, we discovered that they were
generally not aware of the new interim status units in their jurisdiction, entered
into RCRAInfo. While at least one of the four EPA regions we spoke with
appears to closely monitor and control the creation of new interim status units,
there is no national review of new interim status designations to ensure these are
appropriate actions under the RCRA program.
Recommendations
We recommend that the Assistant Administrator for Solid Waste and Emergency
Response:
2-1 Improve the accuracy of GPRA National Permitting Goal measurement by
defining and implementing criteria for changes to the baseline.
2-2 Review current State GPRA National Permitting Goal projections for
2008 and 2011 to identify any opportunities for prioritizing facilities based
on risk, including time in interim status.
2-3 Improve oversight of new interim status units by (1) implementing a
national level review to validate new interim status designations, and
(2) documenting the reason for interim status in RCRAInfo.
Agency Comments and OIG Evaluation
The Agency agreed with our recommendations. The Agency agreed to clarify the
criteria for baseline changes and to describe when baseline changes are
acceptable. The Agency will continue to track all unit baseline changes in
7
-------
RCRAInfo for the 2008 baseline. The Agency agreed to work with States during
their annual planning process to identify opportunities for prioritizing facilities
that have not met the risk-based goal criteria. The Agency also agreed to make
changes to RCRAInfo, Version 4, to identify conditions that must be satisfied
prior to entering interim status codes and to require comments/notes in RCRAInfo
for unusual cases. Finally, the Agency agreed to create a report for program
managers to identify new interim status units. The Agency will assess the validity
of any new interim status units at the end of 2008. In its 90-day response to the
report, the Agency will need to provide a corrective action plan and milestone
dates for agreed upon actions.
The Agency disagreed with our conclusion that insufficient documentation of
baseline changes potentially compromises permitting progress. However, we
continue to believe that insufficient documentation compromises EPA's ability to
validate reported progress.
8
-------
Chapter 3
RCRAInfo Improvements Needed to Effectively Track
and Validate Permitting Progress
RCRAInfo is not fully effective in tracking permit information because
RCRAInfo lacks some controls to ensure data integrity and data quality. The lack
of data integrity and data quality are due to the ability to overwrite legal and
operating status codes, and the lack of system controls to prevent incorrect data
from being entered into RCRAInfo. This potentially compromises EPA's ability
to validate its progress on the National Permitting Goal. Despite data quality
problems, RCRAInfo data are made available for public use and reporting without
appropriate disclaimers.
RCRAInfo Lacks System Controls to Prevent Data Loss and Protect
Data Integrity
RCRAInfo lacks system controls needed to protect data integrity4 and prevent
data loss. The lack of these controls could lead to the corruption of historical
records, and could potentially compromise EPA's ability to reliably track or
ensure accurate reporting of its progress on the National Permitting Goal.
During our review, we received multiple data sets from EPA listing all TSD units
that had ever been in interim status. From these data sets, we extracted and
analyzed data for interim status units on the GPRA baseline. We found that each
set of data contained units not in the other sets. RCRAInfo does not currently
require that historical records be maintained. If historical data were preserved, a
later data set should contain all information in the earlier set.
We also found that legal and operating status codes and effective dates were
modified for a small number of records. According to EPA, it is possible to
(1) overwrite legal and operating status and initial effective dates in RCRAInfo,
and (2) wipe out prior RCRAInfo entries. We uncovered one case of data
overwriting at a Federal facility in interim status. As of October 2004, interim
status units at this facility were listed as part of all units ever in interim status.
However, data extracted for the facility in November 2004 did not show the units
as ever having been in interim status. Additional analyses of the October 2004
and January 2005 Comprehensive Permitting Reports for this facility showed that
the October 2004 legal code of "interim status" was overwritten with the legal
code of "permitted." The overwriting eliminated any history in RCRAInfo that
these units were ever in interim status.
4 Integrity refers to data not changing so the historical record is preserved.
9
-------
Additional RCRAInfo Data Quality Concerns
We found the following data quality issues, which are described in detail below:
(1) unsupported legal and operating status code combinations, (2) illogical code
sequences, (3) effective dates prior to the enactment of RCRA, and (4) Part A and
Part B dates that are missing or out of order. Addressing weaknesses in
RCRAInfo system controls will improve data quality and the valid measurement
of progress on the GPRA National Permitting Goal.
Unsupported Status Codes
Unsupported status codes are code combinations that EPA has determined do not
support typical program operations, and therefore are not appropriate for entry
into RCRAInfo, except perhaps with explanation. The PAA listed various legal
and operating status code combinations as unsupported by typical program
operations, and generally not valid for use. We found over 60 occurrences of
unsupported legal and operating status code combinations for interim status and
permitted units on the GPRA baseline (see Appendix B for unsupported codes).
These occurrences include over 30 units designated as having "controls in place,"
which makes the designation of "controls in place" questionable for these units.
Illogical Sequences
Illogical sequences of status codes are those that logically should not occur.
Examples of illogical sequences of legal and operating codes include changing (1)
from a legal status of permitted to interim status, (2) from a legal status of either
permitted or interim status to a legal status of non-notifier or never regulated, or
(3) from interim status operating to permitted before construction. We found
approximately 100 illogical sequences of legal and operating status codes for
units on the GPRA baseline.
Pre-RCRA Effective Dates
Interim status effective dates prior to the enactment of RCRA are another example
of poor data quality. An effective date is when a unit's legal and operating status
became effective. RCRA requirements determine legal and operational status.
While RCRA regulations for hazardous waste were not effective until 1980, we
found approximately 100 occurrences of effective dates prior to 1980. Poor data
quality compromises the ability to track permitting goal accomplishments.
Part A and Part B Data Quality Concerns
As described earlier, the RCRA permit application process consists of two parts -
Part A and B. Although the Part B process follows Part A, EPA's data show that
25 percent of interim status units listed Part B dates that preceded Part A dates.
10
-------
In addition, only 36 percent of the units in interim status had information for both
the Part A and Part B dates. Effective tracking of interim status units requires an
actual date for the Part A application and a "call in" and "received date" for the
Part B application. Without these dates, it is difficult to determine how long it
takes to get an interim status unit permitted.
RCRAInfo Data Released to the Public Without Needed Cautions on
Data Quality
RCRAInfo data are available to the public through EPA's Envirofacts Data
Warehouse and the Right to Know Network without cautions or disclaimers about
data quality. EPA staff are working to prevent inactive facility data from
populating Envirofacts and are evaluating options to address data quality. These
include making needed modifications to RCRAInfo, creating reports to identify
errors, and/or providing caveats for data accessible by the public.
Recommendations
We recommend that the Assistant Administrator for Solid Waste and Emergency
Response:
3-1 Implement system controls to ensure that permit information in RCRAInfo
cannot be modified unless a history of the modification and its rationale is
also created.
3-2 Improve the current and future quality of RCRAInfo data by:
3-2.1 Generating reports that identify unsupported legal and operating
status codes, illogical sequences of status codes, entries with dates
prior to the inception of RCRA, and missing mandatory data
elements such as Part A and Part B call-in and received dates; and
correcting entries as needed.
3-2.2 Implementing control mechanisms that would prevent these types
of incorrect data entries in the future.
3-3 Provide a caution or disclaimer on data released publicly from RCRAInfo,
until data quality controls are put in place.
Agency Comments and OIG Evaluation
The Agency agreed with our recommendations. The Agency agreed to maintain
unit level history to provide a better historical view of changes. The Agency also
agreed to develop reports to allow RCRAInfo users to verify that data are correct
and complete. The Agency agreed to provide disclaimer language for RCRAInfo
that is available in Envirofacts.
11
-------
EPA also agreed to release data only on active facilities to Envirofacts. In its
90-day response to the report, the Agency will need to provide a corrective action
plan and milestone dates for agreed upon actions.
The Agency disagreed with our conclusion that the lack of some data quality and
integrity controls in RCRAInfo potentially compromises EPA's continued
permitting progress on the GPRA National Permitting Goal. However, we
continue to believe that the potential for incorrect data entry could potentially
compromise EPA's reporting on progress.
12
-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
POTENTIAL MONETARY
BENEFITS (In $000s)
Rec.
No.
Page
No.
Subject
Status1
Action Official
Planned
Completion
Date
Claimed
Amount
Agreed To
Amount
2-1 7 Improve the accuracy of the GPRA National
Permitting Goal, as a measure of progress, by
defining and implementing criteria for changes to
the baseline.
2-2 7 Review State GPRA National Permitting Goal
commitments for 2008 and 2011 to identify any
opportunities for prioritizing facilities based on risk
or time in interim status.
2-3 7 Improve oversight of new interim status units by
(1) implementing a national level review to validate
the use of the interim status designation for new
units, and (2) documenting the reason for interim
status in RCRAInfo.
3-1 11 Implement system controls to ensure that permit
information in RCRAInfo cannot be modified unless
a history of the modification and its rationale is also
created.
3-2 11 Improve the current and future quality of RCRAInfo
data by:
3-2.1 11 Generating reports that identify incorrect data
values associated with unsupported status codes,
illogical sequences of status codes, entries with
dates prior to the inception of RCRA, and missing
mandatory data elements such as Part A and
Part B call-in and received dates; and correcting
entries as needed.
3-2.2 11 Implement control mechanisms that would prevent
these types of incorrect data entries in the future.
3-3 11 Provide a caution or disclaimer on data that is
released publicly from RCRAInfo, until data quality
controls are put in place
Assistant Administrator for
Solid Waste and Emergency
Response
Assistant Administrator for
Solid Waste and Emergency
Response
Assistant Administrator for
Solid Waste and Emergency
Response
Assistant Administrator for
Solid Waste and Emergency
Response
Assistant Administrator for
Solid Waste and Emergency
Response
Assistant Administrator for
Solid Waste and Emergency
Response
Assistant Administrator for
Solid Waste and Emergency
Response
Assistant Administrator for
Solid Waste and Emergency
Response
1 0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is undecided with resolution efforts in progress
13
-------
Appendix A
How is the Permitting Goal for GPRA Met?5
The following lists contain the most common situations that indicate accomplishment of the
GPRA goal:
1. For a unit in the operating permit baseline, approved controls in place means:
a. An operating permit has been issued for the unit, or an existing permit at the facility has
been modified so that the unit in question is considered permitted.
b. The unit has achieved clean closure, as verified by the lead regulatory agency (the unit
was on the operating track, but closed instead).
c. The unit closed before being issued an operating permit, and approved post-closure
controls, as discussed in 2(c) and 2(d) below, are being met.
d. The unit never managed hazardous waste (the unit does not have closure obligations).
e. The unit is proposed and has not yet been built (the unit will need a permit issued before
it is built and manages hazardous waste).
f. The unit has been accepted by the Superfund program (Federal or State equivalent) for
remediation.
2. For a unit in the post-closure baseline, approved controls in place means:
a. A post-closure permit has been issued for the unit, or an existing permit at the facility has
been modified so that the unit in question is subject to the post-closure permitting
standards.
b. The unit has achieved clean closure, as verified by the lead regulatory agency.
c. The unit has properly closed with waste in place, as verified by the lead regulatory
agency, and an approved post-closure plan, or similar enforceable document (such as a
consent order), covers appropriate post-closure obligations including part 264 subpart F
groundwater monitoring requirements (similar to the part of the post-closure rule that
deals with alternate authorities in lieu of permits).
As mentioned previously, this list describes only the most common situations where the GPRA
goal of safe waste management is met.
5 2005 GPRA goal criteria. This page has since been revised for the GPRA 2008 goal.
14
-------
Unique situations will arise that are not captured by this list. Therefore, other controls in place
that prevent dangerous releases to air, soil, and ground water will also be considered on a case-
by-case basis by the region or Headquarters.
(amended 7-19-01)
Baseline units have to be under control for the facility to be under control
A hazardous waste facility in the GPRA Operating Permit Baseline Universe is considered to
have met the GPRA goal when all units identified as part of this baseline have approved controls
in place. Likewise, a hazardous waste facility on the GPRA Post-Closure Baseline Universe is
considered to have met the GPRA goal when all units identified as part of this baseline universe
have approved controls in place.
Criteria to be in the GPRA operating and post-closure baselines
The GPRA Operating Permit Baseline includes only those hazardous waste facilities which had
at least one unit that had or needed an operating permit as of 10/1/97. This baseline universe
excludes units that needed an operating permit prior to 10/1/97, but the permits were
subsequently terminated or expired and the unit(s) clean closed prior to 10/1/97 or were in the
closure workload at that time, or did not need a permit prior to or on 10/1/97.
The GPRA Post-Closure Baseline includes any hazardous waste facilities that had at least one
land disposal unit that ceased operating before 10/1/97 and had not clean-closed or was not in the
operating or closure workload.
15
-------
Appendix B
Legal and Operating Status Code Matrix
Achieving "controls in place" is determined based on legal and operational status codes indicated
in the matrix below. In this matrix, the colors indicate if the combination of codes has achieved
a "controls in place" designation as defined by EPA. A description of controls is provided in
Appendix A. Most of the units have "controls in place" by being permitted operating units,
permitted clean closed units, or interim status units that are clean closed. The boxes with checks
indicate that the codes must be verified before the controls designation is accepted. (See page 17
for a list of legal and operating status codes.)
Legal and Operating Status Codes Matrix
OPERA!
HNG CODES
OP
CN
UC
BC
IN
CC
CP
CO
DC
CA
SF
cv
CR
AB
PF
PI
PC
L
PT
~
~
E
PR
G
IS
~
~
A
LI
-------
Legal and Operating Status Codes
LEGAL STATUS CODES
PI
Operating Permit Issued
PC
Post-Closure Permitted
PT
Permit Terminated/Permit Expired, Not Continued
PR
Proposed
IS
Interim Status
LI
Loss of Interim Status
IT
Interim Status Terminated
TA
T emporary Authorization
PM
Pre-Mod Authorization
LP
Loss of Pre-Mod Authorization
NN
Non-Notifier/lllegal
RQ
Requested but Not Approved
DL
Delisted
RD
Research, Development, and Demonstration Permit
EM
Emergency Permit
SR
State Regulated
NR
Never Regulated as a TSD
RU
Permit by Rule
OPERATING STATUS CODES
OP
Operating, Actively Managing RCRA-Regulated Waste
CN
Constructed, Not Yet Managing Hazardous Waste
UC
Under Construction
BC
Before Construction
IN
Inactive/Closing, Not Yet RCRA Closed
CC
Clean Closed
CP
Closed with Waste in Place
CO
Completed Post-Closure Care
DC
Delay of Closure
CA
Referred to Corrective Action for Closure
SF
Referred to CERCLA
CV
Converted but Not RCRA Closed
CR
Conducting Activities not Requiring a Permit
AB
Abandoned
PF
Protective Filer
17
-------
Appendix C
Agency Response and OIG Comments
October 18, 2006
MEMORANDUM
SUBJECT: Comments on Draft Evaluation Report "EPA's Management of Interim Status
Permitting Needs Improvement to Ensure Continued Progress"
Assignment No. 2005-1259
FROM: Susan Parker Bodine/s/
Assistant Administrator
TO: Bill A. Roderick
Acting Inspector General
Thank you for the opportunity to comment on the draft report. Your September 27, 2006
memorandum requested that we respond to the findings and recommendations, comment on the
factual accuracy of the draft report, and indicate concurrence or non-concurrence.
Attached are our comments on the draft report, as well as our response to your
recommendations. If your staff has any questions on the comments, please contact Vern Myers
(703) 308-8660 in the Office of Solid Waste.
Attachments:
I. Response to OIG Draft Recommendations
II. Comments on Evaluation Report Content
III. GPRA Permitting Baseline Criteria
18
-------
Chapter 2 - Conclusion:
Interim status units continue to exist for many years without formal issuance or denial of a
permit. EPA reported progress in achieving controls in place at interim status facilities through
the GPRA National Permitting Goal. However, insufficient documentation of changes in the
GPRA baseline potentially compromises progress. In addition, EPA does not prioritize its
National Permitting Goal activities based on potential risks posed by units that lack controls.
Further, new interim status units are reported in RCRAInfo every year without a clearly
identified reason, lack of full awareness of new designations, and lack of national oversight.
Agency Response to Conclusion:
1. We acknowledge that some interim status facilities have remained in interim status a
long time without a permit, but the length of time in interim status is not by itself a
meaningful measure of success for all units, since some units can meet their
regulatory conclusion and retain interim status. With the post-closure rule, for
example, some units will remain in interim status yet have achieved their regulatory
goal.
OIG Response
We agree that the length of time a unit is in interim status is not by itself a meaningful
measure of success for units. Time in interim status is one of many possible risk criteria.
We believe that the length of time an interim status unit exists without controls to prevent
releases (i.e., not meeting the GPRA goal criteria) is a meaningful indicator of potential
risk; the report was revised to clarify this point.
2. We disagree with the statement that claims "insufficient documentation of changes in
the GPRA baseline potentially compromises progress."
• Many facilities achieved approved controls in place during the initial tracking period
regardless of the level of baseline documentation. The sentence on page 7 directly
before Table 2-1 is not accurate ("Between 1999 and 2002, EPA reported an increase
in progress towards their goal, although the number of facilities addressed
decreased"). During this three year period of tracking the goal, hundreds of
accomplishments were recorded. These accomplishments took place independent of
the reduction in the baseline.
• In 1999 and 2000, there were very intensive efforts in the regions and states to assess
the facilities in the permitting track to determine if each unit belonged on the baseline
(Table 2-1 does not recognize that the baseline was still being corrected during these
years). This was the first time that the program had a goal that required this level of
scrutiny at the unit level, for each facility on the permitting track. Much of the unit-
specific data was of questionable quality (again, we had generally not tracked data at
the unit level in our data systems) and as the data was improved, many units were
added or removed based on the updated unit status. This was the primary reason for
the decline in the baseline totals. Since there are about 10,000 baseline units, this
was a long process and many corrections were made to the facility status recorded in
RCRAInfo. Many baseline changes resulted. The baseline stabilized in 2001 and
19
-------
remained constant through the 2005 reporting cycle (which Table 2-1 should note).
Please note that facilities removed from the baseline didn't meet the GPRA baseline
criteria at the time of removal because they were not on the permitting track, but
some had in fact met the GPRA goal or "under approved control" (e.g., the units
clean-closed prior to 10-1-97). Please see Attachment III for the baseline
establishment criteria. This information is also at the bottom of this web site:
http://www.epa.gov/epaoswer/hazwaste/permit/apprcntr.htm .
OIG Response
We believe the ability to validate EPA's reported permitting progress is impacted by
insufficient documentation of changes made to the GPRA baseline in 1999, 2000, and
2001. Due to insufficient documentation, we were unable to recreate the original
baseline and confirm which units or facilities were removed from or added to the
baseline. We have revised the final report to clarify our point that insufficient
documentation of changes to the baseline challenges EPA's ability to validate GPRA
progress. In addition, we revised the report to identify that in 2002 the number of GPRA
baseline facilities did not decrease from the 2001 baseline figure of 2,750 facilities, and
that the number of baseline facilities became constant in 2001. Finally, we believe that
report content preceding Table 2-1, adequately discusses the baseline changes due to a
nationwide data cleanup effort, and no further changes are needed.
3. We agree that the GPRA permitting goal criteria do not by themselves prioritize
which facilities to permit based on risk or time in interim status. However, separate
from the GPRA permitting goal criteria, we have generally prioritized facilities for
permitting. We believe there is a general prioritization scheme appropriate to the
level of risk for different unit types. This is described below:
• The program has prioritized facility progress based on the nature of the unit/facility.
For example, most land disposal facilities have been permitted. In addition, EPA's
1994 combustion strategy gave higher priority to those facilities for which a final
permit decision would result in the greatest environmental benefit or the greatest
reduction in overall risk to the public. We note that a number of the units that are now
identified as needing initial controls are boilers that combust hazardous waste. We
generally considered these units to be of a lower priority relative to other combustion
sources because they are not commercial units and because the boiler and industrial
furnace regulations for interim status boilers require compliance with emissions
limitations (thus, to a large extent, these units have controls in place under interim
status).
• For closed/post-closure units, corrective action acts as a surrogate prioritization, since
facilities with closed units that are considered to be major threats would be
categorized as a corrective action priority.
• Regardless of the GPRA permitting goal criteria, the program has and is continuing to
address the units that pose a greater risk, according to the data that show that storage
units (which generally pose less of an environmental risk) are by far the most
numerous unit type that have not met the goal criteria. In addition, the program has
made substantial progress in getting all of the different facility types under control in
20
-------
the past several years. EPA and the states are on track to meet the 2008 GPRA goal of
having 95% of the baseline facilities under approved controls. At that time, the vast
majority of the facilities will be under control, including those considered to be a
higher environmental risk and those facilities that have been in interim status a long
time.
OIG Response
As we reported, the GPRA National Permitting goal lacks a risk prioritization scheme.
Based on the Agency's response, EPA appears to rely on professional judgment in
determining whether they have applied risk-based prioritization criteria to interim status
units. We believe more can be done; there is an opportunity to prioritize the remaining
interim status high-risk units that do not meet the GPRA goal criteria by working with
States to identify higher-risk units for priority handling. The report was revised to clarify
that time in interim status is one of many possible criteria for determining risk.
OIG's Recommendation 2-1:
Improve the accuracy of the GPRA National Permitting Goal, as a measure of progress, by
defining and implementing criteria for changes to the baseline.
Agency Response:
We will clarify the criteria for the baseline to describe when baseline changes are
acceptable. The baseline criteria have been posted on the web for several years and were
recently revised for the 2008 baseline update. (See the end of this web site:
http://www.epa.gov/epaoswer/hazwaste/permit/apprcntr.htm). This information is also in
Attachment III. As noted in the draft report, we have also tracked all unit baseline
changes in RCRAInfo for the new 2008 baseline.
OIG's Recommendation 2-2
Review State GPRA National Permitting Goal commitments for 2008 and 2011 to identify any
opportunities for prioritizing facilities based on risk or time in interim status.
Agency Response:
We will work with states during the annual planning process to identify opportunities for
prioritizing facilities that have not met the goal criteria based on risk. As noted
previously, we do not believe "time in interim status" by itself is a meaningful criterion
for prioritizing which units present the highest risks. The program will review the out-
year projections for the rest of the facilities that have not yet met the goal criteria in order
to identify facilities that should be prioritized.
OIG Response:
We will modify the report so that it is clearer that time in interim status is one of many
possible criteria for determining risk.
21
-------
OIG's Recommendation 2-3
Improve oversight of new interim status units by (1) implementing a national level review to
validate new interim status designations, and (2) documenting the reason for interim status in
RCRAInfo.
EPA Response:
We have plans to make changes in RCRAInfo Version 4 to identify conditions that must
be satisfied prior to entering interim status codes. This is planned to be available toward
the end of 2007. The person entering the data in RCRAInfo will have to agree to and
select one of the standard conditions that would cause interim status to be applied to the
unit before the interim status code can be saved. RCRAInfo will also require the person
entering the data to provide comments/notes for unusual cases if the standard conditions
do not apply (for example, if a unit group split into 2 units that both require interim
status). OSW will create a report that can be used by program managers to identify new
interim status designations. New interim status designations are infrequent, but it is
important for program managers to be aware of new interim status designations that have
been entered into RCRAInfo. At the end of 2008, we will look at new interim status
designations in RCRAInfo and assess their validity.
OIG's Chapter 3 Conclusions
RCRAInfo is not fully effective in tracking permit information. This is because RCRAInfo lacks
some controls to ensure data integrity and data quality. This potentially compromises EPA's
ability to ensure its continued progress on the National Permitting Goal. The lack of data
integrity and data quality are due to the ability to overwrite legal and operating status codes, and
the lack of system controls to prevent incorrect data from being entered into RCRAInfo. Despite
data quality problems, RCRAInfo data are made available for public use and reporting without
appropriate disclaimers.
EPA Response:
We do not agree that data issues in RCRAInfo are of sufficient magnitude to compromise
EPA's ability to ensure its continued progress on the National Permitting Goal.
RCRAInfo currently allows the person entering the data into the system to overwrite
legal and operating status codes to provide an updated status of the facility. While
overwriting a status code would remove the record of the previous entry and prevent EPA
from having a historical record of that status change, it does give an accurate picture of
the current status of the facility and therefore, whether it is or is not under control. At the
same time, OSW has had a long-standing process to identify and address the data needs
in the different program areas in which we capture RCRA hazardous waste data. The
permitting (including interim status) and corrective action program areas were recently
addressed in the WIN/Informed Program Area Analysis (PPA).
The results from that analysis will be implemented in Version 4 (V4) of RCRAInfo,
which is planned to be available toward the end of 2007, and the issue of historical
records will be addressed in the revision.
22
-------
OIG Response
The data quality and integrity issues observed in RCRAInfo potentially compromise
EPA's continued permitting progress under GPRA. Without additional system controls,
the potential for incorrect data entry exists. Overwriting permitting data compromises a
unit's historical record in RCRAInfo and creates opportunities for incorrect data entry.
Because overwriting could be used to confer a control in place status (i.e., meeting the
GPRA goal criteria), and the historical permitting information is deleted, RCRAInfo
cannot be relied on to determine the unit's prior status.
OIG's Recommendation 3-1
Implement appropriate system controls to ensure that permit information in RCRAInfo cannot be
modified unless a history of the modification and its rationale is also created.
EPA Response:
Based on the requirements that came out of the PAA and the agreement by the V4 Design
Team, we will be maintaining history at the unit level so as to provide a better historical
view of the unit.
OIG's Recommendation 3-2
Implement mechanisms to improve the current and future quality of RCRAInfo data by:
Recommendation 3-2.1
Generating reports that identify unsupported legal and operating status codes;
illogical sequences of status codes; entries with dates prior to the inception of
RCRA; and missing mandatory data elements such as Part A and Part B call-in
and received dates, and correct entries as needed.
Recommendation 3-2.2
Implement control mechanisms that would prevent these types of incorrect data
entries in the future.
EPA Response:
The V4 Design team will be developing reports to allow users to view their data and
verify that their data is correct and complete. RCRAInfo's standard is to provide the best
possible choices for data entry through lookup tables and pick lists on the data entry
screens. Implementers will be able to determine which, if any, of their data needs to be
corrected.
OIG's Recommendation 3-3
Provide a caution or disclaimer on data released publicly from RCRAInfo, until data quality
controls are put in place.
EPA Response:
We will provide disclaimer language for RCRAInfo information that is available in
Envirofacts. We have mitigated this problem by limiting the data that are released in the
default reports in Envirofacts to data on the active facilities.
23
-------
Appendix D
Distribution
Office of the Administrator
Assistant Administrator for Solid Waste and Emergency Response
Director, Office of Solid Waste
Audit Followup Coordinator, Office of Solid Waste and Emergency Response
Agency Followup Official (the CFO)
Agency Followup Coordinator
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
Office of General Counsel
Acting Inspector General
24
------- |