U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Stronger Management
Controls Will Improve
EPA Five-Year Reviews of
Superfund Sites
Report No. 12-P-0251
February 6, 2012

-------
Report Contributors:
Carolyn Copper
Patrick Milligan
Kathryn Hess
Abbreviations
CERCLIS	Comprehensive Environmental Response, Compensation and Liability
Information System
EPA	U.S. Environmental Protection Agency
FYR	Five-Year Review
OIG	Office of Inspector General
OSRTI	Office of Superfund Remediation and Technology Innovation
OSWER	Office of Solid Waste and Emergency Response
RPM	Remedial Project Manager
Hotline
To report fraud, waste, or abuse, contact us through one of the following methods:
e-mail:	OIG Hotline@epa.gov	write: EPA Inspector General Hotline
phone:	1-888-546-8740	1200 Pennsylvania Avenue NW
fax:	202-566-2599	Mailcode 2431T
online:	http://www.epa.gov/oiq/hotline.htm	Washington, DC 20460

-------
x^ed sta^
U.S. Environmental Protection Agency	12-P-0251

#	\ Office of Inspector General	February6 2012
s
"V—'—J"
% V|// "
At a Glance
Why We Did This Review
We evaluated the Office of
Solid Waste and Emergency
Response (OSWER)
management controls to ensure
Superfund Five-Year Reviews
(FYRs) are thorough, meet
policy, and lead to well-
supported determinations that
accurately report how well
cleanup remedies protect
human health and the
environment.
Background
In 2007, OSWER set a goal of
reviewing at least 75 percent of
EPA regions" draft FYR reports
to improve quality and
consistency. OSWER's Office
of Superfund Remediation and
Technology Innovation
(OSRTI) reviews the drafts for
private sites. FYRs are required
at sites where remedial action
leaves contaminants at levels
that do not allow for unlimited
use and unrestricted exposure.
Evaluation of the remedy
should be based on and
sufficiently supported by data
and observations.
For further information, contact
our Office of Congressional and
Public Affairs at (202) 566-2391.
The full report is at:
www.epa.aov/oia/reports/2012/
20120206-12-P-0251.pdf
Stronger Management Controls Will Improve
EPA Five-Year Reviews of Superfund Sites
What We Found
The FYR process benefited from OSRTI reviews of draft reports, but
improvements could be made to increase the impact of these reviews. OSRTI's
reviews are intended to ensure protectiveness determinations are accurate,
supported by available information, and consistent in format nationwide. OSRTI
has not established sufficient management controls to make certain that FYR
reports consistently meet quality standards and adhere to guidance. No formal
process is in place to resolve differences when OSRTI and regions disagree on
conclusions before final reports are released to the public. OSRTI reviewers did
not always follow up to determine whether the region implemented
recommendations, and regions sometimes disregarded valid OSRTI comments
that were based on Agency guidance. We found the following cases where
regions did not accept OSRTI's recommendations:
•	Not enough recent data were available on site conditions; OSRTI had
recommended that the region defer making a determination until additional
data were collected.
•	Data did not support the protectiveness determination; OSRTI had
recommended a less protective determination than stated by the region.
•	Construction of remedial actions was initiated at multiple operable units,
but not yet complete; OSRTI had recommended that protectiveness be
determined for each operable unit, consistent with Agency guidance.
•	The remedy was declared to be short-term protective in spite of the region
proposing significant follow-up action; OSRTI had recommended a less
protective determination that agreed with the magnitude of the work to be
conducted.
The lack of controls and procedures for these reviews means that OSRTI does not
know the extent to which the regions implemented its recommendations. As a
result, OSRTI lacks reasonable assurance its oversight is effective.
What We Recommend
We recommend that OSWER establish a process to resolve disagreements with
regions on protectiveness determinations. We also recommend steps to improve
the consistency, thoroughness, and communication of OSRTI reviews and to
better define protectiveness determinations. OSWER agreed to the
recommendations and needs to establish completion dates.

-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
February 6, 2012
MEMORANDUM
SUBJECT: Stronger Management Controls Will Improve
EPA Five-Year Reviews of Superfund Sites
Report No. 12-P-0251
FROM:	Arthur A. Elkins, Jr.
Inspector General
TO:
Mathy Stanislaus
Assistant Administrator for Solid Waste and Emergency Response
This is our report on the subject evaluation conducted by the Office of Inspector General (OIG)
of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe
the problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
Final determinations on matters in this report will be made by EPA managers in accordance with
established audit resolution procedures.
Action Required
In accordance with EPA Manual 2750, you are required to provide a written response to this
report within 90 calendar days. You should include a corrective actions plan for agreed-upon
actions, including milestone dates. Your response will be posted on the OIG's public website,
along with our memorandum commenting on your response. Your response should be provided
as an Adobe PDF file that complies with the accessibility requirements of Section 508 of the
Rehabilitation Act of 1973, as amended. The final response should not contain data that you do
not want to be released to the public; if your response contains such data, you should identify the
data for redaction or removal. We have no objections to the further release of this report to the
public. We will post this report to our website at http://www.epa.gov/oig.
If you or your staff have any questions regarding this report, please contact Elizabeth Grossman
at (202) 566-0838 or grossman.elizabeth@epa.gov. or Carolyn Copper at (202) 566-0829 or
copper.carolyn@epa.gov.

-------
Stronger Management Controls Will Improve
EPA Five-Year Reviews of Superfund Sites
12-P-0251
Table of C
Chapters
1	Introduction		1
Purpose		1
Background		1
Noteworthy Achievements		2
Scope and Methodology		3
2	Oversight of Five-Year Reviews Needs to Be
Better Defined and Implemented 		5
OSRTI Reviews Improve Quality of FYR Reports		5
No Defined Procedure for Resolving Differences Between
OSRTI and Regions for FYR Conclusions		5
Other Improvements to OSRTI Review Process Needed		10
More Accurate and Complete Communication of
FYR Results and Information Needed		11
Conclusions		13
Recommendations		13
Agency Response and OIG Evaluation		13
Status of Recommendations and Potential Monetary Benefits		15
Appendices
A Prior OIG Evaluation Reports of Specific Sites	 16
B Agency Response to Draft Report and OIG Evaluation	 17
C Distribution	 26

-------
Chapter 1
Introduction
Purpose
The purpose of this evaluation was to examine the Office of Solid Waste and
Emergency Response's (OSWER's) oversight of Five-Year Reviews (FYRs) at
contaminated sites that have been remediated or are undergoing remediation.1
Our objective was to determine whether OSWER has effective management
controls to ensure FYRs are thorough, meet policy, and lead to well-supported
determinations that accurately report how well the cleanup remedy protects
human health and the environment.
Background
The Office of Inspector General (OIG) of the U.S. Environmental Protection
Agency (EPA) recently evaluated long-term monitoring at remediated Superfund
sites deleted from the National Priorities List (appendix A). We believe some
issues we identified should have been reported in the site FYRs but were not. For
example:
•	The site monitoring network was inadequate to assess whether offsite
migration of contaminants had been controlled by the remedy, affecting
EPA's ability to make an accurate protectiveness determination.
•	Environmental sampling data did not support the site's protectiveness
determination.
•	Long-term sampling had not sufficiently measured ecological impacts and,
therefore, the protectiveness determination could be challenged.
•	Oversight of long-term monitoring did not detect the collection, reporting,
and analysis of invalid data that was then used in completing the site's
FYR.
FYRs are intended to evaluate implementation and performance of cleanup
remedies at Superfund sites to determine whether each remedy is or will be
protective of human health and the environment. According to Agency guidance,
evaluation of the remedy and determination of protectiveness should be based on
and sufficiently supported by data and observations. An FYR is required by law at
a site where the remedy leaves conditions that do not allow for unlimited use and
unrestricted exposure. A site where remedial action takes 5 or more years to
1 FYRs are conducted at Superfund sites either to meet a statutory mandate under the Comprehensive Enviromnental
Response, Compensation and Liability Act, 121(c), or as a matter of EPA policy. This evaluation is limited to FYRs
conducted at private sites; EPA has different oversight responsibilities for FYRs conducted at federal facilities. As
such, our findings and recommendations do not apply to federal facility FYRs.
12-P-0251
1

-------
complete is subject to an FYR as a matter of Agency policy. FYRs ensure
protection of human health, inform the public on conditions at these sites, and are
conducted at least every 5 years. The responsibility for completing FYRs is
delegated to the EPA regional administrators, but OSWER provides oversight.
EPA is required to report to Congress on the results of FYRs.
Agency Response to 2006 OIG Report
The OIG reported in 2006 (EPA Has Improved Five-Year Review Process for
SuperfundRemedies, But Further Steps Needed, Report No. 2007-P-00006, issued
December 5, 2006) that the Agency needed to improve the overall quality and
completeness of information and conclusions in FYR reports. The problems OIG
identified challenged assumptions that remedies protect human health and the
environment and that Congress and the public receive accurate information on the
protectiveness status of remedies. In response to our 2006 report, the Agency took
action to improve the quality and consistency of FYR reports. In 2007, OSWER
set a goal to review at least 75 percent of draft FYR reports. Regions were asked
to send all draft FYR reports to headquarters for possible review. OSWER's
Office of Superfund Remediation and Technology Innovation (OSRTI) reviews
the draft reports for private sites. OSRTI has made it "a priority to issue
protectiveness statements that are accurate, supported by available information,
and consistent in format nationwide to the extent possible." As such, OSRTI
reviews the entire draft report with a focus on areas that have the greatest impact
on the protectiveness determinations of the remedies. OSRTI also provides
technical assistance to regions with remedy design and construction, site deletion,
operation and maintenance of remedies, and long-term monitoring activities.
Costs for Review of FYR Reports
In fiscal year 2009, EPA spent an estimated $7.6 million conducting FYRs.
OSRTI estimated that each review its staff conducted of draft FYR reports took
an average of 4 hours to complete, and the review time ranged from 2 hours to
more than 16, depending upon the site's complexity. Using the average salary rate
given by OSRTI, we estimated the cost for OSRTI to review each FYR report to
be between $120 and $960, with an average cost of $240 and a total for all fiscal
year 2009 reviews of approximately $48,000. The regions estimated that their
staffs took an average of 5 hours to address the OSRTI review comments, and the
time spent ranged from 2 to 13 hours. Using the average salary rate given by the
regions, we estimated the cost for the regions to address each OSRTI review to be
between $100 and $872, with an average cost of $302 and a total for all fiscal
year 2009 reviews of approximately $60,000.
Noteworthy Achievements
The Agency has taken steps to improve the quality of FYRs. OSRTI issued a
policy memorandum to regions in 2007 establishing program priorities that
12-P-0251
2

-------
included steps to improve quality and consistency of FYR reports. OSRTI also
issued supplements to the 2001 FYR guidance on specific media and issues.
OSRTI reported to us it provided training for regional staff responsible for
drafting FYR reports and reviewed over 95 percent of the draft FYR reports in
fiscal years 2008 and 2009, exceeding its goal of 75 percent. OSRTI conducted its
own internal assessments of FYRs and reported on these assessments at the 2009
and 2010 national meetings of EPA remedial project managers (RPMs). In
comparing the results of its fiscal years 2008 and 2009 reviews, OSRTI noted that
in 2009 there was greater adherence to the national guidance for FYR reports,
including better use of FYR protectiveness statement language.
Scope and Methodology
We conducted this evaluation from January 2011 to September 2011 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the evaluation to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and
conclusions based on our evaluation objective. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions based on
our evaluation objective.
To meet our evaluation objective, we conducted meetings with OSRTI managers
and staff and contacted managers and RPMs in four regions—Regions 1, 2, 4,
and 5. We reviewed appropriate Agency guidance and policy, including the
Comprehensive Five-Year Review Guidance, issued in 2001, and the Agency's
memorandum, Five-Year Review Program Priorities, issued May 3, 2007. We
reviewed OIG's 2006 report on FYRs and the Agency's responses. We also
reviewed relevant documents, including recent EPA reports to Congress on the
results of FYRs, previous OIG evaluation reports on the FYR process and long-
term monitoring (appendix A), EPA planning documents, and results of internal
reviews.
We reviewed 27 percent (49 of 182) of the reviews OSRTI staff conducted of
draft FYR reports at non-federal sites in fiscal year 2009. OSRTI had identified
these 49 reviews in its internal assessment of FYR reports as being all of the cases
OSRTI recommended that the region change the protectiveness determination and
the region did not make the change. We compared the OSRTI review to the final
FYR report to assess whether the region appropriately responded to OSRTI
reviewer comments and recommendations. We sent questions to RPMs, regional
managers, and OSRTI staff on the quality and impact of OSRTI reviews for
18 cases where we identified significant disagreement between OSRTI's review
and the final FYR report. We also obtained responses from OSRTI managers and
staff regarding the 14 cases where the region accepted OSRTI's recommendation
to change the protectiveness determination, but OSRTI's internal assessment
stated that the region did not make the change.
12-P-0251
3

-------
We gathered information on the cost of FYRs from EPA's financial system and
questioned OSRTI and regional managers on the costs incurred by their staff
conducting reviews of draft FYR reports and responding to those reviews.
12-P-0251
4

-------
Chapter 2
Oversight of Five-Year Reviews
Needs to Be Better Defined and Implemented
The reviews OSRTI provided of draft FYR reports improved final reports, but the
impact of these reviews could be increased. OSRTI's reviews are intended to
ensure that protectiveness determinations are accurate, supported by available
information, and consistent in format nationwide. However, OSRTI needs to
establish management controls to make certain that FYR reports consistently meet
quality standards and follow guidance. The Agency lacks procedures for resolving
differences between OSRTI and the regions, and for documenting regional
decisions not to act on OSRTI recommendations. In addition, procedures are
needed to improve OSRTI's reviews and the Agency's dissemination of FYR
conclusions. Improvements should result in protectiveness statements that are
better supported by the information provided in the FYR report and, in some
cases, better protection of human health and the environment.
OSRTI Reviews Improve Quality of FYR Reports
Our evaluation showed that OSRTI reviews of draft FYR reports improved the
quality of many final reports. For example, in many final reports the regions
included more information to support their protectiveness determinations, as the
OSRTI reviewers had recommended. Reviewers also identified problems with
institutional controls as described in the draft reports at several sites. Regions
improved their analyses and supporting information on institutional controls in the
final reports, leading to well-supported protectiveness determinations. Regional
managers and RPMs agreed that the FYR process benefited from the OSRTI
reviews, citing the advantages of having independent assessments of draft reports
and striving for national consistency.
No Defined Procedure for Resolving Differences Between
OSRTI and Regions for FYR Conclusions
The regions and OSRTI did not have a defined procedure to resolve differences.
In many of the cases we reviewed, we found significant differences between
OSRTI's review comments and the information and conclusions in the final FYR
reports, including in all 18 cases where we sent regional managers and RPMs
questions. These cases included situations where:
• Not enough recent data and other information were available on site
conditions; OSRTI had recommended deferring protectiveness
determination until additional data were collected.
12-P-0251
5

-------
•	Data did not support the protectiveness determination; OSRTI had
recommended a less protective determination than stated by the region.
•	Construction of remedial actions was initiated at multiple operable units,
but not yet complete; OSRTI had recommended that protectiveness be
determined for each operable unit, consistent with Agency guidance.
•	The remedy was declared to be short-term protective in spite of the region
proposing significant follow-up action; OSRTI had recommended a
less-protective determination that agreed with the magnitude of work to be
conducted.
OSRTI provided written comments to the site RPMs on their draft FYR reports.
OSRTI reviewers offered to meet further with the regions if clarification on
comments was needed. However, the regions did not always engage the OSRTI
reviewers further. In addition, OSRTI reviewers did not consistently follow up
with the regions to determine whether comments were addressed and how, and
whether the final protectiveness determinations were supported by the revisions.
When conducting an internal assessment of its reviews of draft FYR reports,
OSRTI did not consult the final FYR reports. Instead, OSRTI used information
entered into a database by the regions. Without formal follow-up and resolution,
some final FYR reports may contain unsupported protectiveness determinations.
While the preparation and approval of FYR reports has been delegated to the
regional administrators, the protectiveness determinations in those reports should
be consistent with standards issued by the Agency. In addition, without
determining the results and impacts of its reviews, OSRTI management is limited
in measuring the effectiveness of its oversight work.
Differences Between OSRTI and Regions
Not Enough Recent Data Available on Site Conditions
Regions did not accept OSRTI's recommendation that the protectiveness
determination be deferred until additional information was obtained in two-
thirds of the 18 cases where we sent regional managers and RPMs questions.
In most cases, the region said it did not accept OSRTI's recommendations
because information deficiencies only affected the region's ability to
determine long-term protectiveness, not short-term protectiveness. However,
we were provided no evidence to support the regions' assertions. There were
three cases of insufficient data where a major component of the remedy at
each site was "monitored natural attenuation." Natural attenuation means
chemical and biological conditions in the groundwater should be causing
contaminants to degrade. In each case, the OSRTI reviewer commented in
writing that sufficient data had not been collected to determine whether the
contaminants were degrading as anticipated. The OSRTI reviewer suggested
that the region defer declaring protectiveness until needed data could be
collected and analyzed. However, in each case, the region made a
12-P-0251
6

-------
protectiveness determination without including additional information in the
final FYR.
In some cases, the OSRTI reviewer suggested the region include data that was
missing, but did not cite the FYR guidance that "evaluation of the remedy and
the determination of protectiveness should be based on and sufficiently
supported by data and observations." Further, the guidance states that the
report should "contain the data and information necessary to support all
findings and conclusions." In addition to suggesting the data be included in
the report, the reviewer should have cited the FYR guidance as support for the
suggestion. Citing the guidance could be an effective method for
communicating to the region the importance of including data in the report to
support the findings and conclusions.
Data Did Not Support Protectiveness Determination
We found cases where the protective status declared in the final FYR report
was more protective than could be supported by data and other information
included in the report. OSRTI pointed out these problems in its reviews of the
draft reports, but did not have a procedure in place to resolve issues and
ensure that regions modified their reports accordingly.
•	One case involved a final FYR report that stated the review of
groundwater data, documents, cleanup standards, and risk assumptions
indicated the remedy was not functioning as intended by the record of
decision. The OSRTI reviewer commented that the remedy was not
protective because the remedy did not appear to be functioning as
intended. The final FYR report laid out plans to install new wells,
initiate efforts to capture the migrating groundwater contamination,
and conduct a new water use survey. These are significant actions for a
remedy that EPA declared to be "construction completed" 10 years
previously. However, the final FYR report declared the remedy
protective in the short-term because "there are no exposure pathways,"
which is a statement inconsistent with plans to conduct a new water
use survey.
•	In another case, monitoring showed a contaminant at concentrations
greater than 100 times the drinking water standard seeping from the
groundwater to a river a half-mile upstream from a major city's
drinking water intake. The region declared the remedy protective in
the short-term partly because monitoring results at the intake showed
contaminant concentrations were below the standard. The OSRTI
reviewer recommended a not protective determination in agreement
with the state's position.
12-P-0251
7

-------
In both of these cases, a resolution procedure was warranted to ensure that the
final FYR reports included well-supported declarations on how or whether
cleanup remedies protect human health and the environment.
Regions Did Not Follow Agency Guidance for Remedies Under
Construction
We identified three sites at which the regions did not follow Agency guidance
for remedies under construction. In all three cases, the OSRTI reviewer
recommended that the region change its protectiveness determination.
However, changes were not made, even though recommended changes would
have been consistent with Agency guidance.
According to Agency guidance, the three protectiveness statement options for
a remedy still under construction are:
1.	The remedy is expected to be protective of human health and the
environment upon completion.
2.	The remedy is not protective because of issues described and actions
need to be taken to ensure protectiveness.
3.	A protectiveness determination cannot be made until further
information is obtained.
Protective in the short-term is not provided in the guidance as an example of
protective determinations available for a remedy still under construction.
Nonetheless, we identified two cases where the remedy was declared in the
final FYR report as being protective in the short-term.
In the third case, the region did not include all operable units in the final FYR
report. Agency guidance allows for separate FYRs for operable units at
complex sites. However, the guidance also states that no unit should be
reviewed later than 5 years after initiation of its remedial actions. In this case,
more than 5 years had passed at both operable units. As such, FYRs should
have been conducted on both, as requested by the OSRTI reviewer. Less than
10 months after issuing the final FYR report that included only one unit, the
region reclassified and closed out the special account funds for this site. The
region mistakenly stated in its closeout memorandum that the 2009 FYR
report had concluded that remedies at both units continue to provide adequate
protection of human health and the environment.
Short-Term Protectiveness Not Defined
In over 80 percent of the cases we examined in detail (15 of 18), the region
had declared the remedy to be protective of human health and the environment
in the short-term. The range in conditions varied greatly at these sites and in
examples given in Agency guidance. We found that regions were using
12-P-0251
8

-------
protective in the short-term even when serious issues had been identified at
sites that warranted considerable follow-up action similar to that needed for a
site declared not protective. At the other extreme, we found examples in the
Agency guidance where the constructed remedies were protective as intended,
but other restrictions on site use were yet to be implemented. Current Agency
guidance does not adequately define what is meant by short-term protective.
Consequently, this could result in misuse of short-term protective
determinations.
We raised this issue in our 2006 report, and recommended that the Agency
revise its guidance to more clearly define short-term and long-term
protectiveness. However, the Agency decided not to act on that
recommendation, stating that "the Five-Year Review guidance clearly defines
short- and long-term protectiveness determinations" and "includes several
example scenarios for the appropriate use of protectiveness determinations."
Because of the wide ranging use of short-term protective, we believe that this
determination currently is not clearly defined. OSRTI needs to provide greater
clarity on which situations are acceptable under a short-term protective
determination to prevent misuse of this determination.
OSRTI's Oversight Process Does Not Include Formal Resolution
We found that OSRTI's oversight of FYRs could be improved by including
formal resolution of disagreements between OSRTI and the regions on issues
related to remedy protectiveness. In response to the OIG's 2006 recommendations
on FYRs, the Agency said that increased headquarters' review would address the
FYR quality problems we identified. We recognize that improvements have been
achieved. However, in our opinion, the current review process can be revised to
consistently obtain greater impact by adding formal resolution, as the Agency
currently practices with disagreements over remedy selection in the record of
decision.
We sent questions to RPMs and regional managers for 18 of the cases where we
identified there were important but unresolved disagreements between OSRTI and
regions. Many of the comments OSRTI had made in its reviews of the draft FYR
reports went unaddressed by the regions in the final reports. The regional
responses to our questions still did not provide sufficient information to support
the protectiveness determinations. These cases are examples where regions are
issuing final reports with issues unresolved. A more formal and thorough review
process would call for the regions and OSRTI to resolve disagreements before a
final FYR report is issued and would lead to better supported protectiveness
determinations. Currently, with no formal resolution process, regions proceed
with approving FYR reports without resolving disagreements with OSRTI.
12-P-0251
9

-------
Other Improvements to OSRTI Review Process Needed
To improve the quality of FYR reports, OSRTI reviews of draft reports could be
more standardized and could more consistently address the report elements listed
in Agency guidance. More consistency in how OSRTI staff structure their reviews
can help ensure that all reviewers approach the reviews similarly, comments are
directed at key issues and clearly communicated, and regions receive review
comments in a form that facilitates their attention to the reviewer's major points.
These improvements should lead to FYR reports that better document long-term
stewardship of sites and whether the remedies continue to protect human health
and the environment.
We found some of the 49 written reviews we examined to be more effective than
others in communicating key points and describing significant problems identified
in the draft reports. Some reviewers assembled their comments in a separate
document, listing the most significant comments and recommendations first,
followed by minor technical, grammatical, and format comments. These reviews
effectively communicated problems with support for the protectiveness
determination. Other reviewers arranged comments by page number or embedded
comments in the electronic version of the draft report, making it difficult to
readily identify the reviewer's major points. In these cases, EPA regional staff,
responding to the review, could overlook key comments from OSRTI, and the
final FYR report could be issued without consideration of those comments.
To better document OSRTI's communication with the region and resolution of
differences between OSRTI and the region, OSRTI could develop a brief review
summary sheet to accompany each review. This sheet could include:
•	The reviewer's agreement with the draft determination or the reviewer's
recommended change and the reasons behind that recommendation
•	Basic site and reviewer information, such as the name of the site, the site
RPM, the OSRTI reviewer, and key milestone dates
•	A place for the region's response and resolution of any disagreements
between the regions and OSRTI
A management control such as using this summary sheet would simplify
communication with the regions by providing a consistent, easily understood
summary. The sheet also may improve the final FYR reports by clearly
communicating problems the OSRTI reviewer identified with the draft
protectiveness determination and documenting the review and resolution of
differences between OSRTI and the region.
OSRTI developed a checklist for its internal use when reviewing draft FYR
reports. We found the checklist could be improved. The checklist is similar in
many ways to the FYR report checklist provided in Agency guidance. The review
checklist includes useful information, such as prompts to check whether site
12-P-0251
10

-------
monitoring data are discussed, site data are included in the back of the document,
and data indicate that the remedy is operating and expected to achieve remedial
action objectives. However, key items are missing from the current internal
checklist. For example, there is no reminder to the reviewer to cite specific
occurrences where the draft was not consistent with Agency policy and guidance.
Inclusion of data and site maps is designated as optional in the appendices section
of the checklist, but Agency guidance states that data and information to support
the protectiveness determination should be included in the FYR report.
Further, regular use of the checklist by reviewers should improve the quality of
OSRTI's reviews and promote national consistency. There was insufficient
evidence to determine whether OSRTI staff consistently consulted the checklist.
We were told that use of the checklist is optional by OSRTI staff. We believe that
use of the checklist should be part of the review process, and each completed
checklist should be retained with the review summary sheet and the written
review comments as part of OSRTI's records for the site. This information would
be useful 5 years later when the remedy is again evaluated.
More Accurate and Complete Communication of FYR Results
and Information Needed
The Agency has taken steps to provide results of FYRs to the public:
•	The protectiveness status and corrective actions needed are entered into a
national database for each site.
•	Results of FYRs are reported to Congress.
•	FYR reports are made available online.
•	The front piece of each FYR report contains a summary form.
•	Links to the FYR reports are included on the publicly available, online
Superfund Site Progress Profile for each site.
We identified changes that could be made in each of these efforts to improve the
Agency's communication of the results of FYRs to the public.
We asked OSRTI staff, regional program managers, and RPMs about procedures
for ensuring that accurate data on results of FYRs—protectiveness status and
corrective actions needed— are entered into the national database, the
Comprehensive Environmental Response, Compensation and Liability
Information System (CERCLIS). We were consistently told that regions had
procedures in place to correctly enter and verify data. However, we found data
entry errors with some protectiveness determinations. OSRTI had identified
49 cases where it disagreed with the region on the protectiveness determination,
but our assessment found cases where the 2 parties actually agreed. OSRTI had
referred to CERCLIS for the region's final determination, and in six cases, the
regions had entered the incorrect protectiveness status into the database. In five of
the six cases, the region entered a remedy status that was more protective than
12-P-0251
11

-------
that given in the final FYR report. In the sixth case, the region entered a status
that was less protective. OSRTI has worked with the regions to correct the status
in two cases after we brought the data entry errors to OSRTI's attention.
We also found that the protectiveness status for these sites was incorrectly listed
in the Report to Congress sent by the EPA Administrator on September 28, 2010.
The Superfund law requires that the Agency report to Congress a list of sites for
which FYRs were required, the results of the reviews, and any actions taken as a
result of such reviews. If OSRTI continues to rely on CERCLIS for its required
reporting to Congress, it needs to provide greater assurance of the quality of the
data by verifying the protectiveness status in the database against those in the
final FYR reports.
The Agency does not consistently provide in the online FYR report the site data
and other information used to support its protectiveness determination. We found
13 cases (or 27 percent of the 49 reviewed) where the monitoring data and other
site information was placed in FYR report appendices that were not included in
the report file available online. This information is the foundation for the
technical analysis that supports the protectiveness determination. In an FYR
program initiatives memorandum, the Agency requested that regions offer
complete reports to the public online. We believe the posted information should
include the maps, data tables, and other information contained in the report
appendices.
The summary form at the beginning of each FYR report is intended to
communicate the basic results of the FYR. Included on the summary form is a
place to explain the protectiveness determination, but these explanations in many
of the FYR reports we examined were long and confusing. We found that in many
cases the protectiveness statements in the summaries were reiterations of complex
statements made in the body of the report. As a result, these summaries did not
effectively communicate the protectiveness status of the site remedy to the
community. The summary form should succinctly inform the public of the results
of the FYR and clearly list the protectiveness status.
The Agency provides the public information on the progress of cleanup at each
site through an online Superfund Site Progress Profile. The profile does not
include the results of FYRs in the same manner as information on the progress of
other stages of the cleanup, which is displayed through easily understood
graphics. Only an electronic link to the latest FYR report is provided. The profile
should effectively inform the public of the results of the FYR and include the
protectiveness status as a graphic.
12-P-0251
12

-------
Conclusions
OSRTI lacks procedures to obtain resolution and agreement on protectiveness
determinations before final FYR reports are issued. OSRTI reviews of draft FYR
reports added value to the FYR process by improving the quality of many reports
reviewed. However, additional improvements and management controls are
needed to ensure that the reviews consistently produce accurate and well
supported statements. The Agency has not clearly defined what conditions are
acceptable under a short-term protective determination. Additionally, the
Agency's communication of protective status to the public can be confusing and
could be clearer. These shortcomings reduce the Agency's assurance that the FYR
process is effective and protectiveness determinations are accurate and supported.
Recommendations
We recommend that the Assistant Administrator for Solid Waste and Emergency
Response:
1.	Establish management controls to ensure:
a.	OSRTI and the regions resolve and fully document disagreements
on remedy protectiveness determinations for FYR reports where
OSRTI recommended "not protective" or "deferred"
determinations.
b.	Reviews of draft FYR reports use consistent criteria.
c.	Results of FYR reports are accurately entered into CERCLIS,
correctly reported to Congress, and fully communicated to the
public.
2.	Clearly define protectiveness categories in Agency guidance or policy and
ensure definitions are consistently applied across the Agency.
Agency Response and OIG Evaluation
OSWER agrees with the OIG's recommendations, as modified following
OSWER's written response to the draft report and discussion with OIG personnel.
Where appropriate, the OIG made changes to the report based on OSWER's
response. OSWER's response and the OIG's evaluation of those comments are in
appendix B.
OSWER agrees to take steps to ensure that:
1.	OSRTI and regions resolve and fully document disagreements on remedy
protectiveness for FYR reports where OSRTI recommended "not
protective" or "deferred" determinations
2.	Reviews of draft FYR reports use consistent criteria
12-P-0251
13

-------
3. Results of FYR reports are accurately entered in the Agency database,
correctly reported to Congress, and fully communicated to the public
OSWER also agrees to clearly define protectiveness categories and ensure those
definitions are consistently applied.
The OIG accepts the corrective actions described by OSWER as meeting the
intent of the recommendations, as revised. However, these recommendations are
designated as unresolved with resolution efforts in progress until completion dates
are established. In its 90-day response to the final report, OSWER should provide
milestone dates for completion of the corrective actions.
12-P-0251
14

-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
POTENTIAL MONETARY
BENEFITS (In $000s)
Rec.
No.
Page
No.
Subject
Status1
Action Official
Planned
Completion
Date
Claimed
Amount
Ag reed-To
Amount
13 Establish management controls to ensure:
a.	OSRTI and the regions resolve and fully
document disagreements on the remedy
protectiveness determinations for FYR
reports where OSRTI recommended "not
protective" or "deferred" determinations.
b.	Reviews of draft FYR reports use consistent
criteria.
c.	Results of FYR reports are accurately
entered into CERCLIS, correctly reported to
Congress, and fully communicated to the
public.
13 Clearly define protectiveness categories in Agency
guidance or policy and ensure definitions are
consistently applied across the Agency.
Assistant Administrator for
Solid Waste and
Emergency Response
Assistant Administrator for
Solid Waste and
Emergency Response
1 0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is unresolved with resolution efforts in progress
12-P-0251
15

-------
Appendix A
Prior OIG Evaluation Reports of Specific Sites
Superfund
site
Report
number
Date
Report title
Bruin
Lagoon
10-P-0217
September 2010
EPA Should Improve Oversight of Long-term
Monitoring at Bruin Lagoon Superfund Site in
Pennsylvania
http://www.epa. aov/oia/report s/2010/20100908-10-P-0217.pdf
Delatte
Metals
09-P-0029
November 2008
EPA's Safety Determination for Delatte Metals
Superfund Site Was Unsupported
http://www.epa.aov/oia/reports/2009/20091119-09-P-0029.pdf
Jones
Sanitation
09-P-0243
September 2009
Independent Sampling Generally Confirms EPA's
Data at the Jones Sanitation Superfund Site in New
York
http://www.epa.aov/oia/reports/2009/20090923-09-P-0243.pdf
Neal's
Dump
09-P-0110
March 2009
Results of Independent Groundwater Sampling at
Neal's Dump Superfund Site
http://www.epa.aov/oia/reports/2009/20090304-09-P-0110.pdf
PAB Oil
10-P-0229
September 2010
EPA Should Improve Oversight of Long-Term
Monitoring at PAB Oil and Chemical Services, Inc.,
Superfund Site in Louisiana
http://www.epa.aov/oia/reports/2010/20100921-10-P-0229.pdf
Wheeler Pit
10-P-0218
September 2010
Independent Ground Water Sampling Generally
Confirms EPA's Data at Wheeler Pit Superfund Site in
Wisconsin
http://www.epa.aov/oia/reports/2010/20100908-10-P-0218.pdf
Wheeling
Disposal
11-P-0034
December 2010
EPA Should Improve Its Oversight of Long-Term
Monitoring at Wheeling Disposal Superfund Site in
Missouri
http://www.epa.aov/oia/reports/2011/20101220-1 l-P-0034.pdf
Wildcat
Landfill
10-P-0055
January 2010
Changes in Conditions at Wildcat Landfill Superfund
Site in Delaware Call for Increased EPA Oversight
http://www.epa.aov/oia/reports/2010/20100126-10-P-0055.pdf
Source: EPA OIG.
12-P-0251
16

-------
Appendix B
Agency Response to Draft Report and OIG Evaluation
November 4, 2011
MEMORANDUM
SUBJECT: Response to OIG Draft Audit Report, Stronger Management Controls Will
Improve EPA Five-Year Reviews of Superfund Sites
FROM: Mathy Stanislaus
Assistant Administrator
TO:	Wade Najjum
Assistant Inspector General
Office of Program Evaluation
Thank you for the opportunity to review and respond to the Office of Inspector General (OIG)
draft audit report, Stronger Management Controls Will Improve EPA Five-Year Reviews of
Superfund Sites dated October 4, 2011.
We appreciate the continued dialogue and openness that the OIG team has afforded the Office of
Solid Waste and Emergency Response (OSWER) during the course of this audit.
First, we would like to address each of the recommendations and provide information about
planned or initiated actions related to those recommendations. Second, we provide additional
comments to ensure that the final audit report contains accurate information.
Before addressing the recommendations and the report, it is worth noting how this audit was
initiated. In 2009, staff from the Office of Superfund Remediation Technology and Innovation
(OSRTI) presented an evaluation on five-year reviews at the Annual National Remedial Project
Manager (NARPM) training conference where staff from the OIG was present. While OIG staff
built on this evaluation, OSRTI was already well aware of many of the issues and was working
towards addressing them.
Improving five-year reviews has been a focus of OSWER and OSRTI since the August 27, 2001
Five-Year Review Program Initiatives memorandum (OSWER Directive 9355.7-07) following
previous Inspector General audits in 1995 and 1999. While OSWER does not agree with all the
recommendations made by the OIG, we will continue to work with the OIG on efforts to
strengthen five-year reviews.
12-P-0251
17

-------
OIG RECOMMENDATIONS AND OSWER RESPONSE
OIG Recommendation la: Establish management controls to ensure: OSRTI and the regions
resolve disagreements on the remedy protectiveness determinations and other significant issues
in each draft FYR report and that the resolution process is fully documented.
OSWER Response:
OSWER does not agree with the recommendation as written. We are committed to ensuring that
significant disagreements between OSRTI and the regions related to protectiveness in draft Five-
Year Review (FYR) are resolved. As a result, OSWER will implement a formal, documented
process for resolving significant disagreements where the headquarters' reviewer recommends a
"not protective" determination and the region disagrees. OSWER will document the resolution
process following a process similar to the one described in the Elevating Site-Specific Superfimd
Remedy Selection Issues between the Office of Superfimd Remediation and Technology
Innovation and Regional Superfimd Program Offices memorandum, dated March 31, 2010.
OSWER recommends modifying recommendation la as follows: Establish management controls
to ensure: OSRTI and the regions resolve and fully document all significant disagreements in
each FYR report where the remedy protectiveness determination is evaluated by headquarters' to
be "not protective" and the region disagrees.
OIG Response 1: In a December 12, 2011, meeting, OIG and OSRTI managers agreed that
OSRTI and the regions will resolve and fully document disagreements on remedy protectiveness
determinations for FYR reports where OSRTI recommended "not protective" or "deferred"
determinations.
In its 90-day response to the final report, OSWER should explain how it will establish the
resolution process with the regions and provide milestone dates for when the resolution process
will be implemented. This recommendation is designated as unresolved with resolution efforts in
progress.	
OIG Recommendation lb: Establish management controls to ensure: Reviews of FYR reports
use consistent criteria, format, and communication style.
OSWER Response:
OSWER does not agree with this recommendation as written. OSWER agrees that the use of a
consistent set of criteria for review of draft FYRs is appropriate. As a result, OSWER will revise
and enhance the Headquarters' Review Checklist for Draft Five-Year Review Reports and will
make use of the checklist a routine part of the headquarters' review process. However, using a
structured format and communication style to address key issues in the FYR will not necessarily
lead to better documented FYRs. Achieving those goals is not based on structuring our reviews,
but rather by tailoring our reviews to meet the individual needs of each region.
OSWER recommends modifying recommendation lb to delete the recommendation that
headquarters use a consistent "format, and communication style" when evaluating draft FYRs.
12-P-0251
18

-------
OIG Response 2: We accept OSWER's suggested revision to recommendation lb and revised
the recommendation to read: "Establish management controls to ensure reviews of draft FYR
reports use consistent criteria." In its 90-day response to the final report, OSWER should explain
how it will ensure that reviews of FYR reports use consistent criteria and provide milestone dates
for when the reviews will follow consistent criteria. This recommendation is designated as
unresolved with resolution efforts in progress.	
OIG Recommendation lc: Establish management controls to ensure: Results of FYR reports
are accurately entered into CERCLIS, correctly reported to Congress, and fully communicated to
the public.
OSWER Response:
OSWER agrees with the recommendation. To improve the accuracy of the FYR data entered into
CERCLIS, OSRTI has initiated modifications to the Five-Year Review Summary Form to more
closely reflect the FYR data entry fields in CERCLIS. These modifications should improve the
overall accuracy of the FYR data entered in CERCLIS and minimize errors in the annual Report
to Congress. Additionally, as of fiscal year 2011 the file megabyte limit on EPA web pages has
been substantially increased so that OSWER can now typically make the entire private site FYR
report, including appendices, available to the public on-line. In addition, hard copies of
completed FYR reports are and will continue to be available for public review in local
repositories and the Regional offices. We will also modify the Superfund site progress profiles to
include the protectiveness determination.
OIG Response 3: OSWER's proposed corrective actions meet the intent of recommendation lc.
In its 90-day response to the final report, OSWER should provide milestone dates for completing
revisions to the Five-Year Review Summary Form, making complete reports available to the
public online, and modifying the site progress profiles to include the protectiveness
determination. This recommendation is designated as unresolved with resolution efforts in
progress.	
OIG recommendation 2: Clearly define protectiveness categories in Agency guidance or policy
and ensure definitions are consistently applied across the Agency.
OSWER Response:
OSWER agrees with the recommendation. We will issue a directive to the Regional Superfund
Program offices clarifying the protectiveness categories used in the Comprehensive Five-Year
Review Guidance, OSWER Directive No. 9355.7-03B-P (June 2001), and monitor use of those
definitions during the routine headquarters' review of the draft FYRs.
OIG Response 4: OSWER's proposed corrective actions meet the intent of recommendation 2.
In its 90-day response to the final report, OSWER should provide milestone dates for issuing the
directive to the regions clarifying the protectiveness categories. This recommendation is
designated as unresolved with resolution efforts in progress.	
12-P-0251
19

-------
OTHER OSWER COMMENTS:
At a Glance/What We Found
Comment 1: Second bullet. The phrase "remedies that were less" should be replaced with "a
protectiveness determination that was less."
OIG Response 5: We accept the intent of the requested change and revised the report as follows:
"Data did not support the protectiveness determination; OSRTI recommended a determination
that was less protective than stated by the region."	
Comment 2: Third bullet. The language is not consistent with the language used in the Superfund
program. The Superfund 2001 FYR guidance requires that the region make a protectiveness
statement for each operable unit, not "each completed phase." We recommend replacing the
current language with the following: "Site remedies are implemented as multiple projects which
may occur across multiple operable units. OSRTI recommended that protectiveness statements
be made for each operable unit where construction has been initiated. Likewise, OSRTI
recommended that protectiveness statements are not required for operable units where
construction has not been initiated."
OIG Response 6: We accept the intent of the requested change and revised the report as follows:
"Construction of remedial actions was initiated at multiple operable units, but not yet complete;
OSRTI recommended that protectiveness be determined for each operable unit, consistent with
Agency guidance."	
Chapter 1/Purpose
Comment 1: Page 1, first paragraph, first sentence: OSWER recommends replacing the first
sentence of the paragraph with language that is more consistent with FYR guidance and policy.
For example, "The purpose of this evaluation was to examine the Office of Solid Waste and
Emergency Response (OSWER's) oversight of Five-Year Reviews at Superfund sites conducted
either to meet a statutory mandate under CERCLA 121(c) or as a matter of EPA policy."
OIG Response 7: We added information on statutory mandate and EPA policy to the footnote 1
in the purpose section of the report.	
Chapter 1/Background
Comment 1: Page 1, first paragraph: OSWER disagrees with the second sentence, "Some issues
the OIG identified should have been detected in the site FYRs but were not." It is not clear in the
text of the report, whether the issues identified by the OIG should have been detected by the
headquarters' review of the FYR report or if the issues should have been detected by the regions
during the FYR process. It is not appropriate to presume that data collected during the OIG
review would have changed the issues identified in a FYR conducted at an earlier date. We
12-P-0251
20

-------
recommend this sentence be deleted; it is unclear, misleading and not supported by the
information presented in this report.
OIG Response 8: We accept the intent of the requested change and revised the report as follows:
"We believe some issues we identified should have been reported in the site FYRs but were not."
Comment 2: Page 1, last paragraph: OSWER disagrees with second to the last sentence, "An
FYR is required by law at a site where the remedy leaves conditions thatdoes not allow for
unlimited use and unrestricted exposure."
We recommend replacing this sentence with the following: "Under the Agency's interpretation
contained in the NCP, the FYR requirement in CERCLA §121(c) is triggered when hazardous
substances, pollutants, or contaminants remain on-site above levels that allow for "unlimited use
and unrestricted exposure."
OIG Response 9: We modified the sentence. Reference to CERCLA was added to footnote 1.
See OIG Response 7.	
Chapter 1/Agencv Response to 2006 OIG Report
Comment 1: Page 2: OSWER disagrees with the second to last sentence, "As such, OSRTI
reviews focus on areas of draft FYR reports that have the greatest impact on the protectiveness
determinations."
We recommend replacing this sentence with the following: "As such, OSRTI reviews the entire
FYR report, especially those areas that have the greatest impact on the protectiveness
determination."
OIG Response 10: We accept the change. We revised the report as follows: "As such, OSRTI
reviews the entire draft report with a focus on areas that have the greatest impact on the
protectiveness determinations of the remedies."	
Chapter 1/Scope and Methodology
Comment 1: Page 3, third paragraph: OSWER recommends adding the word "all" to the second
sentence, "This was a judgmental sample comprised of (all) the cases where OSRTI had
recommended the region change the protectiveness determination and the region did not agree.
OIG Response 11: We accept the change of adding "all" to the sentence. It reads as follows:
"OSRTI had identified these 49 reviews in its internal assessment of FYR reports as being all of
the cases OSRTI recommended that the region change the protectiveness determination and the
region did not make the change."	
Comment 2: Page 3, Third paragraph: OSWER recommends making the following change to the
last sentence: "... we had on the quality and impact of OSRTI review for 18 of the 49 cases
where we identified significant disagreement between OSRTI's review and the final FYR report.
12-P-0251
21

-------
Our identified discrepancies were 9.8% (18 of the 182 reviews) of the total FYR reports
reviewed by OSRTI."
OIG Response 12: In a December 12, 2011, meeting between OSRTI and OIG, we explained
that we could not agree to the requested addition because we did not examine all 182 reviews.
Chapter 2
Comment 1: Page 5, first paragraph: OSWER recommends making the following change to the
fourth sentence: "The Agency lacks (formal) procedures for resolving differences between
OSRTI and the regions and for requiring regions to act on OSRTI's recommendations."
The FYR is not a decision document and the FYR process is delegated to the region and does not
include a HQ concurrence role. As such, we cannot "require" the regions to implement our
recommendations. In addition, the regions may have a good reason for not taking our
recommendation(s). Typically, the regions have more specific knowledge of the sites and a better
understanding of the site contaminants, potential exposures and operation of the remedy.
OIG Response 13: We accept the suggested change to remove the verb "require": "The Agency
lacks procedures for resolving differences between OSRTI and the regions and for documenting
regional decisions not to act on OSRTI recommendations."	
Comment 2: Page 5, first paragraph: OSWER recommends modifying the last sentence to read:
"Improvements should result in protectiveness statements that are better supported by the
information provided in the FYR."
Implementation of the reviewer's recommendation will not necessarily result in "better
protection of human health and the environment," they are likely to lead to greater transparency
and a clearer presentation of site conditions.
OIG Response 14: We accept some of the suggested changes, such as adding "... statements that
are better supported by the information provided in the FYR report..."	
Chapter 2/No defined Procedure for Resolving Differences Between OSRTI and Regions
for FYR Conclusions
Comment 1: Page 6, second bullet: Replace the word "phase" with "operable units" or "OU" as
appropriate.
OIG Response 15: We accept the suggested change to replace "phase" with "operable unit."
Comment 2: Page 6, first full paragraph, 5th sentence: "Even when conducting its own internal
assessments, OSRTI did not consult the final FYR reports to identify the protectiveness status and
whether the region corrected problems identified in the OSRTI review."
12-P-0251
22

-------
OSWER recommends that this sentence be clarified to state: "When conducting its own internal
assessment of headquarters' comments on the FYR reports, OSRTI did not consult the final FYR
reports. Instead, OSRTI used the protectiveness determination information entered into
CERCLIS by the regions to conduct their internal review."
OIG Response 16: We accept the suggested change: "When conducting an internal assessment
of its reviews of draft FYR reports, OSRTI did not consult the final FYR reports. Instead, OSRTI
used information entered into a database by the regions."	
Comment 3: Page 6, first full paragraph: OSWER recommends deleting the 6th sentence:
"Without formal follow up and resolution, final FYR reports may contain unsupported
protectiveness determinations."
This is an assertion without a basis in fact. Based on the small number of FYRs reviewed by the
OIG, it would be difficult to draw this conclusion. The regions remain responsible for supporting
their protectiveness determinations.
OIG Response 17: We do not accept OSWER's recommendation to delete the sentence. Our
review of a small number of FYR reports shows cases where the final reports contain
unsupported protectiveness determinations. If OSWER does not implement follow-up and
resolution controls over its reviews, particularly in cases where it has identified unsupported
protectiveness determinations, final FYR reports may contain unsupported protectiveness
determinations. Our report shows that regional responsibility for supporting the protectiveness
determinations, alone, is not an effective management control. To address the Agency's
concerns, we have added "some" to the sentence to indicate that this situation is possible.	
Chapter 2/Differences Between OSRTI and Regions
Comment 1: Page 7, first full paragraph: OSWER recommends deleting this paragraph, it's an
unnecessary additional workload for OSRTI and there is no evidence presented in the report that
citing the FYR guidance would have an impact on either the regions' willingness to take our
comments or their protectiveness determination.
OIG Response 18: We do not accept OSWER's recommendation to delete the paragraph. The
paragraph describes a process where the OSTRI reviewer would be communicating to the region
what the Agency's internal controls are for ensuring FYRs are properly conducted. The
Agency's internal control is the FYR guidance. Under OMB Circular A-123, Management's
Responsibility for Internal Control (see: http://www.whitehouse.gov/omb/circulars a!23 rev),
"management is responsible for establishing and maintaining internal control to achieve the
objectives of effective and efficient operations, reliable financial reporting, and compliance with
applicable laws and regulations." In fulfilling these obligations, management creates and
reinforces a culture that adheres to internal controls by communicating and reinforcing
throughout the organization what the internal controls are.	
Comment 2: Page 8: OSWER recommends replacing "three" with "four" in the following
sentence: "According to Agency guidance, the four options for a remedy under construction."
12-P-0251
23

-------
Please reference the 2001 FYR guidance, Section 4.5.1, Exhibit 4-6, page 4-21. "Protective" is
an option for sites under construction.
OIG Response 19: We do not accept the recommended revision. In the second column of
Exhibit 4-6, the 2001 FYR guidance lists three protectiveness statements, not four. As such, we
did not change the number. We did revise the wording in the report to clearly indicate that we are
referring to protectiveness statements (the second column of Exhibit 4-6).	
Comment 3: Page 8, last paragraph: OSWER recommends deleting the sentence that begins,
"Less than 10 months..The closing of a site special account has no bearing on the FYR
process or the Agency's ability to address any recommendations in a FYR.
OIG Response 20: We do not accept OSWER's recommendation to delete the sentence. We do
not state that the closing of the site special account has any bearing on the FYR process or the
Agency's ability to address any recommendations in a FYR report. However, if the Agency was
relying on special account funds to fund FYRs and the actions recommended in FYRs (which
does occur), closing an account and reclassifying funds prematurely could result in the Agency
having to bear the cost of conducting future FYRs and corrective actions and potentially needing
to rely on Superfund appropriations to fund such activity.	
Comment 4: Page 9, first paragraph: OSWER recommends clarifying the sentence that begins,
"We found that regions..." Although remedies may have "serious issues" and "warranted
considerable follow up action," if the performance issues are not resulting in unacceptable
exposures the site would be considered short-term protective. It is misleading to compare this
situation with a determination of "not protective." Please see, the Comprehensive Five-Year
Review Guidance, OSWER No. 9355.7-03B-P, Section 4.5 (page 4-14).
OIG Response 21: We make the recommendation that OSWER "clearly define protectiveness
categories in Agency guidance or policy." OSWER agreed with this recommendation and said it
would issue a directive to the regional offices clarifying the protectiveness categories. The wide
range of conditions that currently fall under the short-term protective category needs to be
clarified. In our opinion, a single protectiveness status should not be so broad to include both
remedies that have serious issues that warrant considerable follow-up action, and remedies that
are functioning as intended, but need institutional controls in place to ensure future exposures are
limited.
Chapter 2/Other Improvements to OSRTI Review Process Needed
Comment 1: Page 10, first four paragraphs including bullets: OSWER recommends that these
paragraphs be deleted. There are no data that suggest regions will more readily agree to accept
our comments if they are presented in a particular structure, format or order. In fact, too much
emphasis on the format of the headquarters' written reviews may have a negative effect on
comments in other areas of the document and may reduce the overall effectiveness of our review.
This is consistent with OSWER's response to Recommendation lb.
OIG Response 22: In a December 12, 2011, meeting, OIG and OSRTI managers agreed to
retain these paragraphs with minor changes suggested by OSRTI.	
12-P-0251
24

-------
Chapter 2/More Accurate and Complete Communication of FYR Results and information
Needed
Comment 1: Page 12, first paragraph: OSWER recommends clarifying what is meant by
"overstating" or "understating" remedy protectiveness.
OIG Response 23: We accept the suggested change and clarified in the final report: "In five of
the six cases, the region entered a remedy status that was more protective than that given in the
final FYR report. In the sixth case, the region entered a status that was less protective. OSRTI
has worked with the regions to correct the status in two cases after we brought the data entry
errors to OSRTI's attention."
Comment 2: Page 12, third paragraph: OSWER recommends clarifying this paragraph, please
include the citation for the statement "the region needs to offer the complete report to the public
on line." The first sentence implies that site data are not publicly available, while often data are
available from regional websites or national websites. Initially, we were constrained in the size
of files we could post on the web, which prevented us from publishing complete appendices on
line. Tables of data that are routinely found in appendices are typically of little interest to the
general public. If the public is interested, such data are available at the site information
repositories. Additionally, as of fiscal year 2011 the file megabyte limit on EPA web pages has
been substantially increased so that OSWER now can typically make the entire private site
FYRs, including appendices, available to the public on-line.
OIG Response 24: We accept the suggested change to clarify the paragraph. In a Five-Year
Review Program Initiatives memorandum (No. 9355.7-07, dated August 27, 2001), OSWER
requested of the regions that "the complete report [is to] be made available to the general public
through the regional internet web page." We consider the tables and maps to be part of the
complete report. OSWER responded that it could typically make the information available.
We maintain that this information should always be made available. We agree with OSWER's
planned action to provide online all components of FYR reports, including the appendices.
Thank you for the opportunity to review the draft report. If you have any questions regarding our
comments, please contact David Cooper at 703-603-8763 or Johnsie Webster, OSWER Audit
Liaison at 202-566-1912.
cc: Lisa Feldt, OSWER
Barry Breen, OSWER
Barbara Hostage, OSWER/OPM
James Woolford, OSWER/OSRTI
Elizabeth Southerland, OSWER/OSRTI
Robin Richardson, OSWER/OSRTI
Cheryl Upton, OSWER/OSRTI
David Cooper, OSWER/OSRTI
John Smith, OSWER/OSRTI
Mike Hurd, OSWER/OSRTI
Regional Superfund National Program Mangers
12-P-0251
25

-------
Appendix C
Distribution
Office of the Administrator
Assistant Administrator for Solid Waste and Emergency Response
Agency Follow-Up Official (the CFO)
Agency Follow-Up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Education
Audit Follow-Up Coordinator, Office of Solid Waste and Emergency Response
12-P-0251
26

-------