$
<
73
\
Mi)
¦p.. ,\s
PRO'S4-
OFFICE OF INSPECTOR GENERAL
Catalyst for Improving the Environment
Quality Assurance Report
Quality Assurance Review
of Two Assignments:
Assignment No. 2004-000929, an Unissued Report
on Environmental Justice
Assignment No. 2004-000709, Unissued Findings
on a Congressional Request to Review
EPA's Management of Superfund Resources
Report No. 08-A-0074
January 30, 2008
-------
Report Contributor:
Robert K. Bronstrup
Abbreviations
Acting Deputy IG
Acting Deputy Inspector General for Planning, Audit, and Evaluation
AIG
Assistant Inspector General
AT SDR
Agency for Toxic Substances and Disease Registry
CAMD
Clean Air Markets Division
CERCLA
Comprehensive Environmental Response, Compensation, and Liability Act
EJ
Environmental Justice
EPA
U.S. Environmental Protection Agency
HHS
U.S. Department of Health and Human Services
OAR
Office of Air and Radiation
OCPL
Office of Congressional and Public Liaison
OECA
Office of Enforcement and Compliance Assurance
OEJ
Office of Environmental Justice
OIG
Office of Inspector General
OPE
Office of Program Evaluation
ORD
Office of Research and Development
OSRTI
Office of Site Remediation and Technology Innovation
OSWER
Office of Solid Waste and Emergency Response
PEJ
Potential Environmental Justice
PM2.5
Particulate Matter 2.5 (Fine Particulate Matter)
S02
Sulfur Dioxide
-------
^£0SX
i -jQt-7 5 UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
| X\\/V ? WASHINGTON, D.C. 20460
V
OFFICE OF
INSPECTOR GENERAL
January 30, 2008
MEMORANDUM
SUBJECT: Quality Assurance Review of Two Assignments:
Assignment No. 2004-000929, an Unissued Report on
Environmental Justice; and Assignment No. 2004-000709,
Unissued Findings on a Congressional Request to Review
EPA's Management of Superfund Resources
Report No. 08-A-0074
For over a year, the Office of Inspector General (OIG) of the U.S. Environmental Protection
Agency (EPA) has taken steps to enhance its quality assurance process. This includes a
continual look at our products, processes, and activities, in order to identify and address
deficiencies. This quality assurance report is on two OIG assignments.
Part 1 of this report examines Assignment No. 2004-000929, which had a purpose to evaluate the
Agency's environmental justice program. That assignment included a demonstration of the
feasibility of evaluating the potential environmental justice implications using the Acid Rain
Program as a model for analysis. The OIG did not issue a final report, and this quality assurance
report examines the circumstances surrounding that decision. It includes comments by current
and former OIG officials about the decision not to issue the final report.
Part 2 of this report examines Assignment No. 2004-000709. This examines the circumstances
surrounding the OIG's decision not to include a specific chapter as part of the OIG's final report,
"EPA Can Better Manage Superfund Resources," issued February 28, 2006. That report was to
provide answers to a congressional request to evaluate Superfund expenditures at Headquarters
and the regions. The unissued chapter indicated that Superfund program managers had either not
implemented key performance management practices or acted on certain program efficiency and
effectiveness information. As a result, they could have missed opportunities to improve the
effectiveness of the Superfund program and achieve cost savings. As with Part 1 of this quality
assurance report, Part 2 also provides comments by current and former OIG officials.
-------
Previous to the issuance of this quality assurance report, the OIG had already acted to ensure that
decisions about the reporting of issues are thoroughly documented in the OIG working papers
and are transparent to all. This report has resulted in additional measures to enhance our
processes and document our decisions concerning the reporting of issues. Additionally, steps
have been taken by the OIG to more closely monitor Agency employees detailed to the OIG to
ensure and demonstrate their actions are not influenced by prior Agency responsibilities.
Bill A. Roderick
Deputy Inspector General
-------
Quality Assurance Review of Two Assignments
(Assignment Nos. 2004-000929 and 2004-000709)
Table of Contents
Introduction
Purpose 1
How This Review Was Conducted 1
Part 1 - Assignment No. 2004-000929
1 Review of Preliminary Research and Field Work 3
Background 3
Preliminary Research Guide/Design Matrix Prepared 4
Kickoff Meeting with Agency Officials 5
Preliminary Research and Field Work Followed Approved Guide 6
OIG Science Advisor Provided Input in Early 2005 7
2 Review of Meetings during and after Field Work 8
Meetings Held with AIG-OPE during Field Work 8
Contacts with Science Advisor during Field Work 9
Additional Meetings from June through August 2005 11
Meetings with Agency Officials Prior to Draft Report Issuance 14
3 Review of Report Phase 16
Draft Report Called for Agency to Issue EJ Guidance 16
Agency's Response Provided and OIG's Analysis Prepared 17
Final Report Prepared 18
4 Review of OIG Decision Not to Issue Final Report 20
Acting Deputy IG Continued to Note Objections after Draft Issued 20
Actions after December 15, 2005, Led to Cancellation of Final Report 21
Officials Provided Additional Information for this Review 23
5 Subsequent OIG Report Addressed Certain EJ Issues 27
-------
Quality Assurance Review of Two Assignments
(Assignment Nos. 2004-000929 and 2004-000709)
Part 2 - Assignment No. 2004-000709
6 Background 28
Superfund Report Requested 28
First OIG Briefing Report Issued in September 2004 29
7 OIG Decision to Issue a Full Report Rather than
a Second Briefing Report 31
Second Draft Briefing Report Issued in December 2004 31
Agency Response to Draft Briefing Report 32
OIG Analysis of Agency's Response 33
OIG Decision Not to Issue Second Briefing Report 34
8 OIG Development and Issuance of a Draft Narrative Report 36
Early Inspector General Involvement in Report Process 36
Issuance of Draft Narrative Report 38
OIG Working Papers Reflect Congressional Interest in Report Data 39
Agency Response, OIG Analysis, and Involvement of
Inspector General and Acting Deputy IG 40
9 Preparation of a Final Report and Decision to Delete Chapter 3 45
November and December Activities to Prepare Final Report 45
Proposed Final Report Shared with Agency, and
Activity to Finish Final Report 48
Activity on February 28, 2006, that Deleted Chapter 3 from Final Report 51
Activity after February 28, 2006 52
Officials Provide Additional Information for this Review 53
Part 3 - Conclusions and Actions Taken by OIG
Conclusions 56
Actions Taken by OIG 57
-------
Introduction
Purpose
We conducted this quality assurance review to evaluate why the U.S.
Environmental Protection Agency (EPA) Office Inspector General (OIG) handled
two assignments the way it did:
For Assignment No. 2004-000929, we examined the circumstances for
why the OIG did not issue the final report, "EPA Needs Specific Guidance
to Evaluate Potential Environmental Justice Impacts of Its Programs."
For Assignment No. 2004-000709, we examined the circumstances
surrounding an unissued chapter in the report "EPA Can Better Manage
Superfund Resources" published as Report No. 2006-P-00013 on
February 28, 2006. The assignment was conducted as a result of a
congressional request to review EPA's management of Superfund
resources.
For both assignments, the quality assurance review sought to determine whether:
The assignments complied with OIG policies and procedures current at
that time and adhered to professional standards.
The OIG can strengthen its process for ensuring the OIG's decision
making is transparent and documented in the OIG's working papers and
data systems so that internal disagreements are resolved timely.
The OIG initiated the review on Assignment No. 2004-000929 when the current
Assistant Inspector General (AIG) for the Office of Program Evaluation (OPE)
became aware in late June 2007 of the unissued report. This occurred while
preparing testimony for the U.S. Senate's Subcommittee on Superfund and
Environmental Health, Committee on Environment and Public Works. He
advised the then Acting Inspector General, who immediately requested a quality
assurance review of this assignment. Subsequently, the OIG employee who has
quality assurance responsibilities advised the then Acting Inspector General of
another report where a chapter with findings deleted was from the final report.
The then Acting Inspector General directed that this assignment also be reviewed.
How This Review Was Conducted
To conduct the quality assurance review of Assignment No. 2004-000929, we
analyzed the OIG's working papers prepared to determine whether the review was
conducted in accordance with OIG policy and procedures and generally accepted
government auditing standards current at the time. Additionally, we reviewed all
1
-------
write-ups of meetings between OIG officials and the staff assigned to conduct the
review. We reviewed this assignment from July through September 2007. We
focused on OIG staff activities on the assignment from April 2004 through
September 2006.
We interviewed Agency officials involved with this review from the Office of
Environmental Justice (OEJ), within the Office of Enforcement and Compliance
Assurance (OECA), and the Office of Air and Radiation (OAR). In addition, we
interviewed the OIG staff involved, including the following former and current
officials:
Inspector General (former)
Deputy Inspector General (current, previously acting)
Acting Deputy Inspector General for Planning, Audit, and Evaluation
(former) - this person had initially been the OIG Science Advisor
Assistant Inspector General for Program Evaluation (former and current)
Director, Cross Media Issues
To supplement data in the working papers, OIG staff and former officials
associated with the assignment provided supplemental emails and
correspondence. We reviewed correspondence within the OIG's electronic mail
system for any contacts between the Agency and the former Inspector General
about this review, and between the former Inspector General and OIG officials
and staff assigned to the review. We discussed potential conflict of interest issues
with the OIG's Office of Counsel. We were unable to perform a full review of
the former Acting Deputy IG's electronic correspondence during his time with the
OIG because the emails only remained on the Agency server for a limited time.
The review of Assignment No. 2004-000709 likewise included a review of the
OIG's working papers prepared to determine whether the review was conducted
in accordance with OIG policy and procedures and generally accepted
government auditing standards current at the time. We also reviewed write-ups of
meetings between OIG staff and OIG officials and with Agency officials. We
interviewed the same OIG staff that had been involved in the other assignment
except that instead of the Director, Cross Media Issues, we interviewed the
Director, Land Issues.
We conducted this quality assurance review from October 2007 through
December 2007. We focused on OIG staff activities on the assignment from
September 2004 through March 2006.
2
-------
Part 1:
Assignment No. 2004-000929
Chapter 1
Review of Preliminary Research and Field Work
Background
Executive Order 12898, "Federal Action to Address Environmental Justice in
Minority Populations and Low-Income Populations," signed February 11, 1994,
directs agencies to make environmental justice (EJ) part of their mission by
assessing the effects of their programs on minority and low income populations.
In 1995, EPA issued an "Environmental Justice Strategy" to implement the
Executive Order and integrate EJ into its programs, policies, and activities.
In a previous report, issued in March 2004 (Report No. 2004-P-00007), the OIG
concluded that EPA had not developed or established a clear vision for EJ nor
developed guidance on how to carry out the intent of the Executive Order. The
OIG concluded that EPA was not consistently integrating EJ into its day-to-day
operations. Also, EPA had not identified minority and low-income communities,
or defined the term "disproportionately impacted." The result was inconsistency
in determining EJ communities across EPA's regions and programs. The OIG
had made 12 recommendations to the Agency. Key recommendations included:
(1) reaffirm the Executive Order as a priority; (2) establish specific timeframes for
developing definitions, goals, and measurements; (3) develop a comprehensive
strategic plan; and (4) determine if adequate resources were being applied to
implement EJ. EPA disagreed with 11 of the 12 recommendations. EPA did
agree to perform a comprehensive study of program and regional offices' funding
and staffing for EJ to ensure that adequate resources are available to fully
implement its EJ plans.
In April 2004, the OIG initiated a followon review of the Agency's EJ program,
which is the subject of this special quality assurance report. As stated in the draft
report issued September 9, 2005, the overall review objective was to demonstrate
the feasibility of evaluating the potential EJ implications of the Agency's
programs. The review, Assignment No. 2004-000929, specifically sought to
answer the question: "Has EPA developed policy and guidance on how to assess
the Agency's programs pursuant to the responsibilities of Executive Order 12898,
using the Acid Rain Program as the model for analysis?" The Agency provided a
formal response to the draft report on October 24, 2005. The OIG analyzed the
Agency's response and met with Agency officials to prepare a final report, dated
3
-------
December 15, 2005. However, the OIG did not issue the final report. The OIG
had expended about 1,125 staff days on this assignment through January 2006.
A key player in the decision not to issue a final report was the OIG Science
Advisor, who later became the Acting Deputy Inspector General for Planning,
Audit, and Evaluation (Acting Deputy IG). In the early 1980s, the Science
Advisor had served as the Coordinator for the EPA Acid Precipitation Program at
North Carolina State University. He joined EPA in 1985, and until 2000 served in
a variety of science leadership roles in the Office of Research and Development
(ORD). These roles included Associate Director for Ecology in the National
Exposure Research Laboratory, for which he supervised EPA laboratory directors
with Acid Rain Program responsibilities. Starting in 2000 and up until August
2005, this person served as Science Advisor for the OIG. For a brief period of
time he served in this capacity while on a detail from ORD but in February 2001
was formally transferred to the OIG. The Science Advisor became the OIG's
Acting Deputy IG in August 2005, and served in that role until October 2006,
when he left the OIG. After leaving the OIG, the former Acting Deputy IG
returned to ORD, and is now EPA's National Program Director for Ecology.
The OIG team followed supervisors' directions and complied with generally
accepted government auditing standards and the OIG Project Management
Handbook during its preliminary research and field work phases. These phases
involved gathering and analyzing data to answer the assignment objectives. The
Director and the AIG-OPE reviewed and approved the assignment's design and
specific methodology for gathering and analyzing documentation. The team also
complied with OIG procedures for reviewing working papers and ensuring
analysis accuracy and quality. These procedures included discussions with
Agency officials on review results.
Preliminary Research Guide/Design Matrix Prepared
In April 2004, the OIG team met with the AIG-OPE to discuss the focus of the
review and developing a specific design matrix for the assignment. The OIG
evaluation initially was to focus on the impact of emission concentrations on the
areas that were purchasing allowances. The team was to evaluate whether the
Acid Rain Program, which had reduced Sulfur Dioxide (S02) emissions at the
national level, had negatively impacted air quality locally by creating "hotspots"
(increased concentration of pollutants in localized areas). The team was to also
evaluate whether the program had incorporated EJ protections within its
guidelines to provide transparency on which facilities are generating or acquiring
allowances. The team further planned to evaluate the impact on emission
concentrations without trading. The team was to determine whether controls for
reporting emissions by facilities were applied and adhered to on a consistent
basis. There was no evidence of any internal disagreement concerning the review
approach in the working papers, emails, correspondence, or interviews with staff
4
-------
involved on the assignment. The OIG's Science Advisor was not required to and
did not attend this meeting.
The team prepared a formal preliminary research guide for the assignment
resulting from the discussion with the AIG-OPE. The objectives as stated in the
guide were to answer the following questions:
Have EJ guidelines been built into the S02 Acid Rain Allowance Trading
Program?
Does the trading of S02 allowances create hotspots? Are there any trends
in the purchasing of allowances which result in the creation of hotspots?
What is the impact of the Acid Rain Program on local S02 emission
concentrations compared to local S02 emission concentrations in the
absence of the Acid Rain Program?
Does the location of sources trading S02 disproportionately impact
emission concentrations in EJ communities?
The guide reiterated that the team was to determine if there were indicators that
EPA should consider implementing EJ safeguards. The safeguards would be
needed to ensure that cap-and-trade programs do not disproportionately impact
local communities surrounding power plants that use allowances to cover excess
emissions. The guide asked which coal burning power plants employed the cap-
and-trade program to purchase the greatest amounts of allowances for excess
emissions. The guide also asked whether communities adjacent to high emissions
power plants have EJ characteristics.
The team was to conduct its work in OAR, including reviews in OAR's Clean Air
Markets Division (CAMD) and Office of Air Quality Planning and Standards.
Work was also to be done in OECA, which includes OEJ, and ORD. The OIG
team selected Regions 3, 4, and 5 for more detailed analysis. Although, the team
developed a specific methodology, the working papers did not show early contact
with the OIG's Science Advisor or other OIG staff who were knowledgeable in
the air program. The working papers do not indicate the AIG-OPE required the
involvement of the Science Advisor at this stage of the review.
Kickoff Meeting with Agency Officials
On June 21, 2004, the team met with CAMD officials at a kickoff meeting. At
the meeting, the team advised CAMD officials that it was starting the second
phase of its EJ evaluation as detailed in the OIG's multi-year plan. The objectives
of this evaluation, consistent with the directions provided by the AIG-OPE and as
stated in the review guide, were to:
Determine whether the Acid Rain Program, which has been recognized as
a successful cap and trade program for S02 at the national level, may have
achieved regional S02 reductions at the expense of some communities.
5
-------
Determine whether the trading of S02 allowances has created hotspots of
emission concentrations that would not have occurred under a command
and control environment.
The team explained it would evaluate whether the Acid Rain Program negatively
impacted air quality in the areas purchasing the allowances. Also, the team would
evaluate whether the program had incorporated EJ protections within its
guidelines to provide transparency on the program on what facilities are acquiring
allowances. The team said it planned to evaluate what the impact on emission
concentrations would be without trading. Further, the team would determine
whether emission reporting controls are applied and adhered to on a consistent
basis. The discussion with CAMD officials reflected no influence to alter the
objectives or discourage the review.
Preliminary Research and Field Work Followed Approved Guide
Between June 2004 and January 2005, the OIG team conducted data gathering
and analysis to answer the objectives as stated in the review guide. As detailed in
Chapter 3 of this report, the team met periodically with the AIG-OPE and
Director to discuss data gathered and receive additional instructions. The OIG's
working papers show the team met with Agency officials and reviewed EPA's
policies and procedures. Team activities were adequate to determine the extent to
which EPA had issued specific policy and guidance to implement Executive
Order 12898.
As part of the demonstration exercise using the Acid Rain Program, the team
developed and gathered the following data during field work, which was later
presented in Chapter 3 of the OIG's draft report issued on September 9, 2005:
The top 15 power plants that used additional allowances to comply with
the Acid Rain Program.
The top 5 power plants of those 15 in their actual emissions versus
allocated allowances.
Regional methodologies for determining EJ communities.
Potential Environmental Justice (PEJ) areas around the Tennessee Valley
Authority Paradise Fossil Fuel Plant in Kentucky.
Decreased S02, Fine Particulate Matter (PM2.5), and Ozone monitor
values between 1999 and 2004.
Increased/no change monitor values between 1999 and 2004 for S02,
PM2.5, and Ozone monitors,
Increased/no change S02, PM2.5, and Ozone monitor values between
1999 and 2004 in relation to PEJ.
PEJ in relation to monitor sites exceeding National Ambient Air Quality
Standards in 2004.
6
-------
Percent of PEJ S02 and PM2.5 monitors in relation/percent of regional
PEJ areas.
Percent change in number of monitors between 1999 and 2004.
An analysis of the working papers showed that the team gathered the data in
accordance with generally accepted government auditing standards, and
documented and summarized the data in the working papers.
OIG Science Advisor Provided Input in Early 2005
In January 2005, the team contacted the Science Advisor concerning the
methodology for determining whether there were additional health impacts due to
S02 and secondary particle exposure, to see if the methodology was logical. The
Deputy AIG-OPE said this methodology initially tried to use hospital admissions
data for the State of Kentucky. The purpose was to see if admissions for asthma
were higher in counties with coal-fired generators than in other counties. The
approach, however, had uncertainties, including the difficulty of using hospital
admissions data (most hospitals treat asthma in the emergency room) and the
inability to determine if admission to a particular hospital was an appropriate
surrogate for a place of residence. This approach was later abandoned in the
spring of 2005, when the team's focus changed to the placement of air monitors.
The Science Advisor did not immediately respond to the team about their
methodology. Instead, on January 27, 2005, he separately forwarded the team's
proposed methodology in an email to an ORD statistician. In our interview
during this quality assurance review, the Science Advisor said the ORD
statistician had found "multiple flaws" in the OIG's approach. However, the
working papers do not show that these concerns were forwarded to the team for
consideration. On August 29, 2007, during our quality assurance review, the
ORD statistician advised the OIG that he did not remember any details regarding
the review and had no correspondence to the Science Advisor expressing
concerns about the OIG's approach.
On February 9, 2005, the Science Advisor encouraged the team to provide copies
of the report to some experts in the Agency prior to official release of the draft
report. The Science Advisor said the Acid Rain Program was by far one of the
best in EPA, so any science and statistical data presented would have to be sound.
7
-------
Chapter 2
Review of Meetings during and after Field Work
The team conducted field work between June 2004 and July 2005, according to
working papers. During and after field work, the OIG team met with supervisors,
including the AIG-OPE, to discuss results at that time and determine what
additional field work was needed to complete the assignment. Additionally, the
Science Advisor indicated he would like to review the report before it was sent to
the Agency as a draft. The following addresses the more significant meetings
held and correspondence prior to issuing the draft report.
Meetings Held with AIG-OPE during Field Work
January 2005 Meeting
On January 26, 2005, the team met with the AIG-OPE and reported the results of
field work as follows:
Impact of Emission Shifting on Local Areas
The Acid Rain Allowance Trading Program allows for the geographical
shifting of pollution allowances and the potential formation of hotspots.
The shifting of allowances has allowed higher emissions in certain local
areas compared to the program's cap of allowances.
Many of the allowances used for compliance were obtained from out-of-
State facilities.
Retired and shutdown units continue to be allocated allowances annually.
Environmental Justice
CAMD and OAR collected and maintained information on the Acid Rain
Program but did not conduct EJ evaluations.
Responsibility was unclear for conducting EJ evaluations.
No internal policy/guidance by OAR or CAMD existed relating to
incorporating EJ into the Acid Rain Program.
CAMD expressed interest in working with OEJ to better incorporate EJ
into its programs.
CAMD will consider suggesting the Acid Rain Program for a future EJ
action plan in OAR.
There was an opportunity for CAMD to incorporate community
involvement protocols and educational outreach for PEJ areas.
Facility Characteristics
All facilities had PEJ areas within a 30-mile radius.
Low income/poverty was the predominant indicator of PEJ areas.
8
-------
Regions 3, 4, and 5 were able to provide GIS (Geographic Information
System) Maps.
At the regional level, there were no active EJ groups or complaints.
At the State level, there were EJ groups and complaints for (1) Hatsfield
Ferry, Pennsylvania; and (2) Harrison, West Virginia.
March 2005 Meeting
The team held another meeting with the AIG-OPE on March 15, 2005. At that
time, the AIG-OPE outlined the following specific approach that the team adopted
to gather additional data.
Obtain data from all monitoring sites and ascertain the average
improvement over time (if possible).
Determine for all sites the location of the monitors (were they located in
EJ areas or not?).
For monitors above the median reading, ascertain the locations of the
monitors.
Determine if monitors had been removed and, if so, whether they were
removed from EJ areas.
The AIG-OPE advised the team to try to determine whether all groups were
benefiting by the Agency's trading program. The AIG-OPE noted this impact
was going to be difficult to show. The AIG-OPE added that the OIG analysis
may show nothing at all - in effect, that there is not a negative impact from
trading. The meeting included a discussion on how to link the ambient
monitoring data to power plants and, specifically, emissions trading.
Contacts with Science Advisor during Field Work
On May 3, 2005, the Science Advisor requested the Director to provide him a
copy of the discussion draft report prior to it being sent to the Agency program
office. The Director agreed and noted the team was refocusing its review on
trends in ambient air quality rather than emissions.
On May 19, 2005, the Director and certain team members met with the Science
Advisor to discuss information on air monitor numbers, locations, and PEJ areas.
The Science Advisor again expressed concern with the OIG methodology.
Specifically, the Science Advisor said he did not believe "absolute change of
means was credible due to annual variability." The Science Advisor wanted to
know whether the means for each monitor were actually significantly different or
had stayed the same.
In an email to the team dated July 8, 2005, as the team neared completing field
work, the Science Advisor again expressed concerns about the team's analysis:
9
-------
My concern is that the annual-seasonal daily variability is so high
within sites that you are unlikely to detect a statistically significant
reduction in the pollutant concentrations between 2 years.... Just
looking at two averages without accounting for variance just
doesn't seem feasible.
In a subsequent email dated July 12, 2005, the Science Advisor stated:
Did you see a real decrease in ambient concentrations between
time 1 and time 2? Even if true or false, however, we still aren 7 in
a strong position to draw any conclusions unless we look at all
yearsmaybe it is lower at time 2, 4, and 9, but not at time 3, 5, 6
or 7. This begs the question then are we really shedding much
light on the EJ issue via this approach.
The Director replied on July 12, 2005, that:
...the purpose of this study is not to determine whether the acid rain
had a statistically significant different impact on EJ communities as
opposed to non-EJ communities.... Our analysis suggests that
while the program did reduce exposure to pollutants for much of
the population, not all communitiesincluding EJ communities
experienced the same benefits. Our message is simply that OAR
can build on the success of this program by working with OEJ to
identify and address any EJ communities that continue to
experience historic levels of these pollutants of concern.
The Science Advisor concurred that if OAR had advised the OIG that it did not
consider the Executive Order, and OAR had not looked to see which communities
are benefiting and not benefiting from the program, then the OIG should
encourage OAR to do so and let that be the end of the issue. He remained
concerned as to whether the OIG had valid data to suggest that while the program
reduced exposure to pollutants for much of the population, not all communities -
including EJ communities - experienced the same benefits.
In August 2005, the Science Advisor was appointed Acting Deputy IG. In this
position, he was the first line supervisor for the AIG-OPE and second line
supervisor for the Director.
On August 22, 2005, the Acting Deputy IG again expressed concern to the
Director that he remained uncomfortable with the analysis. He added that by
proportionately placing the number of monitors in both PEJ and non-PEJ areas,
EPA can track pollution concentrations to susceptible populations. At this point
he said, "I would feel more comfortable if we had an extramural statistician
confirm our effort here and tell us the possible down sides of what we did."
Although the team did not contact an extramuaral statistician, it did contact and
10
-------
discuss the Science Advisor's concerns with the OIG statistician in OPE. These
additional discussions are further detailed later in this chapter.
Additional Meetings from June through August 2005
On June 7, 2005, the team met with the AIG-OPE and the Science Advisor. The
team reviewed the analysis that has been performed pertaining to ambient air
monitoring. The team noted in its discussion write-up that it was asked to find out
if cap-and-trade causes disproportionate benefit among the nation's population -
meaning that one part of the population benefits from the reductions more than
another. The following were the specific issues discussed:
Data from all monitoring sites, and whether there was a reduction,
increase, or no change at all.
Where the monitors were located and whether they were in PEJ areas.
Whether monitors had been removed or added and, if so, where were they
removed and added in PEJ areas.
Whether the National Ambient Monitoring System tackled EJ issues.
The overall idea was to determine whether all groups were benefiting equally;
if not, was there a disproportionate impact.
The team reported that most communities have seen a reduction in S02, PM2.5,
and Ozone concentrations for 1999 and 2004. Some monitors showed either an
increase or no change. When discussing how most communities experiencing
increases still remain below health-based air quality standards, the AIG-OPE said
EPA should look at local experiences, not just entire regions, for trends. When
comparing PEJ and non-PEJ areas, the team found that a larger proportion of non-
PEJ areas improved, although both had an overall reduction in pollutant
concentrations. The team stated that the overall percent of monitors in PEJ is
proportional to the regional percent for S02 and PM2.5. The Director said this
was "good news," although EPA did not consider EJ when siting monitors. For
Ozone, the team found the percent of monitors in PEJ areas was not proportional
to the percent of regional PEJ, except for Region 3. In regard to whether monitors
had been removed and added disproportionately, the team found PEJ areas tended
to lose more monitors and gain less monitors in many regions.
The AIG-OPE asked whether the report's discussion on removed or added
monitors would include a statistical analysis or a comparison of percentages, and
the team responded it would be the latter. A detailed breakdown of all analyses
was in the working papers.
The team reported that it appeared that the Acid Rain Program is achieving
reductions in air pollutant concentrations. However, some areas - both PEJ and
non-PEJ - have pollutant concentrations that remained the same or increased for
11
-------
1999 and 2004. Additionally, it is more likely that PEJ areas remained the same
or increased for S02 and PM2.5 than for Ozone.
The team discussed with senior OPE officials whether the Agency should
implement safeguards to ensure that cap-and-trade programs do not
disproportionately impact local communities surrounding power plants. EPA did
not monitor point-source specific because the Acid Rain Program is viewed as a
regional program. The current National Ambient Air Monitoring Network is also
regionally based and not designed to address local impact. The OIG's previous
analysis showed shifting of emissions allowances for compliance. Opportunities
also existed for EPA to analyze data for health impacts.
The AIG-OPE advised the team that, when writing the report, it should think of
OAR's perspective and show how the OIG is trying to assist the program (versus
imposing an additional burden on OAR). The AIG-OPE added that the OIG, in
its report, should attempt to minimize an overreaction on OAR's part.
It is important to note that the AIG-OPE directed the team to focus more on EJ in
the Acid Rain Program and less on the Acid Rain Allowance Trading program.
The AIG-OPE explained that the report's audience is all citizens, not just those
directly associated with the Acid Rain Program. The AIG-OPE advised the team
that it should focus on EJ as the report concept instead of the Acid Rain Program
because the program was a highly successful one. The AIG-OPE thought the
team may want to state that it evaluated cap-and-trade using the Acid Rain
Program as the OIG's unit of analysis. The AIG-OPE also commented the
purpose statement could possibly read as follows: Have PEJ areas benefited from
cap-and-trade programs using the Acid Rain Program as a demonstration?
After June 7, 2005, other key meetings occurred and correspondence were sent
between the AIG-OPE, Science Advisor, and team as the team began to refine its
report message. The Science Advisor (and later as Acting Deputy IG) continued
to express concerns with the team's methodology, as noted on the following dates.
July 8, 2005: In response to an email from the Deputy AIG-OPE (who is also the
OIG statistician), the Science Advisor said he could not conduct the needed
analysis to address his concern. His concern was that the annual seasonal daily
variability is so high within sites that one is unlikely to detect a statistically
significant reduction in the pollutant concentration between 2 years. He said a
specific ORD statistician could shed light on this. This ORD statistician was the
one to whom the Science Advisor had sent the OIG's case methodology on
January 27, 2005. As noted, the Science Advisor said the ORD statistician had
indicated to the Science Advisor that there were problems with the OIG's
methodology.
July 12, 2005: The Deputy AIG-OPE told the Science Advisor that she
understood his concerns about seasonal variability. However, exposure to
12
-------
pollutants was calculated based on readings made every hour of the year. The
Deputy AIG-OPE believed there was little chance for bias. The Science Advisor
replied that he believed the Deputy AIG-OPE was "correct on all counts" with
one concern of whether the OIG could actually see "a real decrease in ambient
concentrations... " The Science Advisor continued to believe the OIG was not in
a strong position to draw any conclusions unless it looked at all years.
July 13, 2005: The Science Advisor stated there are three points to the study:
1. OAR officials have told the OIG that they did not consider the EJ
Executive Order in implementing the program.
2. Did the data exist for EPA to implement the EJ program?
3. Does the data resulting from the OIG's analysis actually suggest that we
are seeing anomalies - that while the program did reduce exposure to
pollutants for much of the population, not all communities, including
EJ communities, experienced the same benefits?
August 22, 2005: As previously cited, the Science Advisor (now Acting Deputy
IG) advised the Director that he "remains uncomfortable with the analysis." He
indicated he would feel more comfortable if the OIG had an extramural
statistician confirm the OIG's effort and indicate the possible downside of what
the OIG did. As the Acting Deputy IG, he was now the first line supervisor for
the AIG-OPE and second line supervisor for the Director.
August 22, 2005: The AIG-OPE also met with the Director regarding the report.
The draft report as of August 19, 2005, offered several recommendations to OAR
and one to the Deputy Administrator. The AIG-OPE did not want to offer any
recommendations to OAR, since the report was not about OAR or the Acid Rain
Program. Rather, the AIG-OPE wanted the report to focus on EJ and the lack of
guidance and definitions from OEJ.
August 23, 2005: The Acting Deputy IG stated to the Director, with copies to the
AIG-OPE and Inspector General, that:
We struggle with the best way to present a broadly important issue
to the Agency about EJ and regulatory actions... That without the
team's work, OAMD would not have likely taken any steps to put
this issue on their radar. EJ has so many important implications
for the Agency there is not question of the need to get this message
out.
The Acting Deputy IG believed it important "that the proper guidance from the
Administrator is the next measurable impact of the work."
August 24, 2005: The Acting Deputy IG met with the Director and said that the
OIG looked at two questions but there are many other questions the Agency could
13
-------
ask regarding EJ and the Acid Rain Program. The Acting Deputy IG told the
Director that the OIG should highlight the following in the report:
1) Reductions in S02.
2) Trading implications and how the Agency will benefit from additional
guidance on EJ.
3) No National Ambient Air Quality Standards exceedances for S02.
4) Differences exist in regional definitions of EJ and the distribution of
monitors; the characteristics of communities are all affected by the lack of
EJ guidance.
Meetings with Agency Officials Prior to Draft Report Issuance
As detailed below, the team also met with various Agency officials to discuss the
results of the review prior to the issuance of the draft report.
August 2005 Meeting with CAMD Officials
In August 2005, the team met with CAMD officials to discuss review results. At
the time, the draft report was to focus on whether PEJ areas are evaluated by the
Agency. The draft report was to use the Acid Rain Program as a successful
program and a model to determine whether EJ was addressed. The OIG was to
issue the report to OECA (which includes OEJ), but the OIG met with OAR
officials for informational purposes. The OIG staff explained that, based on
previous meetings with CAMD, it had changed its scope to reflect the lack of
guidance provided to CAMD to evaluate its program for EJ. The OIG used the
Acid Rain Program as an example of a program that has components where EJ
could be implemented but, without guidance, the Agency did not do so. The OIG
had looked at the program from three different perspectives: (1) Allowances,
(2) EJ Characteristics, and (3) Ambient Air Monitors.
Overall, the OIG working papers stated that CAMD agreed with the OIG's
evaluation findings. The team noted that due to previous meetings with CAMD,
CAMD was now looking into how EJ could be incorporated into its program.
August 2005 Meeting with OEJ officials
The team also met with OEJ officials in August 2005. The team advised OEJ
officials that the draft report focused on whether PEJ areas are being evaluated by
the Agency. The draft report was to use the Acid Rain Program as a successful,
model program to determine whether EJ considerations were addressed.
OEJ officials disagreed with the contents of the proposed draft report, and
commented that OEJ did not have the authority to require the Agency to assess
(monitor, analyze, report) EJ impacts in any of its programs. They said they were
not aware of whether any program offices in the Agency had made assessments
14
-------
from an EJ perspective. OEJ officials also said the OIG recommendations must
state that OEJ needs to have the authority, mandate, and resources to carry out the
Executive Order; otherwise, the recommendations would be useless. OEJ also
had problems with the analysis on PEJ and air monitors, and suggested that the
OIG look at percent change rather than absolute change.
After the meeting with OEJ officials, the team met with the AIG-OPE, who
decided that the report should be issued to the Agency's Deputy Administrator.
CADM Officials Briefing to Inspector General
On August 24, 2005, CAMD officials briefed the Inspector General, Acting
Deputy IG, AIG-OPE, Director, and team members on OAR's on-going EJ
assessment of its trading program. At the time, a contractor was in the process of
finalizing a review. Although notes of the meeting are not in the working papers,
the Director provided a copy of the OAR briefing paper. CAMD officials noted
that CAMD had never investigated whether the Acid Rain Program could cause or
contribute to environmental injustices. It was important, CAMD officials said,
that it assess (1) how trading of S02 emissions under the program might affect the
minority and low income communities describe in the Executive Order, and (2)
how trading of S02 emissions impacted emissions at both a regional and local
scale. The briefing paper stated:
The paper analyzes the impacts of the Acid Rain Program on
people of different races, ethnicities, and income levels.... This
report contains air quality modeling analyses using two different
methodologies to estimate the changes in exposure of minority and
low income populations to ambient concentrations ofPM2.5 as a
result of the ARP [Acid Rain Program]. Both analyses show that,
in 2010, each racial, ethnic and income level group studied
received similar average improvement in PM2.5 in the
eastern U.S.
15
-------
Chapter 3
Review of Report Phase
The team responded to the concerns of the Acting Deputy IG and AIG-OPE, and
drafted a report that contained no recommendations concerning the Acid Rain
Program. Instead, the draft report addressed the need for the Agency to issue
overall EJ guidance for all offices to follow. The draft report was independently
referenced in accordance with OIG policy. The independent referencer's
comments were cleared, indicating the report was supported by sufficient and
appropriate evidence. On September 9, 2005, the OIG issued its draft report.
After the Agency provided responses, the OIG prepared a final report that was not
significantly different than the draft report. However, although initially planning
to issue a final report in December 2005, the OIG decided not to issue a final
report (see Chapter 4).
Draft Report Called for Agency to Issue EJ Guidance
The OIG draft report, issued September 9, 2005, demonstrated the feasibility of
carrying out the Executive Order, as well as the need for guidance, by evaluating
the potential EJ implications of the Acid Rain Program. The report indicated the
feasibility of evaluating PEJ implications because data on emissions, allowance
trading activity, ambient air quality, and demographic characteristics on
communities were transparent and available. However, the design and conduct of
the evaluation also demonstrated the need for guidance in identifying program-
specific EJ issues of concern, defining an EJ community, and selecting criteria for
determining a disproportional impact.
With respect to the Acid Rain Program, the OIG demonstration project found
inconsistent EJ considerations within the National Ambient Air Monitoring
Strategy, and regional differences in the portion of air monitors in EJ areas:
The lack of Agency EJ guidance may potentially result in localized
increased concentrations of S02 through the Acid Rain Program, despite
the program's overall national reduction in S02 emissions. As a result,
local communities surrounding facilities that emit additional S02
emissions may bear increased human health and environmental burdens.
In Regions 1 through 5, the proportion of non-PEJ areas with reductions in
S02 pollutant concentrations for 1999 and 2004 was greater than the
proportion of PEJ areas.
The lack of EJ guidance may lead to differences in the location and
removal of monitors in PEJ and non-PEJ areas. Additionally, the
differences in the removal and relocation of monitors may
disproportionately effect monitoring in PEJ and non-PEJ areas.
16
-------
The report stated that air officials had advised the OIG that OEJ never requested
that they perform an EJ assessment, nor had they received any guidance from OEJ
on how to evaluate their program for EJ concerns.
The report noted that while EPA had created OEJ to coordinate the Agency's
efforts to integrate EJ into all policies, activities, and programs, OEJ officials said
they did not have the authority to request assessments or provide guidance at that
time. As a result, Agency program offices lacked the guidance necessary to carry
out the intent of the Executive Order, and determine any disproportionately high
and adverse change in health or environmental effects on minority or low income
populations. The OIG's opinion was that without clear guidance there is no basis
to believe that the Agency has integrated EJ considerations into its programs.
The OIG recommended in its draft report that the Agency delegate the necessary
authority to the OECA Assistant Administrator and OEJ to carry out the
responsibility of Executive Order 12898 to, specifically:
Develop guidance to assess the Agency's program, policies, and activities
for EJ as mandated by Executive Order 12898;
Develop guidance for assessments in accordance with Section 3-302 of the
Executive Order; and
Establish a timetable for completed assessments on major Agency
activities as outlined in the Executive Order.
This special quality assurance review found that the independent referencing was
carried out appropriately and the report data is supported by the working papers.
The recommended actions appear reasonable and in line with the facts stated in
the report.
Agency's Response Provided and OIG's Analysis Prepared
In a response dated October 24, 2005, the Agency (1) reaffirmed its commitment
to integrating EJ into all of its programs, policies, and activities; (2) asserted that
the OIG's methodology was critically flawed and misleading; and (3) stated that
its "Toolkit for Assessing Potential Allegations of Environmental Justice," issued
on November 3, 2004, represented current Agency policy with respect to EJ and
provided for a structured yet flexible framework for conducting EJ analysis. The
Agency disagreed with the OIG's draft report recommendation.
In accordance with the OIG established procedures, the team - including the
Deputy AIG-OPE - met at an exit conference on November 21, 2005, with OECA
and OAR officials, to discuss the issuance of the final report. The team explained
that the Agency's toolkit was not applicable to the recommendations to the draft
report. OIG indicated it would reiterate its recommendation on the need for
guidance.
17
-------
According to the OIG writeup of the meeting, OECA and OAR officials
commented that about a year-and-a-half ago the focus of the review was on
emissions trading. They believed the OIG's focus on EJ came from a complaint
to the OIG from a non-profit organization called the Public Employees for
Environmental Responsibility. The draft report did not mention the Agency's EJ
toolkit at all. OECA officials stated that they realized a couple of months prior to
this meeting that the focus of the report was on EJ and, as a result, the OIG did
not receive documents that would have been useful.
The OIG analyzed the Agency's response and continued to have exchanges with
Agency officials to ensure the report was accurate and reflected the Agency's
position, for balance. To illustrate, on November 28, 2005, in an email, an OAR
official told OIG staff he was pleased to see the OIG ".. .had taken many of our
suggestions, especially the ones that clarify how the air program is designed and
implanted," in the final version of the OIG's final report. However, the official
remained concerned about the "interpretation of the science of atmospheric
chemistry in the OIG's report, particularly the contention that proximity to a
power plant is a good indicator of exposure to fine particles (a secondary
pollutant) from power plants." The official believed the OIG members agreed
"with our response and have changed verbiage as appropriate, of utmost
importance is OAR's position that it remains unjustified to insinuate that any lack
of guidance could result in environmental justice concerns regarding S02
concentrations under the Acid Rain Program." The official believed that the
conclusion lacked credence.
Final Report Prepared
The OIG team's analysis of the Agency's response was included in the final
report that was dated December 15, 2005. The OIG incorporated comments from
the Agency's formal response, and from supplemental comments, including
technical comments sent separately to the Acting Deputy IG by a CAMD official.
The Acting Deputy IG in turn supplied the technical comments to the team.
Where appropriate, the team rebutted comments that the Agency made in its
response. Specifically, the OIG noted that OECA's former Assistant
Administrator had noted in a January 16, 2004, response to the OIG's first EJ
report that"... .the Executive Order requires the Agency to conduct internal
reviews of its programs, policies and activities...," and the Executive Order
specifically states that EPA should "Determine whether their programs, policies,
and activities have disproportionately high and adverse human health or
environmental effects on minority or low income populations."
Additionally, the OIG noted that the EPA toolkit was not mandatory and that the
OEJ Director had advised the OIG that EPA's Office of General Counsel
expressed concern with referring to the document as "guidance." The Agency's
response was fairly presented, analyzed, and summarized for the final report.
18
-------
An analysis of the draft and final reports did not show that the OIG materially
changed its opinion, facts, or recommendation when preparing the final report.
The OIG did incorporate the Agency's comments into the final report. Further,
Chapter 3 incorporated information about OAR's EJ study performed by its
contractor, and a comparison to the OIG demonstration project. The
recommendation remained virtually unchanged. With respect to the Acid Rain
Program, the report specifically noted:
Inconsistent EJ definitions within Regions 1 through 5,
The absence of EJ considerations within the National Ambient Air
Monitoring Strategy, and
Regional differences in the proportion of air monitors in PEJ areas.
It is important to note that the OIG's proposed final report also stated in
Chapter 3:
The intent of our analyses was to determine whether environmental
justice assessments could be performed on the Acid Rain Program
without specific guidance; it was not the intent of our analyses to
determine whether the program actually causes environmental
justice concerns, rather that responsibility resides with EPA 's
OAR
The different methods used in our analyses and OAR's recent
environmental justice report demonstrate the needfor criteria and
specific guidance when monitoring and assessing environmental
justice.
By early December 2005, the AIG-OPE had retired. On December 13, 2005, the
above-proposed final report was independently referenced in accordance with
OIG policy to ensure the accuracy, balance, and tone of the report. However, the
team did not answer the referencing comments. A review of the referencer's
comments showed: discrepancies with factual statements between certain pages of
the report, being unable to find support for a quote from a former Assistant
Administrator, and other insufficient indexing for report statements. The Director
had concluded that, due to the continuing objections of the Acting Deputy IG,
clearing the referencing comments at that time would not have been a productive
activity. Ultimately, as discussed in Chapter 4, the final report was not published.
19
-------
Chapter 4
Review of OIG Decision Not to Issue Final Report
After issuance of the draft report, the Acting Deputy IG continued to have
reservations about the information on the Acid Rain Program. This information,
in Chapter 3 of the draft report, was also included in the proposed final report.
These objections ultimately led to the decision by the Acting Deputy IG to not
issue the final report on December 15, 2005, as planned. The AIG-OPE had
retired on December 3, 2005, and had not reviewed the proposed final report prior
to his retirement. On March 2, 2006, the report was withdrawn from the
Agency's SCOUT tracking system at the direction of the Inspector General.
Acting Deputy IG Continued to Note Objections after Draft Issued
Through correspondence after the draft EJ report was issued, the Acting Deputy
IG continued to object to content in the report, as summarized below.
September 13, 2005: The Acting Deputy IG stated, "I don't think I can live
with Chapter 3 in the EJ report. I still think it is potentially misleading." He
said the language was confusing, and he wanted the chapter modified to stay
only with S02 and show how, without guidance on definitions and other
aspects, the conclusions might change. He expressed concern with the
analysis and said the OIG had not put enough caveats on it for him to be
comfortable.
October 26, 2005: After receiving the Agency's formal comments to the
draft report, the Acting Deputy IG advised the AIG-OPE and Director that the
Agency's comments caused him to worry about the confidence we inspire in
the Agency relative to our work. He believed that some of the Agency's
comments weaken what he thought was a strong argument.
December 13, 2005: The Acting Deputy IG told the OIG team he still had a
problem with the proposed final report. He recommended a memo report
stating what the Executive Order requires, what the EPA Administrator memo
states, what the Agency has done, and how the Agency does not know if
programs are having a disproportionate effect. The Acting Deputy IG said he
still was not inclined to use EPA's or the OIG's analysis for Chapter 3.
December 14, 2005: The Director advised the Inspector General and Deputy
Acting IG of a revised final report due date of January 19, 2006. The Acting
Deputy IG then advised the Inspector General that he did not like the report
because the analyses were not sound and the examples in the report will lead
some people to erroneous conclusions.
20
-------
When we contacted the former Acting Deputy IG for this review, he reiterated the
concerns he expressed during the assignment and drafting of the report in 2005.
He said he believed the Agency needed to issue guidance on EJ, as noted in
Chapter 2 of the final unissued report. However, he reiterated his belief that the
OIG's analysis in Chapter 3 could not be defended. He believed that releasing a
report that clearly demonstrated insufficient knowledge of the science involved
was not in the best interest of the OIG. He said he believed the OIG information
would not pass a scientific peer review. For example, the implications made in
the proposed final report were that the sulfur came from the local source, which
was not defensible with the information the team had. In his opinion, such an
error did not advance the cause of EJ, and could mislead the reader into believing
the reduction in sulfur did not occur because of trading when this was not at all
demonstrated. The former Acting Deputy IG emphasized, for example, that the
assumption that all sulfur concentrations detected in a PEJ community come from
a nearby power plant can be grossly in error. Sulfur from tall stacks can travel
long distances and thus could be from other sources.
The former Acting Deputy IG said that to use the OIG's analysis, more work
should have been done. He said he wanted an outside review of the OIG's
approach, but that would have taken a significant amount of time. He added that
the Director believed the issues in the proposed final report could be addressed in
the next report. The OIG initiated a subsequent review after a February 2006
meeting with the new Deputy Associate Administrator for OECA.
The former Acting Deputy IG said OAR officials were "very upset" with the
OIG's use of the Acid Rain Program. However, OAR, according to the former
Acting Deputy IG, played no role in the choice he and the Director ultimately
made regarding the report. He added that as Science Advisor in the OIG, he had
expressed reservations about the analysis when he became aware of the approach,
but as Science Advisor he did not have a supervisory role.
In subsequent correspondence, the former Acting Deputy IG said that, based on
his past experience in the Agency, EPA staff often believed that OIG staff did not
have the knowledge or background for many of the reviews that involved science.
His charge as the OIG Science Advisor was to help change that perception.
Actions after December 15, 2005, Led to Cancellation of Final Report
As noted, the team had drafted the final report and submitted it to the Director and
Acting Deputy IG for review. The team had planned to issue the final report on
December 15, 2005. Meetings that ultimately led to the report not being issued,
and another effort by the OIG to answer three questions posed to it by OECA's
newly appointed Deputy Assistant Administrator, are summarized below.
December 19, 2005: The Director advised the team that the toolkit provides a
framework for analysis but needs to be augmented. OAR and OIG review
21
-------
showed where better guidance would help. The Director added that EJ
analysis cannot be successfully done in isolation; stakeholder and community
issues need to drive and define the analysis. Regardless of guidance, the OIG
and OAR approaches could never be successful if the EJ issues are not
defined with third party input.
January 18, 2006: OIG staff met with CAMD officials. The OIG project
manager advised CAMD officials that OIG was "re-thrusting" the report.
OIG advised with respect to the report that:
The report was going to be about 95 percent looking at guidance
and direction of the EJ assessments, the other 5 percent will report
on the two analyses. The report will contain an analysis of EJ
assessments that is EJ assessment compared to CAMD's
assessment. The two analyses will also compare themselves to the
toolkit's guidance. The draft report will be reissued at either the
Assistant Administrator level or Administrator's level. The report
will be recommending a clear direction is neededfor conducting
EJ assessment. These assessments will answer the question
whether these programs have resulted in disproportionate impacts
to minority or low income populations. We will also be
determining how many assessments have been conducted.
CAMD officials said they had not been requested by any EPA office to
perform an EJ assessment, but had done so based on OIG's questioning
whether they had performed an EJ assessment per the Executive Order. In
performing the EJ assessment, CAMD officials said they had not provided
their contractors with the toolkit because it was not publicly available.
February 2, 2006: OIG staff met with OECA's new Deputy Assistant
Administrator. The OIG advised officials about prior EJ reviews and OECA's
disagreement with the prior report issued in 2004. With respect to the final
OIG report, the OIG Director explained the OIG was looking at two issues:
(1) EJ and the guidance available for program offices to do EJ assessments,
and (2) the extent to which OIG's analysis on the EJ implications of the Acid
Rain Program may be misinterpreted. The Director said OIG senior
management indicated it was not worth potentially discrediting the Acid Rain
Program due to possible misinterpretation of OIG's demonstration program.
OECA's new Deputy Assistant Administrator expressed interest in knowing
who in the Agency was using the toolkit, whether the toolkit worked for them,
and whether more definitions and guidance were needed. That information
would allow OEJ to make changes to the Agency's EJ program. Three areas
were discussed: (1) are various offices conducting EJ assessments; (2) are
these offices using the Agency's EJ toolkit and, if not, why; and (3) is there a
need for better direction/protocols for conducting assessments?
22
-------
February 10, 2006: The OIG Director advised the Acting Deputy IG of the
meeting with OECA. The Acting Deputy IG stated that the questions sounded
reasonable and he approved as long as the focus remained on the EJ toolkit
and the guidance needed. At this point, the Acting Deputy IG had decided
that the final EJ report would not be issued December 15, 2005, as planned.
March 2, 2006: The Inspector General directed her staff assistant to withdraw
the proposed final report from the SCOUT tracking system. This represented
the Inspector General's determination that the final report should not be issued.
SCOUT is the Agency's database system that identifies and tracks EPA's
significant ongoing work, including significant OIG assignments.
Officials Provided Additional Information for this Review
Since the decision making trail leading to the proposed final report was not
completely documented, we also contacted the following individuals to provide
their perspective on this assignment and the decision not to issue a final report.
Former Inspector General
On August 16, 2007, we contacted the former Inspector General. She said she did
not have a standard procedure to review all draft reports prior to issuance and was
not on the distribution list for draft reports. However, she recalled the briefing by
CAMD officials, and indicated that in essence she was pleased that OAR was
conducting an EJ review of the Acid Rain Program. While she recalled no
specific meetings with the former Acting Deputy IG, she believed he would have
kept her advised of his concerns concerning the draft and final reports. She added
that she would have relied on his assessment, since he had been the OIG Science
Advisor. She said that if the Acting Deputy IG believed the report had
information that would be misleading about the Acid Rain Program, she would
have agreed that a final report should not be issued.
In later correspondence, the former Inspector General added she always wanted to
ensure that what the OIG said in reports was correct and supported by adequate
analysis. She added that OPE had problems in a peer review and did not want this
report to add to that problem, because the OIG needed to maintain its credibility
with its customers and the public.
The former Inspector General did not remember receiving any calls from Agency
officials requesting or expressing an opinion that this specific report should not be
issued. When the decision was made not to issue a final report, she expected that
the project manager would have prepared a writeup to document the decisions
affecting issuing the report. Also, she said that the assignment manager and
Director should have ensured these decisions were documented in accordance
with audit standards.
23
-------
The former Inspector General noted that the Agency's former Assistant
Administrator for ORD, who was also the Agency's senior science advisor,
agreed to let the OIG have access to his science staff if the OIG needed scientific
expertise beyond what it had. The intent was to assist the OIG and ensure its
work was based on sound science. The former Inspector General said that while
this activity may appear to have been a conflict of interest, ORD in many
instances acted as an independent science and research arm for the Agency and at
times did not agree with the Agency's program scientists.
Former AIG-OPE
On August 13, 2007, we contacted the former AIG-OPE. He reaffirmed that the
methodology he had approved for use by the team was appropriate. He said his
objective was to show that through the demonstration project the Agency could
assess EJ in its programs in accordance with the intent of the Executive Order. In
August 2005, during the briefing by CAMD officials, he said he had requested
that OAR's contractor provide the data used, but it was not provided because it
was considered proprietary. Despite that limitation, the team included reference
to that contractor data in Chapter 3 to present the variance in methodologies used
by OIG and OAR's contractor. He said that at no time did any Agency official
request him not to issue the report.
OIG Director
The OIG Director emphasized that the review developed questions based on the
published results of peer-reviewed researchers. The team reached conclusions
regarding the feasibility and challenges of doing specific types of EJ analyses.
The Director said the OAR contractor essentially performed the same type of
analysis of potential local effects as the OIG team. He said the contractor did not
conclude that local impacts could not occur due to trading. Instead, the contractor
found that local emission increases in the studied area are balanced by nearby
emission decreases. The Director said that because comments by the Acting
Deputy IG and the Agency officials continued to support the need for overall
Agency guidance, he had favored issuing the final report on December 15, 2005.
The Director subsequently noted that documenting the change in the assignment's
focus, with the reasons why the December version of the final report was not
issued, would be helpful to peer reviewers. The former Acting Deputy IG would
have been in the best position to perform this step because the Director could not
have cited any documented analysis to explain the decision not to release the
report. Lacking that documentation, the Director said the decision not to release
the report rested on two issues - one technical and one a potential perception.
Regarding the technical issue, the Director said the former Acting Deputy IG had
concerns about the statistical basis for one of the supporting elements in OIG's
analysis (as detailed earlier in this quality assurance report). The Director, in his
24
-------
opinion, believed the former Acting Deputy IG's concern had been expressed as a
question or a concern, not as a statistical argument that contradicted the OIG's
analysis in the final report. As noted by the Director, the OIG's analysis was
endorsed by OPE's internal statistician, had been vetted by Agency staff, and had
passed through established internal quality assurance procedures.
The Director further believed that regardless of the outcome of any statistical
debate, the core message of the demonstration project in Chapter 3 was
compelling, and as the proposed final report stated:
even in the successful Acid Rain Program - a program not thought
to have environmental justice implications because it is a model
for other cap and trade programs and is designed to provide
regional benefits....we found inconsistent environmental justice
definitions.... the absence of environmental justice considerations...
and regional differences....
The Director added that Chapter 2 of the proposed final report by itself was no
more enlightening than the OIG's 2004 report on EJ. Chapter 2's basic theme
was that senior Agency officials had not specifically directed program staff to
assess the EJ implications of their programs, or provided specific guidance on
how to assess programs for EJ considerations. Chapter 3 of the proposed final
report highlighted analytical challenges that needed to be overcome: defining
issues and questions, developing methodologies, and collecting data. The
demonstration project, using the Acid Rain Program, provided a valuable
opportunity to demonstrate to senior leaders, the program offices, and the public
the scope, complexity, and potential implications of implementing the Executive
Order. The fact that the OIG's approach fostered debate about the "right" way to
analyze EJ had motivated OAR's own EJ analysis of the Acid Rain Program
validated the OIG use of the demonstration project.
The Director acknowledged that the former Acting Deputy IG believed the results
of OIG's effort, especially the demonstration of potential EJ issues regarding the
Acid Rain Program, could be misinterpreted. However, the Director believed that
the report had been written to minimize wording that could be taken out of
context and misused.
The Director added that the former AIG-OPE had endorsed the methodology and
the message in the report despite the longstanding concerns of the Science
Advisor. The former Acting Deputy IG did not endorse, in the Director's opinion,
the methodology, or the report message. The Director believed it was important
to note, as also acknowledged by the former Acting Deputy IG previously in this
quality assurance report, why the position of Deputy IG for Planning, Audit, and
Evaluations was created. The former Acting Deputy IG had advised the Director
and others that he was charged with correcting the negative perceptions regarding
OPE's work, which involved credibility, timeliness, relevance/utility, and tone. A
25
-------
key theme, according to the Director, was that the OIG's work needed to be less
confrontational, of greater utility to Agency customers, and have greater
credibility both inside and outside the Agency. In his opinion, the EJ analysis of
the Acid Rain Program was not what the Agency/OAR officials wanted to see and
instead the EJ questions asked by OECA officials were acceptable to the former
Acting Deputy IG.
OIG Counsel
The current OIG Counsel - who had been the Acting Deputy Inspector General -
reviewed the facts presented in this report and certain correspondence of the
Science Advisor/former Acting Deputy IG concerning the assignment. As a result
of the review, the Office of Counsel initiated steps to determine what additional
actions should be taken when the OIG hires or details an Agency employee into
the OIG. The Office of Counsel is drafting a questionnaire to find out more about
the type of work that the employee was engaged in, what major projects the
person was involved with, and who the employee's former supervisors were. This
information will better ensure that potential impairments can be identified and
whether a cautionary memorandum should be issued to the employee.
Agency Officials
Agency officials in OEJ and OAR said they did not contact the Inspector General,
Acting Deputy IG, or AIG-OPE to advise the OIG not to issue the report. They
said they only expressed concerns about data in the report they believed were
incorrect.
26
-------
Chapter 5
Subsequent OIG Report Addressed Certain EJ Issues
As a result of the February 2006 meeting with OECA, the OIG initiated efforts to
address the three questions posed by the OECA Deputy Assistant Administrator
in a separate report. On September 18, 2006, the OIG issued Report No. 2006-P-
00034, "EPA Needs to Conduct Environmental Justice Reviews of Its Programs,
Policies, and Activities," to address those issues.
The report objectives and the work conducted mirrored the questions posed by the
OECA official. As stated in the final report, the questions were:
1. Has there been clear direction from Agency senior management to
perform EJ reviews of EPA programs, policies, and activities?
2. Has the Agency performed EJ reviews?
3. Does the Agency have adequate guidance to conduct these reviews or is
there a need for additional directions or protocols?
As a result of its work, the OIG made the following four recommendations:
(1) Require the Agency's program and regional offices to identify which
programs, policies, and activities need EJ reviews and require these offices
to establish a plan to complete the necessary reviews.
(2) Ensure that EJ reviews determine whether the programs, policies, and
activities may have a disproportionately high and adverse health or
environmental impact on minority and low-income populations.
(3) Require each program and regional office to develop, with the assistance
of OEJ, specific EJ review guidance, which includes protocols, a
framework, or directions for conducting EJ reviews.
(4) Designate a responsible office to (a) compile the results of the EJ reviews,
and (b) recommend appropriate actions to review findings and make
recommendations to the decision-making office's senior leadership.
A review of the working papers showed that the assignment was carried out in
accordance with OIG policy and procedures for conducting evaluations and
generally accepted government auditing standards. The Agency accepted the
recommendations. However, the recommendations do not call for EPA to issue
overall guidance to ensure compliance with the Executive Order.
27
-------
Part 2:
Assignment No. 2004-000709
Chapter 6
Background
Superfund Report Requested
The Comprehensive Environmental Response, Compensation, and Liability Act
(CERCLA) established the Superfund program in 1980. Superfund is the Federal
Government's program to clean up the Nation's uncontrolled hazardous waste
sites. This program is continually the subject of evaluations. From 1995 to the
time of Assignment No. 2004-000709, over 120 internal and external audits,
reviews, and evaluations had been conducted by independent groups, including
the EPA OIG and the Government Accountability Office. Near the time the
assignment began, EPA had released an internal study, "SUPERFUND: Building
on the Past, Looking to the Future."
Congress, as specified in the conference report that accompanied H.R. 2673
(Consolidated Appropriations Act of 2004), asked the OIG to evaluate Superfund
expenditures and recommend options to increase resources directed to extramural
cleanup while minimizing administrative costs. The OIG Director for Superfund
evaluations (heretofore referred to as the Director) noted that while the OIG was
asked to recommend "options" to improve the use of resources, the AIG-OPE
decided to make specific recommendations to make the course of actions the
Agency should take sufficiently clear.
At the outset of the assignment, OIG staff met with staff for the House and Senate
Appropriations Committees. As noted in a February 26, 2004, meeting with the
congressional staffers, the congressional staff believed the Administration needed
to find ways to make the Superfund program more effective. One concern was
that too much money was going to administrative costs and attorneys and not
enough to actual cleanup. From a timing standpoint, both committees wanted the
OIG to complete a report with suggestions by the end of the fiscal year, so they
could use it during the Fiscal Year 2005 budget process. The staffers wanted OIG
to be able to indicate what the impact would be if Congress imposed a percent
limit on administrative costs. Additionally, the staffers asked what the advantages
would be if there was a required dedication of funds. OIG agreed to the following
questions as the focus of this assignment:
28
-------
(1) What are regions' and Headquarters' Superfund program expenditures
for the last 5 years (Fiscal Years 1999-2003)?
(2) How effective are the processes and criteria for determining, allocating,
and optimizing regional and Headquarters' administrative and support
resources?
(3) How effective are the processes and criteria for allocating Superfund
program dollars to program needs?
(4) How effective are EPA's procedures for integrating efficiency and
effectiveness information into the Superfund program?
The Director said that at the OIG's first meeting with congressional staff in
February 2004, the staffers said that if the OIG believed a 2-year project was
appropriate they would like an interim report. The interim report was desired
because Superfund administration was an issue and informal recommendations
would be helpful. The Director said that direction from the congressional staff
impacted the OIG's subsequent scoping and reporting decisions for the
assignment.
First OIG Briefing Report Issued in September 2004
As noted by the Director, because the congressional staff wanted the OIG to
respond quickly, the OIG decided to issues findings in a briefing report format.
The Director explained that this approach would allow the team to streamline the
reporting process and, accordingly, be responsive to the request.
While work was ongoing on the congressional request, OIG issued an initial
briefing report that summarized the work completed at the time to answer part of
Congress' request. This briefing report signed, by the Inspector General on
September 15, 2004, answered the first two questions above. The Inspector
General advised Congress that the OIG would complete its work to answer the
remaining two questions by January 2005.
The first briefing report was accomplished under Assignment No. 2004-000707.
A review of the working papers showed that they were prepared in accordance
with the OIG's Project Management Handbook current at the time of the
assignment. An evaluation guide was prepared, as well as working papers to
answer the steps in the guide. The working papers showed that supervisors
reviewed the work of staff. Meetings were held with Agency officials during
field work, commencing with the kickoff conference with OSWER on March 24,
2004. Officials in OSWER's Office of Site Remediation and Technology
Innovation (OSRTI) expressed concern about how the OIG would categorize
administrative costs, since there was no OSRTI definition of such activities for
Superfund dollars. OSRTI was also concerned about what the OIG might
recommend concerning OSRTI's workload model, which had not been used
since 1990.
29
-------
The OIG obtained points of contact for programmatic, budget, and enforcement
information. The working papers showed that OIG staff initiated field work in
accordance with the guide prepared for the assignment to collect and analyze data
to answer the congressional request. On August 20, 2004, upon completion of
field work, the OIG issued a draft report to the Acting Assistant Administrator for
OSWER. The OIG Director requested OSWER's comments by August 27 on the
factual accuracy of the information collected. The Director noted the briefing to
the appropriations committee staff was scheduled for August 31, with dates being
finalized.
An external review could note that the Completion Table for the assignment
shows certain quality assurance activities as not started - independent referencing
form, work product checklist, and quality assurance checklist. However, the
working papers clearly demonstrated that the OIG staff met with the Agency staff
on August 26, 2004, to discuss the draft report, and received written comments on
that date from OSWER, OECA, and Office of the Chief Financial Officer
officials. The working papers showed that the staff analyzed the comments from
all three offices, with specific notes on changes made to the briefing report to
incorporate those Agency comments. The working papers also showed that on
September 2, 2004, an independent referencer cleared responses to comments on
the final product. No factual material, findings, opinions, or recommendations
were subsequently deleted from the report.
After September 15, 2004, the working papers reflect no additional comments or
concerns by the Agency on the contents of the first briefing report. On
September 16, 2004, OIG staff met with the Senate appropriations staffers to brief
them on the report, and briefed House of Representative staffers on November 10,
2004.
OIG staff continued field work to answer the remaining questions, as described in
the next section of this report.
30
-------
Chapter 7
OIG's Decision to Issue a Full Report
Rather than a Second Briefing Report
OIG's subsequent efforts to address the remainder of the congressional request
were accomplished under Assignment No. 2004-000709. OIG intended to issue a
second briefing report. The following summarizes OIG's efforts to accomplish
that commitment and its subsequent decision to issue a full report instead of a
final second briefing report.
Second Draft Briefing Report Issued in December 2004
On December 14, 2004, the OIG Assignment Manager sent a second draft
briefing report to the Agency. This represented a "rollup" of information on all
four questions in the congressional request. In the transmittal memorandum, the
OIG notified the Agency that, as agreed with the appropriations staff, the OIG
briefing for the second two questions outlined in the notification memorandum
was scheduled for January 2005, with dates being finalized. The OIG requested
Agency comments on the factual accuracy of the data by January 4, 2005.
A review of the report shows the OIG had recommended options in three areas:
To better manage Superfund administrative and support costs
To make effective use of Superfund dollars
To integrate efficiency and effectiveness information into the Superfund
program
The report included as an option that Congress could re-evaluate the CERCLA
requirement that the Agency for Toxic Substances and Disease Registry (ATSDR)
conduct a full Superfund health assessment at every National Priorities List site.
ATSDR is part of the U.S. Department of Health and Human Services (HHS).
The potential savings or amount for redirection under this option was estimated at
$30 million per year. While the OIG could not determine estimated savings for
all recommended options, it did estimate that funding available for Superfund
cleanups resulting from improved deobligation and reallocation processes alone
was over $700 million.
To answer the remaining questions in the briefing report, the OIG had:
(1) Extracted Superfund expenditure data from EPA's Integrated Financial
Management System.
(2) Reviewed external studies of the Superfund program conducted by the
OIG, Government Accountability Office, Resources for the Future, and
31
-------
the Superfund Subcommittee of the National Advisory Council for the
Environmental Policy and Technology; and internal EPA studies.
(3) Identified recurring findings and recommendations for Superfund
improvement made to EPA through internal and external reports since
1995.
(4) Obtained information, including internal policy memos, guidance, and
draft action plans, from EPA Headquarters and regional officials. OIG
also interviewed officials in the Air Force cleanup program.
(5) Reviewed programmatic activities funded by the Removal, Remedial, and
Pipeline Operations "Advices of Allowance." OIG identified and
reviewed the Advices of Allowance funding process at the Headquarters
level and allocation process at the regional level.
The working papers reflect a "piecemeal" independent referencing process by
OPE staff assigned to independently reference the draft briefing report. No final
version of the draft briefing report was issued showing the indexes supporting the
statements in the draft briefing report. However, review of the various email
exchanges between the independent referencer and staff did not show any open
comments at the date of draft report issuance.
Agency Response to Draft Briefing Report
On January 3, 2005, the Agency provided a formal response to the draft report,
which included the following comments:
There is no reference to cost effectiveness of cleanup contracting, the
distribution of Superfund resources among various Agency offices, and
minimizing overhead costs while accelerating cleanups. Most of the
recommendations focus on redirection rather than savings.
The bulk of the resource redirection options relate to funds already
earmarked for cleanups (deobligating and special accounts) in some form
or another and do not address reducing administrative costs. In fact many
recommendations would increase administrative costs.
The greatest areas of "savings" identified by the OIG are those that would
involve deobligating funds from one site that needs the money in later
years, and reallocating to a site that needs funds that current year. Failing
to manage for future site needs by reallocating funds does not result in real
savings.
In some cases, OIG uses speculative assumptions particularly those in
which large amounts of savings are potentially involved. Critical findings
based upon current practices did not have supporting information as to
how it relates to site funding. Examples of incorrect information were
claimed by the Agency.
Some OIG recommendations could not be implemented in the Superfund
program, particularly those related to appropriations, remedy selection,
enforcement, and the preauthorized mixed funding program.
32
-------
Other recommendations to implement systems to monitor performance
would add to the Agency's administrative costs for Superfund. These
recommendations would result in less money being directed to cleanups.
OIG Analysis of Agency's Response
After receiving the Agency's response, OIG staff analyzed the response. Except
for pages 6 and 7 of EPA's response, the OIG analysis rebutted the Agency's
response or agreed to modify the briefing report based on the information the
Agency provided. For example, the Agency criticized the OIG's recommendation
that systems be instituted to better monitor performance because it would result in
less money directed to achieve cleanups. The OIG pointed out in its analysis that
the Agency did not provide support for that statement, specifically noting that "It
is unknown whether these investments would exceed the potential life time
savings associated with the options we recommend."
A working paper prepared on January 14, 2005, reflects the Assignment Manager
providing the Director with the above OIG analysis of the Agency's response.
No further discussion in the working papers about this analysis was provided.
The working papers did not contain evidence of any subsequent meetings between
the Agency and OIG regarding the Agency's January 3 response.
Because the OIG report also focused on ATSDR in an option, ATSDR officials
also received a copy of the draft report from the Agency. On January 5, 2005,
ATSDR provided a letter to the Inspector General. The ATSDR comments on the
OIG report, as well as OIG comments, follow:
OIG had not evaluated in detail expenditures from its Agency. OIG said it
had obtained budget documentation showing that ATSDR was allocating
$30 million for health assessments.
OIG did not distinguish between ATSDR's health assessments and EPA's
risk assessments. OIG said that was accurate but was not relevant for the
purpose of the issues in the OIG report.
OIG used outdated information and incorrect assumptions. OIG said that
assertion was incorrect because information was obtained during its
evaluation.
OIG should remove specific recommendations from its report. OIG said it
did not remove the recommendations because they were directed to EPA
and Congress and not to ATSDR.
The OIG team met with ATSDR and HHS officials on January 13, 2005, to
explain that data collected by the team was based on OIG's survey of all
10 regions and discussions with EPA Headquarters staff. The Director explained
that in the end she and the team's position was that the OIG had support for
recommendations related to ATSDR's use of Superfund money.
33
-------
In a subsequent letter to the OIG Director dated January 19, 2005, ATSDR raised
additional concerns about ATSDR health assessments and the $30 million for
health assessments cited in the OIG's report as a potential amount for savings and
redirection. OIG agreed to clarify the report to state that some part of that amount
could be saved or redirected. According to OIG staff, ATSDR officials had
suggested during the January 13 meeting that some amount could be saved but did
not know the specific amount. As noted later in this quality assurance report, the
AIG-OPE later told the staff that all references pertaining to ATSDR would be
removed from the OIG narrative report.
OIG Decision Not to Issue Second Briefing Report
Although the OIG committed to providing Congress with a second briefing report
to answer the remaining congressional questions, the second briefing report was
not issued. The working papers do not contain any discussion of the reasoning
behind the decision. However, OPE status reports dated in February 2005 and
after do explain the briefing report has been indexed and referenced but other
factors resulted in a decision not to issue the final report:
We received extensive comments on 1/5/05. Due to the nature of the
comments we have added an appendix, with our response to these
comments, to the briefing. In addition, due to Agency comments, we
needed to resolve a legal issue and an issue that required resolution
with ATSDR. These have taken more time than anticipated. All
issues have been resolved and all comments have been addressed.
The briefing report has been indexed and referenced. The time
required to respond to Agency comments on the briefing and index
our response has impacted our final report date.
Due to the significance of the findings and the Agency's response,
and the desire to report such findings in full context, it was decided
that the final briefing report would not be issued. Rather, only the
Final Report will be issued.
There is no explanation in either the status report or working papers on who made
the final decision not to issue the final briefing report. However, correspondence
not in the working papers showed that on January 31 the Assignment Manager
provided the OIG congressional liaison a copy of the final briefing and an
attachment. However, she stated a decision about releasing the report had not yet
been made by the Inspector, the AIG-OPE, or the AIG for the Office of
Congressional and Public Liaison (OCPL). Other correspondence not in the
working papers also reflects that on February 2 the AIG-OPE advised the Director
and team that he had spoken again with the Inspector and "... confirmed that she
does not want to issue this briefing report as a hard copy and she does not want to
have it on the OIG website." He advised the Director and the Assignment
34
-------
Manager that he had approved the briefing charts and that OIG's OCPL staff
should set up a meeting with congressional staff.
The Director said there was a consensus that OIG needed to provide necessary
context for the OIG's findings and recommendations, which could be done in the
narrative report because a narrative report would better address Agency questions
on the origins of some of OIG's findings.
35
-------
Chapter 8
OIG Development and Issuance of a
Draft Narrative Report
In January and February 2005, the OIG staff continued to gather and analyze data
to prepare a final narrative report to fully address the congressional request and
questions outlined in the notification memorandum. In subsequent months,
correspondence shows the Inspector General reviewed the drafts of the narrative
report. The development and issuance of the draft narrative report, including the
Inspector General's involvement, is summarized below based on the records
obtained. All dates are in 2005 unless otherwise noted.
Early Inspector General Involvement in Report Process
February 28: The team provided a draft report to the Director and AIG-OPE,
who subsequently provided it to the Inspector General on the same day. The team
reported that it had made changes as a result of the comments provided by the
AIG-OPE and Inspector General. These included comments on material that
would become Chapter 3 of the proposed final report on February 27, 2006.
March 14: The team provided another draft to the Director. The Director in turn
provided a copy to the AIG-OPE, who forwarded a copy to the Inspector General.
The Director stated that draft does address the Inspector General's comments.
March 23: The AIG-OPE advised the Inspector General's assistant that he had
finished reviewing the report and it was ready for the Inspector General's review.
March 25: The Director advised the AIG-OPE and Inspector General that a
version of the report, likely the December briefing report, had been released
(leaked) to Congress. The Director said the IG and AIG-OCPL had initially
brought this matter to her attention on March 25
April 19: The draft narrative report still had not been released. The Director
advised the AIG-OPE that the Inspector General's comments on the draft report
were in the possession of the editor; the Director said she did not have a copy of
the Inspector General's comments.
April 22: The Inspector General told the ATSDR Director that OIG staff have
worked with ATSDR staff to address and resolve concerns expressed in his
January 5 letter. Those concerns included that OIG should remove its
recommendations to Congress pertaining to ATSDR. The Inspector General said
that based on OIG followup work with ATSDR the OIG had modified its
recommendations.
36
-------
April 29-May 3: The Inspector General asked the AIG-OPE on April 29 whether
he had received the revised the draft. The AIG-OPE told the Inspector General on
May 1 that the Director had the report and would like to work on it before giving
it to him. On May 3, the AIG-OPE told the Inspector General that the report is
much improved and more readable, and he had just a few comments on it.
May 5: The AIG-OPE advised the Inspector General to pay specific attention to
certain sections of the report dealing with ATSDR and the National Institute of
Environmental Health Science. The AIG-OPE believed both sections help to
strengthen the report's theme.
May 6: The Inspector General advised the Assignment Manager and AIG-OPE
that the report is much better. The Inspector General advised that the report
referencer needed to be someone with experience, and noted the Acting Assistant
Administrator for OSWER is very sensitive about the report. The Inspector
General emphasized that "We need to have our ducks in a row and have support
for what we are saying." The Inspector General provided edit comments and
suggestions to strengthen the report. For example:
At the end of the discussion on special accounts, you should be
clear that EPA 's option is to transfer the funds to the Trust Fund
and if appropriated, then it could use the funds for cleanup. Tom
Dunne is very sensitive about how we discuss the special accounts.
Why aren 7 the regions being held accountable for determining
how much if any of the construction complete site funds might be
needed in the future on the site? "
With respect to a specific recommendation, the Inspector General stated:
EPA should hold managers accountable for improving SF program
efficiency and effectiveness in their performance standards... they
could do this under Standard 1 for senior executives in which
execs commit to accomplishing specific things during the
performance period. You may not have time to make this
recommendation more specific now, but you could do so during the
comment period. This kind of a recommendation might be similar
to what the Agency is doing with grants managers to hold them
accountable, worth a look.
May 23: The Assignment Manager told the AIG-OPE and Director that the
Acting Assistant Administrator for OSWER spoke that morning at the National
Association of Remedial Project Managers' Annual Conference. The conference,
in addition to EPA staff, included State officials and consultants. EPA OIG staff
also attended, and noted the Acting Assistant Administrator's remarks to the
attendees included comments about the OIG's yet-to-be issued narrative report.
They said the Acting Administrator advised the conference attendees that the
37
-------
report, although due to be issued the past December, had not been issued yet
because of "factual inaccuracies," and that "this report will be damaging."
OIG staff said he indicated that Superfund special account money is potential
responsible party money and that Agency staff advised the OIG team about this
but the OIG team did not understand. They said the Acting Assistant
Administrator stated that the Agency expected to deobligate $167 million during
the current fiscal year and $550 million had already been deobligated. OIG staff
said the Acting Assistant Administrator stated that, at the risk of losing his
position, he would defend the Agency against the OIG report.
June 14: Subsequent to the Inspector General's review of the draft report at that
time, another review of the draft report occurred within OCPL's Publications
Unit. The supervisory editor advised the AIG-OCPL of certain concerns with the
report. The editor said he believed the report conveyed a lack of understanding of
the principles of effective management, especially concerning measurement. The
editor also commented that the report appeared to convey that the OIG
misunderstood certain concepts such as baseline, benchmarking, and
accountability. As a result, the editor believed related recommendations were
illogical, confusing, and/or unsupported. The AIG-OCPL advised the Inspector
General of the editor's concerns, and noted that the OIG needed a better process
defined regarding responsibility and accountability on reports.
June 29: The Inspector General provided additional comments on the report and
advised the AIG-OPE, Director, and Assignment Manager that she approves of
this version of the report.
June 30: The Director advised the Inspector General that OIG will have to delay
releasing the draft report because the team had not had a chance to brief the
congressional staff on the recommendations and other details. The Director
advised the Inspector General that the OIG could still meet the proposed draft
report date if it obtains the requestors' input on the recommendations at the same
time the draft is out for Agency comments. The Inspector General concurred and
said that OCPL would tell the congressional staff about the recommendations and
request a meeting. The Inspector General also told the AIG-OPE, Director, and
Assignment Manager that the OIG Director for the OIG's financial statement
reports reviewed the report and he believed certain additional edits were needed.
The Inspector General said the OIG is informing Congress that there is even more
money spent in support costs than it realizes.
July 6 : The OCPL supervisory editor advised the AIG-OCPL that differences
with the team on language had been resolved.
Issuance of Draft Narrative Report
The working papers showed that the OIG followed its quality assurance process
current at the time. The draft report was independently referenced by an
38
-------
experienced GS-13 in OPE and all comments were cleared. The independent
referencer noted that certain statements in the draft report were indexed to a
briefing presentation that had already been independently referenced, and his
review was limited to identifying them in the briefing document. Additionally,
the independent referencer noted that because of the need to resolve questions
quickly so that the report could be finalized and issued, he did not go through the
lengthy process of providing written comments and receiving written responses.
Instead, he contacted team members by phone and resolved all questions orally.
The Assignment Manager did, however, document responses to specific final
referencing comments.
On July 11, the OIG issued the draft report to the Agency. The draft report
identified numerous opportunities for EPA to more effectively manage its existing
Superfund resources, as well as provide additional resources for cleanup. Among
the issues it identified were:
EPA is unable to allocate and manage Superfund resources as efficiently
and effectively as possible because of the way the Agency accounts for
program resources, manages by function caps, supplements the program
with other funds, and relies on an outdated workload model.
EPA does not allocate Superfund resources for extramural cleanup to the
greatest program needs because it prioritizes a different measure and uses
outdated management systems.
EPA is losing opportunities to improve the Superfund program's
performance and cost effectiveness, as well as opportunities to reallocate
potential savings to clean up priority sites. This occurred because EPA
was not managing efficiency and effectiveness information.
These and other issues, including the information and recommendation pertaining
to ATSDR, were contained in the four report chapters. Chapter 3, "Superfund
Funding Processes and Criteria Can Be Improved to Effectively Meet Program
Needs," and Chapter 4, "EPA is Not Managing or Measuring Superfund
Efficiency and Effectiveness Information," contained the bulk of the issues to
address the remaining questions agreed to by Congress. Large parts of these two
chapters were subsequently combined and formed the content for Chapter 3 in the
final report.
OIG Working Papers Reflect Congressional Interest in Report Data
The OIG working papers supported the Inspector General's concern about the
need for accuracy in the report and that the report's subject matter was sensitive to
the Agency. The working papers included information showing that on July 15
both senators from the State of Illinois (Durbin and Obama) send a letter to the
Administrator that expressed concern about Superfund sites at which EPA has
determined human exposure is not under control. At the outset, the letter to the
Administrator states:
39
-------
As the Senate considers the pending nominations of Marcus
Peacock, Susan Bodine and Granta Nakayama, we wish to
highlight an issue of grave concern regarding the Superfund
program. Specifically, we are deeply troubled by data revealing
that 103 superfund sites exist nationwide at which EPA has
determined "human exposure is currently not under control. "At
an additional 157 sites, EPA has determined there is not enough
data available to determine whether or not human exposure is
currently controlled, so the number of sites where humans are in
risk of exposure is potentially much higher than 103.
At a July 15 confirmation hearing, Senators Boxer, Lautenberg, and Obama asked
specific questions that dealt with information in the draft report. These questions
were directed to Susan Bodine, nominated to be Assistant Administrator of
OSWER. Some of the comments directed to Bodine included:
Please work with EPA to provide us with a complete list of Superfund
sites in order of current health hazards.
Please indicate how many children live near these sites and how they may
be at risk. Please also indicate any daycares, schools, playgrounds or
other similar places that are near these sites.
Please indicate what emergency or other short-term steps EPA may
conduct at each site to address the risks at that site, and the cost to take
those actions.
Please provide the cost to cleanup all 103 sites where EPA has determined
"human exposure is not under control."
Senator Boxer requested that the information be sent to herself, and Senators
Lautenberg, Durbin, and Obama. The working papers showed that Senator Boxer
believed this information was crucial to the Superfund program that the nominee
would lead. Accordingly, Senator Boxer requested that this information be
received before her Senate confirmation moved forward.
Agency Response, OIG Analysis, and Involvement of
Inspector General and Acting Deputy IG
In August, the Agency responded to the July 11 draft report, and the OIG began
its analysis of the Agency's very critical response.
August 11: The Inspector General told the Director and Assignment Manager
that she had just received a call from the Acting Assistant Administrator for
OSWER advising her that he would be signing a strongly worded response to the
draft report the following week. The Inspector General said that it is unfortunate
that the OIG and Agency do not agree on the facts.
40
-------
August 12: The Director advised the Inspector General that staff had learned that
the Agency was implementing many of the recommendations in the report either
via the Agency's 120-day study or in response to the draft report. Additionally,
EPA's Office of the Chief Financial Officer staff had advised OIG staff that they
had seen the response and had wanted to distance themselves from it. The
Inspector General said that if the response makes OSWER "look bad" she would
talk to the Administrator or the Deputy Administrator about it.
August 17: The Inspector General advised an OSWER official that she had
talked to her staff about the official's thoughts and those of HHS/ATSDR on the
scope of the OIG's work (along with her thoughts). She advised him that the OIG
staff decided to delete the ATSDR discussion from the OIG report.
August 18: The Agency provided its response to the Inspector General. The
response states that the report "continues the legal and factual errors,
misstatements, and misrepresentations that we have repeatedly pointed out to your
office." The Acting Assistant Administrator said the Agency fully nonconcured
with 21 of 30 recommendations in the report. In summary the Agency response
said that the report is "grossly inaccurate," because it contains "flagrant errors"
that have been identified on previous occasions. The Acting Assistant
Administrator said the report does a disservice to the Agency and it would be
irresponsible to finalize it. The Agency's detailed disagreements fell into the
primary areas of (1) the report significantly overstating potential savings and
promoting false conclusions about program resources, and (2) the report reflecting
a fundamental lack of understanding by OIG staff of the Superfund program.
August 24: Regarding the ATSDR findings included in the December draft
briefing report, the AIG-OPE advised a member of the HHS-OIG staff that no
references to ATSDR and HHS will be included. The AIG-OPE said this decision
was the result of a conversation between the Inspector General and an
ATSDR/HHS official. However, there was no explanation in the working papers
or the status reports concerning the decision to remove the information on the
ATSDR data from the draft report prior to receiving the Agency's response. Staff
did provide a copy of the Inspector General's handwritten notes on an undated
copy of the report. The Inspector General stated that "we said we would not
discuss and we should not discuss." She advised the team in the notes to provide
what little the OIG knows about this in a meeting and not in the report. She
advised the team that otherwise OCPL should explain that the OIG did not do
enough work to address, if someone does ask.
August 25: By August 2005, the OIG's Science Advisor had been appointed the
Acting Deputy Inspector General for Planning, Audit, and Evaluation (Acting
Deputy IG), and at this point became involved with the assignment. As noted in
Part 1 of this Quality Assurance Report, he was the direct supervisor for the
AIG-OPE and second line supervisor over the Director for this assignment. The
Director advises the Acting Deputy IG of the staffs plans for responding to the
41
-------
Agency's comments. A written document was to be prepared by the team that
showed the findings where the Agency made a comment, the Agency comment,
and then the OIG response and supporting working paper.
September/October: The working papers showed that in September and October,
leading up to the exit conference, the staff sought clarification and support from
the Agency on certain Agency comments. The OIG staff also requested and
received advice from OIG's Office of Counsel on specific topics and prepared a
detailed analysis of the Agency's comments. On September 20, the Director
provided the Inspector General, Acting Deputy IG, and AIG-OPE a copy of a
table that listed 57 Agency comments on the draft report and the team's
responses. The table included working paper support or references, and notes
concerning whether the team recommended changes to the report or was waiting
for additional information.
The detailed analysis of Agency comments on the recommendations and findings
was provided to the Agency prior to the exit conference held on October 20. The
analysis also showed how the OIG analyzed the Agency's comments and specific
notations of where the OIG would clarify or make changes to the report and
recommendations.
The Acting Deputy IG also inquired of the OIG's Office of Planning, Analysis,
and Results as to whether an internal review could quickly be made of the
assignment's working papers. The Acting AIG for the Office of Planning,
Analysis, and Results replied on September 9 that while the report was
complicated, his staff could probably do a very limited scope review of the quality
of evidence, supervisory review, and referencing for a few selected areas. The
limited scope of the review recognized the requirement to perform the review
quickly. For the purpose of the quality assurance report, the former Acting AIG
for the Office of Planning, Analysis, and Results advised that a request to perform
such a review was never requested and the Office of Planning, Analysis, and
Results did not perform any review of this assignment's working papers.
In reviewing an OIG analysis of the Agency's comments that would be shared at
the exit conference on October 20, the Acting Deputy IG, in handwritten notes,
advised the Director that his handwritten comments are his reaction and to "go
overboard to make as many issues go away as possible by adding information as
needed." The Director explained that she believed the Acting Deputy IG was
emphasizing to the team to provide as much information as possible to support the
OIG's findings and recommendations and to achieve agreement with the Agency.
October 7: An OCPL staff person advised the Director and the AIG-OCPL that
the House Appropriations Subcommittee staff were still interested in meeting with
the OIG to discuss legislative actions in the OIG's Superfund report.
42
-------
October 20: The exit conference was attended by the Acting Deputy IG,
AIG-OPE, Director, Assignment Manager, staff, and an OIG editor. The Agency
officials included the OSWER/Deputy Assistant Administrator, the OECA/Office
of Site Remediation Enforcement Director, the OSRTI Superfund Director, and
other Superfund staff. The writeup of the meeting showed that the attendees
discussed each area of the report, but at a certain point some Agency officials said
there were not many areas of Agency and OIG agreement. The OSWER Deputy
Assistant Administrator said that while he had read the OIG briefing materials
ahead of time, he was not comfortable discussing the issue areas without seeing a
revised draft report. The Director indicated the OIG process did not normally
include issuing a revised draft report.
October 20: Subsequent to the meeting, the Acting Deputy IG advised the
Inspector General that the Agency was clearly unhappy with the report. The
Acting Deputy IG said, "Being an outsider having had this problem before, I was
sympathetic to their arguments - this report will do damage." He summarized the
Agency points and acknowledged that the report would be released on
October 26. However, he said he was "terribly uncomfortable yet from an IG
perspective, just because they convince me they know what they are doing, they
have not produced the paperwork to prove it." The Acting Deputy IG
recommended (1) delay the report, (2) do another draft, (3) do a careful internal
review within the OIG, and (4) give the Agency another few weeks to respond.
He further advised the Inspector General that both the AIG-OPE and Director
believed this would set a bad precedent for other offices and OSWER in the
future. While he concurred, he said he "would also hate to do unnecessary
damage and that is more important to me."
After advising the Inspector General about the exit conference, the Acting Deputy
IG notified the AIG-OPE and Director that the OIG should prepare a very brief
summary of where the Agency and the OIG agree and a brief of the issues. He
said the OIG should take that information to Congress and advise them that the
OIG had completed work but there are "contentious areas of disagreement" that
would benefit from additional discussion and evaluation.
In reflecting on the meeting with the Agency, the Acting Deputy IG also advised
the AIG-OPE and the Director of his primary concerns:
OIG's relationship with the Agency and trust in the future.
The debate that could occur between the OIG and the Agency over factual
support.
His belief that merit existed in arguments on both sides and his uncertainty
on whether issues could be resolved between the OIG and the Agency
given more time.
43
-------
The Director reiterated to the Acting Deputy IG that this was the second
opportunity for the Agency to comment and review the OIG's findings. In
specific instances, during the meeting, the Agency had been unable to provide
documentation to support their statements concerning deobligations. Further,
OIG staff had spent 2 months reviewing the Agency's comments to provide
assurances that OIG did have support for its findings. Additionally, the OIG had
requested a revised Agency response and had not received one. The Director
advised that a revised draft report was to be provided for review by senior OIG
officials on October 21, 2005.
October 27: 2005: An email was sent from an OCPL employee to a
congressional staff person concerning their meeting earlier in the week. The OIG
expressed appreciation for input received on the OIG's legislative
recommendation. The OCPL employee advised that the Inspector General had
agreed to delay the report so that the OIG could look into the status of
reprogrammed Superfund money for Fiscal Years 1999-2002 and 2004-2005.
Also, the OIG would review for what activities that money was spent, as the
congressional staff person had requested. OIG believed this information would be
readily available and included in a final report it expected to issue in
mid-November.
The assignment's working papers reflected that the Agency did respond with
information in a series of communications to the OIG starting in mid-November
and continuing through February 21, 2006, when it provided final information on
carryover funds.
October 31, 2005: The Acting Deputy IG advised the AIG-OCPL that neither he
nor the Inspector General liked the current version of the report and wanted to
make changes. However, he asked the AIG-OCPL what Congress specifically
expects relative to the Superfund work.
44
-------
Chapter 9
Preparation of Final Report and
Decision to Delete Chapter 3
In December, the AIG-OPE retired. The OIG appointed the Director to be Acting
AIG-OPE, which extended through the duration of Assignment No. 2004-000709.
After the exit conference, the team continued its preparation of a final report. The
following summarizes the efforts of the staff and senior OIG officials to develop
the final report and, ultimately, the Inspector General's decision to delete a
chapter and some recommendations from the final report.
November and December Activities to Prepare Final Report
November 28, 2005: The Inspector General asked the Acting Deputy IG when
the next draft of the final report is due. The Acting Deputy IG said December 5.
Subsequently, the Inspector General told the Acting Deputy IG that she noticed
the OIG planned to issue the report by December 29 and it probably should be
delayed for a week because of expected media calls and the Acting Deputy IG
would be needed to handle those calls. The Acting Deputy IG stated that it was
no problem from his perspective to wait. He was not certain how the report
looked and if the OIG was on track.
The AIG-OCPL said for the purposes of this quality assurance report that
correspondence shows she had questioned the selection of the December 29
release date and she had no involvement in the selection of that date. A review of
the OPE status reports for November 23, 2005, confirms that the team had
projected a December 29 release date.
November 29, 2005: The Director advised the Inspector General, Acting Deputy
IG, and AIG-OPE that the Agency's 2005 Performance Report included a
summary of the findings, recommendations, and EPA's planned response to the
OIG's draft report. The Performance Report stated that EPA is working on an
implementation plan to take action on the OIG's recommendations. The summary
did not say that it disagreed with the OIG's findings or recommendations. In
other sections of the report, the Agency did say where it disagreed or agreed with
recommendations in other OIG reports. Among the questions the Director asked
is the relevance of the Agency's August 18 response. The AIG-OPE stated that
someone should speak with the Acting Assistant Administrator for OSWER about
his response to see if he wanted to send the OIG a different set of comments to
indicate that the Agency is working on an implementation plan to take action on
the OIG's recommendations.
45
-------
The Inspector General stated she still wanted to see a revised draft but that the
prior Agency's comments may be unnecessary. The Acting Deputy IG also told
the Director that "the bottom line is that the Agency recognizes they have
problems and in general they cannot argue the big picture. They may still argue
over individual elements that lead us to the bottom line - better management and
accountability is grossly needed in Superfund."
December 8, 2005: The Acting AIG-OPE provided a copy of the report to the
Inspector General for review. The Director explained how comments by the
Acting Assistant Administrator for OSWER and other information had been
handled in the report.
December 11, 2005: The Inspector General advised the Acting AIG-OPE, the
Acting Deputy IG, and the Assignment Manager that Chapter 2 is about ready but
Chapter 3 is not. The Inspector General's comments on Chapter 3 are that it reads
like a laundry list of unconnected items. The chapter contained 14
recommendations. She suggested finding those items that link most closely to the
message and use them to illustrate the OIG's point. The chapter needed to have a
clear and simple message with a few good examples. She specifically noted that
for Technical Outreach for Communities this represented .08T of the Superfund
expenditures and asked whether it is it really where we want EPA to spend its
time.
December 12, 2005: In response to questions from the Acting Deputy IG, the
Inspector General advised that if OIG correspondence contains recommendations
and will be placed on the OIG Web page, then public relations staff should be
aware. She further advised that the draft report should not be issued until after the
first of the year due to time constraints in getting the report referenced and
because the Agency most likely would not have staff available to review the
report. The AIG-OCPL also advised that all OIG work that is public needs to be
shared with OCPL for Web posting and also to keep congressional staff and the
media informed should inquiries be made.
December 14, 2005: The Acting Deputy IG advised the AIG-OCPL that he
approved of Chapter 1 and 2 with edits. He believed Chapter 3 was an "after
thought." In his opinion, the point of Chapter 3 was not only that they are not
managing money as noted in Chapter 2, but they are not really managing the
program either and Chapter 3 is a mix of evidence. The Acting Deputy IG
indicated that the Inspector General and he would like to have the OCPL
supervisory editor rewrite Chapter 3.
December 16, 2005: As the report was being revised, the working papers showed
that as part of the Congressional Record, Senator Boxer advised the President
that:
46
-------
I am releasing the hold I placed on the nomination of Susan
Bodine for Assistant Administrator, Office of Solid Waste and
Emergency Response based on the written comments I have
received from EPA to provide information and documents I have
requested in connection with oversight of the Superfundprogram.
I originally requested information on the Superfund program
immediately after the Ms. Bodine confirmation hearing July.
To date Ms. Bodine and EPA have provided only a partial
response to my request. I want to be clear that every question I
posed to Ms. Bodine and all the information I requested from EPA
on this important public health program should have been
provided to me without restriction as part of the Congressional
oversight process. I ask unanimous consent that the original
questions posed to Ms Bodine be printed in the Record. EPA has
now committed to provide additional information by January 31,
2006. (Note: Among the original questions is "Please provide the
cost to cleanup all 103 sites where EPA has determined human
exposure is not under control. ")
December 21, 2005: The Acting Deputy IG asked the Inspector General that,
"Given the superfund tension," would it be appropriate to share another draft of
the report with the Agency because it is very different.
December 22, 2005: The Acting Deputy IG had advised the Acting Assistant
Administrator for OSWER that OIG had observed that the Agency's 2005
Performance Report included EPA's planned response to the OIG's draft report.
Given the Agency's pronouncement by the Agency on the draft report, he wanted
to work as quickly as possible to bring the report to closure. The OIG had
redrafted the report. The OIG intended to share the revised draft report with the
Agency shortly and would provide the Agency a week to respond with any final
suggestions. The OIG could only accept suggestions that are consistent with the
findings
January 4, 2006: The Inspector General had reviewed another draft of the report,
believed the report was "getting better," and asked the Acting Deputy IG what he
thought. The Acting Deputy IG said he believed the report is also better.
January 27, 2006: The Acting Deputy IG advised the AIG-OCPL that both he
and the Inspector General had agreed to the Agency reviewing the report for
5 days for any inaccuracies and not general revisions.
47
-------
Proposed Final Report Shared with Agency, and Activity to Finish
Final Report
The final version of the report had been changed from the draft issued on July 11,
2005. A February 2, 2006, draft of the final report showed that significant parts
of Chapters 3 and 4 in the July 11, 2005, draft report had been combined into a
new Chapter 3, "Decentralized Management and Lack of Accountability Hinder
EPA from Efficiently and Effectively Managing Superfund Performance." The
revised chapter also contained reference to the 103 sites where EPA had
determined human exposure was not under control:
As of March 7, 2005, of1,530 final and deleted NPL sites, 280 were
classified as "not under control" (103 sites).... In response to a July
2005 request, in October 2005, EPA compiled information on the
risks at sites designated as "not under control". We became aware
of this in January 2006 and requested the information from the
Agency. Due to incomplete data, we were unable to fully
characterize the risks from these 103 sites. (Note: Subsequent drafts
of the final report state that since the OIG received the data on
February 8, 2006, the OIG was unable to fully characterize the risks
from these 103 sites in time to include it in the report).
Chapter 3 as revised contained 14 recommendations representing opportunities
for EPA action to (1) improve managing Superfund resources, (2) provide funds
for Superfund cleanup, and (3) improve other aspects of Superfund program
management. Other elements of Chapters 3 and 4 of the draft had been moved to
Chapter 2 in the proposed final report. This included a section that "EPA Does
Not Have a Fully Effective Process to Guide Decisions to Deobligate
Unliquidated or Use Special Account Funds (the report noted about $639 million).
February 1, 2006: The Acting Deputy IG advised the Director/Acting AIG-OPE
that he preferred the revised Chapter 3 as it was currently organized.
February 6, 2006: The Director/Acting AIG-OPE provided the Agency with
another draft of the final OIG report and requested comments by February 10,
2006.
February 10, 2006: The new Assistant Administrator for OSWER (Susan
Bodine) provided comments back to the Inspector General on the OIG's
February 2 draft of the final report. In this response, the Agency essentially
nonconcurred with Recommendations 3-1, 3-2, 3-3, 3-6, and 3-7. The Inspector
General thanked the Assistant Administrator and stated, "Looks like we are
getting closer to something we can all live with." She said she appreciated
having the response to look over during the weekend.
48
-------
February 11, 2006: The working papers reflected changes to the draft final
report based on the Agency's response to the draft final report given them. The
staff evaluated the Agency's proposed changes and made changes they believed
were appropriate. The draft reflected how the Agency's comments had been
incorporated. Chapter 3, including the 14 recommendations, was still in this
version of the final report.
February 15, 2006: The Acting Deputy IG advised an OSWER official that he
would like to go over the recent Superfund report before finalizing it and sending
it to Congress. The Acting Deputy IG said he remained uneasy about the
Agency's nonconcurrence and believed the OIG and the Agency would have been
further along on an agreement on the report issues. The official responded that
another meeting would be beneficial.
February 16, 2006: The working papers contained another revised version of the
proposed final report; Chapter 3 and its 14 recommendations were still included.
February 21, 2006: The OIG met with Agency officials to further discuss the
report prior to issuance. A detailed review of the meeting writeup in the working
papers showed that the Acting Deputy IG stated at the meeting that "the purpose
of the meeting was for the OIG to better understand the Agency's reasons for the
nonconcurrence with the OIG's recommendations as the OIG was under the
impression that the Agency and the OIG were closer on agreement regarding the
recommendations." The writeup further showed that OSWER and OIG staff
discussed the recommendations on which the Agency disagreed. The OIG
documented the officials' reasons, but at one point noted that the OSRTI Director
stated, "Well, we had to disagree with some of the OIG's recommendations." The
OIG Director/Acting AIG-OPE said that it would be helpful if the Agency had
provided alternative actions to the OIG recommendations. After the meeting, the
Acting Deputy IG explained that for the recommendations for which the Agency
did not concur, the OIG might want to add language to the recommendations to
explain why we recommended what we did.
February 22, 2006: The Director/Acting AIG-OPE advised the Inspector
General, the Acting Deputy IG, and the AIG-OCPL that a positive meeting had
occurred with the OSWER/OECA senior staff and there was a better
understanding of the reasons for nonconcurrences on some of the
recommendations. The Director reported that final changes to the report were
being made and that once the report had been cleared by the Acting Deputy IG
and the editor, it would be forwarded to the Inspector General
February 27: The working papers showed that the final version of the report had
been assigned a publication number (2007-P-00012) and had cleared independent
referencing with all comments responded to. Chapters 3, including the 14
recommendations, were still included. The narrative report was intended to answer
the remaining questions from the original congressional request as summarized in
49
-------
the original notification memorandum, as well as capture information in the first
briefing report. The summary of the revised Chapter 3 stated:
Because EPA does not have a single accountability entity
responsible for Superfundperformance, EPA has not implemented
some key performance management practices or acted on some
program efficiency and effectiveness information. Superfund
managers used Superfund resources for program or activities not
clearly linked to Superfund priorities or greatest risks. Superfund
program managers have missed opportunities to improve the
effectiveness of the Superfund program and to achieve cost
savings.
The chapter's specific sections had the headings:
Superfund Managers Provided Resources for Activities Not Clearly
Linked to or Accountable for Superfund Priorities or Goals
EPA Has Not Implemented Challenging Superfund Performance Goals or
Measures
Decentralized Program Causes Superfund Managers to Miss Repeated
Opportunities to Improve the Program's Efficiency and Effectiveness
Chapter 3 continued to make reference to the 103 sites as not under control. The
14 specific recommendations again were to (1) improve managing Superfund
resources, (2) provide funds for Superfund cleanup, and (3) improve other aspects
of Superfund program management. The report also included a table for
Chapters 2 and 3 showing "Potential Savings or Amount for Redirection."
The Assignment Manager received oral clearance from the independent referencer
(an experienced GS-13 who had been an instructor for the OIG independent
referencing course) on all referencing comments. The working papers showed
that, subsequently, he had verified all factual statements in the work papers to
original source documents or other supporting documentation. The independent
referencer concluded all report statements were reasonably supported by the
information in the working papers. For the purposes of this quality assurance
review, the independent referencer was contacted. He reviewed the independent
referencing memoranda and stated that he was not influenced or pressured at any
point in time to clear comments or accept report statements without supporting
evidence.
At about 4:30 p.m., the Acting Deputy IG advised the Inspector General that
because of the problems with the report earlier, he wanted the recommendations
handled better and asked that they be changed to be unusually explanatory. The
Acting Deputy IG advised that the report chapters were now ready for the
Inspector General's review. At about 5:00 p.m., the Acting Director forwarded
the final report to the Inspector General. The report included Chapter 3 with
50
-------
14 recommendations. Specific recommendations had been revised to be more
explanatory as requested by the Acting Deputy IG.
Activity on February 28, 2006, that Deleted Chapter 3 from Final Report
As reflected in the working papers, a series of emails occurred on February 28,
2006, between the Acting Deputy IG, AIG-OCPL, Director/Acting AIG-OPE, and
staff on the assignment. Commencing with the Acting Deputy IG email at 11:14
a.m., he instructed the Director and team to:
Remove potential cost savings from the table
Drop Chapter 3
Revise the Table of Contents
Get the report to the Inspector General for issuance
Write a summary letter report about additional information now in
Chapter 3 and discuss these matters with congressional staffers.
The AIG-OCPL advised the Acting Deputy IG that, as a result, changes would be
needed to the introduction, the Agency's response, and the OIG evaluation of the
Agency's response ("This will need to be cut down, rewritten as necessary, edited
[and may need additional referencing?])" However, the Acting Deputy IG
advised the AIG-OCPL, Director/Acting AIG-OPE, Assignment Manager, and
other staff that he wanted the report issued on February 28, 2006.
The Director/Acting AIG-OPE then advised the Acting Deputy IG, with a "cc" to
the AIG-OCPL and staff, that:
...there is some confusion, questions and concerns about the
decision to remove Chapter 3. This is your decision and we
respect it from your position. From my perspective, proceeding
this way will produce a report that is not different from the briefing
report we issued 18 months ago and will likely generate the same
public questions and criticisms we got then (Is that all EPA OIG?)
gives the wrong impression that we've done nothing (or are hiding
something), and doesn 7 really present new information. I'm also
concerned about the possibility of a FOIA [Freedom of
Information Act], on the missing Chapter 3, or other records
people may want to see on this job. Recall that earlier versions of
our draft (which were leaked) included the Chapter 3 materials.
The Acting Deputy IG responded that "I hear, I understand, I am sympathetic, and
with time I might even be persuaded but I need to keep decision as is." He
delegated to OPE the responsibility of getting the Inspector General a cleaned-up
copy for signature, and providing the Agency with a heads up that the report will
go out February 28. He said that this was done with EPA's Deputy Chief
Financial Officer. The Acting Deputy IG recommended the team meet its target
51
-------
date so the Inspector General could sign. The Acting Deputy IG added that as
further incentive, if the Inspector General did not sign, the new Acting IG would
want to review the report from the beginning yet again if he were to sign off on
the report.
The OIG dated the final narrative report on February 28, without Chapter 3 and
14 recommendations. Additionally, all tables in the report showing potential
savings or amounts for redirection were deleted. The report was signed by the
former Inspector General and directed to the Assistant Administrator for OSWER.
Copies were also sent to several other Assistant Administrators and the
Congressional Liaison, Office of Congressional and Intergovernmental Relations,
Office of the Administrator.
Activity after February 28, 2006
On March 1, 2006, the OCPL's supervisory editor advised the Inspector General
and the AIG-OCPL that everyone had approved the changes to the report. He
provided the latest version to the Inspector General. Chapter 3 and all
recommendations had been deleted from the report.
The OIG had expended 1,126 staff days on Assignment No. 2004-000709.
Combined with Assignment No. 2004-000707, the OIG had expended a total of
1,901 staff days on the congressional request.
The working papers reflected a call between the Acting Deputy IG and the
Assignment Manager on April 12, 2006. A memorandum prepared by the
Assignment manager reflected the following:
The Acting Deputy IG apologized regarding the decision to remove
Chapter 3 of the report. The memorandum explains the decision to
remove chapter 3 had nothing to do with the Assignment
Manager's or the team's work and that the decision was purely a
management decision. The Acting Deputy IG said that he and the
Inspector General and the AIG-OPE (retired), could never agree
on what was to be included in the report.
The Assignment Manager asked the Acting Deputy IG whether
OIG was going to issue the findings in Chapter 3 as the Acting
Deputy IG had indicated in his email. The Acting Deputy IG said
that the OIG team should discuss how to wrap it up and bring the
report back up in the planning process. The Assignment Manager
subsequently discussed this with the Director and Acting AIG-OPE
and she indicated she wanted to keep the two issues the
Assignment Manager thought most critical in Chapter 3 -
community involvement and HHEI - in future planning. The
Assignment Manager told the Acting Deputy IG that she was
52
-------
planning to have a "'post op " evaluation about what went right
and wrong about the evaluation and the Acting Deputy IG
indicated that both he and the new Acting IG would want to attend
the meeting. The "post op " evaluation did not occur, according to
the Assignment Manager due to other assignment priorities.
The working papers for this assignment contained no other information about
senior OIG officials' decisions or reasons to drop Chapter 3.
May 30, 2006: The Agency provided its response to the final report. The
Agency expressed its appreciation of the OIG's consideration of the comments
made in its August 18, 2005, response to the OIG's draft report and also to the
Agency's comments provided to the OIG's proposed final report dated February
16, 2006. The Agency noted the final report reflected many revisions in
accordance with the Agency's comments. The Agency concurred with all
recommendations except for one. The Agency did not concur with the
recommendation to declare its accounting for administrative and supporting
activities, and its lack of a current workload model, as internal control
weaknesses.
Officials Provide Additional Information for this Review
Former Inspector General
On December 9, 2007, the former Inspector General provided a statement for this
quality assurance report concerning the removal of revised Chapter 3 from the
final report. The former Inspector General explained she had expressed to the
team her concern that there was little audit work done to support some of the
issues. She specifically remembered a meeting where she talked with the team
about how little audit work was performed given the time charged to the
assignment. The issues, as she could best recall, pertained to revised Chapter 3
that she received on February 27, 2006. According to the Inspector General, in
one instance, OIG staff stated one person in Denver had said that EPA could quit
funding some of the community involvement. Similarly, the suggestion to quit
funding ATSDR, she remembered, was based on a comment or comments, not on
audit work that HHS or EPA OIG did to see what was accomplished versus what
was spent. The former Inspector General was sensitive to suggestions using the
lowest level of audit evidence and not doing substantive audit work.
In summary, she said she was not comfortable suggesting changes to the
Superfund program when the OIG staff had not done work to support the
suggestions. Her major frustration was that it was embarrassing that the OIG had
spent so much time and had so little to support the suggestions the OIG staff
wanted to make. It made the OIG look like it was able to do little other than write
down the ideas of others and list those ideas in a report. The former Inspector
General also believed the report could have been written months earlier.
53
-------
In supplemental comments received on December 15, the former Inspector
General had no comments regarding the independent referencing of the report.
The former Inspector General explained that the referencer could accept
testimonial evidence as support for comments in the report. In her opinion, the
overall quality of the report and the level of evidence used to support the findings
were the responsibility of the Assignment Manager, the Director, and the AIG-
OPE.
Former Acting Deputy IG
Comments were not received from the Former Acting Deputy IG. Should the
OIG receive comments they will be made part of the working papers for the
assignment.
Comments from the AIG-OCPL
The AIG said that her involvement in the assignment was to manage the
execution of the editing, and coordinating a timely publication process. She had
advocated in her correspondence the importance of keeping Congress and the
media informed of all OIG public reports. She had no involvement in the
selection of December 29, 2005, as a potential issue date for the report and was
not offering any advice in a specific email. She had questioned December 29 as a
release date for the report. Finally, the AIG-OCPL explained that on February 28,
2006, she had raised concerns about rushing the report in light of all of the last
minute changes. Neither she nor anyone else in OCPL to her knowledge had any
involvement in the decision to drop Chapter 3 from the report.
Comments from Director
The Director made several points regarding this assignment. The Director
believed this review involved a challenging and unprecedented Superfund
evaluation by the OIG in response to a February 2004 congressional request.
Also, the Director believed this quality assurance review documents that, despite
the assignment meeting the quality assurance standards in effect at the time, at
least one senior OIG manager directed the removal of findings and
recommendations from an OIG report. Further, the Director noted there were
significant delays associated with getting the final report released to the public.
The Director stated this quality assurance review showed the team's attempts to
develop a report were responsive to the objectives and the team had met
government auditing standards and OIG style preferences. More specifically, the
Director noted that:
The team was directed to do more than what was needed in responding to
Agency comments and working with the Agency to reach agreements. The
54
-------
Inspector General approved a draft report for release to the Agency on
July 11, 2005. The final report was issued on February 28, 2006. The bulk of
the team's efforts in the 7 month period between draft and final reports were
spent dealing with OIG senior management comments on report form and
having multiple meetings with the Agency, or others, on their comments.
The team was directed to delay its report to accommodate Agency comments
and the resolution of disagreements.
The Inspector General, or other senior OIG managers, did not direct the team
to "hurry up," or reduce staff days. As the Director recalled, the only
expressions of concern about the length of time the assignment was taking
were coming from her and the team.
On occasion, the Inspector General withheld her comments from the team,
and instead shared them with report editors.
Senior OIG managers did not take action to publicly acknowledge or remedy
the Agency's lack of compliance with OIG direction regarding release of draft
OIG reports. As documented in this review, on March 25, 2005, the Director
became aware from the Inspector General and the AIG-OCPL that the OIG's
December 2004 draft briefing report had been leaked. However, no public
action was taken to address the Agency's failure to maintain control of OIG
documents. On May 23, 2005, the Director said she and the AIG-OPE were
informed by the OIG team that the Acting Assistant Administrator for
OSWER had publicly criticized the draft OIG work and had expressed an
intention to prevent the OIG's work from being released. The AIG-OPE had
indicated he informed the Inspector General of this, but no public action was
taken to address the Acting Assistant Administrator's statement. On February
21, 2006, just a week before the final report was issued, an OSWER Director
stated during a meeting to discuss disagreements that, "Well, we had to
disagree with some of the OIG's recommendations." The OSWER Director's
statement was not addressed, or further considered, by senior OIG officials,
including the Acting Deputy IG who attended the meeting.
The Director also stated that one of the finding areas deleted from the OIG's
February 26, 2006, report (i.e., Superfund Alternative Sites) has since been
reported on in a June 6, 2007, OIG report, EPA Needs to Take More Action in
Implementing Alternative Approaches to Superfund Cleanups (Report No. 2007-
P-00026). However, other issues deleted from the February 26, 2006, report have
not been publicly reported on by the OIG. A decision to report on those issues
requires some additional work to determine whether the same conditions
identified earlier continue to exist today.
55
-------
Part 3:
Conclusions and Actions Taken by OIG
Conclusions
Overall, the OIG's decision making for both assignments was not transparent as
documented in the working papers. Specifically, the administration of
Assignment No. 2004-000929 did not ensure that independence and transparency
were upheld. External reviewers could point to several questionable actions:
A drifting scope that went from the Acid Rain Program to EJ;
Over 1,000 staff days expended on the assignment without issuing a report;
Contacting an Agency statistician or scientist to review an OIG
methodology without first assessing the statistician's ability to review the
OIG's methodology impartially;
Allowing a former Agency official to supervise an evaluation of a
program he had closely been associated with;
Not documenting the rationale for the decision to curtail the project and
not issue a report; and
Starting another project under the same control number and representing it
as the result of the project.
Likewise, the rationale behind the decision on Assignment No. 2004-000709 to
delete findings and recommendations immediately prior to issuing the final report
was not documented. As with the assignment on EJ, OIG expended a
considerable amount of time on the congressional request. In addition to the 775
staff days to issue an initial briefing report, the OIG expended another 1,125 staff
day to develop additional findings and recommendations in a narrative report.
Much of that additional information was deleted despite awareness by the OIG
that the Agency did not disagree with all of the recommendations. In fact, the
Agency had stated in its yearly Performance Report that it had developed an
implementation plan to address certain recommendations.
The expressed motive for senior OIG management decisions in both assignments
was the desire to maintain or enhance the credibility of the OIG in reporting on
scientific matters and issues important to Congress. Ultimately, the OIG senior
leaders are left with the responsibility of ensuring that a report has sufficient
evidence to justify a disclosure and make recommendations, or withhold a report
that may mislead Agency decision makers, Congress, and the public. However,
without adequate supporting documentation explaining those actions, the actions
56
-------
had the effect of making the decisions appear questionable. Also, the
circumstances made it appear that the OIG did not report all information it
promised on matters important to Congress. OIG senior leaders believed they had
explained their decisions, including the rationale for their decisions, and that
subordinates would document the decisions. While the working papers reflect
their decisions, the working papers do not fully explain the rationale for the
decisions.
The end results of the situations and circumstances disclosed by this review
reflect negatively on the OIG. Two reports, apparently ready to be made public,
were either significantly altered or not issued. Appropriate documentation or
explanation for the actions were not prepared and included in the OIG's working
papers. These situations may have grown out of legitimate questions involving
methodology or quality of work performed and well intentioned decisions of
senior leaders in the organization. The consequence, however unintended, is the
appearance of unprofessional work and a lessened credibility of the OIG.
Actions Taken by OIG
Since these two assignments were conducted, the OIG has strengthened its quality
assurance program and made revisions to its Project Management Handbook. For
example, effective March 31, 2006, the Handbook was revised to require that for
all assignments a separate section be maintained in the working papers of all
communications with the Agency about the assignment and reporting of issues.
Additionally, as noted earlier in this quality assurance report, the Office of
Counsel has also acted to strengthen the monitoring of Agency employees hired
or detailed to the OIG. Also, decisions by senior OIG officials affecting the
conduct of an assignment and reporting of issues and information are required to
be documented.
57
------- |