Office of Inspector General
Audit Report
SUPERFUND
Region 8 Needs to Further Strengthen Its
Superfund Field Sampling Quality Assurance Controls
Report No. E1SKG7-08-5001-8100082
March 25,1998
-------
Inspector General Division Central Audit Division
Conducting the Audit: Denver, Colorado
Region Covered: Region 8
Program Office Involved: Office of Ecosystems Protection
And Remediation
Office of Technical and Management
Services
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
_ : CENTRAL AUDIT DIVISION
V^V J&WgfiXX* B-CH0mCES
*i PBO^ 1445 ROSS AXENJS. SUITE 1:0;
913-551-7878 DALLAS TEXAS 75202-:-;:-
FAX: 913-551-7837 2i--e-:s-eei'
FAX 21--665-6589
999 18Tn STPEE" SU TE 50C
DENVER C 0,0= ADC 80202-2-:5
March 25, 1998 FAX go^lsS
MEMORANDUM
SUBJECT: Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
Audit Report E1SKG7-08-5001-8100082
FROM: Bennie S. Salem &^~it^ v^J-£^~—.
Divisional Inspector General
TO: William P. Yellowtail
Regional Administrator
Region 8
We have completed our followup audit of the U. S. Environmental Protection Agency (EPA)
Region 8's Superfund field sampling quality assurance activities. Attached is our report entitled Region
8 Needs to Further Strengthen Its Superfund Field Sampling Quality- Assurance Controls. We found
that the Region needed to better define and document its data quality objectives, further strengthen
controls over field sampling activities, improve its quality assurance training to its remedial project
managers, and ensure increased oversight and improve documentation of oversight at Superfund sites.
We met with the Region 8 Superfund Remedial Response program manager and supervisor and the
Regional Quality Assurance program manager who generally concurred with our recommendations and
agreed to begin corrective actions. We incorporated their comments as appropriate.
Action Required
In accordance with EPA Order 2750, you, as the action official, are required to provide us a
written response to the audit report within 90 days of the final audit report date. For corrective actions
planned but not completed by the response date, reference to specific milestone dates will assist us in
deciding whether to close this report.
This audit report contains findings that describe issues the Office of Inspector General (OIG)
has identified and corrective actions OIG recommends. This report represents the opinion of OIG, and
the findings contained in this report do not necessarily represent the final EPA position. Final
determinations on matters in this report will be made by EPA managers in accordance with established
EPA audit resolution procedures. In this audit, OIG did not measure the reviewed offices against all
standards established by the National Contingency Plan. The findings contained in the report are not
binding in any enforcement proceeding brought by EPA or the Department of Justice under Section
RECYCLED
-------
107 of the Comprehensive Environment Response, Compensation, and Liability Act to recover costs
incurred not inconsistent with the National Contingency Plan.
We have no objections to the release of this report to any member of the public. This report
contains no confidential business or proprietary information that cannot be released to the public.
Should you or your staff have any questions regarding this report, please contact me at (913)
551-7831 or Jeff Hart, Audit Manager in our Denver office, at 312-6169. Please refer to the report
number on all related correspondence.
Attachment
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
EXECUTIVE SUMMARY
INTRODUCTION
The Office of Inspector General conducted a followup audit
of Region 8's Superfund field sampling quality assurance
activities. In our January 1995 report, Implementing
Controls Would Improve Region 8's Superfund Field
Sampling Activities (E1SKG4-08-0045-5400034), we
found that Region 8 needed to strengthen pre-field sampling
controls, better train its remedial project managers (project
managers), and ensure an appropriate level of oversight at
Superfund sites. Our followup audit disclosed that similar
problems continued to exist.
The U. S. Environmental Protection Agency (EPA) requires
that each region or major organization have a quality
assurance program to ensure that activities generate data of
known and adequate quality. EPA stated in its 1997
strategic plan that it "will continue to implement its
mandatory Quality Assurance Program" to ensure high
quality environmental data.
OBJECTIVES
Our general objective was to review Region 8's improved
procedures and processes for review, approval, and
implementation of field sampling quality assurance activities
in the Superfund Remedial Program. Specifically, our
objectives were to answer the following questions:
* Is the Region properly using data quality
objectives to support decisionmaking?
* Are adequate controls in place to ensure that
an approved sampling and analysis plan is in
place prior to commencing field sampling
activities?
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
Do project; managers possess the skills
necessary, through training, to approve
sampling and analysis plans?
Is Regional oversight of field sampling
activities adequate to ensure compliance with
sampling and analysis plans and reliable and
defensible data?
RESULTS IN BRIEF
A Consistently Applied Data
Quality Objectives
Process Would Provide For
Better Decisions
Region 8 improved some of its field sampling quality
assurance activities since 1995. However, the Region could
more effectively support EPA's goal for high quality
environmental data by better implementing its quality
assurance program. Project managers could better define,
document, and update data quality objectives to support
environmental cleanup decisions. Also, Region 8's
strengthened controls did not fully achieve their intended
purposes to prevent untimely project plan submissions and
undocumented plan approvals. Supervisors had not ensured
that all parties responsible for field sampling activities
attended quality assurance training. Project managers
continued to perform limited oversight over contractors and
did not routinely look to the Region 8 quality assurance
group for assistance in reviewing project plans.
Project managers could have more consistently applied the
data quality objectives process as specified by EPA policy.
Specifically, they should have better defined, documented,
and updated data quality objectives to support their
environmental cleanup decisions. Data quality objectives in
EPA project plans were too broadly defined to ensure
sufficient and appropriate data was gathered, and project
managers accepted deficient contractor-prepared plans.
Without evidence of decisions based on sound data, EPA is
vulnerable to criticism and risks inadequate, inappropriate, or
untimely sampling and cleanup.
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Supeifund
Field Sampling Quality Assurance Controls
Accountability For
Strengthened Controls
Would Help Eliminate
Planning Problems
Training Would Strengthen
Quality Assurance Skills
Further Actions Could Ensure
Appropriate
And Well-Documented
Oversight
Region 8's strengthened controls did not fully achieve their
intended purposes to prevent untimely project plan
submissions and undocumented plan approvals. Some
project managers continued to allow contractors and
potentially responsible parties to submit late plans and
neglected to ensure their plans included signed approval
pages. Without clearly identified and approved plans, the
Region could not ensure it (1) collected the necessary data to
accurately identify pollution sources and assess risks, or (2)
made the most efficient, effective, or scientifically defensible
cleanup decisions.
Region 8 agreed that project managers needed to attend
mandatory quality assurance training to ensure they had the
necessary skills to properly approve project plans. However,
13 (or 38 percent) of the project managers and other staff
needing the training had not completed the two mandatory
Regional courses. Management should have held project
managers accountable for attending training, and the
Regional quality assurance group needed to improve training
effectiveness. Without well-trained project managers,
Region 8 could not ensure project managers had the
necessary skills to properly review and approve project
plans.
Region 8 had inadequate assurance that data collected during
the remedial cleanup phase was reliable and defensible
because project managers had not significantly changed their
oversight practices to ensure compliance with project plans.
Some project managers continued to perform limited
oversight, relied too heavily on oversight contractors, did not
use the Region 8 quality assurance group's assistance, and
had not appropriately documented their oversight activities.
Project managers had no procedures specifically describing
their oversight and site management responsibilities and
authority. Without appropriate and documented oversight,
Region 8 could not ensure data was reliable and defensible.
E1SKG7-08-5001-8100082
in
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
RECOMMENDATIONS
We recommend that the Regional Administrator (RA)
develop Regional controls to ensure that Regional staff
appropriately develop data quality objectives^ document the
process in the project plans, and hold Superfund supervisors
accountable for ensuring that project managers have
actually reviewed and properly approved project plans.
Further, the RA should direct supervisors to identify staff
who have not completed required training and hold staff
accountable for attending. The RA should also direct
supervisors to hold project managers accountable for
following existing Region 8 policy and determine whether
additional oversight direction is needed.
AGENCY COMMENTS
AND OIG EVALUATION
Region 8 generally concurred with our recommendations
and presented an action plan that outlined how the Region
planned to address our recommendations and generally
satisfied the intent of our recommendations. We agree that
Region 8 had improved its field sampling quality assurance
controls since our last report. We also agree that the
Region can further improve its quality assurance efforts in
its Superfund field sampling planning. We reviewed Region
8's specific comments to our draft report and made some
minor modifications to the final report as we determined
necessary.
E1SKG7-08-5001-8100082
IV
-------
Region 8 Needs to Further Strengthen Its Super-fund
Field Sampling Quality Assurance Controls
TABLE OF CONTENTS
Page
EXECUTIVE SUMMARY i
ABBREVIATIONS vii
CHAPTERS
1 INTRODUCTION 1
Purpose 1
Background 1
^ Scope And Methodology 4
Prior Audit Coverage 4
2 A CONSISTENTLY APPLIED DATA QUALITY OBJECTIVES
PROCESS WOULD PROVIDE FOR BETTER DECISIONS 6
EPA Specifies Using The Data Quality Objectives Process 6
Managers Needed to Demonstrate Use of The Data Quality
Objectives Process 8
Project Managers Accepted Deficient Plans 11
Conclusion 11
Recommendations 12
3 ACCOUNTABILITY FOR STRENGTHENED CONTROLS WOULD
HELP ELIMINATE PLANNING PROBLEMS 13
EPA Requires Timely Reviewed and Approved Plans 13
Project Managers Did Not Properly Apply Additional
Controls 14
Accountability Had Not Been Enforced 18
Conclusion 19
Recommendations 19
4 TRAINING WOULD STRENGTHEN QUALITY ASSURANCE SKILLS 21
EPA and Region 8 Require Quality Assurance Training 21
Staff Had Not Completed Required Training 22
Management Did Not Regularly Emphasize Quality Assurance Training 23
Project Managers Did Not Value Quality Assurance Training 24
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
Conclusion 25
Recommendations 26
5 FURTHER ACTIONS COULD ENSURE APPROPRIATE AND
WELL-DOCUMENTED OVERSIGHT 27
Project Managers' Responsibilities Include Oversight 27
Oversight Activities Had Not Significantly Improved 28
Oversight Responsibilities and Authority Were Unclear 30
Project Managers Did Not Value Quality Assurance
Assistance 31
Conclusion 32
Recommendations 33
EXHIBITS
1 THE DATA QUALITY OBJECTIVES PROCESS COMPARED TO
THE SCIENTIFIC METHOD 34
2 SCOPE AND METHODOLOGY 35
3 EPA COMMENTS AND OIG EVALUATION 37
APPENDICES
I EPA RESPONSE 38
H DISTRIBUTION 43
E1SKG7-08-5001-8100082
vi
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
ABBREVIATIONS
EPA U. S. Environmental Protection Agency
OIG Office of Inspector General
Project Manager Remedial Project Manager
E1SKG7-08-5001-8100082
vii
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
CHAPTER 1
INTRODUCTION
PURPOSE
The Office of Inspector General (OIG) conducted a
followup audit of Region 8's Superfund field sampling
quality assurance activities. Our general objective was to
review Region 8's improved procedures and processes for
review, approval, and implementation of field sampling
quality assurance activities in the Superfund Remedial
Program. Specifically, our objectives were to answer the
following questions:
*• Is the Region properly using data quality
objectives to support decisionmaking?
*• Are adequate controls in place to ensure that
an approved sampling and analysis plan is in
place prior to commencing field sampling
activities?
*• Do project managers possess the skills
necessary, through training, to approve
sampling and analysis plans?
* Is Regional oversight of field sampling
activities adequate to ensure compliance with
sampling and analysis plans and reliable,
defensible data?
BACKGROUND
The U. S. Environmental Protection Agency (EPA)
requires that each region or major organization have a
quality assurance program to ensure that activities generate
data of known and adequate quality. EPA stated in its 1997
strategic plan that it "will continue to implement its
mandatory Quality Assurance Program" to ensure high
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
quality environmental data. The quality assurance program
goal is to measure and minimize sources of error and to
monitor conditions of sampling, storage, and transport.
Quality assurance and quality control operate hand-in-hand
to help managers meet EPA's quality assurance goal.
Quality assurance is a management function that should
provide policy and an integrated system of management
controls for planning, implementation, and review of
environmental data collection activities. Quality control is a
system designed to measure the attributes and performance
of an activity against specific standards and verify that they
meet the standards. Quality control should be implemented
at the project level and include such scientific precautions as
calibrations and duplications.
EPA Order 5360.1, Policy and Program Requirements to
Implement the Mandatory Quality Assurance Program,
dated April 1984, applies to all EPA programs including
Superfund. It provides for each EPA organization to
implement the requirements of EPA's quality assurance
program. EPA Order 5360.1 requires data be of known
quality and states that data are of known quality when data
"...are thoroughly documented, such documentation being
verifiable and defensible."
EPA developed a "data quality objectives" process as its
systematic planning tool to help ensure it consistently
collected the quality of data needed to support technical
decisions. The data quality objectives process is based on
the scientific method and thus contributes to EPA's goal of
sound science. EPA requires project plans to include a data
quality objectives section to document the project's defined
goal and planning outputs.
The National Contingency Plan regulates the Superfund
Remedial Program through Part 300 of 40 Code of Federal
Regulations. Sections 300.420 and 300.430 require
sampling and analysis plans that provide a process for
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
obtaining environmental data of sufficient quality and
-quantity to support decisionmaking. EPA must review and
approve sampling and analysis plans at most Superfund
sites. Sampling and analysis plans consist of: (1) a field
sampling plan, and (2) a quality assurance project plan. A
field sampling plan describes the number, type, and location
of samples and the type of analysis required. The quality
assurance project plan describes policy, organization, and
functional activities necessary to develop adequate data.
The data is used for planning and documenting a removal
action, site evaluation, and hazard ranking system activities.
EPA staff often use the terms "sampling and analysis plan,"
"field sampling plan," and "quality assurance project plan"
interchangeably. Throughout this report, we have used the
term "project plan" to refer to all these types of plans except
when we have described a specific example.
EPA's quality assurance program requires each organization
that collects environmental data to implement an approved
quality management plan. Region 8's Quality Management
Plan, dated September 1996, is Region 8's statement of and
commitment to quality assurance activities. It describes
Region 8 quality management policies, objectives,
principles, responsibilities, and accountability.
Region 8's Office of Ecosystems Protection and
Remediation and Region 8's Montana Office are responsible
for Superfund cleanup and oversight. On-scene
coordinators, site assessment managers, and remedial
project managers (project manager) are responsible for
ensuring field sampling integrity at their sites and ultimately
report to the Superfund program managers located in
Denver and Montana.
Region 8's Office of Technical and Management Services
has responsibility for the quality assurance program. The
quality assurance group oversees the quality of data-
generating activities throughout the Region.
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Super-fund
Field Sampling Quality Assurance Controls
SCOPE AND
METHODOLOGY
We performed a followup audit of EPA Region 8's
Superfund field sampling quality assurance activities to
determine whether Region 8 improved its controls over its
field sampling activities as a result of our report,
Implementing Controls Would Improve Region 8's
Superfund Field Sampling Activities, E1SKG4-08-0045-
5400034, dated January 27, 1995. We performed our audit
in accordance with Government Auditing Standards (1994
Revision) issued by the Comptroller General of the United
States. Other than the issues discussed in this report, no
other significant issues came to our attention that warranted
expanding the scope of our review. The scope of our 1995
report and our followup audit work was limited to the
Superfund Remedial Program. See Exhibit 2 for additional
information about our scope and methodology.
PRIOR AUDIT
COVERAGE
The only relevant prior audit coverage was the report cited
above. We have summarized relevant portions of the report
below.
We found that project managers inconsistently approved
project plans and did not denote final, approved plans.
Project managers previously allowed contractors to submit
project plans for review and approval without providing 30
days for EPA review and approval prior to a sampling
event. At the time, Region 8 allowed verbal approvals.
Consequently, project managers had not documented their
review and approvals in project plans. Some project
managers prepared approval letters, but did not attach the
letters to the approved plan or identify the approved plan in
the letter.
Our 1995 report disclosed that project managers recognized
they had not fully complied with mandatory training
requirements within specified timeframes or understood
quality assurance requirements even after having attended
Regional quality assurance courses. Project managers
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
identified weaknesses in training and requested better
presentations to include identifying quality assurance
experts for project managers to contact. None of the
project managers had completed all three mandatory quality
assurance courses required by the Region as of January
1994. Also, the Region needed to better document training
attendance.
We further found that project managers relied on
contractors to conduct sampling with limited oversight, did
not always adequately oversee activities to ensure proper
contractor performance, and did not enhance their own
experience by observing site activities. Project managers
had not used field audits to help them oversee their sites.
In response to our 1995 report, the Region instituted
strengthened controls over its field sampling activities by
distributing two internal policy memorandums. The first
memorandum to all Region 8 office directors instructed
them to use an existing Region 8 sampling brokering system
for all analytical work. Project managers were to attach
approved and signed project plans to their requests for
analytical services. Requests were required 1-6 weeks
before a sampling event. The Region agreed that project
managers and contractors should have followed a 30-day
approval rule and suggested that staff include the
requirement in contractor project planning schedules. The
second memorandum instructed project managers to
approve project plans in writing prior to field sampling and
include the approval pages within the document. The
Region further directed its project managers to take the
mandatory quality assurance training. The memorandum
advised project managers of other related quality assurance
and program training available to enhance their skills. The
memorandum emphasized contractor oversight and
provided suggestions of where to get additional technical
expertise to review project plans, how to use field audits to
fulfill oversight responsibilities, and how to document site
decisions.
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Super-fund
Field Sampling Quality Assurance Controls
CHAPTER 2
A CONSISTENTLY APPLIED DATA QUALITY OBJECTIVES
PROCESS WOULD PROVIDE FOR BETTER DECISIONS
Project managers could have more consistently applied the
data quality objectives process. EPA policy and guidance
specifies using the data quality objectives process to
support site decisionmaking. Project managers should
have better defined, documented, and updated data quality
objectives to support their environmental cleanup
decisions. Region 8's project plans did not provide
evidence that Region 8 used the data quality objectives
process or another systematic planning approach to
support decisions. Project managers stated they routinely
reviewed contractors' project plans, but our review
indicated they routinely accepted deficient plans. Without
evidence of decisions based on sound data, EPA is
vulnerable to criticism by Congress, states, potentially
responsible parties, and the public for site mismanagement
and unnecessary expenditures. Additionally, it risks
inadequate, inappropriate, or untimely sampling and
cleanup.
EPA SPECIFIES USING
THE DATA QUALITY
OBJECTIVES PROCESS
The Deputy Administrator directed in a May 1984
memorandum, Data Quality Objectives, that "Beginning in
September 1984, data quality objectives should be an
integral part of each significant new data collection
activity." The Deputy Administrator further emphasized
that "each National Program Manager is responsible for
establishing data quality acceptance criteria (i.e., data
quality objectives) for all of their projects...."
A subsequent Deputy Administrator continued to focus on
the importance of data quality objectives in a November
1986 memorandum, Agency Institutionalization of Data
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
Quality Objectives. This Deputy Administrator directed
that EPA implement the data quality objectives process.
The November 1986 memorandum stated that the data
quality objectives process was "sound management
planning" and a key element of EPA's quality assurance
program. The memorandum also stated that data quality
objectives are explicit statements of the quality of data
needed to support a decision. The Deputy Administrator
stated that institutionalization of this process would save
money and improve data quality.
General EPA Quality Assurance Division guidance and
specific Superfund program guidance provide detailed
instructions for project managers to implement the data
quality objectives process to support decisionmaking. The
Quality Assurance Division's September 1994 Guidance
for the Data Quality Objectives Process (QA/G-4) details
a seven-step process to help project managers collect
appropriate data. The data quality objectives process is a
systematic planning tool based on the scientific method for
determining the type, quantity, and quality of data that will
be sufficient and appropriate for the data's intended use.
Although project managers are not required to document
each of the seven steps in project plans, the plans must
adequately explain the results of the process. The Office of
Emergency and Remedial Response also specifies using the
process for all Superfund cleanups in its September 1993
Data Quality Objectives Process for Superfund (EPA 540-
R-93-071). Region 8's Quality Management Plan requires
project managers to develop and adequately document data
quality objectives in their project plans that are appropriate
for the site circumstances.
w
Because the data quality objectives process is based on the
scientific method, it "...improves the legal defensibility of
site decisions by providing a complete record of the
decision process and criteria for arriving at conclusions."
The scientific method provides principles and processes
necessary for scientific investigation and a means for
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
independent observers to reach the same conclusion. See
Exhibit 1 for a more detailed description of how the data
quality objectives process builds upon the scientific
method.
Data quality objectives determine quality control
requirements for data collection, sampling, and analysis.
Data quality objectives provide the purpose and focus as
well as the qualitative and quantitative criteria that:
(1) define the most appropriate type of data to collect, and
(2) specify the limits on decisionmaking errors to identify
acceptable risk. The seven steps in the data quality
objectives process are shown in Exhibit 1.
In accordance with the September 1994 data quality
objectives guidance, data quality objectives must be
continually assessed and revised as site circumstances
change. Also, the Quality Assurance Division's August
1994 EPA Requirements for Quality Assurance Project
Plans for Environmental Data Operations (QA/R-5)
requires project managers to review project plans, at least
annually, for long duration projects or multi-year projects.
For example, Superfund sites sometimes take 10 years or
more to clean up. During such a lengthy cleanup process,
much data is collected, many decisions are made, and
public interests may change. In addition, new technology
or new information on health risks may become available.
Finally, site conditions may change as a result of
unanticipated outcomes of other site decisions.
MANAGERS NEEDED TO
DEMONSTRATE USE OF
THE DATA QUALITY
OBJECTIVES PROCESS
Project managers did not use or adequately document the
use of the data quality objectives process in making field
sampling decisions. The data quality objectives for specific
projects did not adequately support decisionmaking
because project managers had not adequately: (1) defined
data quality objectives, (2) appropriately documented data
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
quality objective results in their project plans, and (3)
updated project plans as site circumstances changed.
Although project managers stated they applied the data
quality objectives process, the objectives in their project
plans were too broadly defined to ensure sufficient and
appropriate data was gathered. Also, prior to 1992, the
quality assurance manager approved project plans but had
not identified deficient data quality objectives or ensured
the data quality objectives were well documented. We
examined 36 project plans and determined that the data
quality objectives in 8 plans (22 percent) were adequate,
while 28 (78 percent) were deficient. The eight adequate
plans were all prepared prior to the issuance of the 1994
data quality objectives guidance. These eight plans
appeared to follow the EPA data quality objectives process
and appropriately documented the specific project needs by
explaining the results of the planning process. The 28
deficient project plans included data quality objectives that
were broadly defined and were only task or project
objectives which did not directly contribute towards
specific site decisionmaking. While the majority of the 28
deficient plans were prepared prior to issuance of the 1994
guidance, 8 of the 28 were prepared after guidance
issuance.
As an example of the deficiencies, we noted the data
quality objectives for the California Gulch Superfund site
plan clearly stated a need to conduct a sampling study.
However, the objectives did not identify the point at which
enough information would be collected and a decision
could be made. Appropriate data quality objectives for this
project should include statements of the problem (i.e., the
project/task objectives); specific inputs needed; a decision
statement (i.e., a point at which remedial action should be
taken); risks associated with the decision, if the critical
sample results were too close to the action level; and an
optimal design.
E1SKG7-08-5001-8100082
-------
Region 8 Needs to Further Strengthen Its Super-fund
Field Sampling Quality Assurance Controls
Region 8 quality assurance staff told us that project plans
they reviewed over the last 2 years generally either did not
specify or incorrectly specified limits on acceptable
decision errors. According to the Region 8 quality
assurance manager and the staff statistician, project
managers generally had not adequately identified limits on
decision errors (data quality objective step six). Project
managers had not accurately calculated the total error and
could not have known the quality of the collected data.
While some plans identified an acceptable laboratory error
rate, they did not generally identify an acceptable field
error rate. The plans generally implied that the laboratory
error rate was equal to the total error rate. Therefore, the
plans did not demonstrate that implementation would result
in data of known quality.
Project managers did not update project plans to account
for changing site circumstances. All of the plans should
have contained a discussion of how the data quality
objectives evolved. However, only 1 of the 36 plans was
updated with this information. Plans for sites that
consisted of multiple operable units or phases referenced a
generic quality assurance project plan that was developed
many years before the most current sampling event. For
example, for the Silver Bow Creek/Butte, Montana site, a
1996 remedial design/remedial action sampling event
referred to the overall Site Remedial Investigation/
Feasibility Study Quality Assurance Project Plan dated
February 1992. This practice may have been considered
efficient for multiple remedial investigation/feasibility study
sample events. Using the 1992 project plan may also have
saved time and resources to prepare a new quality
assurance project plan for each event. However, multi-
year plans should be evaluated annually because the old
data quality objectives may no longer be relevant. In this
case, the 1996 sampling event was related to remedial
design/remedial action activities that are generally
significantly different from remedial investigation/feasibility
study objectives.
E1SKG7-08-5001-8100082
10
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
PROJECT MANAGERS
ACCEPTED DEFICIENT
PLANS
Although five of seven project managers stated that they
understood and applied the data quality objectives process,
the project plans we reviewed showed that project managers
did not understand what they were reviewing or chose to
accept plans with deficient data quality objectives sections.
Generally, project managers used contractors to prepare the
project plans we reviewed, including the data quality
objectives section. Project managers stated that they
provided the contractors with EPA guidance and
requirements documents to use when the contractor
prepared the project plans. Six of seven project managers
explained that they looked for data quality objectives
sections in their project plans during their reviews of the
plans. However, although contractors included the required
data quality objectives section in project plans, our review
of project plans showed that project managers had not
identified basic deficiencies in the data quality objectives
sections of the project plans or ensured that any data quality
objectives steps taken were adequately documented in the
project plans.
Some project managers did not understand that the data
quality objectives process could be modified to
accommodate site-specific needs. Some project managers
stated they believed the seven-step data quality objectives
process was unnecessary and inflexible. They explained that
not all seven data quality objective steps needed to be
completed for every data collection activity but did not
understand that not completing all steps was an acceptable
practice, if reasons for omitting steps were documented.
Instead, they viewed the process as inflexible and not
tailored to site-specific needs and phases of cleanup.
CONCLUSION
Region 8 unnecessarily risked inadequate or inappropriate
sampling and cleanup because it did not use or adequately
document application of the data quality objectives process.
Without adequate planning, EPA may waste resources
E1SKG7-08-5001-8100082
11
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
developing sampling plans that are never used, collecting
unnecessary samples, delaying cleanup, or cleaning up sites
beyond what is cost effective. While we believe project
managers should complete all seven steps, the process is
flexible and should be tailored to the significance of the
sampling or other site decision to be made. The Region 8
Quality Management Plan recognizes the process does not
have to be complicated and laborious. Without proper
documentation to support project managers' decisions, EPA
is vulnerable to criticism by Congress and other outside
parties. EPA must be able to clearly demonstrate that it has
developed data of known quality, properly assessed risks,
and that its decisions are based on the best scientific
information available. To accomplish this, project managers
must better define and document data quality objectives to
adequately support their decisions.
RECOMMENDATIONS We recommend the Regional Administrator:
2-1. Develop Regional controls to implement an effective
data quality objectives process. Controls should be
designed to ensure that Regional staff appropriately
develop data quality objectives and adequately
document the process in the project plans.
2-2. Direct project managers to review data quality
objectives periodically throughout the cleanup
process and replace or modify project plans to
account for changing site circumstances.
2-3. Direct project managers to reject contractors'
project plans that do not include an acceptable data
quality objectives section.
2-4. Include in the controls for Recommendation 2-1,
steps to ensure that contractor-prepared project
plans include adequate data quality objectives.
E1SKG7-08-5001-8100082
12
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
CHAPTER 3
ACCOUNTABILITY FOR STRENGTHENED CONTROLS
WOULD HELP ELIMINATE PLANNING PROBLEMS
Region 8's strengthened controls did not fully achieve their
intended purposes to prevent untimely project plan
submissions and undocumented plan approvals. EPA
regulations and current Regional policy require timely and
approved plans. However, some project managers
continued to allow contractors and potentially responsible
parties to submit plans late without consequence, and some
project managers neglected to ensure their plans included
signed approval pages. Senior managers allowed staff to
disregard improved controls developed in response to the
1995 audit without being held accountable for results.
Therefore, Region 8 could not be certain that project
managers had adequate time to review plans or that proper
sampling was done in accordance with approved plans.
Without clearly identified and approved plans, the Region
could not ensure it collected the necessary data to
accurately identify pollution sources and assess risks. Also,
the Region could not ensure it made the most efficient,
effective, or scientifically defensible cleanup decisions.
EPA REQUIRES TIMELY
REVIEWED AND
APPROVED PLANS
EPA regulations and guidance and Region 8 policy require
approved project plans and records that are defensible and
verifiable. EPA Region 8 policy also specifies plans be
timely. The Superfund National Contingency Plan
regulations and Region 8 policy require that EPA review
and approve all project plans. EPA Order 5360.1 requires
written records that are defensible and verifiable. The Order
holds program managers responsible and requires that
information withstand reasonable challenges related to
accuracy or truthfulness and results that can be
substantiated. EPA Requirements for Quality Assurance
E1SKG7-08-5001-8100082
13
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
Project Plans for Environmental Data Operations (QA/R-
5) states that plans must be approved prior to initiating site
work. Regional policy memorandums specified that project
managers should ensure they receive project plans in
advance of sampling activities to allow adequate time for
thorough reviews. The policy memorandums instituted 30
days as a standard.
The Region provides for accountability for fulfilling quality
assurance requirements through performance agreements.
Superfund remedial performance agreements for staff and
supervisors require adherence to quality assurance policies,
procedures, guidelines, and regulations. Specifically,
project managers must comply "...with data quality
assurance policies, procedures, and guidelines to assure that
data collected are of the quality and type needed to support
decisions." Supervisors' job elements require them to
establish, implement, and maintain a quality assurance
program and ensure "...compliance with all quality
assurance related Agency policies, procedures, and
regulations."
PROJECT MANAGERS
DID NOT PROPERLY
APPLY ADDITIONAL
CONTROLS
Although Region 8 strengthened its controls in response to
our 1995 report, the controls did not fully achieve their
intended purposes to prevent untimely submissions and
undocumented plan approvals. Although the Region
implemented a modified sample brokering system to help
eliminate untimely plan submissions, some project managers
did not use the control effectively. Also, project managers
did not always follow Regional policy instituted in 1995
requiring approval signature pages in their plans. Some
project managers neglected to ensure their plans included
signed approval pages.
Six of seven project managers did not use the Region 8
quality assurance group's modified sample brokering system
effectively. The Region implemented a modified sample
E1SKG7-08-5001-8100082
14
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
brokering system in August 1995 that required the sample
broker to ensure a project manager had an approved project
plan attached to the project manager's request for analytical
services. Although the sample brokering system's primary
purpose was to identify and coordinate Region 8's internal
analytical service requests, the system also provided a
control to help eliminate untimely project plan submissions.
According to the quality assurance staff, the premise was to
have project managers better plan all phases of their
sampling activities. By working backwards, project
managers could determine when they needed their
contractors, other federal agencies, or states to submit the
project plans for EPA's review. Project managers knew
that Regional policy required them to request analytical
services approximately 1-6 weeks prior to sampling and that
approved project plans had to be attached to their request
for analytical services. With this knowledge, project
managers could better project timelines.
However, project managers circumvented this control by
not appropriately using the sample brokering system either
to obtain needed analytical services or provide notification
of external analysis. The project managers allowed their
contractors, other federal agencies, and states to request
sampling analysis outside the Regional sample brokering
system. Project managers should have provided the sample
broker copies of approved project plans and notified the
sample broker of internal and external laboratory analysis.
Five of seven project managers explained that they use a
tiered approach when managing their contractors and based
their concerns on their level of trust with their contractors.
If a project manager had used a contractor for several years
and was comfortable with the contractor's expertise, the
project manager would not require the contractor to comply
with the 30-day approval rule. However, as emphasized in
our 1995 report, project managers increased the risk of
inappropriate sampling because contractors may realize that
a final draft plan submitted just prior to a sampling event
E1SKG7-08-5001-8100082
15
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
may not receive a thorough review due to the late
submission.
Four of seven project managers stated they did not
unconditionally require their contractors and potentially
responsible parties to submit plans 30 days in advance.
Project managers said they were not concerned about
having 30 days to review plans because they were
comfortable with the contractors' and potentially
responsible parties' levels of experience and expertise. They
explained they held discussions throughout the project and
believed all parties were in agreement and the project plans
would contain the needed information. Also, project
managers stated they were not concerned that site activities
had significantly changed to warrant a 30-day review
period.
However, the Region wasted money and time on at least
one inappropriate sampling plan that may have been
prevented had the Region required the contractor to follow
the 30-day requirement. The project manager canceled a
sampling event and rejected a contractor-prepared project
plan for the Summitville, Colorado Superfund site because
the plan was inadequate. However, the project manager
had not received the project plan until the day before the
sampling event and did not review the plan until that time.
Although the contractor never implemented the plan, the
Region was still obligated to pay for it as a deliverable
product. While the project manager prevented the
contractor from implementing the inadequate sampling plan,
Region 8 paid about $28,000 for this plan that it could not
use. In addition, Regional staff prepared a "last-minute"
plan for the sampling event. The $28,000 did not include
Regional staff costs.
Some project managers had not diligently implemented the
Region's approval policy and continued to produce plans
that did not demonstrate adequate review and did not have
approval signature pages included within the plan. We
E1SKG7-08-5001-8100082
16
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
reviewed 8 of the 36 project plans in our sample that were
developed and implemented after our 1995 report.
Although we reported the lack of approval signatures in
1995, we found that 5 of the 8 project plans (63 percent)
still were not signed by the project managers and only 1 of
the 8 project plans received a quality assurance review. Of
the 5 unsigned plans, 4 (80 percent) did not include
approval signature pages within the plan, while 1 (20
percent) had a signature page that was not completed. The
Region researched its files and found 2 of the 4 missing
approvals after we issued our official draft report.
However, neither of the 2 approvals were included in the
project plans as required by Regional policy. In addition,
the Region approved 1 of the 4 project plans' missing
approvals based on our draft report findings.
Project managers had not ensured that all plans met the
requirement for approval signature pages. Although only
three of seven project managers stated that they had seen
the Region's 1994 policy memorandum requiring signature
pages, all seven explained that they knew of the approval
requirement. However, we found that they did not
consistently review their contractor-prepared project plans
to ensure a signature page was included. Without signature
pages, we could not easily confirm that plans had been
reviewed or approved by the project manager. Also, as we
reported in 1995, because there is often more than one
version of a particular plan, project managers or others
would find it difficult to determine which version was
approved for contractor use for sampling without approval
documentation having been included in the plan.
E1SKG7-08-5001-8100082
17
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
ACCOUNTABILITY HAD Project managers disregarded supervisory suggestions to
NOT BEEN ENFORCED incorporate the 30-day review period into project planning
schedules and as contract deliverable language for new
contractor work. Project managers continued to allow
contractors and potentially responsible parties to submit
untimely project plans to project managers for review,
comment, and approval without consequence. Project
managers did not include language in contracts specifying
the 30-day advance plan submission. Although project
managers had penalty provisions available in consent
decrees and other contractual documents, some project
managers had not used these authorities to ensure timely
submissions. As a result, project managers did always not
hold contractors or potentially responsible parties
accountable.
Four of seven project managers ignored the 30-day approval
requirement because they did not believe the requirement
was always necessary. Project managers explained that
because they had seen previous iterations of the project
plans, the final draft was usually not a surprise. They also
stated that sometimes the Region requested a contractor to
prepare a project plan in less than 30 days to meet urgent
requirements at a site. They stated that neither project
managers nor contractors should be held accountable for
meeting the 30-day requirement in such cases. However,
supervisors had not confirmed that project managers met
project plan review and approval requirements.
Although Superfund remedial office supervisors' and staffs
performance agreements required quality assurance
compliance, supervisors did not ensure staff complied with
all Regional requirements and staff did not meet all quality
assurance requirements. Remedial office supervisors stated
they empowered their staff to perform their duties as "they
see fit" and explained that project managers were
professional staff tasked with many competing priorities.
Supervisors had notified their staff of the Region's policy
requiring documented approvals within project plans and
E1SKG7-08-5001-8100082
18
-------
Region 8 Needs to Further Strengthen Its Super-fund
Field Sampling Quality Assurance Controls
said they believed staff were complying. One supervisor
said that unless he heard of a major problem, he did not
believe there was a problem. As a result, unless notified of
problems, supervisors did not consciously evaluate project
managers' approval actions to ensure they were complying
with the quality assurance requirements.
CONCLUSION
Region 8's strengthened controls did not fully achieve their
intended purposes to prevent untimely project plan
submissions and undocumented plan approvals. The Region
issued a revised policy but did not ensure that the policy
was implemented. As a result, little actually changed since
our 1995 review. Without the required approval signature
page, the Region could not be certain that the correct plan
version was used and that sampling was done in accordance
with the most current, approved plan. As a result, the
Region could not ensure it collected all the necessary data
to accurately identify pollution sources and assess risks.
Also, the Region could not ensure it made the most
efficient, effective, or scientifically defensible sampling and
cleanup decisions. The Region needs to ensure that project
managers comply with the revised Regional policy for
timely submissions and written approval. Top'management
needs to hold staff accountable for properly approving
plans. Although we agree that staff should be empowered
to perform their duties without undue supervisor scrutiny,
supervisors must ensure that staff meet EPA and Regional
policy requirements. Top management also needs to hold
contractors and potentially responsible parties accountable
for submitting timely plans.
RECOMMENDATIONS
We recommend the Regional Administrator:
3-1. Hold Superfund supervisors accountable for
ensuring that project managers timely and properly
E1SKG7-08-5001-8100082
19
-------
Region 8 Needs to Further Strengthen Its Super-fund
Field Sampling Quality Assurance Controls
review and approve project plans prior to sampling.
3-2. Implement existing penalty policies regarding late
deliverables when a pattern of delay becomes
evident.
3-3. Hold Superfund supervisors accountable for
accurately evaluating project managers' quality
assurance compliance in their performance
appraisals.
E1SKG7-08-S001-8100082
20
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
CHAPTER 4
TRAINING WOULD STRENGTHEN
QUALITY ASSURANCE SKILLS
Region 8 agreed that project managers needed to attend
mandatory quality assurance training to ensure they had the
necessary skills to properly approve project plans.
However, 13 (or 38 percent) of the project managers and
other staff needing the training had not completed the two
mandatory Regional quality assurance training courses.
Regional policy prescribes two mandatory quality assurance
courses for project managers to ensure they gather sound
data. The Superfund program managers had not regularly
emphasized the importance of quality assurance training or
ensured project managers attended. Some project managers
believed the training was ineffective and therefore a low
priority. Management should have held project managers
accountable for attending training, and the Regional quality
assurance group needed to improve training effectiveness.
Without well-trained project managers, Region 8 could not
ensure project managers had the necessary skills to properly
review and approve project plans.
EPA AND REGION 8
REQUIRE QUALITY
ASSURANCE TRAINING
EPA and Region 8 policies require certain staff to take
mandatory quality assurance training. EPA Order 5360.1
and Region 8's 1996 Quality Management Plan require all
individuals (staff and their supervisors) directly involved in
data collection activities to have a working understanding of
quality assurance and quality control activities. Specifically,
Region 8's Quality Management Plan requires training for
all project managers with environmental data operation
responsibilities including planning, design, collection, or
assessment activities. The Quality Management Plan further
suggests that project managers' immediate supervisors I
attend the training. Region 8's training is provided by the
quality assurance group.
E1SKG7-08-5001-8100082
21
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
Regional policy prescribes two mandatory quality assurance
courses for project managers to ensure they gather sound
data. The mandatory courses include a 2-day, 12-hour EPA
course, entitled Regional Quality Assurance Planning
Requirements; and a 1-day, 6-hour EPA course, entitled
Data Quality Objectives. In addition, individuals are
required to take a refresher course once every 2 years. The
Region 8 Quality Management Plan states that these
training requirements were effective on January 19, 1996.
Affected staff had until September 30, 1996, to complete
the required quality assurance training or lose their quality
assurance plan approval authority.
Project managers must take mandatory training for three
reasons: (1) to comply with EPA and Regional policy, (2) to
maintain their project plan approval authority, and (3) to
enhance their skills to serve as project managers.
STAFF HAD NOT
COMPLETED
REQUIRED TRAINING
Although Region 8 Superfund supervisors notified and
encouraged staff to take mandatory quality assurance
training, 13 Region 8 Superfund remedial staff had not
completed the required training. Thirty-five remedial staff
(nine in the Montana office) were subject to the mandatory
quality assurance training including Superfund remedial
program managers, supervisors, project managers, and
Senior Environmental Employees. However, 13 of the 35
staff (38 percent) had not completed the mandatory quality
assurance training including the 2 Superfund remedial
program managers, 9 project managers, and 2 Senior
Environmental Employees. Seven of the 13 (54 percent),
including the 2 program managers, had not taken any
quality assurance training.
Although the Regional Quality Management Plan does not
require that the two Superfund remedial program managers
take quality assurance training, Region 8 agreed that
"...some familiarity with QA [quality assurance]
E1SKG7-08-5001-8100082
22
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
requirements is important for managers...." Having the two
Superfund program managers attend quality assurance
training would enhance their ability to oversee the
Superfund program and send an important message to staff
that quality assurance is essential and valuable in meeting
program goals.
Nine project managers who had not completed training by
September 30, 1996, were out of compliance with Regional
policy and should have had their project plan approval
authority revoked after that time. Without proper training,
these nine project managers could not ensure they possessed
the skills necessary to appropriately approve project plans.
However, at least two of the project managers continued to
approve plans. Region 8 should have revoked the signature
authority for the nine project managers until the project
managers had completed training. At a minimum,
supervisors should have monitored these project managers'
activities. One supervisor was aware that one project
manager had not completed training and assigned the
"untrained" project manager to work with a project
manager that had completed all training requirements. The
"untrained" project manager completed training in February
1997 and had not approved project plans between
September 30, 1996, and February 1997. However, the
remaining project managers had not been individually
monitored. Without proper training, project managers
could not ensure they possessed the skills necessary to
approve project plans and, therefore, had not fulfilled their
responsibilities as project managers.
MANAGEMENT DID
NOT REGULARLY
EMPHASIZE QUALITY
ASSURANCE TRAINING
According to quality assurance staff, Region 8's two
Superfund program managers had not regularly emphasized
the importance of quality assurance training to staff by'
identifying those staff needing training and had not ensured
all staff attended the training. To fulfill Regional quality
assurance training requirements, the quality assurance
E1SKG7-08-5001-8100082
23
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
program manager must first identify staff requiring training.
In a December 9, 1996, memorandum to all Region 8
"Program Directors," the quality assurance program
manager requested a list of staff requiring quality assurance
training to be submitted by January 15, 1997.
Although one of the two Superfund program managers had
responded in a previous year to a similar request, neither
responded to the December 1996 request that would have
identified 1997 training needs. Quality assurance staff said
their perception, based on the Superfund program
managers' unresponsive actions, was that Superfund
program managers did not consider quality assurance
important. Quality assurance staff were concerned that
project managers had reached the same conclusion.
PROJECT MANAGERS
DID NOT VALUE
QUALITY ASSURANCE
TRAINING
Project managers wanted better quality assurance training,
but did not value the current Regional training offered
because they believed the training was ineffective and,
therefore, a low priority. Most of the project managers we
interviewed expressed dissatisfaction with the current
training provided by the Region 8 quality assurance group.
In general, training course evaluations suggested:
(1) providing more coverage of where to get more
information, (2) relating quality assurance to the program
and not the program to quality assurance, (3) providing
laboratory tours to enhance quality assurance
understanding, and (4) discussing more "horror stories" to
make a point of quality assurance importance.
Some project managers said the quality assurance training
was too general and needed to be more program specific.
They explained that some examples used in training did not
accurately depict the problems they often encountered at
their sites because their sites were at different cleanup
phases than the examples used in training. Also, project
managers criticized that presentations were often read from
E1SKG7-08-5001-8100082
24
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
overheads and not presented in a manner that would have
enhanced an inexperienced project manager's knowledge.
Some of the Regional quality assurance training, including
the data quality objectives course, did not effectively convey
the underlying quality assurance concepts that would help
project managers better oversee their sites. Project
managers stated that Region 8 quality assurance group
trainers had not linked quality assurance to their program
areas in a manner that enhanced their quality assurance
understanding. The Regional quality assurance program
manager acknowledged that the group was trying to
improve its training presentations. He stated that he had
met with staff to discuss alternative presentation approaches
that would provide clearer and more relevant information to
program staff. He had also met with other Regional
program managers to determine specific staff needs and
planned to meet with other managers to assess training
needs.
CONCLUSION Thirteen (38 percent) of Region 8's Superfund remedial
program staff had not completed mandatory quality
assurance training, 9 of which were project managers with
the responsibility to review and approve project plans.
Without well trained project managers, Region 8 could not
ensure project managers had the necessary skills to properly
review and approve project plans. We believe a direct
correlation exists between senior management emphasizing
the importance of quality assurance and project managers
attending training. When the two Superfund program
managers did not respond to the quality assurance training
request, the message delivered to the quality assurance
group and, possibly to project managers, was that quality
assurance is unimportant. The Region needed to ensure
attendance policies are adhered to by all affected staff and
that training is effective.
E1SKG7-08-5001-8100082
25
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
RECOMMENDATIONS We recommend the Regional Administrator:
4-1. Direct the Superfund remedial program managers to
identify staff who have not completed required
training and hold staff accountable for attending.
4-2. Involve program and project managers in the design
and timing of quality assurance training.
4-3. Direct the quality assurance staff to routinely solicit
customer comments from program staff and evaluate
comments to improve services.
4-4. Direct Superfund program staff to promote quality
assurance through initiatives such as inviting quality
assurance staff to attend Superfund program staff
meetings to explain the benefits of quality assurance
to the Superfund program.
4-5. Obtain training for the quality assurance staff in
presentation techniques to help quality assurance
staff better articulate how quality assurance adds
value.
E1SKG7-08-5001-8100082
26
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
CHAPTER 5
FURTHER ACTIONS COULD ENSURE APPROPRIATE
AND WELL-DOCUMENTED OVERSIGHT
Region 8 had inadequate assurance that data collected
during the remedial cleanup phase was reliable and
defensible because project managers had not significantly
changed their oversight practices to ensure compliance with
project plans. Regional policy, project managers' position
descriptions, and performance agreements outline oversight
responsibilities and good management practices. However,
some project managers continued to perform limited
oversight, relied too heavily on oversight contractors, and
did not use the Region 8 quality assurance group's
assistance. Also, project managers had not appropriately
documented their oversight activities. Project managers had
no procedures specifically describing their oversight and site
management responsibilities and authority. Project
managers needed more specific direction and clearer
authority. When project managers rely too heavily on
contractors and do not observe contractor activities, they
cannot ensure contractors perform as intended, are fully
knowledgeable of site activities, or enhance their own
experience. Without appropriate and documented
oversight, Region 8 could not ensure data was reliable and
defensible.
PROJECT MANAGERS'
RESPONSIBILITIES
INCLUDE OVERSIGHT
Regional policy outlines project manager oversight
responsibilities and good management practices. As a result
of our prior review, the Region issued a 1995 policy
memorandum that provided some general suggestions to
project managers on contractor oversight and identified
other technical personnel available to assist them. The
policy memorandum emphasized that oversight of
contractors was one of project managers' most important
responsibilities and that project managers should use
available Regional expertise to review workplans. The
E1SKG7-08-5001-8100082
27
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
memorandum also instructed project managers to evaluate
contractor performance, such as through field audits. The
policy memorandum also directed project managers to
document activities in "records of communication" as a
good management practice and to periodically observe field
sampling activities.
Project manager oversight responsibilities had not changed
since our 1995 review and are addressed in generic position
descriptions and performance agreements. Project manager
position descriptions at General Schedule levels 9-13 still
included several statements applicable to oversight
activities, including: (1) ensuring technical soundness of
remedial efforts accomplished by contractors, states, and
potentially responsible parties; and (2) directing, overseeing,
managing, and evaluating contractor actions. Generic
performance agreements specifically required project
managers to oversee compliance with orders and decrees
and take appropriate action in cases of noncompliance;
monitor contractor performance through invoices and
supporting documentation; maintain appropriate records;
communicate clearly, concisely, timely, and responsively
with colleagues and managers; and prepare written work
and presentations that are clear, relevant, concise, well-
organized, and appropriate to the audience.
OVERSIGHT
ACTIVITIES HAD NOT
SIGNIFICANTLY
IMPROVED
Although we criticized Region 8 project oversight in our
1995 review, project managers did not significantly revise
their oversight activities to ensure contractors complied
with project plans and site history files supported decisions
and justified resources. Project managers continued to rely
too heavily on oversight contractors and six of seven had
not appropriately documented their oversight activities.
Project managers did not periodically observe contractor
field activities and, therefore, could not independently
confirm contractors complied with project plans. Also,
E1SKG7-08-5001-8100082
28
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
some project managers did not maintain adequate site
history files that would support site management decisions.
Some project managers did not revise their oversight
activities. Three of seven project managers continued to
rely on their field sampling and oversight contractors to
conduct sampling and provide oversight of other
contractors and did not personally observe field sampling
activities as directed in the Regional policy memorandum.
Although project managers received contractor- or state-
prepared progress reports to help them monitor site
activities, they did not regularly schedule site visits to
observe contractor performance or sampling activities. As
such, project managers could not ensure contractors
performed as expected in workplans, and project managers
did not broaden their experience and knowledge about their
sites' sampling activities. As we reported in 1995,
inadequate oversight increased the risk that contractors
might not be performing in accordance with statements of
work and workplans. Also, contractors could have
circumvented sampling requirements since project managers
did not periodically observe contractors' field activities.
Six of seven project managers had not appropriately
documented their oversight actions and maintained a
complete site history file to support field sampling and other
decisions, resources expended, and site transitions between
project managers or between EPA and states. One project
manager voiced concern about assuming responsibility for a
site that had an incomplete site history file because a
previous project manager did not prepare memorandums-of-
record to document what had transpired at the site. The
project manager explained that file documentation should
provide evidence that supports: (1) what actions occurred,
(2) what agreements were reached, (3) how funds were
spent, and (4) how resources were used when transitioning
a site from one project manager to another or when
transitioning a site from EPA to the state. Without
documentation that supported oversight actions, newly
assigned project managers were less efficient and effective.
E1SKG7-08-5001-8100082
29
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
OVERSIGHT
RESPONSIBILITIES AND
AUTHORITY WERE
UNCLEAR
Project managers had no procedures clearly describing their
oversight and site management responsibilities and
authority. The Regional policy memorandum was too
general, and did not provide specific oversight procedures
that directed project managers' actions and site management
activities. Because some project managers did not believe
they had specific procedures, they stated they performed
their project manager responsibilities in accordance with
other directives, such as consent decrees. They explained
that consent decrees provide specific requirements or
actions, such as defining documentation requirements and
stipulating timelines for construction activities. Project
managers explained that they wanted to maintain their
authority to apply their professional judgment when making
site decisions but would welcome better direction. They
explained that part of their reliance on contractors resulted
from competing priorities and insufficient time to complete
all their work. They also explained that they knew they had
authorities to enforce contractor compliance with workplans
but were unsure whether they would get management
support if they tried to take action against a contractor for
poor performance.
Some project managers explained that they used a tiered
approach when managing contractors. They stated that if
they had extensively used a contractor and had established
confidence in the contractor's abilities, the project managers
would allow the contractor more discretion and not exercise
as much oversight as would be necessary if the contractor
was inexperienced or new to the site.
Program supervisors did not ensure staff properly executed
oversight responsibilities. One supervisor stated that he
allowed his staff to use their professional judgment as to site
visits and what information needed to be documented. He
believed his staff was fulfilling his expectations. Another
supervisor explained that he counseled staff about
documentation requirements but did not believe in micro-
managing staff. Both supervisors believed their staff was
appropriately performing oversight.
E1SKG7-08-5001-8100082
30
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
PROJECT MANAGERS
DID NOT VALUE
QUALITY ASSURANCE
ASSISTANCE
Project managers did not value the quality assurance
group's assistance and believed that the quality assurance
group was unresponsive to their needs because the group's
comments on project plans were not always constructive.
Also, they believed the quality assurance group's
implementation of quality assurance rules was often too
rigid. They stated the group needed to be more customer-
oriented. As a result, project managers did not routinely
solicit the group's input on project plans or consistently use
the Region's sample brokering system. The Regional
quality assurance program manager acknowledged that the
quality assurance group had a negative image.
Some project managers did not consistently use the quality
assurance group to assist them during their reviews because
they did not consider quality assurance comments to be
constructive or value-added. Project managers stated that
some quality assurance staff did not understand project
managers' needs. Project managers also said they believed
quality assurance staff required them to include information
in plans that was not relevant to the particular site.
However, some project managers provided very positive
feedback about one quality assurance staff member who had
a strong project manager background. On the strength of
her background, this quality assurance staff member
understood project managers' needs and provided timely,
effective feedback. For example, she explained "line by
line" any revisions to project plans that she believed were
necessary. According to one project manager, when this
staff member left the quality assurance group, subsequent
quality assurance comments on project managers' plans did
not provide the same specificity. Subsequent quality
assurance comments were sometimes generic and required
certain formats be followed before the quality assurance
group would agree with the project plan contents. Asa
result, project managers did not readily accept subsequent
quality assurance comments and believed quality assurance
requirements were too rigid.
E1SKG7-08-5001-8100082
31
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
Project managers explained, and the quality assurance group
agreed, that the sample brokering system was sometimes
rigid and not user-friendly. For example, one project
manager experienced problems scheduling analytical
services. Regional policy required written requests for
analytical services. However, the sample broker verbally
authorized the project manager's request, but the Region 8
laboratory had not received the authorization and would not
analyze her samples without a written authorization. The
project manager had to obtain funding to use a commercial
laboratory because the Region 8 laboratory would not
analyze the samples on such short notice. The perception
left with the project manager was that the requirement for a
written authorization was too rigid. The project manager
stated that other project managers also resist having to use a
rigid sample brokering system or being told they must use
this system. The quality assurance group acknowledged
some scheduling problems and was aware that it needed to
be more flexible. The sample broker and the quality
assurance program manager have met with project
managers to discuss how to improve the process.
The Regional quality assurance program manager
recognized that the quality assurance group's image was
negative. He explained that the group had been working
closer with program offices to change project managers'
perceptions and help meet their program goals. Because the
quality assurance group can be instrumental in helping
project managers effectively perform their jobs, the quality
assurance group must ensure its staff effectively
communicate to promote quality assurance practices.
CONCLUSION Regional project managers did not properly implement
Region 8's controls or improve their oversight activities.
Project managers had not significantly changed their
oversight practices to ensure compliance with project plans.
When project managers rely too heavily on contractors and
E1SKG7-08-5001-8100082
32
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
do not observe contractor activities, they cannot ensure
contractors performed as intended, be fully knowledgeable
of site activities, or enhance their own experience. Without
appropriate and documented oversight, Region 8 cannot
ensure data is reliable and defensible. Controls may need to
be reexamined to provide for improved contractor oversight
and more responsible project manager actions.
RECOMMENDATIONS We recommend the Regional Administrator:
5-1. Hold project managers accountable for following
existing Region 8 policy and determine whether
additional oversight direction is needed.
5-2. Schedule and budget for a minimum number of site
visits to observe contractors' field sampling
activities and oversight of other contractors.
5-3. Specify the type of site file documentation (e.g.,
agreements reached) required in site files to fully
document field sampling decisions.
5-4. Assure staff that management will support staff
decisions to take appropriate action if contractors
perform poorly.
5-5. Determine whether project managers should be
required to obtain Regional quality assurance
assistance before approving a project plan.
5-6. Provide sample brokering standard operating
procedures to project managers.
E1SKG7-08-5001-8100082
33
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Oualitv Assurance Controls
EXHIBIT 1
THE DATA QUALITY OBJECTIVES PROCESS
COMPARED TO THE SCIENTIFIC METHOD
The scientific method and the data quality objectives process provide tools to plan activities.
EPA's quality system is based on the scientific method which consists of three stages: planning,
implementation, and assessment. The 7-step data quality objectives process is comparable to the
planning stage of the scientific method (steps 1-4). We compared the planning stage of the
scientific method to the data quality objectives process. The following tables correlate the steps
of the scientific method with those of the data quality objectives process.
SCIENTIFIC METHOD
1. DEFINE A PROBLEM.
2. GATHER PERTINENT
HISTORICAL/BACKGROUND
INFORMATION.
3. FORM A WORKING HYPOTHESIS OR
EXPLANATION.
4. DO EXPERIMENTS TO TEST THE
HYPOTHESIS.
5. INTERPRET THE RESULTS.
6. DRAW A CONCLUSION AND
MODIFY THE HYPOTHESIS.
DATA QUALITY OBJECTIVES PROCESS
1. STATE THE PROBLEM.
2. IDENTIFY THE DECISION.
3. IDENTIFY INPUTS TO THE DECISION.
4. DEFINE THE STUDY BOUNDARIES.
5. DEVELOP A DECISION RULE.
6. SPECIFY LIMITS ON DECISION
ERRORS.
7. OPTIMIZE THE DESIGN FOR
OBTAINING DATA.
Scientific
Method
Data Quality
Objectives
STEPS
1
1,2
2
3
3
5,6
4
*4,7
5
—
6
—
* Data quality objectives steps 4 and 7 should be continually assessed throughout a project.
E1SKG7-08-5001-8100082
34
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
EXHIBIT 2
SCOPE AND METHODOLOGY
We conducted our fieldwork from January 1997 through November 1997 in accordance with
Government Auditing Standards (1994 Revision) issued by the Comptroller General of the United
States. We designed our followup audit to gather information that would determine whether the
Region's corrective actions improved its field sampling quality assurance activities after issuance
of our January 1995 report.
We judgmentally selected 10 (approximately 25 percent) Region 8 Superfund sites from EPA
Region 8's National Priority JJst. Region 8's list included 5 states with a total of 38 general
Superfund sites and 9 federal facilities sites. We did not include the federal facilities sites within
our universe. We judgmentally selected two sites from each of the five states except South
Dakota and Colorado. South Dakota only listed two sites of which one was a federal facility site.
In place of the South Dakota site, we selected a third site from Colorado, as Colorado had the
majority of sites listed. We judgmentally selected the sites based on size, money spent, and time
listed on the National Priority List. We judgmentally selected 36 project plans for these sites that
included sampling and analysis plans, quality assurance project plans, and field sampling plans.
We selected the plans based on site index sheets provided by the Superfund Records Center. We
evaluated the project plans to determine whether Region 8 adequately implemented and
documented its data quality objectives process.
We reviewed Regional policy memorandums issued in response to our 1995 report and examined
the same 36 project plans to determine whether project managers had complied with the Region's
additional controls. Because we were interested hi Regional improvements after our 1995 report,
we narrowed those plans to 8 that the Region developed and implemented after our 1995 report.
The additional Regional controls included having signed signature pages within the project plans
and requiring all analytical service requests to go through the Region's sample brokering system.
Also, we interviewed 7 project managers that managed 8 of the 10 sites hi our review and 2
Superfund remedial program supervisors about their awareness of the additional controls and
staffs' implementation of the Region's revised policies.
We interviewed the same seven project managers and supervisors about their mandatory quality
assurance training attendance and understanding of quality assurance concepts, especially related
to the data quality objectives process. We relied on training records from the regional quality
assurance program manager, the remedial program training coordinator, and the Montana training
coordinator to determine whether the appropriate project managers and other remedial staff had
attended training. Region 8 changed its training requirements from its 1992 to 1996 Quality
E1SKG7-08-5001-8100082
35
-------
Management Plans. We used the Region's 1996 Quality Management Plan to assess staffs
compliance with training requirements.
We reviewed Regional policy memorandums issued in response to our 1995 report. We
interviewed the same seven project managers about their oversight responsibilities, available
guidance, and implementation of revised Regional policy memorandums to determine whether
project managers had changed their past oversight practices in accordance with the revised
Region 8 policy.
E1SKG7-08-5001-8100082
36
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
EXHIBIT 3
EPA COMMENTS AND OIG EVALUATION
Region 8 generally concurred with our recommendations and presented an action plan that
outlined how the Region planned to address our recommendations. The action plan generally
satisfied the intent of our recommendations. We agree that Region 8 had improved its field
sampling quality assurance controls since our last report. We also agree that the Region can
further improve its quality assurance efforts in its Superfund field sampling planning. We
reviewed Region 8's specific comments to our draft report and made some minor modifications to
the final report as we determined necessary.
Region 8 made some comments for which we made no changes to the report. We did not revise
our final report to include Region 8 comments about the basin-wide quality assurance project plan
because we still conclude that the site-specific sampling and analysis plan needed specific data
quality objectives. We also did not change our position on the sample brokering system. We
agree that the Region cannot direct contractors to select a specific subcontractor for laboratory
analysis. However, the Region may direct contractors to use the Region 8 laboratory for analysis
which would require use of the sample brokering system. Furthermore, Regional policy requires
project managers to notify the sample broker before any sampling is done and requires project
managers to provide the sample broker with an approved sampling plan regardless of who is
doing the laboratory analysis. We also did not revise our discussion of the untimely Summitville
sampling and analysis plan because we believe it exemplifies the issue of timely plan review. We
also did not change our position on inadequate oversight because inappropriate levels of oversight
have persisted since our last audit.
See Appendix I for the full Agency response.
E1SKG7-08-5001-8100082
37
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
REGION VIII
999 18th STREET - SUITE 500
DENVER, COLORADO 80202-2466
APPENDIX I
Page 1 of 5
FEB 1 3 1998
Ref: 8TMS-G
MEMORANDUM
SUBJECT:
FROM:
TO:
Comments on Draft Report, Region 8 Needs to Further
Strengthen Its Superfund Field Sampling Quality
ssurance Controls,Audit Report E1SKG7-08-5001-XXXXXXX
elldtotai
Regional Administrator
1
Bennie S. Salem
Divisional Inspector General For Audit
Central Audit Division
As requested in the December 23, 1997, cover memorandum to
the above-referenced report, we are providing comments on the
findings of the report, a concurrence or conditional concurrence
with each of the recommendations presented, and an action plan
that addresses each of the recommendations.
We agree that further improvements can be made to quality
assurance efforts in our Superfund field sampling planning.
However, we also believe that significant improvements have been
made since the previous Office of Inspector General (OIG) report
issued in January 1995 and that project managers are implementing
the program in a sound and responsible manner. We believe the
draft report unfavorably describes our quality assurance program,
drawing overall conclusions based on very narrow instances where
we have fallen short. We suggest that more recognition be made
of the actions taken in response to the previous report. We do
appreciate the suggestions made by the OIG on how improvements
could be made in our current processes and program controls.
Despite our concerns with the overall project approach and
the manner in which the findings are presented, we are planning
to address the recommendations made, with the conditions listed
below. An action plan is attached that provides information on
how we plan to address those recommendations.
38
Printed on Recycled Paper
-------
APPENDIX I
Page 2 of 5
Our response to the report recommendations is as follows:
2-1 Concur, provided that the last sentence of the recommendation
is deleted. See Action Plan. ,(
2-2 Concur. See Action Plan.
2-3 Concur. See Action Plan.
2-4 Concur. See Action Plan.
3-1 Concur. See Action Plan.
3-2 Concur. See Action Plan.
3-3 Concur, provided that the recommendation is modified to state
something such as, "When a pattern of delay becomes evident,
implement existing policies regarding late deliverables at a
minimum." See Action Plan.
3-4 We understand that this recommendation is being deleted and
will not appear in the final report.
3-5 Concur. See Action Plan.
4-1 Concur. See Action Plan.
4-2 Concur. See Action Plan.
4-3 Concur. See Action Plan.
4-4 Concur. See Action Plan.
4-5 Concur. See Action Plan.
5-1 Concur. See Action Plan.
5-2 Concur. See Action Plan.
5-3 Concur, provided that the phrase, "and other site decisions,"
is deleted.
5-4 Concur. See Action Plan.
5-5 Concur. See Action Plan.
5-6 Concur. See Action Plan.
Please note that specific comments on the report are listed
in Attachment 1. The Action Plan is described in Attachment 2.
We would like to thank you for the opportunity to provide
comments on the draft report. For questions, please call Barbara
Rodriguez, Regional Audit Coordinator, at (303) 312-6360.
39
Printed on Recycled Papar
-------
APPENDIX I
Page 3 of 5
Attachment 1
Specific Comments on Report
Chapter 2: Throughout this chapter, it should be made clear that
project managers have had the responsibility for approval of
quality^assurance documents only since 1992. Prior to that time,
the Regional Quality Assurance Officer had the responsibility.
For example, the first sentence on p. 7 should be clarified.
p. 7, last paragraph. The Quality Assurance Project Plan
developed for the entire Clark Fork Basin was approved on
May 20, 1992. It addressed DQO's in an adequate manner with the
understanding that each subsequent SAP would include any site-
specific DQO's as appropriate. Much of the design work included
in the Streamside Tailings Operable Unit was to further
characterize the area as an extension of the RI/FS, and thus the
decision was made that the basin-wide QAPP should be the
governing plan. The State of Montana is now the lead for
construction of the remedy. DQO's for that aspect of the remedy
should be part of the planning process for the state's work under
a cooperative agreement.
p. 11, 2nd paragraph. The sample brokering program was designed
only for "analytical work to be performed for all EPA Region VIII
programs ... through lAGs, States, and contractors."
Consequently it does not apply to PRP-lead sites. Also, the
brokering system does not apply to those instances where
laboratory services were being subcontracted out. That
understanding is reflected in the Regional Quality Management
Plan. The brokering system thus applies only to situations where
EPA itself or EPA through an IAG or contract is directly
responsible for obtaining analytical services through the
Regional Laboratory, the Contract Laboratory Program, or other
vehicle.
p. 11, 2nd paragraph. The sample broker does not review each
sampling plan. He only knows that the plan is approved. He then
receives- a copy of the analytes.
P. 12, 1st full paragraph. This example of the Summitville
sampling plan does not demonstrate a situation that would be
helped by requiring a 30-day review period. The actual situation
is that the contractor was tasked with preparing a sampling plan.
Generally these plans are prepared with significant data user
input. However, in this case the EPA data user was not able to
spend the time necessary to allow preparation of an adequate
document. Thus when the draft plan came in, there had been a
project management change that resulted in a new data user
40
-------
APPENDIX I
Page 4 of 5
2
becoming involved. This new person decided that the plan did not
meet EPA needs and asked that another plan be prepared. It was
estimated that the costs incurred to date were $28,000. We do
not believe that a 30-day requirement on a draft plan could have
helped a situation that was directly related to the data user
needs changing toward the end of the planning effort.
p. 12, 2nd paragraph. First sentence should begin, "Some project
managers," rather than the blanket statement, "Project managers"
which makes the reader think that all or a majority of, project
managers were not diligent in carrying out their duties.
p. 12, 2nd paragraph. We have searched our records and have
found approval letters for two of the five plans that did not
have approval signatures. Approval documentation for these is
attached as Attachments 1-A and 1-B. A third plan, for Butte
Priority Soils, was approved based on the findings in the OIG
draft report (Attachment 1-C). These approval letters were not
attached to the sampling plans. One additional plan, for
Streamside Tailings, was never implemented because EPA took back
the work and gave it to the State. Therefore, the State would
have the responsibility, for approving the plan. Thus, there is
only one plan that has not been approved. ->
p. 13, top of page. Region 8 already has in place mechanisms for
holding PRPs and contractors accountable. PRP work is done under
orders or decrees that specify the work to be done. In
agreements there are stipulated penalties for late submission of
deliverables. In unilateral orders, there is the threat of
enforcement. Penalties are determined by a number of factors and
are at the discretion of the project coordinator and signatory.
For contracts the contract document itself describes the
procedure to be followed in cases of lack of performance. Work
assignments are evaluated and awards can be reduced by the
Performance Evaluation Board based on the criteria in the
contract.
p. 16, 3rd paragraph. Attachment 1-D is a table that summarizes
our findings on QA training needs. We believe that seven project
managers, including a new-hire, currently require at least some
training. As of September 30, 1997, we believe there were eight
project managers needing training. One person, Ken Wangerud, who
was identified in the report as needing training, has been
exempted from some training according to the Rick Edmonds
transition plan memo of January 9, 1996. Another, Lisa Lloyd,
completed her training after September 30. Those who are
currently untrained have been counseled about not signing plans
until they are fully trained. We agree that the two SEE
employees should be trained. Although not required by the
Regional QAMP, we agree that some familiarity with QA
requirements is important for managers of project managers. The
41
-------
APPENDIX I
Page 5 of 5
QA Program will be developing a short course specific to this
group.
p. 18. first sentence as continued from p. 17. We disagree with
the statement that managers are delivering a message that QA is
not important, using the example of not responding to the request
for information. Without supporting information, it is
presumptuous of the OIG to conclude that the reason for not
responding is a view that QA is not important. Even though there
may be a perception of this view by some people in the QA Office,
in fact, a memo was distributed to all RPMs in the Superfund
Remedial Response Program from the Program Director on January
16, 1996, that clearly states the importance of receiving this
training. This was one week after the transition plan memo was
issued.
p. 20, second paragraph under "Oversight..." It is not clear how
widespread this problem is. We believe there may be a few
project managers who may not have been present at their sites
during a field sampling episode. However, project managers
routinely perform site visits for one purpose or another and
personally, or with state staff assistance, oversee contractor or
PRP work. Site-specific travel is routinely approved and
encouraged as needed to assure proper oversight.
p. 20, third paragraph under "Oversight..." Same as above.
42
-------
Region 8 Needs to Further Strengthen Its Superfund
Field Sampling Quality Assurance Controls
APPENDIX II
DISTRIBUTION
Office of Inspector General
Inspector General (2410)
Deputy Assistant Inspector General for
Internal Audits (2421)
Headquarters Audit Liaison (2421)
Divisional Inspectors General for Audit
Headquarters Office
Assistant Administrator for Solid Waste and Emergency Response (5101)
Director, Office of Emergency and Remedial Response (5201)
Division Director, Analytical Operations and Data Quality Center (5202G)
Division Director, Regions 5/7 Accelerated Response Center (5202G)
Assistant Administrator for Research and Development (8101)
Director, National Center for Environmental Research and Quality Assurance (8701)
Division Director, Quality Assurance Division (8724R)
Agency Followup Official (2710)
Agency Followup Coordinator (3304)
Associate Administrator for Regional Operations and State/Local Relations (1501)
EPA Region 8 Office
Assistant Regional Administrator for
Ecosystems Protection and Remediation (8EPR)
Assistant Regional Administrator for
Technical and Management Services (8TMS)
Director, Montana Operations Office (8MO)
Program Manager, Superfund Remedial Response - Denver (8EPR-SR].
Program Manager, Superfund Remedial Response - Montana (8MO)
Program Manager, Quality Assurance (8TMS-Q)
Audit Coordinator (8TMS-G)
E1SKG7-08-5001-8100082
43
------- |