-------
Inspector General Division
Conducting the Audit: Mid-Atlantic Division
Philadelphia, PA
Program Offices Involved: Hazardous Waste Management Division
Philadelphia, PA
Environmental Assessment and
Protection Division
Philadelphia, PA
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
e° S7i OFFICE OF THE INSPECTOR GENERAL
MID-ATLANTIC DIVISION
841 Chestnut Building
Philadelphia, Pennsylvania 19107-4431
(215) 566-5800
March 27, 1 997
MEMORANDUM
SUBJECT: Final Report of Audit on Region 3
Superfund Field Sampling Activities
Audit Report Number E1SKF6-03-01 04-7 1001 40
FROM: Carl A. Jannetti
Divisional Inspector General for Audit (3AIOO)
TO: W. Michael McCabe
Regional Administrator (3RAOO)
Attached is our audit report on Region 3 Superfund Field Sampling Activities. The
overall objectives of this audit Were to determine whether the Region: (1)
established adequate quality assurance and quality control procedures over
environmental data; (2) properly reviewed and approved sampling and analysis
plans; and (3) appropriately trained Remedial Project Managers. This report
contains important findings and recommendations.
This audit report contains findings that describe problems the Office of the
Inspector General (DIG) has identified and corrective actions the OIG recommends.
This audit report represents the opinion of the QIG. Final determinations on
matters in this audit report will be made by EPA managers in accordance with
established EPA audit resolution procedures. Accordingly, the findings contained
in this audit report do not necessarily represent the final EPA position, and are not
binding upon EPA in any enforcement proceeding brought by EPA or the
Department of Justice.
ACTION REQUIRED
In accordance with EPA Order 2750, you as the action official are required to
provide this office a written response to the audit report within 90 days. Your
response should address all recommendations, and include milestone dates for
corrective actions planned but not completed.
-------
We have no objection to the release of this report to the public. Should you have
any questions about this report, please contact Michael Wall at 215-566-5800.
Attachment
-------
EXECUTIVE SUMMARY
PURPOSE
BACKGROUND
The purpose of this audit was to determine if Region 3:
• Established adequate quality assurance and quality
control procedures over environmental data.
• Properly reviewed and approved sampling and analysis
plans.
» Appropriately trained Remedial Project Managers.
Each year hundreds of millions of dollars are spent on the
collection of sampling data. Because the success of
environmental projects depends on the reliability of this
data, EPA requires that it be monitored and measured, i.e.,
undergo quality assurance. Thus, the Agency issued EPA
Order 5360.1, dated April 3, 1984, entitled Policy and
Program Requirements to Implement the Mandatory Quality
Assurance Program. This Order required each region to
develop a Quality Management Plan to govern regional
quality assurance activities, and directed that the Office of
Research and Development review each regional plan for
approval. Such plans are important because they serve as
the framework for applying quality assurance and quality
control procedures to environmental data operations. In
effect, they are the first step in the process of implementing
a comprehensive and effective quality assurance system.
Samples collected under the Superfund program are
governed by the National Oil and Hazardous Substances
Pollution Contingency Plan (HCP) as found in 4CXCFR 300.
The NCP requires the development of a site-specific
sampling and analysis plan (SAP) made up of a field
sampling plan and a quality assurance project plan. The
purpose of the SAP is to determine: (1) the number, type,
and location of the samples to be collected; (2) the type of
analysis the samples will undergo; and, (3) the objectives,
Rppprt No. E1SKF6-03-0104-7100140
-------
RESULTS-IN-BRIEF
RECOMMENDATIONS
REGION 3
RESPONSE
organizations, functions, and quality control activities to be
involved. The SAPs we reviewed had been submitted for
the Region's approval by the parties responsible for cleaning
the sites, and were generally the result of legal agreements
between these parties and the Government.
Regional Remedial Project Managers (RPMs) are responsible
for reviewing and approving the SAPs. According to the
NCP these individuals are to be properly trained to carry out
their duties in a legally and technically sound manner. In
addition, the Office of Solid Waste and Emergency Response
requires the regions to document that the RPMs have
received all mandatory training.
Over a 12-year period, until August 1996, Region 3 did not
comply with an Agency requirement to assemble a Quality
Management Plan, because it was deemed a low priority by
regional personnel.
Region 3 personnel did not document their review of SAPs
used to conduct remedial investigations and feasibility
studies at Superfund sites.
Region 3 RPM training records were inaccurate, and RPMs
did not attend all mandatory training courses. Specifically,
the records maintained by the regional Human Resources
Management Branch failed to show what training courses
the RPMs had attended. Also, 5 of the 10 RPMs we
interviewed had not attended all required courses.
We recommend the Region 3 Administrator: (1) fully
implement the Quality Management Plan; (2) require RPMs
to sign and date SAPs as evidence of their approval; and,
(3) ensure that RPMs attend all mandatory training courses.
In regard to the SAPs, the Region stated that it agrees and
will include procedures to ensure that the site file clearly
reflects the review and approval status of the SAP
documents. The Region will examine procedures to ensure
that documents are signed and final approved document
Report No. E1SKF6-03-0104-7100140
-------
dates are confirmed. In regard to training records, the
Region indicated the reconciliation process has begun.
OIG EVALUATION During the exit conference, Superfund personnel indicated
that the Region will implement new procedures to sign and
date the SAP documents. The action planned by the Region
for reconciling the RPM training records appears to meet the
intent of our recommendations.
iii Report No. E1SKF6-03-0104-7100140
-------
[This page was intentionally left blank]
iv Report No. E1SKF6-03-0104-7100140
-------
TABLE OF CONTENTS
EXECUTIVE SUMMARY . . . ........... i
CHAPTER 1 1
INTRODUCTION 1
PURPOSE 1
BACKGROUND . 1
Agency Quality Assurance Policy 1
Region 3 Quality Assurance 2
Sampling and Analysis Plans 2
Training Requirements 3
SCOPE AND METHODOLOGY 4
PRIOR AUDIT COVERAGE , . . 6
CHAPTER 2 . / 7
QUALITY ASSURANCE MUST BE EMPHASIZED . 7
Absence of Region 3 Quality Management Plan . 8
Status of Quality Management Plan at Time of Audit ; .... 10
CONCLUSION .'..-... 10
RECOMMENDATIONS . - .11
REGION 3 RESPONSE AND OIG EVALUATION 11
CHAPTER 3 13
THE SAP REVIEW PROCESS IS INCOMPLETE 13
Region 3 Review Process 13
Rationale for Not Signing SAPs 13
CONCLUSION . 15
RECOMMENDATIONS 15
REGION 3 RESPONSE AND OIG EVALUATION 16
CHAPTER 4 17
. TRAINING RECORDS SHOULD BE MORE PRECISE 17
RECOMMENDATIONS 18
REGION 3 RESPONSE AND OIG EVALUATION 18
APPENDIX A - REGION'S RESPONSE TO DRAFT REPORT . 19
APPENDIX B - DISTRIBUTION '. 27
Report No. E1SKF6-03-0104-7100140
-------
[This page was intentionally left blank]
Report No. E1SKF6-03-0104-7100140
-------
CHAPTER 1
INTRODUCTION
PURPOSE
BACKGROUND
Agency Quality
Assurance Policy
The purpose of this audit was to determine if Region 3:
• Established adequate quality assurance and quality
control procedures over environmental data.
• Properly reviewed and approved sampling and analysis
plans.
• Appropriately trained Remedial Project Managers.
Many Superfund decisions require the collection and
evaluation of site-specific environmental data. Major
activities associated with acquiring this data include
planning, sample collection and analysis, and data quality
assessment. EPA policy requires the development and
implementation of quality assurance (QA) programs to
ensure that these activities generate data of known quality.
The overall goal of a QA program is to measure and
minimize systematic sources of error and to monitor
conditions of sampling, storage, and transport.
EPA Order 5360.1 entitled, Policy and Program
Requirements to Implement the Mandatory Quality
Assurance Program, established mandatory QA
requirements for Agency environmental data collection
activities. The National Oil and Hazardous Substances
Pollution Contingency Plan (NCP) mandated specific
Superfund QA requirements. Both of these documents
emphasize two requirements. First, environmental data
must be of known quality, i.e., all components associated
with its derivation are to be thoroughly documented and
reviewed. Second, QA plans that are based on generic or
site-specific procedures are necessary in order to fulfill the
first requirement.
1
Report No. E1SKF6-03-0104-7100140
-------
Region 3 Quality
Assurance
Sampling and
Analysis Plans
In 1994, the Office of Research and Development (ORD)
directed its Quality Assurance Division to initiate a
management assessment program. This Division oversees
the Agency's Quality Assurance program by conducting
Management System Reviews (MSRs) in each major Agency
component (including the Regional Offices) every three or
four years. The MSRs are independent assessments of
management's quality assurance practices and address the
effectiveness of management controls in regard to:
• Achieving and assuring data quality,
• The adequacy of resources and personnel devoted to
QA functions,
• The effectiveness of training, and
• The applicability of data quality requirements.
In August 1994, ORD conducted an MSR of Region 3 and
found that: (a) the Region did not have a Qifality
Management Plan; and, (b) although the quality assurance
performed by the laboratory in Annapolis, Maryland was
good, the quality assurance performed by the rest of the
Region was sporadic at best. In January 1995, the MSR
team issued its report, which recommended that Region 3
prepare and implement a Quality Management Plan.
However, the Region did not respond to the report.
Samples collected under the Superfund program are
governed by the NGP as found in 40 CFR 300. The NCP
requires the development of a site-specific sampling and
analysis plan (SAP) made up of a field sampling plan and a
quality assurance project plan. The purpose of the SAP is
to determine: (1) the number, type, and location of the
samples to be collected; (2) the type of analysis the samples
will undergo; and, (3) the objectives, organizations,
functions, and quality control activities to be involved. The
SAP ensures that sampling data collection activities will be
comparable to, and compatible with, previous data
collection activities performed at the site while providing a
mechanism for planning and approving field activities. The
Report No. ElSKp6-03-0104-7100140
-------
plan also serves as a basis for estimating costs of field
efforts for inclusion in the work plan.
The SAPs describe each project objective in detail. This is
usually achieved by describing the specific decisions to be
made with the data and involving the decision-maker from
the beginning. The plan should include a description of the
monitoring network, the locations where samples will be
collected, the sampling frequency, the types of analyses for
each sample, and the rationale for the design. All factors
that will influence the eventual decision should, to the
extent practical, be evaluated and specified at the beginning
of the process.
The SAPs are prepared by contractors working for the
parties responsible for cleanup of the sites and are to be
reviewed and approved by EPA.
Training EPA Order 3500.1 requires all inspectors, including
Requirements Remedial Project Managers (RPMs), to meet mandatory
training requirements. RPMs currently meet inspector
training requirements by:
• Completing the 40-hour Health and Safety training
offered in all Regions, followed by annual 8-hour
refresher courses; and,
• Attending program specific courses such as the
Fundamentals of Superfund, Remedial Process,,
and Enforcement Process.
In addition, the Office of Solid Waste and Emergency
Response (OSWER) Directive 9285.9:06 requires that RPMs
attend Community Relations in Superfund: Basic Concepts
and Skills for Response Staff. OSWER Directive 9285.9-05,
entitled Mandatory Training Requirements for OSCs and
RPMs, requires RPMs to complete an 80-hour annual
training requirement, which must be tracked for reporting
purposes. Regions are to track RPM training by using a
Standard Form 182 and should ensure that these forms are
submitted to the Human Resources Management Branch for
entry into the Agency's training data base.
Report No. E1SKF6-03-0104-7100140
V
-------
SCOPE AND We performed this audit according to the Government
METHODOLOGY Auditing Standards (1994 Revision) issued by the
Comptroller General of the United States as they apply to
program audits. Our review included tests of the program
records and other auditing procedures we considered
necessary.
To accomplish our objectives we reviewed records and
interviewed officials at: Region 3 offices in Philadelphia,
Pennsylvania and in Annapolis, Maryland; and at ORD's
Quality Assurance Division in Washington, D.C. and in
Edison, New Jersey, fn Region 3, we focused primarily on
the Environmental Assessment and Protection Division, the
Hazardous Waste Management Division, the Human
Resources Management Branch, and the Central'Regional
Laboratory in Annapolis, Maryland.
We reviewed EPA orders, directives, regulations, and
various policies and procedures related to quality assurance,
Superfund field sampling, and Remedial Project Manager
training. We also reviewed the Region 3 Quality
Management Plan and the ORD Management System
Review reports.
While aj ORD, we interviewed the Director of the Quality
Assurance Division regarding the development of Regional
Quality Management Plans and the SAP review process.
While at the Central Regional Laboratory, we interviewed
the Quality Assurance Team's Leadership Coordinator
regarding Region 3's Quality Assurance Program, Quality
Management Plan, and SAP review process.
To further evaluate the SAP process we judgmentally
selected files from the Hazardous Waste Management
Division for 13 sites that had remedial investigations and
feasibility studies performed between fiscal years 1993 and
1995. These files contained 21 SAP documents and
correspondence from the RPMs, Agency QA personnel, and
the parties responsible for site cleanup. Due to the
complexity of the files, we obtained technical assistance
from our Engineering and Science Staff. We also
interviewed 10 of the 42 regional RPMs.
4 Report No. E1SKF6-03-0104-7100140
-------
On October 16, 1996, we visited a Superfund site in
Emmaus Borough, Pennsylvania., Several wells at the
facility had been used for the disposal of various wastes,
including waste water from copper and zinc plating. We
observed sample collection from groundwater monitoring
wells at two off-site locations. We noted that the sampling
team followed appropriate protocols for measuring, bailing,
and sampling groundwater from the wells, and that the
samples were collected and documented in accordance with
the SAP.
On October 28, 1996, we visited a second Superfund site in
Baltimore, Maryland. This 9.7 acre site had been used for
disposal and burning of a variety of wastes, including
hazardous materials. We observed soil sampling and the
installation of groundwater monitoring wells and concluded
that the work followed appropriate protocols specified in the
SAP.
We evaluated training by reviewing training records for 10
RPMs. These records were obtained from OSWER in EPA
Headquarters and from Region 3's Human Resources
Management Branch, Hazardous Waste Management
Division, Safety and Health Manager, as well as from the
RPMs themselves. We interviewed the Region 3 Training
Officer, the former Hazardous Waste Management Division
Training Coordinator, the Regional Safety and Health
Manager, the RPMs, and representatives from OSWER's
Technology Innovation Office.
Our audit disclosed several areas needing improvement that
are discussed in Chapters 2, 3, and 4. We recommended
that the Region implement the Quality Management Plan.
We also recommended that RPMs be required to sign and
date SAPs as evidence of their approval and that RPMs
attend all mandatory training courses.
We reviewed internal'controls and procedures specifically
related to our objectives. However, we did not review the
internal controls associated with the input and processing of
information into the Human Resources Management
Branch's training data base or any other automated records
system. The controls reviewed included: EPA's Federal
5 Report No. E1SKF6-03-0104-7100140
-------
Managers' Financial Integrity Act Reports for Fiscal Years
1992 through 1995; the Management System Review
reports produced by ORD; SAP review and approval
procedures; the Quality Management Plan; and, the record
keeping practices of the Human Resources Management
Branch.
Our survey began on January 10, 1996 and ended on
March 28,N1996. As a result of the survey, we initiated an
in-depth review on March 28, 1996. We completed
fieldwork by November 27, 1996 and issued position papers
to obtain a preliminary response to the issues in this report.
We received the Region's response on December 23, 1996.
We issued a draft report on January 23, 1997, with a
response due by February 24, 1997. Region 3 received two
time extensions and provided a response on March 20,
1997. After evaluating this response we made minor
modifications; however, our position remains unchanged on
the major issues discussed in this report. An exit
conference was held on March 21, 1997.
At the exit conference, Superfund personnel indicated that
the Region will implement new procedures that would
comply with our recommendations in regard to the SAP
approval process. The RPMs will indicate approval by
signing and dating final SAP documents and labeling these
documents as "Final." These actions, if included in the
Region's official response to this audit report would correct
the issues noted in Chapter 3.
The comments by Region 3, as well as our evaluation of
these comments, are presented at the end of each chapter.
The Region's complete response is included in Appendix A.
PRIOR AUDIT EPA DIG Special Report (E1SKG4-08-0045-5400034) issued
COVERAGE on January 27, 1995 discussed weaknesses found in
Region 8's field sampling activities. Region 8 needed to
strengthen pre-field sampling controls, better train RPMs,
and ensure an appropriate level of oversight at Superfund
sites.
Report No. E1SKF6-03-0104-7100140
-------
CHAPTER 2
QUALITY ASSURANCE MUST BE EMPHASIZED
Over a 12-year period, Region 3 did not comply with an
Agency requirement to assemble a Quality Management
Plan, because it was deemed a low priority by regional
personnel. Such plans are important however, because they
serve as the framework for applying quality assurance and
quality control procedures to environmental data operations.
In effect, they are the first step in the process of
implementing a comprehensive and effective quality
assurance system.
On April 3, 1984, the Agency issued Order 5360.1, Policy
and Program Requirements to Implement the Mandatory
Quality Assurance (QA) Program^ The purpose of the Order
was to address data collected across the spectrum of EPA
programs. Specifically, the Order required each EPA
organization that conducted environmentally-related
measurements to develop and implement a Quality
Management Plan. These plans were to describe the
responsibilities and controls to be used to design,
implement, and assess quality assurance programs related
to the collection and use of environmental data. Because
EPA obtains such data primarily by taking samples of water,
soil, air, and other materials, and then having the samples
analyzed in a laboratory, any weaknesses in the sampling or
-analysis procedures can result in data that is of insufficient
quality or fhat is unsupportable.
The Order also directed EPA's Office of Research and
Development (ORD) to approve each organization's plan, as
well as to oversee the implementation of the individual
quality assurance programs. Unfortunately, ORD had no
enforcement power to assure that individual Agency
components complied.
Report No. E1SKF6-03-0104-7100140
-------
Absence of Region 3 The Regional Administrators have overall responsibility for
Quality Management quality assurance within their organizations. In Region 3,
Plan the responsibility for developing the overall regional Quality
Management Plan was delegated to the Environmental
Assessment and Protection Division and to the Regional
Quality Assurance Officer. These in turn were responsible
for monitoring the development of individual plans by
regional divisions and program offices. In theory then, the
Region employed a decentralized approach to quality
assurance, requiring each of its components to develop their
own unique plans, presumably to be assembled into an
overall regional Quality Management Plan. In fact however,
such plans for the most part did not exist; other than the
Superfund Program, no regional component had an
approved plan.
During the 12 years between 1984 and 1996, Region 3 was
the only regional office never to have had an approved
organization-wide Quality Management Plan. Although ORD
repeatedly reported this deficiency to Region 3
management, the response was always the same, i.e., the
Plan was being developed. Apparently the Region was
being somewhat less than forthright, for according to ORD:
... for 15 years Region 3 had a unique personality-
driven quality assurance program .... The Region's
quality assurance program has been people expecting
others to do the work. Management never forced the
issue of a Quality Management Plan.
In fiscal years 1992-1995, the Agency reported its
environmental data quality as a material weakness in its
Federal Managers' Financial Integrity Act Report to the
President. Even though the Agency weakness resulted, in
part, because of the Region's lack of an approved plan, the
Region still declined to assemble the Quality Management
Plan. However, as seen in the chart following, the Region
repeatedly committed to do so.
8 Report No. E1SKF6-03-0104-7100140
-------
December
1991
Region Commits Plan for
June 1992
June 1992
Region Commits Plan for
September 1992
November
1993
Region Commits Plan for
December 1993
March 1994
July 1995
Region Submits Draft Plan
Rejected
No Plan
Region Commits Plan for
April 1996
March 1996
Region Commits Plan for
June 1996
August 1996
Region Submits Revised
Draft
In August of 1994, ORD conducted a Management System
Review of the Region and concluded that although the
quality assurance performed by the laboratory in Annapolis,
Maryland was good, the quality assurance performed by the
rest of the Region was sporadic at best.
Specifically, ORD reported that:
• Agency quality assurance requirements were not
being observed in many of the regional programs.
Discussions with staff at all levels showed that ". . .
„ the importance of documentation was not
understood."
• The decentralization of quality assurance within the
Region had not effectively established roles and
responsibilities. Some aspects of a quality system
were absent in some organizational units.
Report No. E1SKF6-03-0104-7100140
-------
• Management was perceived as not recognizing the
importance of the role of oversight in quality
assurance.
Although the Management System Review team
recommended improvements, the Region never provided a
response.
Status of Quality As shown in the chart, the Region's March 1994 draft
Management Plan at Quality Management Plan was rejected. According to the
Time of Audit Regional Quality Assurance Officer this had not been the
only Plan; however, he was unable to provide any
documents to support this assertion. He also said that he
was unfamiliar with the personnel responsible for quality
assurance within the Region's divisions and offices.
At the time of our audit, Region 3 submitted a revised
Quality Management Plan, which was approved on
September 27, 1996. According to this Plan, the Region
should have submitted a Quality Assurance Annual Report
and Work Plan to ORD by November 1, 1996. However,
the report was not sent. Considering the Region's lack of
adherence to past deadlines, we believe that management
must make quality assurance a high priority.
CONCLUSION
An organizational-wide quality assurance plan should have
been the cornerstone of decision-making in Region 3 since
1984. It took the Region 12 years to submit an approvable
Plan. Since approval, the Region has missed its first due
date under the Plan by not submitting their Quality
Assurance Annual Report and Work Plan. An approved
Quality Management Plan is only the first step in setting up
an effective quality assurance system. Because the Plan is
not expected to be fully implemented until August 1997, the
Region established a Quality Assurance Task Force to
coordinate the completion of the Plans for the individual
office/division components. We believe that to ensure
timely implementation, the Region should establish
accountability measures and evaluate progress on a
quarterly basis. This in turn would enable management to
initiate swift corrective action as problems are encountered.
10
Report No. E1SKF6-03-0104-7100140
-------
Quality assurance is a vital component in all environmental
programs. To quote from the Region's own Plan:
Managers and staff at EPA Region 3 make daily
decisions which affect the lives and livelihoods of
millions of people who reside in the Middle Atlantic
states. The quality of air, water and land on which
these people depend is the focus of our mission. The
quality of decisions made by EPA and its state
counterparts depends heavily on the quality of the
information used to make those decisions .... If that
data is not adequate to support its intended uses,
then the decisions suffer commensurately.
RECOMMENDATIONS We recommend that the Region 3 Administrator:
1) Fully implement the Quality Management Plan (QMP).
2) Prepare and implement the Quality Assurance Annual
Report and Work Plan.
3) Ensure the Quality Management Plan is circulated,
updated, and understood by all quality assurance
personnel. ,
4) Ensure that quality assurance personnel perform their
required duties as outlined in the Quality Management
Plan.
5) Establish and enforce accountability measures for quality
assurance managers.
REGION 3 RESPONSE AND OIG EVALUATION
Region Response Region 3 believes consideration Should be given to
additional statements about the report findings on
environmental data quality. The Region would like to show
that a "draft" QMP was completed by June 30, 1996 and
that listing environmental data quality as a material
weakness under Federal Management Financial Integrity Act
T1 Report No. E1SKF6-03-0104-7100140
-------
should not be attributed to the delay in completing Region
Ill's QMP.
OIG Evaluation According to ORD, Region 3 was the last Region to prepare
a QMP, and this QMP was not submitted in its entirety until
August 26, 1996. However, we agree that other EPA
organizations contributed to the Agency's weakness in
regard to data quality. We did not mean to imply that
Region 3 was solely responsible. That is why we stated
that the Region's lack of a Quality Management Plan for 12
years was in part responsible for this weakness. We
verified this fact with ORD, which had prepared the FMFIA
statement.
1 2 Report No. E1SKF6-03-0104-7100140
-------
CHAPTER 3
THE SAP REVIEW PROCESS IS INCOMPLETE
Region 3 personnel do not document their review of
sampling and analysis plans (SAPs) used to conduct
remedial investigations and feasibility studies at Superfund
sites. According to the NCR and an EPA document entitled
The Requirements! for Quality Assurance Project Plans for
Environmental Data Operations, the SAPs are to be
reviewed for approval by the Agency. Although it appears
that Region 3 personnel do in fact review the SAPs, they do
not properly document this review, i.e., they do not sign
and date the actual documents.
Region 3 Review
Process
Rationale for Not
Signing SAPs
I
We reviewed 21 SAPs submitted between 1993 and 1995
for 13 Superfund remedial sites, and found that none of the
documents were signed as being approved by the site
Remedial Project Managers (RPMs). Moreover, we could
not ascertain whether these SAPs were draft or final
documents, except in the one instance where the RPM
changed the word "draft" to "final." We then interviewed 10
of the 42, regional RPMs and learned that rather than
indicating their approval by signing the SAP documents,
they sent separate approval letters to the potentially
responsible parties. However, when we requested the
approvals for the 21 SAPs in our review, the RPMs were
unable to provide 7 of the letters.
In addition to not being aware that they were to sign the
SAPs, some RPMs were under the impression that Regional
Counsel had advised against doing so, or at least, not to
sign off on the quality assurance project plan component.
In 1994, ORD conducted a Management System Review
(MSR) of the Region's quality assurance program. In its
final report, ORD noted the absence of RPM signatures.
ORD also noted that:
The RPMs apparently have been advised by lawyers
within Region 3 to avoid signing the documents.
13
Report No. E1SKF6-03-0104-7100140
-------
Instead they reportedly are advised to "accept" the
documents. Although a comprehensive search was
not undertaken, only a quality assurance project plan
from 1988 could be found that was approved, signed,
and dated by the quality assurance officer and RPM.
ORD stated that this was a finding of a critical nature and
recommended that the Region take steps to review and
approve (sign and date) quality assurance project plans
before data collection begins. ORE) requested that the
Region provide a schedule for completion of this
recommendation; however, the Region did not respond to
the recommendation.
We contacted the Regional Counsel lawyers associated with
Superfund. They stated that: (1) they had never advised
RPMs to sign approval letters rather than SAPs; and, (2)
there was no valid reason why RPMs should not sign the
SAPs, v - '
We also met with ORD officials. They agreed that in order
for the SAPs to be considered approved, the documents
must be signed and dated. Otherwise, there is a question of
accountability, and the possibility that a contractor could
use an unapproved plan in sampling the site. One ORD
official stated that:
!
Sampling with unapproved plans could lead to designs
that would not work. Also the physical way
contractors take samples will affect the lab analysis if
the wrong procedure is used. Eighty percent of the
uncertainty/error in the site characterization is related
to field operations, taking samples from the wrong
locations, using the wrong sampling technique or
preservation, etc. Quality assurance and quality
control should focus on field activities, before the
samples are sent to the lab. If they take the samples
wrong, because someone used the wrong version of
the sampling and analysis plan, it does not matter
how good quality assurance is at the lab. The
samples will still be worthless. It is inefficient and a
waste to perform field activities if things that should
1 4 ' Report No. ElSKF6-Q3-0104-7i00140
-------
CONCLUSION
RECOMMENDATIONS
be done, such as signing the approved sampling and
analysis plan, are not done up-front.
\
Moreover, the EPA quality assurance manager for the
Superfund Program also agreed that the SAP must be
signed, explaining that it was the intent of the National
Contingency Plan for approved sampling and analysis plans
to be signed in order to preclude a contractor from
performing site sampling with an unapproved plan.
Finally, of the 10 RPMs we interviewed, only 2 stated that
they generally review the SAPs when visiting their sites.
Otherwise, it was up to the contractor employees
performing the sampling to make sure that they were using
the correct version of the plan.
We believe that Region 3's process of approving Superfund
sampling and analysis plans needs improvement. Although
RPMs said they wrote approval letters, they did not sign and
date the actual SAP document itself. If a contractor uses an
unapproved plan to perform sampling, the data obtained
may be unreliable, which in turn could adversely affect
future site decisions. Therefore, it is incumbent upon EPA
that sampling be performed according to the approved SAP.
We recommend that the Region 3 Administrator take action
to ensure that:
1) RPMs, as well as the preparing organization's project
manager and quality assurance officer, sign and date
both components of approved sampling and analysis
plans before the collection of site samples.
2) RPMs label approved sampling and analysis plans as
"Final" versions.
3) On enforcement-lead sites where Agency contractors
perform direct oversight of sampling and analysis
activities, RPMs should ensure that oversight contractors
verify that the field personnel are following the final
approved SAPs.
15
Report No. E1SKF6-03-0104-7100140
-------
4) On fund-lead sites or on any site where the RPM is
directly overseeing the sampling and analysis activities,
the RPM should verify that the field personnel are
following the final approved SAPs.
REGION 3 RESPONSE AND OIG EVALUATION
Region Response to The Region agrees and will include procedures to ensure
Recommendation 1 that the site file clearly reflects the review and approval
status of the SAP documents. The Region will examine
procedures to ensure that documents are signed and final
approved document dates are confirmed.
During the exit conference, Superfund personnel indicated
that the Region will implement new procedures to comply
with this recommendation.
OIG Evaluation The action planned meets the intent of the recommendation.
Region Response to The Region agrees and the site file will clearly show that
Recommendation 2 SAP documents are "draft" or "final."
During the exit conference, Superfund personnel indicated
that the Region will implement new procedures to comply
with the recommendation.
\
OIG Evaluation The action planned meets the intent of the recommendation.
Region Response to The Region agrees and will include a standard oversight
Recommendation 3 function for enforcement-lead sites in the Superfund Quality
Management Plan.
OIG Evaluation The action planned should meet the intent of the
recommendation. Please provide us with a copy of the
SupeffundQuality Management Plan when it is updated.
Region Response to The Region agrees and will include this function in the
Recommendation 4 Superfund Quality Management Plan.
OIG Evaluation The action planned should meet the intent of the
recommendation. We will review Superfund Quality
Management Plan when it is updated.
16 Report No. E1SKF6-03-0104-7100140
-------
CHAPTER 4
TRAINING RECORDS SHOULD BE MORE PRECISE
Region 3 Remedial Project Manager (RPM) training records
were inaccurate. Specifically, the records maintained by the
regional Human Resources Management Branch (HRMB) did
not show the training courses RPMs attended.
Several Agency orders and directives require RPMs to attend
various training courses, which include a 40-hour Health and
Safety course to enable them to enter Superfund sites. In
general, EPA requires that; (a) RPMs receive a minimum of
80 hours training each year; and (b) the records of this
training be maintained in the regional HRMB training office
(OSWER Directive 9285.9-05 of September 29, 1989).
During our audit we found that the Region 3 training records
for RPMs were incomplete. Specifically, we reviewed the
records for 10 of the 42 regional RPMs, and noted that
HRMB had no record of any of these individuals having
attended any of the required courses. Through interviews
with the 10 RPMs, the regional Safety and Health Manager,
and Headquarters personnel, we learned that the RPMs had
in fact attended many courses; however, because the
Hazardous Waste Management Division (HWMD) failed to
forward the prpper forms to HRMB, the official training
records were inaccurate.
In November 1996, subsequent to our review, HRMB
initiated a reconciliation of the training records for all Region
3 employees going back to 1984. This in turn prompted us
to again contact the 10 RPMs, who informed us that over
100 courses had never been recorded by HRMB. Moreover,
some of these courses included mandatory annual Health
and Safety refresher training, despite the fact that this
information had been provided to HRMB by the Region's
Safety and Health Manager.
17 Report No. E1SKF6-03-0104-7100140
-------
RECOMMENDATIONS We recommend that the Region 3 Administrator ensure that:
r ' • • •
1) Region 3 RPMs attend all mandatory courses.
2) HWMD training records and the Health and Safety
records are reconciled with the official records in HRMB.
3) HWMD notifies HRMB of all future training attended by
RPMs.
REGION 3 RESPONSE AND OIG EVALUATION
Region Response
OIG Evaluation
The process to reconcile records with HRMB has begun.
Official training will be properly reflected.
In regard to recommendations 2 and 3, the actions planned
by Region 3 should meet the intent of these
recommendations. We will review the Superfund Quality
Management Plan when it is updated.
18
Report No. E1SKF6-03-0104-7100140
-------
APPENDIX A - REGION'S RESPONSE TO DRAFT REPORT
19 Report No. E1SKF6-03-0104-7100140
-------
[This page was intentionally left blank]
20 Report No. E1SKF6-03-0104-7100140
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
REGION III
841 Chestnut Building
Philadelphia, Pennsylvania 19107
SUBJECT: Draft Audit Report of Region III Superfund Field
Sampling Activities MAR 20 1997
FROM: j(f, Michael McCabe < ^_«,
/^Regional AdministraTor (3RAOO)
TO: Carl A. Jannetti
Divisional Inspector General for Audit (3AIOO)
Thank you for the opportunity to review and comment on the report. The Region has
reviewed the report and responded to specific recommendations relating to Hazardous Waste
Management Division (HWMD) as follows:
1. Remedial Project Managers (RPMs) sign and date both components of approved Sampling
and Analysis Plans (SAPs) before the collection of site samples.
The Region agrees and will include procedures to ensure that the site file clearly reflects the
review and approval status of theSAP documents. The Region will examine procedures to
ensure that documents are signed and final approved document dates are confirmed.
2. RPMs label approved SAPs as "Final" versions.
The Region agrees and the site file will clearly show that SAP documents are "draft" or
"final."
3. On enforcement-lead sites, RPMs should ensure that Agency oversight contractors verify
that the field personnel are following the final approved SAPs.
The Region agrees and will include a standard oversight function for enforcement-lead sites
in the Superfund Quality Management Plan (QMP).
4. On fund-lead sites, the RPM should verify that field personnel are following the final
approved SAPs.
The Region agrees and will include this function in the Superfund QMP.
5. Region HI RPMs attend all mandatory courses.
\
The Region agrees and the Superfund QMP will include SAP training.
21
Celebrating 25 Years of Environmental Progress
-------
6. HWMD records and the health and safety records are reconciled with the official records in
Human Resources Management Branch (HRMB).
The Region agrees and has started the process to reconcile records with HRMB.
7. HWMD notifies HRMB of all future training attended by RPMs.
The Region agrees and will ensure that the official Regional training properly reflects the
RPMs' training.
In addition, the Environmental Assessment and Protection Division (EAPD) believes
consideration should be given to additional statements about the report findings on environmental
data quality. EAPD would like to show that a "draft" QMP was completed by June 30,1996 and
that listing environmental data quality as a material weakness under Federal Management Financial
Integrity Act should not be attributed to the delay in completing Region Ill's QMP.
We believe that this response addresses specific recommendations identified in the audit. If
you should have any questions on this matter, please contact Robert Reed at 6-5410.
Attachments:
HWMD & EAPD Comments
22
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
REGION III
841 Chestnut Building
Philadelphia. Pennsylvania 19107
SUBJECT: Draft Audit of Region 3 "Superfund Field Sampling DATE: P£[J ]£
Activities"
FROM: Abraham Ferdas, Associate Division Director
Office of Superfund Programs (3HW02)
I
TO: RobertG. Reed,'Jr.; Chief -n7 rr^ ,0
^y / l~ **• 14 I I/ ^^ j<•' —
Grants and Audit Management Branch (3PM7D) L> ' '• 'nt)
We appreciate the opportunity to review and comment on the Inspector General's (IG)
January 23, 1997 Draft Audit Report, "Region 3 Superfund Field Sampling Activities." We
share the IG's concern for the quality of the environmental data that we use to make decisions
in the Superfund program. We are encouraged that the audit did not reveal the use of any
inappropriate field data in the Superfund program. We are also pleased that the audit noted
that the Superfund program did have an approved Quality Management Plan (QMP), even
when the Region as a whole did not. We appreciate the IG's comments on the quality
assurance systems that were observed during the investigation last year and we plan to address
the recommendations as we update our procedures in the Office's QMP.
' •
• ,-\
We believe that the IG's recommendations about document approval records, training
documentation, and verifying the use of appropriate plans in the field are constructive
suggestions. The Office of Superfund was an active participant in the workgroup that
prepared the Regional Quality Management Plan that was approved last year and we
recently submitted a revised QMP for Superfund to EAPD for review. Many of the issues
noted in the audit are addressed in our revised Plan, or can be included before it is finalized.
We plan to have the Office QMP finalized by June 1997.
Here are our specific responses to the Draft Report's recommendations for Superfund
(we did not respond to the general Regional recommendations on page 11):
Responses to Specific Recommendations for Superfund
1. RPMs, as -well as the preparing organization's project manager and quality assurance
office, sign and date both components of approved sampling and analysis plans before the
collection of site samples. -
"'-*,.• (
The Superfund Office is committed to developing good administrative records for all
Regional Superfund sites. We are concerned with the Draft Audit's finding that seven of the
21 sites reviewed did not have good records for the SAP review and approval process. We
will address this issue as we update our QMP and will include procedures to see that the site
file clearly reflects the review and approval status of the SAP documents. The options we
23
Celebrating 25 Years of Environmental Progress
-------
wijl examine include signing the document itself and confirming in the file the date of the
final, approved document. It should be noted that no other site documents (such as work
plans, RI/FSs, etc.) are actually signed by the RPMs during the review and approval process
in the Superfund program and that there are some questions about the benefits of doing that
for the SAPs. Our procedures will also specify that field sampling activities are not to take
place until a final, approved SAP is in place.
2. RPMs label approved sampling and analysis plans as "Final" versions.
The site file will clearly reflect the draft or final status of the SAP documents.
3. On enforcement-lead sites where Agency contractors perform direct oversight of sampling
and analysis activities, RPMs should ensure that oversight Contractors verify that the field
personnel are following the final approved SAPs.
We agree that this should be a standard oversight function for enforcement-lead sites
and will include this activity in our Superfund QMP.
4. Onjund-lead sites or on any site where the RPM is directly overseeing the sampling and
analysis activities, the RPM should verify that the field personnel are following the final
approved SAPs.
We agree that this an important activity for the RPM and will include it in our
Superfund QMP,
5. Region 3 RPMs attend all mandatory courses.
We agree and the Superfund QMP will include SAP training.
6. HWMD records and the Health and Safety records are reconciled with the official records
inHRMB. '
This issue was first noted in your office's November 25, 1996 Position Paper. The
Office of Superfund responded by initiating efforts to reconcile RPM training records with
KRMB. The Office of Superfund has submitted three training record updates to HRMB since
our December 17th, 1996 response to the Position Paper. The Office intends to have this
issue resolved with HRMB by the end of this fiscal year.
7. HWMD notifies HRMB of all future training attended by RPMs.
Our Office will continue the process noted above to make sure that the official
Regional training records properly reflect the training take by RPMs.
If you have any questions about our remarks, please contact Eric Johnson, Chief,
Superfund Technical Support Section, at x3313.
24
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
REGION III
841 Chestnut Building
Philadelphia, Pennsylvania 19107
SUBJECT: Comments on OIG Draft Audit Report on Region 3 DATE: jyjAp i i
SuperfunchField Sampting Activities MAK ' '
FROM: John
Environmental Assessment & Protecion .Division (3EPOO)
TO: Carl A. Janetti
Divisional Inspector General for Audit (3AIOO)
Below are EAPD's comments concerning a January 23,1997 OIG Audit Report
entitled Draft Report of Audit on Region 3 Sv^erfund Field Sampling Activities (Audit
Report Number E1SKF6-03-0104). Part of this Audit included a discussion of the
Regional Quality Assurance Program, for which this Division is responsible. Our
comments are restricted to the QA portion of the audit, although we appreciate the
opportunity to review the document in its entirety.
1. The Table at the top of Page Nine of the report indicates a May, 1996
Regional commitment to prepare a Quality Management Plan (QMP) by June 1996, and
further indicates that No Plan was prepared. In fact, the commitment made was to
produce a Draft QMP by June 30,1996. Such a Draft Plan was completed by this
target date, and this accomplishment was entered into the RSIMS tracking system. I
had noted this error previously in my memo to P. Ronald Gandolfo dated December 23,
1996.
2. Although the Agency has declared for several years a material weakness
under the Federal Management Financial Integrity Act (FMFIA) for environmental data
quality (reference on page 8), this was not related in any direct way to the lack of a
Region III Quality Management Plan (QMP): In fact, despite approval of the Regional
Plan in September, 1996, the Agency still lists data quality as a material weakness in
its most recent FMFIA report for FY 97. Many other EPA organizations besides Region
III have contributed to this declared weakness, and remain without approved QMP's or
with substandard QA programs. The implication that this declared weakness was
attibutable to the delay in completing Region Hi's QMP should be removed.
Thank you for the opportunity to provide comments. If you have any questions
on the comments above, please contact Diann Sims (410/573-2746) or Tom Slenkamp
(215/566-2750) of my staff.
cc: Ken Kryszpzun (3HW40)
25
Celebrating 25 Years of Environmental Progress
-------
[This page was intentionally left blank]
26
-------
APPENDIX B - DISTRIBUTION
Headquarters
Office of Inspector General - Headquarters (2410)
Agency Audit Followup Coordinator (3304)
Agency Audit Followup Official (3101)
Assistant Administrator for Research and Development (8101)
Assistant Administrator for Solid Waste and Emergency Response (5101)
Assistant Administrator for Enforcement and Compliance Assurance (2201 A)
Associate Administrator for Congressional & Legislative Affairs (1301)
Associate Administrator for Communications, Education & Public Affairs (1701)
Associate Administrator for Regional Operations & State/Local Relations (1501)
EPA Library (3404)
EPA Region 3
Regional Administrator (3RAOO)
Director, Hazardous Waste Management Division (3HWOO)
Director, Environmental Assessment & Protection Division (3EPOO)
Director, Office of External Affairs (3EAOO)
Chief, Human Resources Management Branch (3PM40)
Chief, Grants & Audit Management Branch (3PM70)
Regional Library (3PM52)
Other
Office of Inspector General — Divisional Offices
General Accounting Office
27 Report No. E1SKF6-03-0104-7100140
------- |