I
3J
\.
I Office of Inspector General
Report of Review
Special Review of EPA Region 9
Data Quality Oversight at
The Aerojet Superfund Site
Rancho Cordova, California
E1SKG5-09-0110-6400044
March 28, 1996
-------
Inspector General Division Western Audit Division
Conducting the Audit: Sacramento, California Branch Office
Region Covered: Region 9
Program Offices Involved: Hazardous Waste Management Division
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
^ i OFFICE OF THE INSPECTOR GENERAL FOR AUDITS
"tp"olt WESTERN DIVISION
75 HAWTHORNE STREET
19TH FLOOR, MAIL CODE 1-1
SAN FRANCISCO, CA 94105-3901
March 28, 1996
MEMORANDUM
SUBJECT: Review Report No. E1SKG5-09-01 10-6400044
Special Review of EPA, Region 9
Oversight of Laboratory Data Quality
at the Aerojet Superfund Site
FROM: Truman R. Beeler /+^y //.
Divisional Inspector General for Audit
Western Division
TO: Felicia Marcus
Regional Administrator
EPA, Region 9
Attached is our report titled Special Review of EPA Region 9 Oversight of Laboratory Data
Quality at the Aerojet Superfund Site.
A position paper summarizing the results of our review was provided to the Region on
February 16, 1996. The Region responded to the position paper on March 14, 1996. The
Region's response to our draft report and our evaluation of the response are included in the body of
the report. Based on this response, we made appropriate changes to our final report.
The report identifies corrective actions the Office of Inspector General (OIG) recommends
involving data quality at the Aerojet Superfund site. As such, it represents the opinion of the OIG.
Final determinations on the matters in the report will be made by EPA managers in accordance
with established EPA audit resolution procedures. Accordingly, the findings described in this
report do not necessarily represent the final EPA position and are not binding upon EPA in any
enforcement proceeding brought by EPA or the Department of Justice.
Under EPA Order 2750, you are designated as the action official for this report, and must
provide our office with a written response to the audit report within 90 days of the report date.
The response should address all recommendations. For corrective actions planned but not
completed by the response date, your reference to the specific milestone dates will assist us in
deciding whether to close this report. We have no objection to the release of this report to the
public.
Should you or your staff have any questions about this report, please contact Geary Pena,
Audit Manager, at (916) 498-6530. I can be reached at (415) 744-2445.
Attachment
Primed on Recycled Paptr
-------
Table of Contents
Executive Summary
Laboratory Data
Quality at the
Aerojet Superfund
Site Not Monitored
Eage.
i
Purpose, Scope, and Methodology 1
Background 1
Summary of Review Results 5
Lack of Data Quality Objectives 5
QAPP Not Approved 5
QAPP Missing Effective QA Activities 6
Data Validation Requirements Not Met 8
QAPP and Consent Decree Not Followed 11
Recommendations 13
Appendix A - Acronyms 15
Appendix B - Superfund Cleanup Process 17
Appendix C - Report Distribution 19
-------
(This page intentionally left blank.)
-------
Executive Summary
PROBLEM
WE FOUND THAT
Lack of Data Quality
Objectives and QAPP
Approval
QAPP Missing
Effective QA
Activities
Region 9 (the Region) had not monitored the quality of
laboratory data obtained at the Aerojet General Corporation
(Aerojet) Superfund site, although EPA spent nearly $2 million
for oversight of the cleanup. As a result, some of the data
obtained during the Aerojet cleanup was of unknown quality. This
was particularly significant since Aerojet has already spent about
$100 million on studies and cleanup, without fully ensuring the
quality of the underlying data. Since additional data is required to
complete the remedial investigations and feasibility study, it is
important that the quality of the data be verified. In addition, the
data quality requirements of Aerojet's consent decree were not
being fully complied with.
The Region had not
ensured that data
produced for the
Aerojet site was of
known quality.
" rogram
" rioted meoisuremerits. s.prdduce: data *&
':: ''.. ."'<' ." -::." -'.:. '.".-:;.'. .":-.' ::: .: '':'*"*::''':. .:. : ''-.-: : :': ".' :::':v
:::^#7ow^to3^^
'
Specifically, effective
data quality objectives had not been established, although they are
to be used as a basis for the preparation of the Quality Assurance
Project Plan (QAPP). Further, the Region had not approved the
QAPP. Such approval is necessary, since the QAPP is the blueprint
for ensuring laboratory data is of sufficient quality and quantity for
decision-making.
Additionally, the QAPP lacked the following effective quality
assurance activities:
Data validation equivalent to that required by EPA
national functional guidelines;
Independent laboratory audits;
Double-blind performance evaluation samples,
according to a prescribed frequency; and
-------
Data Validation
Requirements Not
Met
NET Data Needs
Close Scrutiny
QAPP and Consent
Decree Not Followed
A requirement to provide magnetic tape data, if
necessary.
While the remedial investigation/feasibility study workplan
contained some data validation requirements, the Region did not
ensure that these requirements were fully implemented. The data
validation requirements were the basis for determining if the data
was of sufficient quality.
One of the laboratories performing sample analyses at the Aerojet
site, National Environmental Testing, Inc. (NET), was suspended
by EPA from any new Federal contract work on March 19, 1996.
We believe that the Region should validate NET's sample analyses
at Aerojet according to EPA national functional guidelines to
ascertain their accuracy.
We found that the
Region was not
monitoring
compliance with the
requirements of the
QAPP. As a result,
the QAPP was not
fully complied with at
the Aerojet site.
; Activities Not Performed But
; ;Required by the QAPP
Annual laboratory audits,
Double-blind performance
evaluation samples. .
ity assurance reports since
April:1993.: ='W-^ :' :
:.. ' : . . ' ,..,.,.:...... .:
(Quality assurance audits..
This deficiency also
affected the consent
decree since it
required that all work
be conducted in
accordance with the
QAPP. Under the 1989 consent decree, Aerojet agreed to
complete a remedial investigation and feasibility study, identify
potential remedies, and monitor contamination.
The QAPP called for annual audits of laboratories by a qualified
chemist who was independent of the chemist performing the
analyses. Since Aerojet used its own laboratory to analyze samples,
we believe that a conflict of interest situation resulted whenever
Aerojet audited its own laboratory.
-------
Problems With
Computer Database
Responsibilities for
Oversight Not Set
"The quality of data is known when
all componeni&^.are thoroughly
documented, such documentation
being verifiable and defensible."
-EPA Order 5360.1
Our review found that
much of the historical
data collected from 1979
to 1988 was of unknown
quality and inaccurately
reflected in Aerojet's
database. Specifically,
the procedure used to
validate historical data did not consider equipment calibrations.
Also, the database included samples taken by superseded sampling
methods, which should have been deleted from the database.
With respect to data collected after 1988, we noted that problems
found by data validation, performed in accordance with the
workplan, were not included in the data base. Further, some of the
data validation requirements established in the workplan were not
met.
Another problem with monitoring data quality was that EPA and
the state regulatory agencies had not established their respective
responsibilities for data quality oversight. As a result, none of the
oversight agencies were evaluating the quality of data.
RECOMMENDATIONS
Our detailed recommendations are included on pages 13 and 14 of
this report. However, in summary, we are recommending that the
Regional Administrator, Region 9, monitor the quality of laboratory
data produced for the Aerojet cleanup by:
1. Making sure the data quality objectives are used to establish the
type, quantity, and quality of data.
2. Documenting approval of the most current quality assurance
project plan.
3. Having Aerojet to revise its QAPP to include requirements for
data validation by EPA national functional guidelines,
independent laboratory audits, double-blind PE samples, and
the maintenance of magnetic media, available to the Region
upon request.
4. Validating the laboratory analyses produced by the National
ui
-------
5. Verifying that the QAPP is complied with.
6. Ensuring Aerojet's database accurately reflects the quality of
data.
7. Delineating regional and state responsibilities for data quality
oversight.
IV
-------
Laboratory Data Quality At The Aerojet
Superfund Site Not Monitored
PURPOSE, SCOPE,
AND
METHODOLOGY
BACKGROUND
We recently completed an audit titled Environmental Data Quality at
POP Superfund Sites in Region 9r Audit Report No. E1SKF5-
0031-5100505, dated September 1995. The audit found that
although serious laboratory problems were identified, the Region did
not strengthen its laboratory data quality oversight program. The
purpose of this special review at the Aerojet Superfund site was to
evaluate the Region's oversight of laboratory data quality at a
responsible party cleanup site.
Our review was conducted between August 15, 1995 and February
9, 1996, and covered management procedures in effect as of August
15, 1995.
Our conclusions are based on interviews with regional and state
officials, and Aerojet management. We also reviewed quality
assurance planning documents, and documentation of quality
assurance activities.
Special reviews are short-term studies of EPA activities. They are
not designed to be detailed audits. Rather, they are information-
gathering studies that seek to identify issue areas for top
management attention. The goal of a special review is to produce
timely, constructive change while minimizing the resources invested
in studying and documenting the issue areas. Due to the limited
scope of this review, it was not an audit according to Government
Auditing Standards issued by the Comptroller General of the United
States. Instead, the work represented a special review which was
conducted in accordance with provisions of OIG Manual Chapter
150.
Aerojet General Corporation is a major Department of Defense
contractor that develops and manufactures solid waste and liquid
rocket motors in Rancho Cordova, California. In 1979,
groundwater contamination was discovered in several private wells
surrounding Aerojet's 8,500-acre facility. The wells contained
volatile organic compounds, including trichloroethylene, a solvent
used in rocket manufacturing. In 1981, Aerojet started
-------
constructing facilities to control the off-site migration of the
contaminated groundwater.
Location of Aerojet General facility
-General Accounting Office graphic
National Priorities
List
Under the Comprehensive Environmental Response,
Compensation, and Liability Act, commonly known as Superfund,
Aerojet must clean up the sites that it polluted, whether or not any
wrongdoing was committed. EPA is responsible for implementing
Superfund laws, including the enforcement of cleanup obligations.
In 1982, EPA listed the Aerojet site on its National Priorities List
as one of the 10 highest risk sites in the United States. The site
has several plumes of contamination, ranging from lh mile to 3
miles in length. The contamination was caused by the use of
hazardous substances dating as far back as the 1950's. One plume
has reached the underground aquifer, and extends under the
nearby American River, moving toward a public drinking well.
-------
.->-. *> . -.., .^sh rr i _' . ;_%
Site of Plume Extending Under the American River
Partial Consent
Decree
Sampling and
Laboratory Analyses
In a 1989 partial consent decree with EPA and the State of
California, Aerojet agreed to: (1) complete a remedial investigation
and feasibility study to determine the nature and extent of the
contamination; (2) identify potential remedies and costs; (3) operate
groundwater treatment facilities to prevent further off-site migration;
and (4) monitor private wells and the American River for
contamination. According to OSWER Directive 9837.2B, the
Region is responsible for verifying that Aerojet has complied with
the consent decree.
Aerojet has spent about $100 million on cleanup-related work, and
has set aside approximately $68 million for a 20-year cleanup
of the site. Aerojet has completed the first part of its remedial
investigation, but has not started the second part nor the feasibility
study. The second part of the remedial investigation includes
collecting additional field data to support the evaluation of the
alternatives. Further, data is collected throughout the remediation
process to ensure cleanup is complete. Thus, considerable
laboratory analysis remains to be completed. Most of the laboratory
analyses have been done by the company's own laboratory.
Environmental data is obtained by sampling contaminated water,
soil, air, and other materials and having the samples analyzed by a
laboratory. EPA Order 5360.1 requires environmental
measurements to be of known quality, verifiable, and defensible.
The quality of environmental data can be degraded by weaknesses in
-------
sampling, laboratory analyses, and the validation of results. Poor
quality data can adversely impact the decision-making process.
There are two major steps to ensure the quality of laboratory
analyses for a site. First, data quality objectives must be determined.
These objectives define how data will be used. Second, a quality
assurance project plan (QAPP) must be developed in accordance
with 40 CFR 300.430. The quality assurance activities necessary to
achieve the data quality objectives are incorporated into a QAPP.
This plan is a blueprint for ensuring laboratory analyses produce data
of sufficient quality and quantity for decision-making throughout the
cleanup process.
In overseeing a responsible party cleanup, EPA is responsible for
verifying work complies with Superfund laws, workplans, quality
assurance project plans, consent decrees, and generally accepted
scientific and engineering methods.
Groundwater Extraction Treatment Facility
The Superfund cleanup process is described in Appendix B of this
report. Generally, a detailed workplan describes, among other
things, the types of sampling activities to be conducted. It is
prepared before a remedial investigation or feasibility study starts.
-------
SUMMARY OF
REVIEW RESULTS
LACK OF DATA
QUALITY
OBJECTIVES
QAPP NOT
APPROVED
datawitt le used for: J) the
assessment of risks to public health and the
environment; 2) the development and initial
screening of remedial alternatives; and 3)
the eventual design of remedial measures,
where necessary,"
-Phase 1 RI Report, Zone 1, Volume 1
August 1993
Our review dis-
closed that the
Region had not
monitored the
quality of
laboratory data
at the Aerojet
site, although $2
million of
oversight costs
were expended.
As a result, we believe that some of the data collected to date
is of unknown quality. This data is significant, since it will be used
for public health risk assessments, developing cleanup alternatives
and remedial design.
In our opinion, the Region's lack of monitoring of the quality of
laboratory data has contributed to: (1) inadequate data quality
objectives; (2) the omission of several quality assurance activities
from the QAPP; and (3) data validation requirements not being met.
We also found that there was a general lack of compliance with the
QAPP and related consent decree. Further, the Region and state
agencies had not established responsibilities for monitoring the
quality of laboratory data.
We found that adequate data quality objectives had not been
developed. Data quality objectives should be the basis for preparing
the QAPP. The process of defining data quality objectives allows
decision makers to define their data requirements and acceptable
levels of decision errors before data is collected. In our opinion,
more attention to the data quality objectives process would help
ensure quality assurance efforts were both efficient and effective.
Both the National Contingency Plan and the Aerojet consent
decree require a QAPP for remedial investigations. Our review
was based on the most recent QAPP, dated February 1993.
Although this was the latest QAPP, the Region had not approved
it. Without this approval, it was unclear whether the Region
-------
QAPP MISSING
EFFECTIVE QA
ACTIVITIES
Data Validation Not
Required by QAPP
agreed that the planned quality assurance activities would achieve
the data quality objectives.
An approved QAPP is important because it is a blueprint for
ensuring laboratory analyses produce data of sufficient quality and
quantity for decision making. Generally, a QAPP describes the
specific quality assurance and quality control activities designed to
achieve the data quality objectives of the project. The Region is
responsible for ensuring the QAPP is acceptable and that
environmental data is of known quality.
The Region did not require that the following quality assurance
(QA) activities to be included in the QAPP.
Data validation according to EPA guidelines.
Independent laboratory audits before and during the
sampling.
Frequent double-blind performance evaluation samples.
Submission of magnetic media, when necessary.
Our 1995 audit of QAPPs at five Federal facilities in Region 9
showed that the above QA activities were particularly effective in
detecting unreliable laboratory data.
Data validation is used to ensure that laboratory data is of known
quality. It involves reviewing data against a set of criteria to
provide assurance that data is adequate for its intended use. The
needed quality of data should be specified before data is collected
in terms of data quality objectives.
Data validation requirements were not specified by the QAPP, but
instead were included in the workplan. We believe data quality
objectives should have been specified in the QAPP rather than in
the workplan, because the QAPP is the tool used to ensure that
data is of sufficient quality and enforceable. The consent decree
also specifies that laboratory work be done according to the
approved QAPP.
A comparison of the control features between the QAPP and the
workplan is shown below.
-------
Data Validation Requirements In QAPP
Provides More Control Features
Control Feature
Serves as the blueprint for QA activities
Reviewed by QA Branch for adequacy
Ensures data is of sufficient quality to meet data
quality objectives
QAPP
Yes
Yes
Yes
Workplan
No
No
No
Laboratory Audits
Not Adequate
Double-Blind PE
Sample Requirement
Could Be
Strengthened
While the QAPP required annual laboratory audits, it did not
require that an audit be performed by an independent activity
before samples were analyzed. It is our opinion that laboratories
should be audited before they analyze samples to help ensure a
laboratory is capable of performing the required analyses. The
audits are designed to identify technical areas which may cause
laboratories to improperly identify or quantify chemicals. The
audits normally evaluate a laboratory's technical expertise,
standing operating procedures, facility and equipment sufficiency,
and possible sources of sample contamination.
The annual audits required by the QAPP are to be performed by a
qualified chemist, independent of the chemist performing the
analyses. We noted that Aerojet uses its own laboratory for some
sample analyses. In our opinion, it is a conflict of interest for
Aerojet to audit its own laboratory.
The QAPP did not adequately
specify the frequency of
performance evaluation (PE)
samples or require an
independent review of the
results. PE samples are
prepared by "spiking" a
known concentration of
chemicals into a contaminate-
free media, such as water or
soil. A laboratory's analyses
of PE samples are used to
evaluate its ability to produce
accurate results.
"The fdouble-blindj PE samples
are made la look as similar to
field samples as possible, and are
submitted as part of afield
sample shipment so that the
iKTravis AFB ModeLQAPP
-------
Magnetic Media
Should Be Available
DATA
VALIDATION
REQUIREMENTS
NOT MET
PE samples can be administered by two methods: "blind" or
"double-blind". When a PE sample is blind, the laboratory is aware
the sample is for PE purposes, but does not know the chemical
concentration levels. When a sample is double-blind, the PE sample
is submitted as part of a field sample shipment. In this situation, the
laboratory is not only unaware of the concentration levels, it is also
unaware that the sample is for PE purposes.
The QAPP provided for periodic submission of double-blind PE
samples. However, it did not specify the frequency of PE samples
or which compounds would be tested. Also, there was no
requirement for an independent review of the results.
The QAPP did not require that magnetic media (tape or disks) be
provided, when necessary. Magnetic tape audits are routinely
conducted by EPA in monitoring its contract laboratories. The tape
audits assist in determining if the laboratory is complying with the
contract, the integrity of the laboratory's computer systems, and the
appropriateness of any software editing. Magnetic tape audits were
instrumental in the Region's detection of fraud at Eureka
Laboratories during the analyses of March Air Force Base samples.
Although it may not be suitable for all types of analytical data, we
believe magnetic media audits should be performed if major
deficiencies are found by other methods, such as data validation or
PE samples. However, in order to perform the audits, the Region
must be able to obtain the magnetic data. This means including the
magnetic tape availability as an integral part of the QAPP.
The Region did not ensure that data validation requirements were
met, although these requirements were the basis for determining if
data was of sufficient quality. Specifically, data validation re-
quirements included in the workplan were not fully implemented.
There were two categories of data used at the Aerojet cleanup:
1. Historical Data. From 1979 to 1988, Aerojet collected
over 29,000 water quality samples, 17,000 soil samples, and
nearly 190,000 hydrologic samples. According to the
consent decree, Aerojet was to evaluate and summarize the
historical information in a scoping report.
2. New Remedial Investigation Data. After the consent
decree was signed, more samples were collected to support
the investigations and studies.
-------
Quality of Historical
Data Unknown
Both categories of
data were stored in
a computerized data
base. This data was
being used to char-
acterize the nature
and extent of chem-
ical substances that
may pose a risk at
Aerojet.
The quality of much of the historical data was unknown and inaccu-
rately reflected in the computer database. In our opinion, this was a
very significant problem since the data base was being used to
characterize the contamination at the site.
"Historical data for groundwater and
soils entered in ihe database were, :
evaluated in concert with the new Rf data
to provide the interpretations presented
in this report,"
Aerojet Site Phase 1 RI Report,
Zone 1, Volume I, August 1993
"The Aerojet [data validation] procedures
^differ from EPA procedures in that they Jo
not evaluate instrument tuning and
calibration or surrogate recovery data. 7»
addition, Aerojet did not report blank results
with samples.:." .';;. ;
: ::: :-Region 9's comments on Aerojet*s Scoping
;::%;'.; . < '-";'; ;;.;:>: 'Report, March 22;:1990
According to the
consent decree,
Aerojet was to
evaluate the
historical data and
assign data quality
levels. Although
the process set out
in the consent
decree was
followed, it lacked
procedures to
evaluate instrument calibrations. Such evaluations are a major
determinant of data quality. In 1990, the Region recognized that
such procedures were not provided for.
Without considering instrument calibrations, the quality of data
obtained is not known. The Region, in its review of the scoping
report, commented on the lack of calibration, and indicated that
other quality checks, such as surrogate recovery data, should be
considered. Thus, we question whether the historical data can be
used for important decisions, such as determining the extent of
contamination, risk assessments, and engineering design.
It should be noted that the scoping report was required by the
consent decree to assess the site conditions and identify the
information necessary to complete the first part of the remedial
investigation and feasibility study.
-------
Region 9 Did Not
Validate Historical
Data
Data Base May
Include
Inappropriate
Samples
Validation
Incomplete for More
Recent Data
According to the Region's review of the scoping report, it planned to
independently validate a portion of the data deemed of high quality
(Level 3 and 4) using EPA procedures. However, this review was
not performed. Thus, there was no independent assessment of the
quality of the historical data.
According to the scoping report, the historical data from 1980 to
1985 included many samples taken by sampling methods superseded
by other methods. The report advised that this data should be
deleted before any statistical evaluations were made. We were
unable to document that this data was deleted. Thus, the data for
some samples may be unreliable.
We found that validation requirements for the more recent data
were not met. The workplan specified that about 25 percent of the
data was to be validated during the first two project stages.
According to data provided by Aerojet, this requirement was
generally met for the water analyses, but not for most soil analyses.
Soil Analysis: Less Than 25 Percent Validated
'....-... Analysis;;:; ;
Base Neutral & Acids
Dioxin & Furan
Mercury
Metals & Minerals
PCBs
Potassium
Petroleum
Hydrocarbons
.;' NoLVof Samples ;
1,202
204
1,164
1,281
178
1,139
719
Percent Validated
12.2
14.2
8.5
7.7
5.6
8.7
0
*j%£apprpxmiaJefy25 percent of the &/a|4;.v.'
-^--'-- *'*'*- validatedihaccorafacW^ :
According to the
workplan, data
validation was the basis
for ensuring data col-
lected was of adequate
quality. Since the
required data validation
was not performed, it is
questionable whether data is of acceptable quality.
^'..W&'^'ify
'' ::'' :''\ - '::::>'.;.;.''.°:^haSel-RI/FS.;wbrIc-planv
::;|-.::; :-/v =l^-':^ ^^^^1,;': Stage 1 jApril 1991
10
-------
Problems Not In-
cluded in Data Base
Data Validation Not
Independent
NET Data Needs
Close Scrutiny
QAPPAND
CONSENT
DECREE NOT
FOLLOWED
Data problems found by validation were not reflected in the data
base. Normally problem data is "flagged" with a code that denotes
it has limited use or cannot be used for any purpose. However, the
data flags were not entered into the computer data base. Thus,
conclusions could be drawn using problem data without being aware
of limitations of the data use.
We also observed that data validation was being performed by
Aerojet, rather than by an independent third party. At our request,
the Region's Quality Assurance Management Section reviewed one
data validation package and found that the validation was done using
the required procedures. The Region also agreed with the
conclusions.
The Region's validation of one package provides some assurance
regarding Aerojet's work. However, we believe that, as long as
Aerojet performs its own data validation, the Region should review
a representative sample of the packages periodically to ensure that
validation reports are reliable.
Aerojet officials informed us that some of the data used for the
cleanup was being analyzed by National Environmental Testing
(NET) Inc., Santa Rosa, California. On March 19, 1996, this
laboratory was suspended from any new Federal contract work. The
suspension action was taken by EPA because the laboratory
allegedly took improper short cuts when analyzing groundwater and
soil samples, and falsified test results.
It is recognized that the laboratory has not been found guilty of any
charges and has a right to respond to the charges. Nonetheless, we
believe it would be prudent for the Region to validate NET's
Aerojet laboratory analyses, according to EPA national functional
guidelines. An audit of NET's magnetic data would also be
advisable.
In addition to the data val-
idation problems, we also
noted that the QAPP was
not fully complied with in
the following areas:
Annual labora-
tory audits were
not competed;
i"Aerojet:shaJI conduct all work
pursuant to this Decree /w
accordance with the approved
QAPP,"
- Partial Consent Decree
between Aero jet, EPA, and
th£ Statelbf California
11
-------
Laboratory Audits
Not Done as Re-
quired
Double-Blind PE
Samples Not Done
Quality Assurance
Reports
Quality Assurance
Audits
Responsibilities for
Oversight Not Set
Double-blind performance evaluation samples were not
done;
Quality assurance reports had not been prepared since
April 1993; and
No quality assurance audits were done.
Since the consent decree required Aerojet to conduct all work in
accordance with the QAPP, the requirements of the consent decree
were also not being followed.
Annual laboratory audits
were not done as required
by the QAPP. The only
laboratory audits performed
were completed in March
1993, after much of the
sampling had already been
completed.
iofEDLfAerojet's
laboratory} would be conducted
M early Feb. *92;" - /
; r Aero jet's Quality Assurance
Report, dated November II, 1991
Aerojet stated that the laboratory audits were postponed until the
1993 QAPP revisions had been made and the appropriate
government agencies had been contacted. We do not believe that
this was a valid reason for postponement, since samples were
continuing to be sent to the laboratory for analyses. In our opinion,
laboratory audits were needed to ensure the laboratories were
properly functioning and following the current QAPP.
Although the QAPP required double-blind PE samples, these
samples were not submitted. The only PE samples done were
single-blind PE samples taken in June and July 1993, although
samples for the remedial investigation have been taken since 1991.
According to the QAPP, Aerojet was to provide quarterly quality
assurance reports to the government oversight agencies upon
request. However, the reports have not been available since 1993.
Further, neither the Region or the other oversight agencies had
requested reports. These reports are important tools for keeping
oversight agencies informed about potential quality assurance issues.
Aerojet did not conduct any QA audits although they were required
annually by the QAPP.
Another problem with monitoring data quality was that the Region
and the state regulatory agencies had not established responsibilities
for monitoring data quality oversight. We contacted both of the
12
-------
state agencies that were parties to the consent decree: the
Department of Toxic Substances Control and the Regional Water
Quality Control Board. Representatives from both agencies
indicated they had not taken any specific steps to monitor data
quality, such as using double-blind performance evaluation samples.
In fact, it was perceived to be EPA's responsibility to ensure data
quality was sufficient. We believe responsibilities for data quality
oversight should be clearly delineated between the oversight
agencies to ensure that proper data quality is obtained for decision
making.
RECOMMENDATIONS We recommend that the Regional Administrator, Region 9 improve
the monitoring of the quality of laboratory data produced for the
Aerojet cleanup by:
1. Making sure the data quality objectives process is used to set
sound data quality objectives and corresponding activities in
quality assurance project plans.
2. Documenting approval of the most current quality assurance
project plan (QAPP).
3. Ensuring that the QAPP includes requirements for:
a. Data validation by EPA's national functional guidelines.
b. Independent third party laboratory audits before samples
are sent to the laboratory and annually thereafter.
c. Double-blind PE samples, including the frequency and
contaminants to be tested. Require these samples to be
evaluated by an independent third party.
d. Magnetic media to be maintained and available to the
Region upon request.
4. Validate laboratory analyses produced by the National
Environmental Testing, Inc. (NET) using EPA national
functional guidelines. Additionally, an audit of NET's magnetic
data should be performed.
5. Verifying that the QAPP is complied with. In making this
determination, the Region should obtain appropriate supporting
13
-------
MANAGEMENT
COMMENTS
documentation, including laboratory audits reports, data validation
reports, quarterly quality assurance reports, and annual quality
assurance audits.
6. Ensuring Aerojet's database accurately reflects the quality of
data by:
a. Independently evaluating historical data categorized as
high-quality.
b. Making sure samples taken by superseded methods are
deleted from the data base.
c. Insuring required data validation is completed and
problem data is flagged in the data base.
d. Reviewing a representative sample of data validation
reports to ensure their reliability.
7. Delineating regional and state responsibilities for data quality
oversight.
On February 21, 1996, we issued a position paper to the Region
summarizing the results of our special review. The Region's March
14, 1996 response provided the following comments which we
consider relevant to our final special review report.
Although there were some quality assurance problems
with data, the Region did not consider the data
completely useless. According to the Region, it is often
difficult, if not impossible, to fully validate historical data.
This does not necessarily mean that the data has no
value.
The Region's Quality Assurance Management section
had reviewed Aerojet's QAPP in 1993, but did not
receive a revised plan for further review.
OIG EVALUATION
The Region's comments are acknowledged, and have been
considered in our changes to the final report. It remains our
conclusion that the quality of much of the historical data is
unknown.
14
-------
APPENDIX A
Acronyms
Acronym
FS
OSWER
PCB
PE
QA
QAPP
RI
Name
Feasibility study
EPA Office of Solid Waste and Emergency Response
Polychlorinated biphenyls
Performance evaluation (samples)
Quality assurance
Quality assurance project plan
Remedial investigation
15
-------
(This page intentionally left blank.)
16
-------
APPENDIX B
Superfund Cleanup Process
Preliminary
Assessment
Site
Assessment
The initial stage of the cleanup program is an installation-wide
study to determine if sites are present that pose hazards to public
health or the environment. Available information is collected on
the source, nature, extent, and magnitude of actual and potential
hazardous substance releases at sites on the installation.
The next step consists of sampling and analysis to determine the
existence of actual site contamination. Information gathered is
used to evaluate the site and determine the response action
needed. Uncontaminated sites do not proceed to later stages of
the process.
Remedial
Investigation
Feasibility
Study
Remedial
Design
Remedial investigation may include a variety of site investigative,
sampling, and analytical activities to determine the nature, extent,
and significance of the contamination. The focus of the
evaluation is determining the risk to the general population posed
by the contamination. Generally, a workplan detailing the work
to be conducted, including the types of data collection activities
and analytical methods to evaluate sampling data, is prepared
before the remedial investigation and feasibility study are
conducted.
Concurrent with the remedial investigations, feasibility studies
are conducted to evaluate remedial action options for the site to
determine which would provide the protection required. The
feasibility study emphasizes data analysis and is generally
performed in an interactive fashion with the remedial
investigation using data gathered during the remedial
investigation.
Detailed design plans for the remedial action option chosen are
prepared.
17
-------
Remedial The implementation of the chosen remedial option is
Action implemented.
Interim Remedial actions can be taken at any time during the cleanup
Remedial process to protect public health or to control contaminant releases
Action to ^e environment-
Remedy in The remedial action is functioning properly and performing as
Place and designed. These include such actions as the operation of pump
Functionina an^ treat systems that will take decades to complete cleanup.
18
-------
APPENDIX C
Report Distribution
Distribution
Individual or Activity
Office of Inspector
General
Inspector General (2410)
Deputy Inspector General (2410)
Assistant Inspector General for Audit (2410)
EPA Headquarters
Assistant Administrator for Enforcement and Compliance Assurance
(2211)
Assistant Administrator for Solid Waste and Emergency Response
(5101)
Associate Administrator for Regional Operations and State/Local
Relations (1501)
Agency Followup Official, ATTN: Assistant Administrator for
Administration and Resource Management (208)
Agency Followup Coordinator, Attn: Director, Resource
Management Division (3304)
Region 9
Regional Administrator
Director, Hazardous Waste Management Division (H-l)
Assistant Regional Administrator, Office of Policy and Management
(P-l)
19
------- |