V
United States Office of Solid Waste and
Environmental Protection Emergency Response
Agency Washington, D.C. 20460
Superfund
® EDA Guidance for
w crM Data Useability in
Risk Assessment
Office of Emergency and Remedial Response
HazardousSite Evaluation Division, OS-230 Quick
9285.7-05FS
September 1990
Reference Fact Sheet
EPA is establishing national guidance for minimum data quality requirements to optimize the useability of data
collected under the Comprehensive Environmental Response, Compensation, and Liability Act of 1980 (CERCLA).
Data useability is the process of assuring or determining that the quality of data generated meets the intended use. This
guidance is designed to provide data users with a nationally-consistent basis for making decisions about the minimum
quality and quantity of environmental analytical data that are sufficient to support Superfund decisions, regardless of
which parties conduct the investigation. EPA workgroups are defining the current uses and associated quality
requirements of Superfund data, and developing minimum requirements for each data use category. Data use categories
include site assessments, risk assessments, and removal and remedy selection for remedial and enforcement actions.
Detailed data useability guidance is being prepared for each data use category; risk assessment is the prototype.
This fact sheet provides an overview of Guidance for Data Useability in Risk Assessment (EPA/540/G-90/008),
highli ghts key points of the manual, and details where additional guidance is found. Copies of the manual can be obtained
by calling EPA's Center for Environmental Research at 513-569-7652 (FTS 684-7562).
What Is This Manual?
The guidance manual provides direction for plan-
ning and assessing analytical data collection activities
for the baseline human health risk assessment, con-
ducted as part of the Remedial Investigation (RI) proc-
ess.
The manual provides guidance on the following:
• How to design RI sampling and analytical ac-
tivities that meet the data quality and data
quantity needs of risk assessors.
• Procedures for assessing the useability of the
data obtained in the RI.
• Options for combining data of varying levels of
quality from different sources and incorporat-
ing them into the risk assessment.
• Procedures for determining the degree of con-
fidence in the risk assessment based on the un-
certainty in the environmental analytical data.
• Guidelines for timing the execution of the vari-
ous activities.
• Appendices requested by risk assessors and
RPMs that assist in selecting analytical meth-
ods to meet required detection limits.
The manual complements guidance provided in
Risk Assessment Guidance for Superfund Volume I
Human Health Evaluation Manual (Part A) (RAGS),
Guidance for Conducting Remedial Investigations and
Feasibility Studies under CERCLA, and Data Quality
Objectives for Remedial Response Activities. RAGS
provides the framework for making data quality assess-
ments in baseline risk assessments. This manual supple-
ments and strengthens important technical details of this
framework by providing minimum requirements for the
environmental analytical data used in baseline risk as-
sessments. As such, it complements and builds upon
Agency guidance for the development and use of data
quality objectives in all data collection activities. The
guidance does not address the use of environmental data
for purposes other than baseline risk assessments for
human health.
Printed on Recycled Paper
-------
Who Will Use This Manual?
The guidance manual is addressed primarily to
RPMs who have the principal responsibility for leading
the data collection and assessment activities that support
the human health risk assessment. Assistance is also
provided to risk assessors who must effectively commu-
nicate their data needs to RPMs and use the data
provided to them. Chemists, quality assurance special-
ists, hydrogeologists, statisticians and other technical
experts involved in the RI process can use this guidance
to optimize the useability of data collected in the RI for
use in baseline risk assessments.
Data collected only with a
view towards identifying the
"nature and extent" of contami-
nation at a Superfund site may
result in rejection of the risk as-
sessment or the need for an
additional round of sampling to
supporttheriskassessmenL Ac-
cordingly, risk assessors must
be an integral part of the RI
planning process to ensure that
the environmental analytical
data collected during the RI meet
their needs. They should work
closely with the RPM to iden-
tify and recommend sampling
designs and analytical methods
that will maximize the quality
of the data collected for the
baseline risk assessment for
human health within the site-
related and budgetary con-
straints of the RI.
RPMs oversee the prepa-
ration of work plans and sam-
pling and analysis plans (S APs)
for RI data collection. It is im-
portant for them to understand
the types, quality, and quantity
of data needed by risk asses-
sors, and the impact that their
data collection and analysis de-
cisions have on the level of
certainty ofbaseline risk assess-
ment for human health. This
manual provides guidance in
these areas. Highlight 1 sum-
marizes each chapter of the
manual.
DATA QUALITY ISSUES IN RISK
ASSESSMENT
Five basic environmental data quality issues are
frequently encountered in risk assessment These issues
affect both the planning for and the assessment of ana-
lytical data for use in the RI risk assessment.
• Data sources
Practical tradeoffs among detection limits, re-
sponse time, documentation, analytical costs
and level of confidence should be considered
prior to selecting analytical methods and serv-
Hlghlightl
ORGANIZATION OF THE MANUAL
Chapter 1
Introduction and Background
Presents critical data useability Issues.
Specifies audience to be primarily RPMs and
risk assessors.
Defines scope and specifies organization of manual.
Chapter:
The Rlak Aaaeaament Process
Explains the elements of a risk assessment and the Impact of
analytical data quality on each element.
• Defines the uncertainties In the risk assessment process.
• Descrbes the rotes of the risk assessor, RPM and others
Involved with the risk assessment planning and assessment
process.
Chapters
Criteria for Evaluating Data Useability In Baseline
Rlak Asaeaaments
• Defines six criteria for assessing data useability.
• Applies criteria to sampling and analytical Issues.
Chapter 4
Steps for Planning for the Acquisition of uaeable
Environmental Data In Baseline Rlak Aaaeaamenta
• Provides guidelines for designing sampling plans and selecting
analytical methods.
• Provides worksheets to support sampling plan design and
analytical method selection.
Chapters
Aaaesament of Environmental Data for
Uaeablllty In Baseline Rlak Aaaeaamenta
• Descrbes minimum requirements for useable data.
• Explains how to determine actual performance compared to
objectives.
• Recommends corrective actions for critical data not meeting
objectives.
• Descrbes options for combining data from different
sources and of varying quality Into the risk assessment.
Chapter 6
Application of Data to Rlak Assessments
• Provides procedures to determine the uncertainty of the
analytical data.
Explains how to distinguish site from background levels of
contamination and determine presence (absence) of
contamination.
Discusses how to characterize exposure pathways.
APPENDICES
Provide technical reference tables for sampling and analysis.
Descrbe data review packages and meanings of selected
data qualifiers.
-------
ice providers. Many analytical sources are able
to serve the needs of the Superfund program.
RPMsandrisk assessors should seek the source
of data that best meets the data quality needs of
the risk assessment
Advances in technology, accompanied by ap-
propriate quality control measures, allow field
data to be used more frequently and with more
confidence in risk assessments. By using field
data, RPMs can increase sample numbers to
better characterize the site, provided accept-
able data quality is maintained.
Detection limits
Selecting the analytical method for optimal de-
tection limits is
fundamental to
the useability of
analytical data in
risk assess-
ments. In addi-
tion, the type of
detection limit,
such as method
detection limit
(MDL) or
sample quantita-
tionlimit(SQL),
used in making
data quality de-
cisions affects
the confidence
of the risk as-
sessment
Qualified data
Qualified data
must be appro-
priately used in
risk assess-
ments. Data are
almost always
useable in the
risk assessment
process, as long
as the level of un-
certainty in the
data and its im-
pact on the level
of confidence of
the risk assess-
ment are thor-
oughly ex-
plained.
Background samples
Analytical data reported near method detection
limits and sample results qualified during data
review complicate the use of background sample
data to determine site contamination. Planning
for the collection of a sufficient number of
background samples increases the confidence
in decisions about the presence or absence of
site contamination.
Consistency in data collection
Consistency must be maintained among all
parties conducting Superfund baseline risk
assessments for human health. The guidance
provided in RAGS and this manual ensures that
Highlight 2
IMPACT OF ANALYTICAL ISSUES
ON RISK ASSESSMENT
ANALYTICAL
ISSUE
Chemicals of
Potential Concern
(3.3.1)
Library Search/
Tentatively Identified
Compounds
(3.3.2)
Identification and
Quantitation
(3.3.3)
Detection Limits
(3.3.4)
Media Variability
(3.3.5)
Sample Preparation
(3.3.6)
Fixed Laboratory
vs. Field Analyses
(3.3.7)
Laboratory Performance
Problems
(3.3.8)
IMPORTANCE
Chemicals of potential
lexicological significance may be
omitted.
Identification and quantitation do
not have high confidence.
False negatives may occur when
analytes are present near the
method detection limit
Risk levels may be at
concentrations lower than
measureable.
Variability and bias may be
introduced to analytical
measurements.
Variability and bias may be
Introduced to analytical
measurements.
Tradeoffs required with regard to
speed, precision, accuracy,
personnel, Identification,
quantitation and detection limits.
Quality of data may be
compromised.
SUGGESTED ACTION
Examine existing data and site history
for industry-specific wastes to
determine analytes for measurement
Perform broad spectrum analysis.
Be prepared to request further
analyses If potentially toxic
compounds are discovered during
screening. Compare results from
multiple samplings or historical data.
Use technique with definitive
identification (e.g., GC-MS).
Alternatively, use technique with
definitive identification first, followed
by another technique (e.g., GC) to
achieve lower quantitation limits.
Review available methods for
appropriate detection limit
Use environmental samples as QC
samples to determine recovery and
reproduclbility in the sample media.
Select analytical methods based on
sample medium and strengths of the
sample preparation technique.
Consider options and set priorities.
Select experienced laboratory and
maintain communication.
-------
baseline risk assessments for hu-
man health are conducted consis-
tently while being protective of
public health.
Guidance for Data Useability in Risk
Assessment addresses these issues in detail.
Procedures, minimum requirements, andcor-
rective actions are provided to resolve the
impact the issues have on the confidence in
the risk assessment.
MAKING DECISIONS WITH
ENVIRONMENTAL DATA
Four fundamental decisions for risk as-
sessment are made with the data acquired
during the Remedial Investigation.
• What contamination is present and
at what levels?
The selection of analytical meth-
ods, laboratory performance, and
type and level of data review affect
the probability of false negatives
and false positives for both site and
background samples.
• Are site concentrations sufficiently
different from background?
Site concentrations must be distin-
guished from background levels to
support an evaluation of increased
risk for human health on the basis
of the site contamination. Both sampling
and analytical designs are considered.
• Are all exposure pathways identified and
examined?
All exposure pathways must be identified
and exposure routes examined. Decisions
concerning exposure pathways primarily
involve identifying and sampling media
of concern. Sampling must be representa-
tive.
• Are all exposure pathways fully charac-
terized?
The final decision involves the characteri-
zation of exposure pathways. Sampling
must be representative and satisfy per-
formance objectives determined during
the planning process. A broad spectrum
analysis must be available in order to
characterize the pathways and avoid false
negatives.
Highlight 3a
AUTOMATED SYSTEMS to SUPPORT
ENVIRONMENTAL SAMPLING
SYSTEM
Data Quality Objective
fTraining) - Expert
System
ESES
Environmental Sampling
(Plan Design) - Expert
System
GEO- EAS
Geostatlstlcal
Environmental
Assessment Software
SCOUT
Muttivartate Statistical
Analysis Package
ASSESS
Assessment of Errors In
Sampling of Soils
EPA CONTACT
Dean Neptune
USEPA
Quality Assurance
Management Staff
(202) 475-9464
Jeff Van Ee
Exposure Assessment Div.
USEPA, EMSL-LV
(702) 798-2367
Evan Englund
Exposure Assessment Div.
USEPA, EMSL-LV
(702) 798-2248
Jeff Van Ee
Exposure Assessment Div.
USEPA, EMSL-LV
(702) 798-2367
Jeff Van Ee
Exposure Assessment Div.
USEPA. EMSL-LV
(702) 798-2367
DESCRIPTION
Training system designed to assist
In planning of environmental
Investigations based on data
quality objective process.
Expert system designed to assist
In planning sample collection.
Includes models that address
statistical design, QC, sampling
procedures, sample handling,
budget, and documentation.
Current system addresses metal
contaminants In a soil matrix.
(Expanded application under
development, contact EMSL-LV.)
Collection of software tools for
two-dimensional geostatistlcal
analysis of spatially distributed
data points. Programs include file
management, contour mapping,
krlging, and vartogramanalysis.
A collection of statistical programs
that accept GEO-EAS files for
muftlvarlate analysis.
System designed to assist in
assessment of error in sampling of
soils. Estimates measurement
error variance components.
Presents scatter plots of quality
control data and error plots to
assist In determining the
appropriate amount of quality
control samples.
1 All systems will run on any IBM compatfole PC AT with 640K RAM (minimum).
A fixed disk Is recommended.
Uncertainty in chemical identification and quanti-
tation is determined based on decisions made during
planning. This analytical data uncertainty affects the
level of confidence of the final risk assessment.
Using Criteria for Planning and
Assessing Data Useability
Six criteria assure the useability of environmental
data in risk assessments. The criteria are:
• Data sources
• Documentation
• Analytical methods and detection limits
• Data quality indicators
• Data review
• Reports to risk assessors.
The manual explains how to use these criteria to
plan data collection efforts that maximize the useability
-------
Highlight 3b
AUTOMATED SYSTEMS f O SUPPORT
METHOD SELECTION
SYSTEM
List of Lists
Smart Method
Index
Geophysical
Techniques
Expert System
EPA Sampling
and Analysis
Data Base
CONTACT
W. A. Telllard
USEPA
Office of Water
(202)382-7120
John Nocerino
Quality Assurance Dlv.
USEPA. EMSL-LV
(202)798-2110
Aldo Maggella
Advanced Monitoring
Div.
USEPA, EMSL-LV
(202) 798-2254
Lewis Publishers
1-800-272-7737
DESCRIPTION
An automated Sorting and
selection software package that
currently contains 150 methods
and 1 ,700 analytes. These are
cross-referenced to facilitate
selection based on required
needs (e.g., analyte detection
limit, Instrument).
Natural language expert system
prototype that provides
Interactive queries of databases
cross-referenced by method,
analyte, and performance
features.
An expert system that suggests
and ranks geophysical
techniques, Including soil gas, for
applicability of use based on
site-specific characteristics.
A three-volume set of diskettes
and a printed manual provides a
search of sampling and analytical
method summaries from a
menu-driven program of 150
EPA-approved methods. The
database can be searched by
method, analyte, matrix, and
various quality assurance
considerations.
1 All systems will run on any IBM compattole PC AT with 640K RAM (minimum).
A fixed disk is recommended.
of environmental analytical data in baseline risk assess-
ments. Highlight 2 details analytical issues which im-
pact the risk assessment and describes actions needed to
resolve them. Guidance tools include a Sample Design
Selection Worksheet and a Method Selection Work-
sheet. Step-by-step instructions for using the worksheets
assist the RPM or risk assessor in planning RI sample
collection and analysis to produce data meeting risk as-
sessment needs. Highlights 3a and 3b list automated
systems which may help in this planning. Regional En-
vironmental Services Divisions (ESD) can also provide
assistance.
ASSESSING ENVIRONMENTAL DATA
FOR USEABILITY
Conducting the Data Assessment
Examine (he data, documentation, and reports to
determine if performance is within the limits required by
the planning objectives. If no performance objectives
have been specified or the specification is incom-
plete, the minimum acceptable requirements for
the data useability criterion should be used for the
minimum performance objectives. The manual
presents minimum requirements for each data usea-
bility criterion. In evaluating the criteria, perform
the following activities:
• Identify or determine minimum data require-
ments and performance objectives.
• Determineactualperformancecomparedtoob-
jectives.
• Determine and execute any corrective action
required.
Take corrective action when actual perform-
ance fails to meet the objectives for data critical to
the risk assessment. Highlight 4 gives several cor-
rective action options for resolving problems with
data not meeting performance requirements.
Highlight 4
CORRECTIVE ACTION OPTIONS
WHEN DATA DO NOT MEET
PERFORMANCE OBJECTIVES
Retrieve missing information.
Resolve technical or procedural problems
by requesting additional explanation or
clarification from the technical team.
Request reanalysis of sample(s) from
extract.
Request construction and re-interpretation
of analytical results from the laboratory or
project chemist.
Request additional sample collection and
analysis for site or background
characterization.
Model potential impact on risk assessment
certainty using sensitivity analysis to
determine range of effect.
Adjust or impute data based on approved
default options and imputation routines.
Qualify or reject data for use in risk
assessment.
-------
Highlight 5
MINIMUM REQUIREMENTS, IMPACT, AND CORRECTIVE
ACTIONS FOR DATA USEABILITY CRITERIA
DATA USEABILITY
CRITERION
5.1 Reports To Risk
Assessor
5.2 Documentation
5.3 Data Sources
5.4 Analytical
Method and
Detection Limit
5.5 Data Review
5.6 Data Quality
Indicators
MINIMUM
REQUIREMENT
• Site description
• Sample design with sample
locations
• Analytical method and
detection limit
• Results on per-sample basis,
qualified for analytical
limitations
• Sample-specific quantitation
limits (SQLs) and detection
limit for non-detects
• Field conditions for media
and environment
• Preliminary reports
• Sample results related to
geographic location
(chain-of-custody records,
SOPs, field and analytical
records)
• Analytical data results for
one sample per medium
per exposure pathway
• Broad spectrum analysis for
one sample per medium
per exposure pathway
• Reid measurements data
for media and environment
• Routine methods used for
critical samples and
chemicals of potential
concern
• Detection limit less than
20% of concentration of
concern
• Correctness of analytical
results reviewed
• Sampling variability
quantified for each analyte
> QC samples required to
identify and quantify
precision and accuracy
• Sampling and
analytical precision and
accuracy quantified
IMPACT ON RISK
ASSESSMENT
• Unable to
perform
quantitative risk
assessment
• Unable to assess
exposure
pathways
• Unable to identify
appropriate
concentration for
exposure units
• Potential for false
negatives and
positives
• Increased
variability in
exposure
modeling
• Invalidated
precision and
accuracy
• False negatives
• Potential for false
negatives
or false positives
• Increased
variability and
bias due to
analytical
process,
calculation or
transcription
errors
• Unable to
quantify
confidence
levels for
uncertainty
• Potential for false
negatives
or false positives
CORRECTIVE i
ACTION
• Request missing
information
• Perform qualitative
risk assessment
• Request locations
identified
• Resampling
• Resampling or
reanalysis for
critical samples
• Reanalysis
• Resampling and
analysis for critical
samples
• Documented
caveats for non-
critical samples
• Perform data
review
• Resampling for
critical samples
• Perform qualitative
risk assessment
• Perform
quantitative
risk assessment
for non-critical
samples with
documented
discussion of
potential limitations
-------
Organizing the Data
Assessment
For each performance measure within
an assessment phase, determine whether the
actual performance for the data involved is
satisfactory, questionable, or unsatisfactory.
The manual includes a worksheet to assist in
applying the useability criteria to the data.
For each criterion, the worksheet requires a
decision to be made: accept, accept with
qualification, or reject the information in-
volved for use in the risk assessment. Record
the justification for each decision on the
worksheet. Highlight 5 summarizes mini-
mum requirements for each data useability
criterion, lists its potential impact on the risk
assessment, and identifies corrective actions
that may be taken.
APPLYING DATA TO RISK
ASSESSMENT
The level of confidence associated with
the actual data that are obtained affects the
ability to answer the four basic questions
being addressed in the risk assessment proc-
ess, as shown in Highlight 6. The final sec-
tions of the manual provide procedures to be
followed to determine the level of certainty
for each decision, given the results of the
assessment of performance measures.
Uncertainty results from each of the components of
risk assessment, and the results for each component
should be presented with an explicit statement of the
degree of confidence. These measures are the bases for
the estimation of the degree of confidence in the risk as-
sessment.
NEED MORE HELP?
Superfund Toxic Integration Coordinators are
located in each region. Questions regarding site-specific
Superfund risk assessment issues should be referred to
the appropriate individuals listed in Highlight 7. The
Toxics Integration Branch, Office of Energy and Reme-
Hlghllght 6
UNCERTAINTY IN DATA COLLECTION AND EVALUATION DECISIONS
AFFECTS THE CERTAINTY OF THE RISK ASSESSMENT
Are Site
Concentrations
Sufficiently
Different From
Background
Arannnmt
Quantitative
(Uncertainty
evenly stated);
t
Quantitative
(Uncertainty not
known)
t
Qualitative (No
uncertainty
estimate)
dial Response (OERR), may also be contacted at (202)
475-9486 or FTS 475-9486 for technical information
sources and assistance with this guidance. Potential
sources for technical assistance are Regional ESDs and
Quality Assurance Officers. EMSL/LV may be a source
for assistance on sampling or statistical issues.
Where to Obtain the Guidance Manual
To order a copy of the manual, call or write toEPA's
Center for Environmental Research at 513-569-7652 or
FTS 684-7562.
Additional copies of this fact sheet can be obtained
by calling the Superfund Docket 202-382-6940 or FTS
382-6940.
-------
Highlight 7
REGIONAL TOXICS INTEGRATION COORDINATORS
Region Name and Address Phone Number
I Sarah Levinson FTS 833-1504
Waste Management Division (HSS-CAN-7) 617-573-9662
EPA Region I
John F. Kennedy Federal Building
Boston, MA 02203
II Peter Grevatt FTS 264-8775
Program Support Branch 212-264-6323
ERR Division
EPA Region II
26 Federal Plaza
New York, NY 10278
III Richard Brunker FTS 597-0804
Hazardous Waste 215-597-0804
Management Division (3HW15)
EPA Region III
841 Chestnut Street
Philadelphia, PA 19107
IV Elmer Akin FTS 257-1586
Waste Management Division 404-347-1586
EPA Region IV
345 Courtiand Street, NE
Atlanta, GA 30365
V Steve Ostrodka FTS 886-3011
Technical Support Unit (5HSM-12) 312-886-3011
EPA Region V
230 South Dearborn Street
Chicago, IL 60604
VI Jon Rauscher FTS 255-2198
EPA Region VI (6H-SR) 214-655-2198
First Interstate Bank Tower
1445 Ross Avenue
Dallas, TX 75202-2733
VII David Crawford FTS 276-7052*
EPA Region VII 913-551 -7052
726 Minnesota Avenue
Kansas City, KS 66101
VIII Chris Weis FTS 330-7655
EPA Region VIII (8HWM-SR) 303-294-7655
999 18th Street, Suite 500
Denver, CO 80202-2405
IX Gerald Hiatt FTS 484-1914
Technical Support Section (H-8-4) 415-381 -8917
Superfund Program
EPA Region IX
1235 Mission Street
San Francisco, CA 94103
X PatCirone FTS 399-2138
EPA Region X (ES-098) 206-442-1597
1200 Sixth Avenue
Seattle, WA 98101
* Caller must have FTS 2000. If not, use commercial number.
------- |