United States
Environmental Protection
Agency
Office of Solid Waste and
Emergency Response
Washington, D.C. 20460
92S5.7-05FS
September 19SO
Superfund
&EPA
Guidance for
Data Useability in
Risk Assessment
Off Iceol Emergency and Remedial Response
Hazardous Site Evaluation Division, OS-23O
Quick Reference Fact Sheet
EPA is establishing national guidance for minimum data quality requirements to optimize the useability of dala
collected under the Comprehensive Environmental Response, Compensation, and Liability Act of 1980 (CMRCLA),
Data useability is the process of assuring or determining that the quality of data generated meets the intended use. This
guidance is designed to provide data users with a nationally-consistent basis for making decisions about the minimum
quality and quantity of environmental analytical data that are sufficient to support Superfund decisions, regardless of
which parties conduct the investigation. EPA workgroups are defining the current uses and associated quality
requirements of Superfund data, and developingminimumrequirements for each dala use category. Data use categories
include site assessments, risk assessments, and removal and remedy selection for remedial and enforcement actions,
Detailed data useability guidance is being prepared for each data use category; risk assessment is the prototype.
This fact sheet provides an overview of Guidance for Data Useability in Risk Assessment (EPA/54G/G-90/QQ8),
highlightskey points of the manual, and details where additional guidanceis found. Copies of the manual can beobtained
by calling EPA's Center for Environmental Research at 513-569-7652 (FTS 684-7562).
What Is This Manual?
The guidance manual provides direction for plan-
ning and assessing analytical data collection activities
for the baseline human health risk assessment, con-
ducted as part of the Remedial Investigation (RI) proc-
ess.
The manual provides guidance on the following;
• How to design RI sampling and analytical ac-
tivities that meet the data quality and data
quantity needs of risk assessors.
* Procedures for assessing the useability of the
data obtained in the RI.
* Options forcombining data of varying levels of
quality from different sources and incorporat-
ing them into the risk assessment
« Procedures for determining the degree of con-
fidence in the risk assessment based on the un-
certainty in the environmental analytical data,
* Guidelines for timing the execution of the vari-
ous activities.
* Appendices requested by risk assessors and
RPMs that assist in selecting analytical meth-
ods to meetrequired detection limits.
The manual complements guidance provided in
Risk Assessment Guidance for Superfund Volume I
Human Health Evaluation Manual (Part A) (RAGS),
Guidance for Conducting Remedial Investigations and
Feasibility Studies under CERCLA, and Data Quality
Objectives for Remedial Response Activities. RAGS
provides the framework for making data quality assess-
mentsin baselinerisk assessments. TMsmanual supple-
ments and strengthens important technical details of this
framework by providing minimum requirements for the
environmental analytical data used in baseline risk as-
sessments. As such, it complements and builds upon
Agency guidance for the development and use of date
quality objectives in all data collection activities. The
guidance does not address the use of environmental data
for purposes other than baseline risk assessments for
human health.
Printed on Roqecled Paper
-------
Who Will Use This Manual?
The guidance manual is addressed primarily to
RPMs who have the principal responsibility for leading
the data collection and assessment activities that support
the human health risk assessment. Assistance is also
provided toriskassesscns who must effectively commu-
nicate their data needs to RPMs and use the data
provided to them. Chemists, quality assurance special-
ists, hydrogeologists, statisticians and other technical
experts involved in the RJ process can use this guidance
to optimize the useabffity of data collected in the RI for
use in baseline risk assessments.
Data collected only with a
view towards identifying the
"nature and extent" of contami-
nation at a Superfund site may
result in rejection of the risk as-
sessment or the need for an
additional round of sampling to
support the risk assessment. Ac-
cordingly, risk assessors must
be an integral part of the RI
planning process to ensure that
the environmental analytical
daia collected during theRl meet
their needs. They should work
closely with the RPM to iden-
tify and recommend sampling
designs and analytical methods
that will maximize the quality
of the data collected for the
baseline risk assessment for
human health within the site-
related and budgetary con-
straints of the RI.
*
1; RPMs oversee the prepa-
ration of work plans and sam-
pling and analysis plans (S APs)
for RI data collection. It is im-
portant for them to understand
the types, quality, and quantity
of data needed by risk asses-
sors, and the impact that their
data collection and analysis de-
cisions have on the level of
certainty ofbaseline risk assess-
ment for human health. This
manual provides guidance in
these areas. Highlight 1 sum-
marizes each chapter of the
manual.
DATA QUALITY ISSUES IN RISK
ASSESSMENT
Five basic environmental data quality issues are
frequently encountered in risk assessment. These issues
affect both the planning for and the assessment of ana-
lytical data for use in the RI risk assessment.
• Data sources
Practical tradeoffs among detection limits, re-
sponse time, documentation, analytical costs
and level of confidence should be considered
prior to selecting analytical methods and serv-
HlghllghU
ORGANIZATION OF THE MANUAL
ChafMtrl
Introduction »nd Background
Presents a
f paellas audience to be priority RPMS ana
itok assessors,
Daflnm Mope and specifies organization of manual.
Ch.pt.f2
The Htak A**n*n
ntProc***
Explains the elements of m. risk assessment and Uia impact cH
(nalyilcal data, quality on oaeh element.
Dellnaa the uncertalntiat fci the risk assessment process,
DtBcrbfts the rotea of the risk assessor, RPM and Dinars
Involved wtih the rtt asmsmarrt planning and assessment
Criteria for ivdunUng D*ti UMOJMltty in B«e1ine
m*k AMMsmanta
* DeltOBs ah criteria tor assessing data uaeabflr^..
• Applas criteria to «air?l Ing and an ar/ilcel bsuas.
Chap»M
Stop* for Planning far the tequMHon M WesMe
Environment!) D«t* In Baaelliie Hl»k Awaonienti
» Pravdss guidsVnws for designing 6arrpll",g plans and sa'acilng
anaJytliaJ methods.
• ProyWosiwfkBhaeistoEupport sampling pl«n design and
analylM nwhod selection,
CtupMrS
HUMMHWM of EnrimumMU D»L» for
ItaMUlHy In eaMltlW Rtak AMW»morrt»
• Describes minimum rsqulrsments for usaabla dsti.
* Explains how to detarrnlna actusl perfomianca comptnxi to
objacivoc.
• Recommends coifeclive actions for crBloal dels not meeting
objectlvas,
' DeesrbM options for combining data from different
sources and of varying quality Into tha riaX assessment.
Chapters
Appi1s»Uon ol Datt ta Bt«k Amtuncrrtm
* ProvUasprocaduresKi determine the urtcenalniyaf the
Exp'alns how ta distinguish sto (ram background teve's ol
contamination and determine presence (absence) of
coma/nkiata.
Dlicueaas how to chaiaclarlzeeipaeupe pathways,
APPENDICES
* P-ovido tecnnicial reference tables for sampling and arayel
* Deserbe data revbw packages and mowings of selected
data qualifiers.
-------
ice providers. Many analytical sources are able
to serve the needs of the Superfand program.
RPMs and risk asses sore should seek the source
of data that best meets the data quality needs of
the risk assessment
Advances in technology, accompanied by ap-
propriate quality control measures, allow field
dala to be used more frequently and with more
confidence in risk assessments. By using field
data, RPMs can increase sample numbers to
better characterize fiie site, provided accept-
able data quality is maintained.
Detection limits
Selecting the analytical method for optimal de-
tection limits is
fundamental to
the useability of
analytical data in
risk assess-
ments. In addi-
tion, the type of
detection limit,
such as method
detection limit
(MDL) or
samplequantita-
donlimii(SQL),
used in making
data quality de-
cisions affects
the confidence
of the risk as-
sessment.
Qualified data
Qualified data
must be appro-
priately used in
risk assess-
ments. Data arc
almost always
useablc in the
risk assessment
process, as long
as the levelof un-
certainty in the
data and its im-
pact on the level
of confidence of
the risk assess-
ment are thor-
oughly ex-
plained.
Background samples
Analytical dalareportednear method detection
limits and sample results qualified during data
review complicate the useof background sample
data to determine site contamination. Planning
for the collection of a sufficient number of
background samples increases the confidence
in decisions about the presence or absence of
site contamination.
Consistency in data collection
Consistency must be maintained among all
parties conducting Swperfund baseline risk
assessments for human health. The guidance
provided in RAGS and this manual ensures that
Highlight 2
IMPACT OF ANALYTICAL ISSUES
ON RISK ASSESSMENT
ANALmCAL
ISSUE
Chemicals of
Potential Concern
(3.3.1)
Library Search/
Tentatively Identified
Cornpoynds
P-3,2)
Identification and
Quanttatfon
(3,3.3)
Detection Limits
(3,3.4)
Media Variability
(3.3.5)
Sample Preparation
(3JJ)
Fixed Laboratory
vs; Raid Analyses
(3.3.7)
Laboratory Performance
Problems
(3.3.8)
IMPORTANCE
Chemicals of potential
toxicologies! significance may be
omtttad.
identification and quanttaten do
not have high confidence.
false negatives may occur when
arialytes are present near the
method detection limit
FM levels may be at
concentrations lower ihan
meesureable.
Variability and bias may be
introduced to analytics!
measurements.
Variability and bias may be
introduced to analytical
measurements.
Tradeoffs required with regard to
speed, precision, accuracy,
personnel, identification,
quanttetico and detection limits.
Quality of data may be .
compromised.
SUGGESTED ACTION
Examine existing data and ste history
tor industry-specific wastes to - '
determine amlytw tor measurement
Perform broad spectrum analysis.
Be prepared to request farther
analyses if potentially toxic
compounds are discovered during
screening. Compare results from
multiple samplings or historical data.
Use technique with definitive
identification (e.g., GC-MS).
Alternatively, uee technique with
definitive identification first, tallowed
by another technique (e.g., GC) to
achieve lower quantitation limits.
Review available methods for
appropriate detection Unit
Use environmental samples m QC
samples to determine recovery and
repmducibility in tha sample media.
Selaa analytical methods based on
sample medium and strengths of the
sample preparation technique.
Consider options and set priorities.
Select experienced laboratory and
maintain communication.
-------
baseline risk assessments for hu-
man health are conducted consis-
tently while being protective of
public health.
Guidance for Date Useability in Risk
Assessment addresses these issues in detail.
Procedures,minimum tequiDements.andcor-
rective actions are provided to resolve the
impact the issues have on the confidence fa
the risk assessment.
MAKING DECISIONS WITH
ENVIRONMENTAL DATA
Four fundamental decisions forrisk as-
sessment are made with the data acquired
during the Remedial Investigation.
« What contamination is present and
at what levels?
the selection of analytical meth-
ods, laboratory performance, and
type and level of data review affect
the probability of false negatives
andfalse positives for both site and
background samples.
« Are site concentrations sufficiently
different from background?
Site concentrations must be distin-
cguished from background levels to
support an evaluation of increased
risk for human health on the basis
of the site contamination. Both sampling
and analytical designs are considered.
» Are all exposure pathways identified and
examined?
All exposure pathways must be identified
and exposure routes examined. Decisions
concerning exposure pathways primarily
involve identifying and sampling media
ofeoncem. Sampling must be representa-
tive.
* Are all exposure pathways fully charac-
terized?
The final decision involves the characteri-
zation of exposure pathways. Sampling
must be represenlauve and satisfy per-
formance objectives determined during
the planning process. A broad spectrum
analysis must be available in order to
characterize the pathways and avoid false
negatives.
Highlight 3a
AUTOMATED SYSTEMS to SUPPORT
ENVIRONMENTAL SAMPLING
SYSTEM
DHa Quality Objective
(Training) -apart
System
ESiS
Environmental Sampling
(Plan Design} - Expert
System
GEO- EAS
BeostaBslkal
Environmental
Assessment Software
SCOUT
Mi.rtJYar.ate Statistical
Analysis Partage
ASSESS
Assessment of Errors In
Sampling «f Sols
EPA CONTACT
Dean Neptune
USEPA
QysJ'rfy Assurance ,
ManaQonor'. Slntf
(Z9Z) 475-94W
Jeff Van B
Exposure Assessment D!v.
USEPA. EHSL-LV
(702) 798-8367
Evan Englund
Exposure Assessment Dt*.
USiPA,EM8L-LV-
(703) 793-224S
Jeff Van E«
Exposure Assessment Diy.
USEPA, EMSL-LV
jToa) ?9i-2M7
Jeff Van to
Exposure Assessment Oiv.
USEPA, EMSL-LV
(702) 738-1367
DESCRIPTION
Tralnhg syatam desfcnsd to assist
In ptariln§ of envlraninantaJ
invBMlgattons baewd on dwa
quslKy ob|eciive procsss.
Expert splsn designed to mskst
In planning sarapto col'ocdor.
Includes rnsdete lhat address
stalMtel dasign, QC, sampling
praosdures, eampto handling,
budget, and docurmntaticn.
Current sjr3»rn addresses metBl
conta^nlnants in asoil mairix,
(E^iwidod application under
development, contact EMSL-LV.)
Coltactlon of eafnwe Kxsls tor
WiK)[rnansloned geasrMMesI
analysis d spatially dsMHned
datspuinta. Programs inoluds Ills
managsnwnt, eontour mapping,
*'"giri3. and variogramanalysis.
A oolectfonof siattaksal proarams
-J-.-ti Kcepi G EO-EAS film -for
muKvailaw tinnlysls.
System designed te assist In
assessment at error In sampling of
soils. Estimates measurement
error variance components.
P/esefits sestter plots ®f quality
control data and e rror ptots in
assist in determining the
approprWo araotint of quality
control samples.
" fit systems will run on ant IBM eompatbte PC AT with 64C K RAM (minimum).
A!iM>d disk Is racornmendoc.
Uncertainty in chemical identification and quanti-
tation is determined based on decisions made during
planning. This analytical data uncertainty affects the
level of confidence of the final risk assessment.
Using Criteria for Planning and
Assessing Data Useability
Six criteria assure the Useability of environmental
date in risk assessments. The criteria are:
« Data sources
• Documentation
• Analytical methods and detection limits
• Data quality indicators
* Data review
• Reports to risk assessors.
The manual explains how to use these criteria to
plan data collection efforts that maximize the useabUity
-------
Highlight 3b
AUTOMATED SYSTEMS fo SUPPORT
METHOD SELECTION
SYSTEM
LbtofUttB
Smait Method
Inde*
Geophysical
Techniques
Expert System
'EPA Sampling
and Analysis
Data Bsss
CONTACT
W. A. Talliard
USEPA
Offlcs at Water
(202)362-7120
John Nocarino
Qtiahy Assurance Olv.
USEPA, EMSL-LV
(802)795-2110
Aldo Maggelia
Advansed Monitoring
OH.
USEPA, EMSL-LV
(202) 78B-2254
Lewis Piiblfehais
1-806-272-7737
DESCRIPTION
An automated Sorting and
selection eoftwire package that
currently ranlalnelSQrnsttais
and 1,700 anarftes, These are
cross-raferancad to facilitate
Eofaelten based on required
ttsads (e.g., analyls detection
limit, inslfumerwi.
Natural language expert system
pratttype th- cinV! 1 1 1 prograin of 1 SO
EPA-appiwad nwlhods. Tho
database can be eeaf chnd by
method, anilyta, mat fh . and
varbus quality Bswranea
conEldBraiinns.
1 All sysema MI run on uny IBM eompaMWs PC AT with 6.1 OK BAM (mMmyrnJ.
A fixed disk le recommended.
of environmental analytical data in baseline risk assess-
ments. Highlight 2 details analytical issues which im-
pact the risk assessment and describes actions needed to
resolve them. Guidance tools include a Sample Design
Selection Worksheet and a Method Selection Woric-
sheet S tep-by-step instructions for using the woiksheets
assist the RPM 01 risk assessor in planning RI sample
collection and analysis to produce data meeting risk as-
sessment needs. Highlights 3a and 3b list automated
systems which may help in this planning. Regional En-
vironmental Services Divisions (BSD) can also provide
assistance.
ASSESSING ENVIRONMENTAL DATA
FOR USEABILITY
Conducting the Data Assessment
Examine the data, documentation, and reports to
determine if performance is within the limits reqeiredby
the planning objectives. If no performance objectives
have been specified or the specification is incom-
plete, the minimum acceptable requirements for
the data useability criterion should be used for the
minimum performance objectives. The manual
presents minimumrequirements for each data usea-
bility criterion. In evaluating the criteria, perform
the following activities:
» Identify or determine minimum data require-
ments and performance objectives,
• Determine actualperformance compared to ob-
jectives.
* Determine and execute any corrective action
required.
Take corrective action when actual perform-
ance fails to meet the objectives for data critical to
therisk assessment. Highlight 4 gives several cor-
rective action options for resolving problems with
data not meeting performance requirements.
Highlight 4
CORRECTIVE ACTION OPTIONS
WHEN DATA DO NOT MEET
PERFORMANCE OBJECTIVES
* Halriwfe missing tnforniaiian.
* Resolve technical or procedural ombfems
by requiting additional explanation or
darif calion from the technical team.
- Request raanalpls of ssniplafs) from
axtrad.
» Request conBlwetlon and ^-interpretation
of analytical results from the laboratory or
project chmntot
. * Request aMianel sample collection and
analysis for site or background
characterization. . '
• Model potential impact on risk aesesement
cwtalnijr using sansttMty analysis to
dstermine range of effect.
* Adjust or impute data based on approved
default options and imputation routines.
« Quality or rajeel data for use in risk
assessment.
-------
Highlight S
MINIMUM REQUIREMENTS, IMPACT, AND CORRECTIVE
ACTIONS FOR DATA USEABILITY CRITERIA
'-
DAT A USEABILITY
CRITERION
5,1 Reports To Risk
Assessor
5.2 Documentation
S.3 Dab Sources
5,4 Analytical
Method and
Detection Umlt
5.5 DmtaFteiriew
5,6 Data Quality
Indicator
MINIMUM
REQUIREMENT
• Site description
» Sample design with sample
locutions
* Analytical method and
detection limit
* Results on per-sample basis,
qualified for analytical
. limitations
* Sample-specific quantftation
limits (SOLA) and detection
limit foe nan-detects
• Field conditions lor media
and environment
* Preliminary reports
• Sample results related to
geographic location
(chaln-of-CUStBdy records,
SOPs, field and analytical
records;
• Analytical data results for
one sample per medium
per exposure pathway
* Bread spectrum analysis tor
one sample per medium
per exposure pathway
* Reid measurements data
for media and environment
• Routine methods used tor
critical samples and
chemicals of potential
concern
• Detection limit less fan
20% of concentration of
concern
• Correctness of analytical
results reviewed
* Sampling variability
quantified far each aralytB
• QC samples required to
identify and quantity
precision and accuracy
• Sampling and
analytical precision and
accuracy quantified
IMPACT ON RISK
ASSESSMENT
* Unable lo
perform
quantitative risk
assessment
> Unitte to assess
exposure
pathways
* Unable to identify
appropriate
concentration for
exposure units
• Potential lor false
negatives and
positives
* Increased
variability in
exposure
modeling
• Invalidated
precision and
accuracy
» False negatives
* Potential for laJse
negatives
or false positives
• Increased
variability and
bias due to
analytical
process,
calculation or
transcription
errors
» Unable to
quantify
confidence
levels for
uncertainty
• Potential for false
negatives
or false positives
CORRECTIVE |
ACTION
• Request missing
information
* Perform qualitative
risk assessment
* Request locations
Identified
• Resampling
• Resampling or
reanaiysls for
critical samples
• Rearatysia
« Resampling and
analysis for critical
samples
• Documented
caveats far nort- •
critical samples
« Perform data
review
' Resampling lor
critical samples '
• Perform qualitative
risk assessment
• Perform
quantitative
risk assessment
for non-critical
s am pies with
documented
discussion of
potential limitations
-------
Organizing the Data
Assessment
For each performance measure within
an assessment phase, determine whether the
actual performance for the date involved is
satisfactory, questionable, at unsatisfactory.
The manual includes a worksheet to assist in
applying the useability criteria to the data.
For each criterion, the worksheet requires a
decision to be made: accept, accept with
qualification, or reject the information in-
volved for use in the risk assessment. Record
the justificaiion for each decision on the
worksheet. Highlight 5 summarizes mini-
mum requirements for each data useability
criterion, lists its potential impact on ihe risk
assessment, and identifies corrective actions
that may be taken,
APPLYING DATA TO RISK
ASSESSMENT
"The level of confidence associated with
the actual data that are obtained affects the
ability to answer the four basic questions
being addressed in the risk assessment proc-
ess, as shown in Highlight 6, The final sec-
tions of the manual provide procedures to be
followed to determine the level of certainty
for each decision, given the results of the
assessment of performance measures.
Uncertainty results from each of the components of
risk assessment, and the results for each component
should be presented with an explicit statement of the
degree of confidence. These measures are the bases for
the estimation of the degree of confidence in the risk as-
sessment
NEED MORE HELP?
Superfund Toxic Integration Coordinators are
located in each region. Questions regarding site-specific
Superfund risk assessment Issues should be referred to
the appropriate individuals listed in Highlight 7, The
Toxics Integration Branch, Office of Energy andReme-
Hlfl Wight 6
UNCERTAINTY IN DATA COLLECT JON AMD EVALUATION DECISIONS
AFFECTS THE CEHTAIMTY Of THE HBK ASSESSMENT
(Urxtctanty
i (Une«t»irt|r nol iii^jpi
5 known) ™SS
" iiiii
dial Response (OERR), may also be contacted at (202)
475-9486 or FTS 475-9486 for technical information
sources and assistance with this guidance. Potential
sources for technical assistance are Regional ESDs and
Quality Assurance Officers, EMSt/LVmaybeasource
for assistance on sampling or statistical issues.
Where to Obtain the Guidance Manual
Toorder acopy of themaaBal,callor write toEPA's
Center for Environmental Research at 513 -569-7652 or
FTS 684-7562,
Additional copies of this fact sheet can be obtained
by ctlling the Superfund Docket 202-382-6940 or FTS
382-6940,
-------
Highlight 7
REGIONAL TOXICS INTEGRATION COORDINATORS
Region Neme and Address Phone Number
I Sarah Lavlnson FTS 833-1504
Waste Management Division (HSS-CAN-7) 617-573-9662
EPA Region I
John F, Kennedy Federal Building
Boston, MA 02208
II Peter Qrevalt FTS 264-8775
Program Support Branch 212-264*323
ERR Division
EPA Region II
26 Federal Plaza
New York, NY 10278
III Hichart Brunter FTSS97-QB04
Hazardous Waste , 215-597-0804
Management Division {3HW15)
EPA Region III
841 Chestnut Sfreet
Philadelphia, PA 19107
IV Elmer AWn FTS 257-1586
Waste Management Division 404-347-1586
EPA Region IV
345 Courtland Street, NE
Atlanta, GA 30365
V Steva Ostrodka FTS 886-3011
Technical Support Unit (5HSM-12) 312-886-3011
EPA Region V
230 South Dearborn Street
Chicago, IL 60604
VI Jon Rauseher FTS 255-2188
EPA Region VI (8H-SR) 214-655-2188
First Interstate Bank Tower
1445 Ross Avenue
Dallas, TX 75202-2733
VII David Crawford FTS 276-7052*
EPA Region Vil 913-551-7052
726 Minnesota Avenue
Kansas CHy, KS 66101
Vlli Chris Weis FTS 330-7655
EPA Region VIII (8HWM-SR) 303-294-7655
999 18th Slieat, Suite 500
Denver, CO 60202-2405
IX Gerald Hiatt FTS 484-1914
Technical Support Section (H-8-4) 415-381-8917
Supertund Program
EPA Region IX
1235 Mission Street
San Francisco, CA 34103
X PalCirone , FTS 389-2138
EPA Region X (ES-098) 206-442-1597
1200 Sixth Avenue
Seattle, WA 98101
* Caller must have FTS 2000. If not, use commercial number.
------- |