United States
Environmental Protection
Agency
Office of
Solid Waste and
Emergency Response
Publication 9355.9-01
EPA540-R-93-071
PB94-963203
September 1993
Superfund
xvEPA
DATA QUALITY OBJECTIVES
PROCESS FOR SUPERFUND
Interim Final Guidance
-------
9355.9-01
EPA540-R-93-071
PB94-963203
September 1993
DATA QUALITY
OBJECTIVES PROCESS
FOR SUPERFUND
Interim Final Guidance
U.S. Environmental Protection Agency
Region 5, Library (PL-12J)
77 West Jackson Boulevard, 12th Floor
Chicago, IL 60604-3590
Office of Emergency and Remedial Response
U.S. Environmental Protection Agency
Washington, DC 20460
Printed on Recycled Paper
-------
NOTICE
The procedures set forth in this document are intended as
guidance to employees of the United States Environmental
Protection Agency (EPA) and other government agencies.
These guidelines do not constitute EPA rulemaking and cannot
be relied upon to create any substantive or procedural rights
enforceable by any party in litigation with the United States.
EPA reserves the right to act at variance with the policies and
procedures in this guidance, based on analysis of site-specific
circumstances. EPA also reserves the right to modify this
guidance at any time without public notice.
11
-------
TABLE OF CONTENTS
Page
LIST OF FIGURES v
LIST OF TABLES v
FOREWORD vii
r>
§ LIST OF ACRONYMS viii
° INTRODUCTION 1
/) OVERVffiW AND PURPOSE OF THIS DOCUMENT 1
O BENEFITS OF THE DQO PROCESS 4
^ THE DQO PROCESS AND STATISTICS 4
IMPLEMENTING THE DQO PROCESS 5
A HOW THE DQO PROCESS FITS INTO INTEGRATED SITE
^ ASSESSMENT/SACM 6
WHERE TO FIND MORE INFORMATION ABOUT THE DQO PROCESS 7
CHAPTER 1. STEP 1: STATE THE PROBLEM 9
1.1 BACKGROUND 9
1.2 ACTIVITIES 10
1.3 OUTPUTS 12
CHAPTER 2. STEP 2: IDENTIFY THE DECISION 13
2.1 BACKGROUND 13
2.2 ACnVTTIES 14
2.3 OUTPUTS 15
X
CHAPTER 3. STEP 3: IDENTIFY THE INPUTS TO THE DECISION 17
3.1 BACKGROUND 17
3.2 ACTIVITIES 18
3.3 OUTPUTS 19
CHAPTER 4. STEP 4: DEFINE THE BOUNDARIES OF THE STUDY 21
4.1 BACKGROUND 21
4.2 ACTIVITIES 22
4.3 OUTPUTS 25
ill
-------
TABLE OF CONTENTS • CONTINUED
Page
CHAPTER 5. STEP 5: DEVELOP A DECISION RULE 27
5.1 BACKGROUND 27
5.2 ACTIVITIES 27
5.3 OUTPUTS 28
CHAPTER 6. STEP 6: SPECIFY LIMITS ON DECISION ERRORS 29
6.1 BACKGROUND 29
6.2 ACTIVITIES 31
6.3 OUTPUTS 34
CHAPTER 7. STEP 7: OPTIMIZE THE DESIGN 37
7.1 BACKGROUND 37
7.2 ACTIVITIES 38
7.3 OUTPUTS 42
7.4 SUPERFUND DATA CATEGORIES 42
CHAPTER 8. STEP 8: BEYOND THE DQO PROCESS 45
8.1 OVERVIEW 45
8.2 SAMPLING AND ANALYSIS PLAN DEVELOPMENT 46
8.3 DATA QUALITY ASSESSMENT 47
APPENDICES
I TECHNICAL SUPPLEMENT TO THE DATA QUALITY OBJECTIVES PROCESS ... 49
H APPLICATION OF DATA QUALITY OBJECTIVES TO SUPERFUND SITES
(EXAMPLES) 81
Section A: Ground-water Example 81
Section B: Removal Program Example 93
Section C: Remedial Program Example 103
IE GLOSSARY Ill
IV BIBLIOGRAPHY 115
IV
-------
LIST OF FIGURES
Page
Figure 1. The Data Quality Objectives Process 2
Figure 2. QA Planning for Superfund Data Collection 3
Figure 3. Repeated Application of the DQO Process 6
Figure 4-1. Defining Spatial Boundaries 23
Figure 6-1. An Example of a Design Performance Goal Diagram - Baseline Condition:
Parameter Exceeds Action Level 35
Figure 6-2. An Example of a Design Performance Goad Diagram - Baseline Condition:
Parameter is Less Than Action Level 36
Figure 7-1. An Example of a Power Curve 41
Figure 8-1. QA Planning and the Data Life Cycle 46
Figure 8-2. The Data Quality Assessment Process 47
LIST OF TABLES
Table 6-1. Decision Error Limits Table Corresponding to Figure 6-1 35
Table 6-2. Decision Error Limits Table Corresponding to Figure 6-2 36
-------
FOREWORD
The U.S. Environmental Protection Agency (EPA) undertakes cleanup activities at abandoned
hazardous waste sites under the Comprehensive Environmental Response, Compensation, and Liability
Act (CERCLA), also known as the Superfund program. Many of the activities involve the collection
and evaluation of site-specific environmental data. EPA has developed and implemented a mandatory
Agency-wide program of quality assurance for environmental data, including a process for developing
Data Quality Objectives (DQOs), as an important tool for project managers and planners to determine
the type, quantity, and quality of data needed to make defensible decisions.
The Office of Emergency and Remedial Response (OERR) is promoting a common
understanding of the quality assurance requirements for site-specific data collection activities. The
DQO Process is an effective means by which managers and technical staff can implement the
mandatory Superfund quality assurance requirements. The Agency has developed this guidance on
Data Quality Objectives Process for Superfund to replace the earlier guidance, Data Quality Objectives
for Remedial Response Activities (EPA 540/G-87/003, OSWER Directive 9355.0-7B) and the five
analytical levels introduced in that document.
It is the goal of the Superfund program and the regulated community to collect data of
appropriate quality for environmental decisions while minimizing expenditures related to data
collection by eliminating unnecessary duplication or unnecessarily detailed data. The most effective
way to accomplish this is to implement the DQO Process.
c
j$£_Henry L. Longest II, Director
Office of Emergency and Remedial Response
lond, Director
Programs Enforcement
vn
-------
LIST OF ACRONYMS
ARAR
CERCLA
CFR
DQO or DQOs
EE/CA
ESI
EU
FS
HRS
MCL
NCP
NPL
OSC
OSWER
PA
PRO
PRP
QAPP
RD
RDT
RI
RME
RPM
RU
SACM
SAM
SEA
SI
Applicable or Relevant and Appropriate Requirement
Comprehensive Environmental Response, Compensation, and Liability Act
Code of Federal Regulations
Data Quality Objectives
Engineering Evaluation and Cost Analysis
Expanded Site Investigation
Exposure Unit
Feasibility Study
Hazard Ranking System
Maximum Contaminant Level
National Oil and Hazardous Substances Pollution Contingency Plan
National Priorities List
On-Scene Coordinator
Office of Solid Waste and Emergency Response
Preliminary Assessment
Preliminary Remediation Goal
Potentially Responsible Party
Quality Assurance Project Plan
Remedial Design
Regional Decision Team
Remedial Investigation
Reasonable Maximum Exposure
Regional Project Manager
Remediation Unit
Superfund Accelerated Cleanup Model
Site Assessment Manager
Site Evaluation Accomplished
Site Inspection
Vlll
-------
INTRODUCTION
OVERVIEW AND PURPOSE OF THIS DOCUMENT
This document provides guidance on developing Data Quality Objectives (DQOs) for
Superfund sites. This guidance replaces EPA/540/G-87/003, Data Quality Objectives for Remedial
Response Activities: Development Process.
Each year the U.S. Environmental Protection Agency (EPA) and the regulated community
spend approximately $5 billion collecting environmental data for scientific research, regulatory
decision making, and regulatory compliance. While these activities are necessary for effective
environmental protection, it is the goal of EPA and the regulated community to minimize expenditures
related to data collection by eliminating unnecessary, duplicative, or overly precise data. At the same
time, they would like to collect data of sufficient quantity and quality to support defensible decision
making. The most efficient way to accomplish both of these goals is to begin by ascertaining the
type, quality, and quantity of data necessary to address the problem before the study begins.
What is the DQO Process? The DQO Process is a series of planning steps based on the Scientific
Method that is designed to ensure that the type, quantity, and quality of environmental data used in
decision making are appropriate for the intended application. The steps of the DQO Process are
illustrated in Figure 1.
What are DQOs? DQOs are qualitative and quantitative statements derived from the outputs of each
step of the DQO Process that:
1) Clarify the study objective;
2) Define the most appropriate type of data to collect;
3) Determine the most appropriate conditions from which to collect the data; and
4) Specify acceptable levels of decision errors that will be used as the basis for
establishing the quantity and quality of data needed to support the decision.
The DQOs are then used to develop a scientific and resource-effective sampling design.
The DQO Process was developed by EPA to help Agency personnel collect data that are
important to decision making. The process allows decision makers to define their data requirements
and acceptable levels of decision errors' during planning, before any data are collected. Application
of the DQO Process should result in data collection designs that will yield results of appropriate
quality for defensible decision making.
Why was this document developed for Superfund? Mandatory quality assurance (QA) requirements
for EPA environmental data collection activities are established in EPA Order 5360.1, Policy and
Program Requirements to Implement the Quality Assurance Program. Additionally, the National Oil
and Hazardous Substances Pollution Contingency Plan (NCP; 40 CFR Part 300) mandates specific
Superfund QA requirements. Both documents emphasize that Superfund environmental data must be
of known quality and require the development of Quality Assurance Project Plans (QAPPs) for all
environmental data collection activities to achieve this goal. The NCP mandates the development of a
"Decision errors occur when variability or bias in data mislead the decision maker into choosing an incorrect course of
action. Decision errors are discussed in detail in Chapter 6: SPECIFY LIMITS ON DECISION ERRORS.
1
-------
1. State the Problem
Summarize the contamination problem that will require new environmental
data, and Identify the resources available to resolve the problem.
*
2. Identify the Decision
Identify the decision that requires new environmental
data to address the contamination problem.
*
3. Identify Inputs to the Decision
Identify the information needed to support the decision, and
specify which inputs require new environmental measurements.
*
4. Define the Study Boundaries
Specify the spatial and temporal aspects of the environmental
media that the data must represent to support the decision.
*
5. Develop a Decision Rule
Develop a logical 'if... then..." statement that defines the conditions that
would cause the decision maker to choose among alternative actions.
*
6. Specify Limits on Decision Errors
Specify the decision maker's acceptable limits on decision errors, which are
used to establish performance goals for limiting uncertainty in the data.
7. Optimize the Design for Obtaining Data
Identify the most resource-effective sampling and analysis design
for generating data that are expected to satisfy the DQOs.
Figure 1. The Data Quality Objectives Process
-------
Sampling and Analysis Plan (SAP), which
specifies acceptable data quality goals, defines
responsibility for achieving these goals, and
includes as its key elements a field sampling
plan and a QAPP. Figure 2 illustrates the
elements of QA planning for Superfund.
The DQO Process requires site managers
to specify acceptable data quality goals by
establishing acceptable limits on decision errors.
The DQO Process outputs, including the
acceptable limits on decision errors, provide the
information necessary to develop the SAP. The
DQO Process and the SAP requirements satisfy
EPA Order 5360.1 and the NCP's mandate. This
guidance document revises the Superfund
program's approach to developing DQOs to be
consistent with the following Agency-wide QA
requirements and guidance documents:
EPA Quality System Requirements for
Environmental Programs. EPA/QA/R-1.
1993.
Interim Draft EPA Requirements for Quality
Management Plans. EPA/QA/R-2.
1992.
Data Quality Objectives Process
OUTPUTS
/ n^K, / / SamP«n9 /
/ Quality / / D , /
/ Objectives / / ues'9n /
INPUTS
Sampling and Analysis Plan
Development
/Quality
Assuranc
Project
Plan
/ 1 Reid /
e / / Sampling /
/ / Plan /
J
ELEMENTS
<
Sampling and SINGLE
Analysis Plan INTEGRATED
' DOCUMENT
Figure 2. QA Planning for
Superfund Data Collection
EPA Requirements for Quality Assurance Project Plans for Environmental Data Operations.
EPA/QA/R-5. 1993.
Guidance for Planning for Data Collection in Support of Environmental Decision Making Using the
Data Quality Objectives Process. EPA/QA/G-4. 1993.
Guidance for Conducting Environmental Data Quality Assessments. EPA/QA/G-9. 1993.
How is this document organized? This document is organized as follows: Chapters 1 through 7
describe procedures for implementing the DQO Process at Superfund sites. Each of these chapters
describes a step of the DQO Process, and includes a background section that explains the purpose of
that step, activities for developing the outputs of that step, and a list of expected outputs. Chapter 8
discusses the relationships between the DQO Process, the Sampling and Analysis Plan, and Data
Quality Assessment.
This guidance is supported by several appendices. Appendix I describes in more detail
selected topics relating to DQO development activities. Appendix n provides three examples of DQO
development: a pre-remedial program (site inspection) ground-water example, a removal program soil
example, and a remedial program soil example. Appendix HI contains a glossary of terms used in this
guidance document, and Appendix IV contains a bibliography of documents used in the development
of this guidance.
-------
BENEFITS OF THE DQO PROCESS
The DQO Process is a planning tool to help site managers decide what type, quality, and
quantity of data will be sufficient for environmental decision making. The outputs of the DQO
Process can be used to develop a statistical sampling design and to effectively plan field investigations
that can stand up to rigorous review.
By using the DQO Process, a site manager provides criteria for determining when data are
sufficient for site decisions. This provides a stopping rule — a way for site managers to determine
when they have collected enough data. In addition, the DQO Process:
Improves Sampling • helps site managers streamline field investigations and decide how many
and Analysis Designs samples and analyses are required to support defensible decision making;
• helps site managers define where and when samples should be collected;
• provides the QA community with a scientific basis for defining the right
type and number of quality control and quality assessment samples and
associated analytical precision and recovery requirements;
Saves Money and Time • helps field personnel identify resource-efficient sample collection
methods;
• helps laboratory analysts identify resource-effective analytical methods;
• can drastically reduce overall project costs by improving the quality of
information for decision making (for example, defining areas of the site
that require remediation) and by eliminating expensive rework;
Improves Decision • helps site managers develop a statistical sampling design that controls
Making decision errors;
• provides a structure for clarifying multiple study objectives into specific
decisions;
• encourages the participation and communication of data users and
relevant technical experts in planning, implementation, and assessment.
The DQO Process is based on the scientific method, and therefore improves the legal
defensibility of site decisions by providing a complete record of the decision process and criteria for
arriving at conclusions.
It is important to remember that there is a tradeoff between the desire to limit decision errors
and the cost of reducing decision errors. Reducing decision errors can be costly because more samples
and more analyses are often required. One of the goals of the DQO Process is to help decision makers
strike the best balance between acceptable limits on decision errors and the cost of meeting those
decision error limits.
THE DQO PROCESS AND STATISTICS
The DQO Process has both a quantitative and a qualitative aspect. The quantitative aspect
seeks to use statistics to design the most efficient field investigation that controls the possibility of
making an incorrect decision. The qualitative aspect seeks to encourage good planning for field
investigations and complements the statistical design. Users of this guidance are encouraged to pursue
both aspects of the DQO Process. A field investigation can always benefit from good planning, even
if planning does not lead to a statistical design.
-------
Generally, the quantitative aspect and subsequent statistical design are important when site
contaminant levels are close to an action level, or when variability in the data is so great that the
results are inconclusive. In such cases, a statistical design can provide quantitative estimates of the
level of uncertainty in the data and, therefore, help the decision maker understand and control the
probability of making an incorrect decision based on the data.
The statistical procedures used in the DQO Process provide:
• a scientific basis for making inferences about a site (or a portion of a site) based on
information contained in environmental samples;
• a basis for defining data quality criteria and assessing the achieved data quality for
supporting integrated site assessment decisions;
• a foundation for defining meaningful quality control procedures that are based on the
intended use of the data;
• quantitative criteria for knowing when site managers should stop sampling (i.e., when the
site has been adequately characterized); and
• a solid foundation for planning subsequent data collection activities.
Non-probabilistic or subjective (judgmental) sampling approaches can be useful and
appropriate for satisfying certain field investigation (study) objectives. For instance, if the study
objective is to locate and identify potential sources of contamination, a subjective identification of
sampling locations may be the most efficient method to employ.2 If the objective is to establish that a
threat exists in a complete exposure pathway by confirming the presence of a hazardous substance
associated with the site or process, a judgmental sampling approach can be used. However, because of
the subjective nature of the selection process, data generated from non-probabilistic samples should not
be used if the goal of the study is to characterize some property of the site as a whole.
IMPLEMENTING THE DQO PROCESS
The scoping team should follow each step of the DQO Process for each medium of concern.
Once the scoping team has gone through the process completely for one medium, it becomes easier
and quicker to develop additional sets of DQOs in other media. For example, typically at Superfund
sites the contaminants of concern identified in the early assessment phase remain the focus of
subsequent field investigations in the advanced assessment, even though the decision and the action
level may change. Similarly, the areas of concern that are directly related to the geographical
boundaries of the study usually do not vary much through the site assessment process. Therefore,
much of the DQO outputs generated in the early assessment will be applicable in advanced assessment
planning.
The DQO Process is flexible and iterative. Often, especially for more complicated sites, the
scoping team will need to return to earlier steps to rethink or better focus the output. These iterations
through the earlier steps of the DQO Process can lead to a more focused design that can save
resources in later field investigation activities.
2 An important caveat here is that if contamination is not found, then without a statistical approach very little can be said
about the probability of having missed the source of contamination.
-------
The DQO Process should be used repeatedly during the life cycle of a project. Early in the
project, a more preliminary and qualitative application of the DQO Process may be appropriate to meet
the site manager's needs. As more details and decisions about the site develop, a more thorough and
quantitative application of the DQO Process usually is warranted. Figure 3 illustrates this point
graphically. During early assessment, a site manager may decide to apply only the more qualitative
aspects of the DQO Process, rely less on the quantitative aspect, and not use a statistical sampling
design, especially since this is not a decision that requires a full assessment of health or environmental
risks. In the advanced assessment phase, the possibility that uncertainty in environmental data may
lead to incorrect decisions becomes more critical and a site manager may place more emphasis on the
quantitative aspects of DQO development.
PEMHJM7KW
QOM.
ACCOMPLISHED
INCREASING LEVEL OF EVALUATION EFFORT
Figure 3. Repeated Application of the DQO Process
HOW THE DQO PROCESS FITS INTO INTEGRATED SITE ASSESSMENT/SACM
The DQO Process provides a logical framework for planning multiple field investigations,
thereby fulfilling the integrated site assessment goal of cross-program response planning and allowing
optimal cross-program data useability. By emphasizing the need to place limits on the probability of
taking incorrect actions, the DQO Process complements the integrated site assessment objective of
evaluating the need for action. The DQO Process places a worthwhile investment on planning, which
results in timely and efficient cleanups, thereby increasing the chances of taking the correct action.
For these reasons, the DQO Process is an effective approach for accomplishing and satisfying the goals
of the Superfund Accelerated Cleanup Model (SACM). This guidance document is the primary
document for planning site assessment field investigations. However, users should consult other
relevant Superfund guidance that provide more detailed information on specific site assessment
activities. Appropriate references are included throughout this guidance, and Appendix IV provides a
summary of references organized by DQO topic.
-------
WHERE TO FIND MORE INFORMATION ABOUT THE DQO PROCESS
A DQO training course is available through the EPA Training Institute at U.S. EPA
Headquarters in Washington, D.C.
Additional documents on DQO applications can be obtained from the Quality Assurance
Management Staff at EPA Headquarters.
EPA regional and national program office quality assurance managers can provide assistance in
learning more about the DQO Process.
-------
CHAPTER 1
STEP 1: STATE THE PROBLEM
THE DATA QUALITY OBJECTIVES PROCESS
Identify Inputs raJie Decision
Define the Study Boundaries
Develop a Decision Rule
Specify Limits on Decision Errors
Optimize the Design for Obtaining Data
STATE THE PROBLEM
Purpose
Summarize the contamination problem that will
require new environmental data, and identify
the resources available to resolve the
problem.
Activities
• Identify members of the scoping team.
• Develop/refine the conceptual site model.
• Define the exposure scenarios.
• Specify available resources.
• Write a brief summary of the contamination
problem.
1.1 BACKGROUND
The purpose of this step is to:
• establish the DQO scoping team;
• provide a brief description of the contamination problem that presents a potential
threat/unacceptable risk to human health and the environment; and
• identify resources available to address the problem.
Stating the problem typically involves a description of the source and/or location of
contamination including physical and chemical factors associated with the site that could result in
contaminant release or unacceptable exposures. The description should include the regulatory and
programmatic context of the problem, such as the regulatory objectives and basis for the field
investigation. The description of the potential contamination problem should also include appropriate
action levels for evaluating and responding to releases or exposures, and appropriate response actions.
-------
The scoping team is a multidisciplinary group of experts. They develop or refine a conceptual
site model that describes and illustrates the known and suspected sources of contamination, potential
migration pathways, and potential human and environmental receptors. The scoping team begins by
collecting and evaluating all historical site data to formulate the conceptual site model and assess the
extent to which the available historical site data support exposure scenarios that are developed later in
the site assessment process. These descriptions aid in understanding the relationship among potential
contaminant releases, sources of contamination, and physical and environmental targets.
1.2 ACTIVITIES
Identify Members of the Scoping Team
The creation of the scoping team is a two-step process. The first step is to identify the
decision maker for the site. The decision maker (usually the site manager) and his technical staff
identify the other members of the scoping team based on a preliminary understanding of the nature of
the contamination problem (e.g., potentially affected media). The site manager' delegates
responsibility for accomplishing planning tasks to the other members of the scoping team. However,
the site manager makes the final decisions at the site.
The second step is to choose the members of the scoping team. The team should include
representatives who are knowledgeable about several project phases, including QA specialists,
samplers, chemists, modelers, technical project managers, human health and ecological risk assessors,
toxicologists, biologists, ecologists, administrative and executive managers, data users, Natural
Resource Trustees, and a statistician (or someone knowledgeable and experienced with environmental
statistical design).
Every member of the scoping team will support or actively participate in all steps of the DQO
Process. Their roles will include interpreting historical site data and preparing their team members for
accomplishing DQO activities. They will also attend meetings to help generate DQO outputs that will
guide the field investigation data collection designs.
Develop/Refine the Conceptual Site Model
Collect all available historical site data, including QA/QC documentation associated with
previous environmental data collection activities. Use the information to develop a diagram that
illustrates the relationships between:
• locations where contamination exists or contaminant/waste sources,
• types and concentrations of contaminants,
• potentially contaminated media, migration pathways,
• potential physical and environmental targets or receptors.
Presenting historical site data in this manner provides a foundation for identifying data gaps and
focusing on where the problems of potentially unacceptable contamination may or may not exist.
More information on developing the conceptual site model (CSM) can be found in Appendix I,
Section A. For more extensive information sources, refer to the Guidance for Performing Site
Inspections Under CERCLA, and the Guidance for Conducting Remedial Investigation and Feasibility
Studies Under CERCLA.
'Throughout this document, the site manager is assumed to be the decision maker.
10
-------
Define Exposure Pathways and Exposure Scenarios
The goal of this step is to define site conditions that indicate or could lead to an unacceptable
threat or exposure at the site. Use the conceptual site model and relevant information on migration
pathways as a base for accomplishing this task. For the early phases of site assessment activities, it is
necessary to establish that a complete exposure pathway exists. In general, identify currently
contaminated media to which individuals or sensitive ecosystems may be exposed. Following
identification of the media of concern, identify potential contaminants of concern based on historical
site use, analytical data, or anecdotal information. Next, define the current and future land use.
Following this, determine the local/state applicable or relevant and appropriate requirements (ARARs)
for the site. For cases where multiple contaminants exist and ARARs are not available for all the
contaminants, develop risk-based contaminant-specific preliminary remediation goals (PRGs).
Chemical-specific PRGs are concentrations based on ARARs or concentrations based on risk
assessment. PRGs should also be developed even when ARARs are available for all contaminants and
meeting all ARARs is not considered protective. For each medium and land use combination, identify
complete exposure pathways and assemble all this information into exposure scenarios that are
expected to represent the highest exposure that could reasonably occur at the site. More detailed
information on accomplishing the above activities during scoping can be found in the Risk Assessment
Guidance for Superfund: Volume I - Human Health Evaluation Manual (Part B, Development of Risk-
based Preliminary Remediation Goals), EPA/540/R-92/004.
It is efficient to evaluate the potential for an unacceptable ecological threat during the human
health evaluation. The following text discusses important relationships between human health and
environmental evaluations:
Environmental evaluation and human health evaluation are parallel activities in the
evaluation of hazardous waste sites. Much of the data and analyses relating to the
.nature, fate, and transport of a site's contaminants will be used for both evaluations.
At each point of these common stages, however, analysts should be sensitive to the
possibility that certain contaminants and exposure pathways may be more important for
the environmental evaluation than for the health evaluation, or vice versa. It is also
important to recognize that each of the two evaluations can sometimes make use of the
other's information. For example, the potential of a contaminant to bioaccumulate
may be estimated for a health evaluation but be useful for the environmental
evaluation. Similarly, measurement of contaminant levels in sport and commercial
species for an environmental evaluation may yield useful information for the health
evaluation.2
For additional information on Exposure Assessment issues and ARARs refer to the Risk
Assessment Guidance for Superfund, Volume I-Human Health Evaluation Manual, Part A and Part B;
Risk Assessment Guidance for Superfund, Volume II-Environmental Evaluation Manual; Framework
for Ecological Risk Assessment; EPA Risk Assessment Forum (Feb. 1992); A Review of Ecological
Assessment Case Studies from A Risk Assessment Perspective; EPA Risk Assessment Forum
(May, 1993); CERCLA Compliance with Other Laws Manual; and Guidance for Data Useability in
Risk Assessment (Part A).
*Risk Assessment Guidance for Superfund, Volume II • Environmental Evaluation Manual, p. 3.
11
-------
Specify the Available Resources
(1) Define the budget. Specify the approximate monetary budget for the field investigation.
This estimate should account for developing DQOs and for carrying out the potential
sampling and analysis activity under consideration.
(2) Define the time constraints. Determine the time constraints, such as the Superfund
recommended time frame, for completing the various required site evaluations. Other
factors to consider include political factors such as public concern and the timeliness of
addressing health and ecological risks.
Write a Brief Summary of the Contamination Problem
Summarize relevant background into a concise description of the problem to be resolved.
1.3 OUTPUTS
The main output of this step is a complete description of the contamination problem that
includes the regulatory and programmatic context of the problem. This description typically consists
of:
• a list of the known and suspected contaminants in each medium and estimates of their
concentration, variability, distribution, and location;
• the conceptual site model and exposure pathways;
• a summary of the outcome and status of any previous response(s) at the site, such as early
actions or previous data collection activities;
• the site's physical and chemical characteristics that influence migration and associated
human, environmental, and physical target(s); and
• an estimate of the budget, schedule, and available personnel necessary to implement the
appropriate response for the site.
12
-------
CHAPTER 2
STEP 2: IDENTIFY THE DECISION
THE DATA QUALITY OBJECTIVES PROCESS
Identify the Decision
Identi
tifyNqputs to
the Decision
Define the Study Boundaries
-x
Develop a Decision Rule
Specify Limits on Decision Errors
Optimize the Design for Obtaining Data
IDENTIFY THE DECISION
Purpose
Identify the decision that requires new
environmental data to address tiie
contamination problem.
Activities
• Identify the key decision for the current
phase or stage of the project
• Identify alternative actions that may be
taken based on the findings of the field
investigation.
• Identify relationships between this
decision and any other current or
subsequent decisions.
2.1 BACKGROUND
The purpose of this step is to identify the decision that will use environmental data to address
the potential contamination problem and to state the actions that could result from the resolution of
each decision statement. This is how the scoping team defines the objective of the field investigation.
Generally, environmental field investigations may be designed to satisfy a broad array of
objectives, such as demonstration of regulatory compliance, research, monitoring for trends, or
estimation of average characteristics. For Superfund, however, most field investigations are designed
to support the site manager's selection of appropriate response actions (i.e., recommend the Site
Evaluation Accomplished (SEA) or further assessment or even a removal/remedial response action).
Since the field investigation objective can be viewed as a choice between alternative actions, this
document describes the objectives as being synonymous with the decision and associated actions. This
chapter presents four major site assessment decisions and associated actions. The site assessment
decisions and associated actions listed below address the most important Removal and Remedial data
collection activities. Site managers who are addressing at least one of these major site assessment
decisions should proceed directly to that section below and identify the decision and corresponding
actions. For site managers who are not addressing one of the major decisions, this guidance provides
activities to help develop project-specific decision statements below.
13
-------
Stating the decision will help focus the efforts of the scoping team toward a common
objective. The actions taken will be based on the outcome of the field investigations and will lay the
foundation for defining the data quality requirements. The decision statement and alternative actions
together provide an initial confirmation of the assumption that environmental data are needed to help
resolve the potential contamination problem.
2.2 ACTIVITIES
Identify the Key Decision for the Current Phase or Stage of the Project
Review the list of decisions presented below and select the appropriate decision for the current
phase of the site assessment process.
EARLY ASSESSMENT DECISION
Determine whether the release poses a potential threat to human health or the environment.
ADVANCED ASSESSMENT DECISION, PHASE I
Determine whether the concentration of contaminants of concern exceed ARARs or exceed
contaminant concentrations corresponding to the preliminary remediation goal for the site.
ADVANCED ASSESSMENT DECISION, PHASE O
(EXTENT OF CONTAMINATION)
Determine the volume of media that exceeds action level(s) (i.e., ARARs, concentrations
corresponding to the preliminary remediation goal, removal action levels, or final
remediation levels).
CLEANUP ATTAINMENT DECISION
Determine whether the final remediation level(s) or removal action level(s) have been
achieved.
If a decision other than one from the list above will be addressed, perform the following
activities:
(1) Consider the actions that EPA, the potentially responsible parties, or another collective
group will take based on the outcome of the field investigation. For example, what will be
done to resolve the potential contamination problem? Is it necessary to collect data on
contaminant concentrations in order to decide if the site-related contamination exceeds
regulatory standards, including ecological screening levels?
(2) Examine the regulatory objectives for this phase of the remedial process. For example,
when a site is listed on the National Priorities List (NPL), but a baseline risk assessment
has not been conducted, then the regulatory objective is to determine the nature and
magnitude of contamination.
(3) Perform a consistency check by assessing whether the decision will be responsive to the
potential contamination problem.
14
-------
Identify Alternative Actions that May Be Taken Based on the Findings of the Field Investigation
Select the actions that will be taken based on the outcome of the field investigation that
correspond with the selected decision above.
Actions based on early assessment decision
(i) Recommend the site evaluation accomplished (SEA) response for the site; or
(ii) Recommend that the site warrants consideration of further assessment or a possible
response action.
Actions based on advanced assessment decision. Phase I
(i) Recommend the SEA response for the site; or
(ii) Recommend that the site warrants consideration of further assessment or a possible
response action.
Actions based on advanced assessment decision. Phase II
(i) Designate the area/volume for remediation; or
(ii) Do not designate the area/volume for remediation.
Actions based on cleanup attainment decision
(i) Recommend the SEA response and proceed with delisting procedures; or
(ii) Recommend that further response is appropriate for the site.
Confirm that the actions associated with the list of decisions above will help to resolve the
contamination problem by determining if actions are consistent with and satisfy regulatory objectives.
Also, based on the statement of the problem and decision, assess if the range of actions helps to
achieve the goal of protecting human health and the environment.
Identify Relationships Between This Decision and Any Other Current or Subsequent Decisions
If several decisions will be made, identify each decision and establish the relationship among
them and their order of priority. Then, identify the actions that are associated with each decision and
determine a logical sequence for these actions. Use this information to determine if it would be more
efficient to conduct the field investigation in stages.
2.3 OUTPUTS
The outputs of this step are:
• a statement of the decision that will use Superfund environmental data; and
• a list of the actions that will be taken toward remediation or removal of the potential
contamination problem based on the outcome of the field investigation.
15
-------
CHAPTER 3
STEP 3: IDENTIFY THE INPUTS TO THE DECISION
THE DATA QUALITY OBJECTIVES PROCESS
State the Problem
*
Identify thejjeeislbn
^-"•""^ \
Identify Inputs to the Decision
\. \
DefinetheStudy Boundaries
^s^
Develop a Decision^RiJle
^
X
* \
Specify Limits on Decision Errors
IDENTIFY INPUTS
Purpose
Identify the information needed to support the
decision, and specify which inputs require new
environmental measurements.
Activities
• Identify the informational inputs needed to
resolve the decision.
• Identify sources for each Informational input
and list those inputs that are obtained
through environmental measurements.
• Define the basis for establishing
contaminant-specific action levels.
• Identify potential sampling approaches and
appropriate analytical methods.
Optimize the Design for Obtaining Data
3.1 BACKGROUND
The purpose of this step is to:
• identify the informational inputs needed to support the decision; and
• specify which inputs will require new environmental measurements.
The conceptual understanding of the site (i.e., conceptual site model), developed in Step 1:
STATE THE PROBLEM, relates sources and retention or transport media to receptors. This
conceptual understanding of the contamination problem and the decision statement defined in Step 2:
IDENTIFY THE DECISION are previous outputs that are important to consider during this step. The
action level, such as an ARAR or preliminary remediation goal(s), is another important input that will
be considered during this step.
17
-------
3.2 ACTIVITIES
The following subsections describe suggested activities that will help identify inputs to the
decision.
Identify the Informational Inputs Needed to Resolve the Decision
It is important to determine whether monitoring, modeling, or a combination of these
approaches will be used to support the decision. The decision inputs depend on the approach selected.
For example, data on soil characteristics and hydrogeology could be useful for calibrating a computer
model of contaminant transport and dispersion through ground water. When decisions are supported
by modeling, it may be useful to consider the conceptual site model as a frame of reference. The
conceptual site model summarizes how the site-related contamination may pose a risk to human health
and the environment. Some components of the conceptual site model may be estimated using
mathematical equations and assumptions (i.e., modeling), and other components will be estimated by
directly measuring some characteristic of the site (i.e., monitoring). The conceptual site model concept
was discussed in Step 1: STATE THE PROBLEM. Based on the selected approach, list all of the
informational inputs needed to support the decision. Diagramming techniques may be used to help
organize the list of inputs into categories and show logical or temporal relationships.
Identify Sources for Each Informational Input and List Those Inputs That are Obtained
Through Environmental Measurements
Identify existing sources for information that can support the decision. Sources may include
historical records, regulations, directives, engineering standards, scientific literature, previous site field
investigations, or professional judgement.
Determine the Basis for Establishing Contaminant-Specific Action Level(s)
Determine if ARARs are available for the potential contaminants or if preliminary remediation
goals have been developed for the site. If no regulatory threshold or standard can be identified during
this step, the decision maker will need to decide how to develop a realistic concentration goal to serve
as an action level for the field investigation design and evaluation. These action levels will be used as
targets for developing and evaluating the study designs in the last step of the DQO Process.
Identify Potential Sampling Techniques and Appropriate Analytical Methods
Review the decision and associated regulatory objectives identified in Step 2: IDENTIFY THE
DECISION. Use the list of contaminants identified earlier in this step and contaminant-specific action
levels as a preliminary basis for identifying the most appropriate analytical methods. The decision on
analytical methodology will be made in Step 7: OPTIMIZE THE DESIGN when more information
about sampling and measurement error is available. Finally, identify potential sampling techniques
and associated equipment.
Further discussion of these decision-specific activities is included in Appendix I, Section C.
18
-------
3.3 OUTPUTS
The outputs that will result from the activities above include a list of informational inputs
needed to make the decision and a list of environmental variables or characteristics that will be
measured. There is a potential for confusion at this point because the outputs of this step are actually
the inputs to the decision.
Example List of Advanced Assessment Decision, Phase I, Inputs
(1) List of Inputs Needed to Support the Decision:
• potential contaminants
— concentrations in space and time
— slope factors or dose/response relationships
• exposure pathways
-- media (e.g., soil, surface water, ground water, air, biota, sediments)
— rates of migration (within and between media)
-- rates of dispersion/accumulation
• receptors
-- types/subpopulations
— ecosystems
- sensitivities
~ numbers/densities
-- activity levels/patterns
• preliminary remediation goal/ARARs
• site's physical and chemical characteristics that influence technology applicability (e.g.,
presence of organic components, soil permeability, and depth to impervious formation)
(2) List of Inputs That Require New Environmental Measurements:
• contaminant concentrations in space and time for each media of concern
• small- and large-scale variability in potential contaminant concentrations
• other measurements related to risk assessment, such as fate and transport model
parameters.
19
-------
CHAPTER 4
STEP 4: DEFINE THE BOUNDARIES OF THE STUDY
THE DATA QUALITY OBJECTIVES PROCESS
State the Problem
*
Identify the Decision^
\^
Identify Irjpdtsto the Decision
^ \
Define the Study Boundaries
^^ \
Developa^Qecision Rule
* \
X
^,
Specify Limits on Decision Errors^ •
DEFINE BOUNDARIES
Purpose
Specify the spatial and temporal aspects of
the environmental media that the data must
represent to support the decision.
Activities
• Define the geographic areas of the field
investigation.
• Specify the characteristics that define the
population of interest.
• Divide the population into strata having
relatively homogeneous characteristics.
• Define the scale of decision making.
• Determine the time frame to which the
decision applies.
• Determine when to collect samples.
• Identify practical constraints that may
hinder sample collection (reconsider
previous steps as necessary).
Optimize the Design for Obtaining Data
4.1 BACKGROUND
The purpose of this step is to define the spatial and temporal boundaries of the study, so as to
clarify the domain of what the samples are intended to represent. In addition, Step 4: DEFINE THE
BOUNDARIES provides guidance on how to partition a site so as to prevent inappropriately pooling
and averaging data in a way that could mask potentially useful information.
In order for samples to be representative of the domain or area for which the decision will be
made, the boundaries of the study must be precisely defined. The purpose of this step is to clearly
define the set of circumstances (boundaries) that will be covered by the decision. These include:
• Spatial boundaries that define what should be studied and where the samples should be
taken; and
• Temporal boundaries that describe when samples should be taken and what time frame the
study data should represent.
21
-------
These boundaries will be used to ensure that the study design incorporates the time periods in
which the study should be implemented, areas that should be sampled, and the time period to which
the study results should apply. This will help ensure that the study data are representative of the
objects or people being studied.
Practical constraints that could interfere with sampling are also identified in this step. A
practical constraint is any hinderance or obstacle that may interfere with the full implementation of the
study design.
Applicable information from previous DQO steps that will be necessary to develop boundaries
includes:
• site contaminant(s) identification;
• potential migration pathways and exposure routes and potential receptors;
• the site's physical and chemical characteristics that enhance or decrease the likelihood of
contaminant distribution movement within and among media;
• future use of the site;
• the decision(s) identified in the Step 2: IDENTIFY THE DECISION; and
• the "sampling and analysis action level" or "final remediation/removal action level."
4.2 ACTIVITIES
Define the Spatial Boundary of the Decision.
Figure 4-1 is a representation of this step.
(1) Define the domain or geographic area within which all decisions must apply. The domain
or geographic area is a region distinctively marked by some physical features (i.e., volume,
length, width, boundary) to which the decision will apply. Some examples are property
boundaries, operable units, and exposure areas.
(2) Specify the characteristics that define the population of interest. The "population" is a
term that refers to the total collection of objects or people to be studied, and from which
the sample is to be drawn. For instance, a population may be PCB concentrations in soil
at a Superfund site, or blood lead levels in the exposed human population. Clearly define
the attributes that make up the population by stating them in a way that makes the focus of
the study unambiguous. For example, "the top 12 inches of soil" is less ambiguous than
merely "surface soil".
Some of the considerations in defining the media of concern are:
• What medium was originally contaminated?
• What inter-media transfer of cross-contamination is likely to have occurred (i.e.,
leaching, transport, etc.)?
22
-------
1. Define Geographic Area
of the Investigation
Property Boundaries
2. Define Population
of Interest
Surface Soil
Subsurface Soil
3. Stratify the Site
Area of Low-Intensity
Activity x
Area of High-Intensity
Activity
4. Define Scale of
Decision Making
Figure 4-1. Defining Spatial Boundaries.
23
-------
(3) When appropriate, divide the population into strata that have relatively homogeneous
characteristics. Using existing information, stratify1 each medium or set of objects into
subsets of categories that exhibit relatively homogeneous properties, such as contaminant
concentrations. Stratification is desirable for studying sub-populations or reducing the
complexity of the problem by breaking it into more manageable pieces. The decision
maker can choose to make separate decisions about each stratum or the entire population.
(4) Define the scale of decision making. The scale of decision making is the smallest area,
volume, or time frame of the media in which the scoping team wishes to control decision
errors. The goal of this activity is to define subsets of media that the scoping team will
make decisions about in order to evaluate health and environmental risks and the cleanup
goals of the site, and, at the same time, meet the constraints of the DQOs. The size may
range from the entire geographic boundaries of the site to the smallest size area that
presents an exposure to the receptor. The size of the scale of decision making is generally
based on:
(A) Risk: Here, the scale of decision making is determined by the relative risk that
exposure presents to the receptor (i.e., the size of the scale is correlated with the
risks that it poses to the receptor). The scale of decision making that is based on
risk is referred to as an "Exposure Unit" (EU). An example of an EU could be a
V^-acre potential homestead on a remediated site.
(B) Technological considerations: Here, the scale of decision making is based on the
most efficient area or volume of medium that can be removed or remediated with
the selected technology. These areas or volumes are called Remediation Units '
(RUs). An example of an RU is the area of topsoil that can be removed by one
pass of a bulldozer.
(C) Other considerations: Here, the scale of decision making is based on practical
factors or a combination of risk and technological factors that dictate a specific
size. These factors may include "hot spots" whose size should be based on
historical site use.
As an example, consider a study of contaminated soil where the goal is to protect future
residents from exposure and where the future land use is residential. The planning team may set the
scale of decision making to a 14' by 14' area (EU) if the children derive most of their exposure from
an outdoor play area of this size. Consequently, the decision that will be made at the site would be
protective of children, a sensitive population in exposure assessment.
Define the Temporal Boundaries of the Decision.
(1) Determine the time frame to which the study data apply. It may not be possible to collect
data over the full time period to which the decision will apply. Therefore the scoping
team must determine the most appropriate time frame that the data should reflect (e.g., the
study data will reflect the condition of contaminant leaching into ground water over a
period of a hundred years).
'Stratification is used to reduce the variability of contaminant concentrations and therefore reduce the number of samples needed to meet
the limits of decision error that will be defined in Chapter 7. Decisions are generally made about an area the size of the stratum or smaller.
24
-------
(2) Determine when to collect samples. Conditions may vary over the course of a study due
to weather or other factors. Moreover, the study decision may be influenced by the
seasons. For example, a study to measure exposure to volatile organic compounds from a
contaminated site may give misleading information if the sampling is conducted in the
colder winter months rather than the warmer summer months. Therefore the scoping team
must determine the most appropriate time period to collect data that will reflect the
conditions that are of interest.
Identify any Practical Constraints on Data Collection.
These constraints include seasonal or meteorological conditions when sampling is not possible
and the unavailability of personnel, time, or equipment. For example, it could occur that surface soil
samples could not be taken beyond the east boundaries of a site under investigation because access to
that area had not been granted by the owner of the adjacent property.
Further discussion of the scale of decision making, including examples, is included in
Appendix I, Section D.
4.3 OUTPUTS
The outputs of this step are:
• a detailed description and physical representation (map) of the geographic limits
(boundaries) of each environmental medium (soil, water, air, etc.) within which the
decision(s) will be made;
• a detailed description of the characteristics that define the population of interest;
• definition of the time period in which samples will be taken and to which decisions will
apply;
• the most appropriate scale of decision making for each medium of concern; and
• description of practical constraints that may impede sampling.
25
-------
CHAPTER 5
STEP 5: DEVELOP A DECISION RULE
THE DATA QUALITY OBJECTIVES PROCESS
State the Problem
*
Identify the Decision
*
Identify Inputs to the Decjaion
4/^
Define the'SUjdy Boundaries
^ \
Develop a Decision Rule
-
^^^_ \
Specify Limits on DeclsieojErrors
-"-^
DEVELOP A DECISION RULE
Purpose
Develop a logical "if...then...' statement
that defines the conditions that would cause
the decision maker to choose among
alternative actions.
Activities
• Specify the parameter of interest
(such as mean, median, maximum,
or proportion).
• Specify the action level for the decision.
• Combine the outputs of the previous DQO
steps into an 'if...then...' decision rule
that includes the parameter of interest,
the action level, and the alternative actions.
Optimize the Design for Obtaining Data
5.1 BACKGROUND
The purpose of this step is to integrate the output from the previous steps of the DQO Process
into a statement that defines the conditions that would cause the decision maker to choose among
alternative actions. The outputs from earlier steps include the actions and the decision from Step 2:
IDENTIFY THE DECISION, the action level from Step 3: IDENTIFY THE INPUTS TO THE
DECISION, and the scale of decision making from Step 4: DEFINE THE STUDY BOUNDARIES.
5.2 ACTIVITIES
Specify the Statistical Parameter that Characterizes the Population of Interest
The statistical parameter of interest is a descriptive measure (such as a mean, median,
proportion, or maximum) that specifies the characteristic or attribute that the decision maker would
like to know about the statistical population. Review the study objectives to determine if a particular
statistical parameter is implied or stated. Consult other members of the planning team, such as a risk
27
-------
assessor or person with statistical training, to determine the most appropriate statistical parameter for
the problem.
Appendix I, Section E, contains additional information on choosing a population parameter.
Specify the Action Level (Final Remediation Level or Removal Action Level) for the Decision
The action level is the contaminant concentration which, if exceeded, would indicate that
action should be taken at the site (the action prescribed in Step 2: IDENTIFY THE DECISION).1
If the decision maker believes that the final remediation level could be one of two different
levels, then the more stringent one should be chosen for the action level. A more stringent action
level will require analytical methods (detection limits) that would satisfy the less stringent action level
as well. If multiple contaminants are of concern and ARARs are not available or not sufficiently
protective, risk-based PRGs need to be developed. Refer to the Risk Assessment Guidance for
Superfund, Volume 1-Human Health Evaluation Manual, Part B, Development of Preliminary
Remediation Goals.
Combine the Outputs from the Previous DQO Steps and Develop a Decision Rule
Recall the actions specified in Step 2: IDENTIFY THE DECISION. Combine the actions,
sampling and analysis action level, and the parameter of interest (including the scale of decision
making) in a statement that describes the conditions that would lead to a specific course of action. An
example of a decision rule for a Superfund site is, "If the mean PCE concentration of each
downgradient well is greater than the upgradient well, then further assessment and response is
required; otherwise recommend SEA."
5.3 OUTPUTS
The output for this step is an "if...then..." statement that defines the conditions that would
cause the decision maker to choose among alternative courses of action. It should include the
decision, the actions, the parameter of interest, the action level, and the scale of decision making. For
example, if the mean concentration of contaminants in sediments within the stream reach the
ecological screening level(s), then recommend that the site warrants consideration of further assessment
on a response action.
'This action level is not the final remediation level. The final remediation level is not determined until the ROD. Rather, this action
level is an assumption made during planning based on the decision maker's expectation of the final remediation level. The action level is
only an assumption, and does not bind the decision maker to a specific value for the final remediation level.
28
-------
CHAPTER 6
STEP 6: SPECIFY LIMITS ON DECISION ERRORS
THE DATA QUALITY OBJECTIVES PROCESS
State the Problem
Identify the Decision /
Identify Inputs to the/Decision
Define the Study Boundaries
__.
Develop a Decision Rule
Specify Limits on Decision Errors
Optimize the Design for Obtaining Data
SPECIFY LIMITS
ON DECISION ERRORS
Purpose
Specify the decision maker's acceptable limits
on decision errors, which are used to
establish appropriate performance goals for
limiting uncertainty in the data.
Activities
• Determine the possible range of the
parameter of interest.
• Define both types of decision errors and
identify the potential consequences of each.
• Specify a range of possible parameter values
where the consequences of decision errors
are relatively minor (gray region).
• Assign probability values to points above and
below the action level that reflect
the acceptable probability for the
occurrence of decision errors.
• Check the limits on decision errors to ensure
that they accurately reflect the decision
maker's concern about the relative
consequences for each type of decision error.
6.1 BACKGROUND
The purpose of this step is to specify the site manager's acceptable decision error rates based
on a consideration of the consequences of making an incorrect decision. These limits will be used in
Step 1: OPTIMIZE THE DESIGN to generate the most resource-effective sampling design.
Site managers are interested in knowing the true state of some feature of a site. Since
measurement data can only estimate this state, however, decisions that are based on measurement data
could be in error (decision error). Therefore, the goal of the scoping team is to design a sampling plan
that limits the chance of making a decision error to an acceptable level. This step of the DQO Process
will help the site manager define what constitutes acceptable limits on the probability of making a
decision error.
There are two reasons why the site manager cannot know the true value of a population
parameter:
(1) The population of interest almost always varies over time and space. Limited sampling will
miss some features of this natural variation because it is usually impossible or impractical to
measure every point of a population or to measure over all time frames. Sampling error
29
-------
occurs when sampling is unable to capture the complete scope of natural variability that exists
in the true state of the environment.
(2) A combination of random and systematic errors inevitably arises during the various steps of
the measurement process, such as sample collection, sample handling, sample preparation,
sample analysis, data reduction, and data handling. These errors are called measurement errors
because they are introduced during measurement process activities.
The combination of sampling error and measurement error is called total study error, which is directly
related to decision error.
The probability of making decision errors can be controlled by adopting a scientific approach.
The scientific method employs a system of decision making that controls decision errors through the
use of hypothesis testing. In hypothesis testing, the data are used to select between one condition of
the environment (the baseline condition or null hypothesis, HJ and the alternative condition (the
alternative hypothesis, HJ. For example, the site manager may decide that a site is contaminated (the
baseline condition) in the absence of strong evidence (study data) that indicates that the site is clean
(alternative hypothesis). Hypothesis testing places the greater weight of evidence on disproving the
null hypothesis or baseline condition. Therefore, the site manager can guard against making the
decision error that has the greatest undesirable consequence by setting the null hypothesis equal to the
condition that, if true, has the greatest consequence of decision error.
A decision error occurs when the measurement data lead the site manager to reject the null
hypothesis when it is true, or to fail to reject the null hypothesis when it is false. These two types of
decision errors are classified as false positive errors and false negative errors, respectively.
False Positive Error — A false positive error occurs when sampling data mislead the site
manager into believing that the burden of proof has been satisfied and that the null hypothesis (H0 or
baseline condition) should be rejected. Consider an example where the site manager presumes that
concentrations of contaminants of concern exceed the action level (i.e., the baseline condition or null
hypothesis is: concentrations of contaminants of concern exceed the action level). If the sampling
data lead the site manager to incorrectly conclude that the concentrations of contaminants of concern
do not exceed the action level when they actually do exceed the action level, then the site manager
would be making a false positive error. A statistician usually refers to the false positive error as alpha
(a), the level of significance, the size of the critical region, or a Type I error.
False Negative Error — A false negative error occurs when the data mislead the site manager
into wrongly concluding that the burden of proof has not been satisfied so that the null hypothesis (HJ
is not rejected when it should be. A false negative error in the previous example occurs when the data
lead the site manager to wrongly conclude that the site is contaminated when it truly is not. A
statistician usually refers to a false negative error as beta (p), or a Type II error. It is also known as
the complement of the power of a test.
While the possibility of making decision errors can never be totally eliminated, it can be
reduced. To reduce decision errors, the scoping team must develop an acceptable estimate of the
population parameter. This can be accomplished by collecting a large number of samples (to reduce
sampling error) and by analyzing individual samples several times using more precise laboratory
methods (to reduce measurement error). Better sampling designs can also be developed to collect data
that more accurately and efficiently represent the population of interest. Reducing decision errors,
however, generally increases costs. In some cases, reducing decision errors is unnecessary for making
30
-------
a reasonable decision. For instance, if the consequences of decision errors are minor, a reasonable
decision could be made based on relatively crude data. Similarly, if the consequences of decision
errors are severe, the site manager will want to develop a sampling design that eliminates as much
sampling and measurement error as possible (within budget constraints).
A site manager must balance the desire to limit decision errors to acceptable levels with the
cost of reducing decision errors. To find the best balance and thereby efficiently determine whether to
reduce sampling and/or measurement error, the site manager must define acceptable probabilities of
decision errors. Once the acceptable probabilities of decision errors are defined, then the effort
necessary to reduce sampling and measurement errors to meet these limits can be quantified in Step 7:
OPTIMIZE THE DESIGN. It may be necessary to iterate between Step 6 and Step 7 more than once
before an acceptable balance between limits on decision errors and the cost of a sampling design can
be achieved.
6.2 ACTIVITIES
The combined information from the activities section of this chapter can be graphically
displayed onto a "Design Performance Goal Diagram" (Figures 6-1 and 6-2), or charted in a "Decision
Error Limits Table" (Tables 6-1 and 6-2). The activities section will refer to these figures and tables
to help the reader understand the relationships between the activities and the outputs of this step.
Determine the possible range of the parameter of interest.
Establish the possible range of the parameter of interest by estimating its upper and lower
bounds. This means defining the lowest (typically zero in environmental studies) and highest
concentrations at which the contaminant(s) is expected to exist at the site. This will help focus the
remaining activities of this step on only the relevant values of the parameter. Use historical data,
including analytical data, if available. For example, the range of the parameter shown in Figures 6-1
and 6-2 and Tables 6-1 and 6-2 is between 0 and 210 ppm. Note that when interpreting the Design
Performance Goal Diagram, the concentration values on the horizontal axis represent the true
concentration of the parameter of interest.
Define both types of decision errors and identify the potential consequences of each.
Using the action level specified in Step 5: DEVELOP A DECISION RULE, designate the
areas above and below the action level as the range where the two types of decision errors could
occur. The process of defining the decision errors has four steps:
(1) Define both types of decision errors and establish which decision error has more severe
consequences near the action level. For instance, the threat of health effects from a
contaminated hazardous waste site may be considered more serious than spending extra
resources to remediate the site. Therefore, a site manager may judge that the consequences of
incorrectly concluding that the concentrations of site-related contaminants do not exceed the
action level are more severe than the consequences of incorrectly concluding that the
concentrations of site-related contaminants exceed the action level.
31
-------
(2) Establish the true state of nature for each decision error. In the example above, from the site
manager's perspective, the true state of the site for the more severe decision error will be that
the concentrations of site-related contaminants exceed the action level. The true state of nature
for the less severe decision error is that the concentrations of site-related contaminants do not
exceed the action level.
(3) Define the true state of nature for the more severe decision error as the baseline condition or
null hypothesis (H0= the site is contaminated), and define the true state of nature for the less
severe decision error as the alternative hypothesis (H,= the site is not contaminated). Since
the burden of proof rests on the alternative hypothesis, the data must demonstrate enough
information to authoritatively reject the null hypothesis and conclude the alternative.
Therefore by setting the null hypothesis equal to the true state of nature that exists when the
more severe decision error occurs, the site manager is guarding against making the more
severe decision error.
(4) Assign the terms "false positive" and "false negative" to the proper decision errors. A false
positive decision error corresponds to the more severe decision error and a false negative
decision error corresponds to the less severe decision error. The definition of false positive
and false negative errors depends on the viewpoint of the decision maker and the actions that
are taken. Consider the viewpoint where a person has been presumed to be "innocent until
proven guilty" (i.e., H0 is: innocent; H, is: guilty). A false positive error would be convicting
an innocent person; a false negative error would be not convicting the guilty person. From a
decision maker's viewpoint the errors are reversed when a person is presumed to be "guilty
until proven innocent" (i.e., H0 is: guilty; H, is: innocent). Here, the false positive error
would be not convicting the guilty person and the false negative error would be convicting, the
innocent person.
Define and evaluate the potential consequences of decision errors at several points within the
false positive and false negative ranges. For example, the consequences of a false positive decision
error when the true parameter value is merely 10% above the action level may be minimal because it
would cause only a moderate increase in the risk to human health. On the other hand, the
consequences of a false positive error when the true parameter is ten times the action level may be
severe because it could greatly increase the exposure risk to humans as well as cause severe damage to
a local ecosystem. In this case, site managers would want to have less control (tolerate higher
probabilities) of decision errors of relatively small magnitudes and would want to have more control
(tolerate small probabilities) of decision errors of relatively large magnitudes.
The action level has been set at 100 ppm in Figures 6-1 and 6-2. (Note that the action level is
represented by a vertical dashed line at 100 ppm.) Figure 6-1 shows the case where a site manager
considers the more severe decision errors to occur above the action level. Figure 6-2 shows the case
where the site manager considers the more severe decision error to occur below the action level. The
hypothesis test for the second case is the reverse of the first case, so the false positive and false
negative errors are on opposite sides of the action level. This chapter will focus on Figure 6-1 for
illustrative purposes.
32
-------
Specify a range of possible parameter values where the consequences of decision errors are
relatively minor (gray region).
The gray region is a range of points (bounded on one side by the action level) where the
consequences of a false negative decision error are relatively minor. Establish the general location of
the gray region by evaluating the consequences of wrongly concluding that the baseline condition (the
null hypothesis) is true.
The gray region establishes the minimum distance from the action level to which the site
manager would like to control decision errors. In statistics, this distance is called delta (8), and is an
essential part of the calculations needed to determine the number of samples that need to be collected.
The width of the gray region reflects the site manager's concern for decision errors. A more narrow
gray region implies a desire to conclusively detect the condition when the true parameter value is close
to the action level. When the sample estimate of the parameter falls within the gray region, the site
manager may have a high probability of making a decision error (i.e., the data may be "too close to
call"), and may wrongly conclude that the baseline condition is true.
The gray region is an area where it will not be feasible or reasonable to control the false
negative decision error rate to low levels because the resources that would be required would exceed
the expected costs of the consequences of making that decision error. In order to determine with
confidence whether the true value of the parameter is above or below the action level (depending on
the more severe decision error), the site manager would need to collect a large amount of data,
increase the precision of the measurements, or both. If taken to an extreme, the cost of collecting data
can exceed the cost of making a decision error, especially where the consequences of the decision
error may be relatively minor. Therefore, the site manager should establish the gray region by
balancing the resources needed to "make a close call" versus the consequences of making that decision
error.
In Figure 6-1, the gray region has been set below the action level in the area where the site
manager has determined that the decision errors have the least consequence. The width of the gray
region indicates that the site manager does not wish to control decision errors when the true
concentration at the site is between 80 and 100 ppm.
Assign probability values to points above and below the action level that reflect the acceptable
probability for the occurrence of decision errors.
Assign probability values to points above and below the action level that reflect the site
manager's acceptable limits for making an incorrect decision. The most stringent limits on decision
errors that are typically encountered for environmental data are .01 (1%) for both the false positive and
false negative decision errors (a and (3). This guidance recommends using .01 as the starting point for
setting decision error rates.1 The most frequent reasons for setting limits greater than .01 are that the
consequences of the decision errors may not be severe enough to warrant setting decision error rates
that are this stringent. If the decision is made to relax the decision error rates from .01 for false
positive and false negative decision errors, the scoping team should document the rationale for setting
the decision error rate. This rationale may include potential impacts on cost, human health, and
ecological conditions.
1 The value of .01 should not be considered a prescriptive value for setting decision error rates, nor should it be
considered as the policy of EPA to encourage the use of any particular decision error rate.
33
-------
Repeat this activity for both sides of the gray region. Generally, the acceptable limits for
making a decision error should decrease as the consequences of a decision error become more severe
further away from the action level.
Figure 6-1 shows that from the action level to a true value of 150 ppm for the parameter of
interest, the site manager will tolerate a 5% chance of deciding that the true value is below the action
level, based on field investigation data. If the true value is greater than 150 ppm, the site manager
will tolerate only a 1% chance of deciding the true value is really below the action level. Below the
action level, from 60-80 ppm the site manager will tolerate deciding the true value is above the action
level 10% of the time, and between 40-60 ppm the site manager will allow a false negative decision
error rate of 5%.
Check the limits on decision errors to ensure that they accurately reflect the site manager's
concerns about the relative consequences for each type of decision error.
The acceptable limits on decision errors should be smallest (i.e., have the lowest probability of
error) for cases where the site manager has greatest concern for decision errors. This means that if
one type of error is more serious than another, then its acceptable limits should be smaller (more
restrictive). In addition, the limits on decision errors are usually largest (high probability of error can
be tolerated) near the action level, since the consequences of decision errors are generally less severe
as the action level is approached. Verify that the site manager's acceptable limits on decision errors
are consistent with these principles.
The Design Performance Goal Diagram (which is sometimes called a "Decision Performance
Curve") can be refined by breaking the "steps" of decision errors into smaller units. This would have
the effect of adding rows of information to its corresponding Decision Error Limits Table. The
information from the diagram will be used in the final step of the DQO Process (Step 7: OPTIMIZE
THE DESIGN) in order to construct a statistically based evaluation of how well the sampling design
will meet the DQOs. This evaluation involves the construction of a power curve, which is a graphical
description of a sampling design's expected performance. If the power curve lies within the
acceptable regions of the Design Performance Goal Diagram, then the corresponding sampling design
satisfies the site manager's acceptable limits on decision errors.
Appendix I, Section F, contains additional information on specifying limits on decision errors.
6.3 OUTPUTS
The outputs from this step are the site manager's acceptable decision error rates based on a
consideration of the consequences of making an incorrect decision. These limits on decision errors
can be expressed in a Decision Error Limits Table or in a Design Performance Goal Diagram.
34
-------
l!
O) _
c g
"6 -.is
'§ 3
O a)
"6 ~
•^ "P
— a)
I |
s ^
sXXX^ ^v Acceptable
>OOCv \ False
xxXX/i Posttlve
xxxX> Decision
^xxxxl Error Rate8
^oS^S^So
x>oSS
)OOvv oray noyiun _
OOOO^ (Relatively Large
A/Vvx! Decision Error
XjQVCt "Rates are
vvS?V:! Considered
CXXXX Acceptable.)
§888QOO<^
Decision XCKXyXC*
• Error Rates XXxSCJ
\ oSSo^
| \ ^OT
xVyVi
50 I 70 I 90 1 110 130 I 150 170 I 190 I
60 80 100 120 140 160 180 200
OQ^
1 573
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
' Action Level
True Value of the Parameter (Mean Concentration, ppm)
Figure 6-1. An Example of a Design Performance Goal Diagram
(Baseline condition: parameter exceeds action level)
True
concentration
50 to 60 ppm
60 to 80
80 to 100
100 to 150
150 to 200
Correct decision
does not exceed
action level
exceeds action
level
Acceptable
probability of making
an incorrect decision
(a decision error)
5%
10%
gray region—no
probability specified
5%
1%
Table 6-1. Decision Error Limits Table Corresponding to Figure 6-1
35
-------
Acceptable Probability of Deciding that
the Parameter Exceeds the Action Level
>
looooooppo— '
'-•.hocoj^bibj-jcoio
. .
U.VIO
0
•>v Acceptable
-^ False
^Negative
— Decision
Error Rates
Gray Rogion
(Relatively Large
Decision Error
Rates are
Considered
Acceptable.)
50 1 70 1 90 I 110 1 130 I 150 170 I 190 I
60 80 100 120 140 160 180 200
"• — Action Level
True Value of the Parameter (Mean Concentration, ppm
1
OOK
.93
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
)
Figure 6-2. An Example of a Design Performance Goal Diagram
(Baseline condition: parameter less than action level)
True
concentration
50 to 60 ppm
60 to 100
100 to 120
120 to 150
150 to 200
Correct decision
does not exceed
action level
exceeds action
level
Acceptable
probability of making
an incorrect decision
(a decision error)
5%
10%
gray region—no
probability specified
20%
5%
Table 6-2. Decision Error Limits Table Corresponding to Figure 6-2
36
-------
CHAPTER?
STEP 7: OPTIMIZE THE DESIGN
THE DATA QUALITY OBJECTIVES PROCESS
State the Problem
*
Identify the Decision >
* /
Identify Inputs to the Decision
*/
Define the Study Boundaries
/*
Develop a Decision Rule
/ *
/
Specify Limits on Decision Errors
OPTIMIZE THE DESIGN
Purpose
• Identify the most resource-effective sampling
and analysis design for generating data that are
expected to satisfy the DQOs.
Activities
• Review the DQO outputs and existing
environmental data.
• Develop general sampling and analysis
design alternatives.
• For each design alternative, verify that
the DQOs are satisfied.
• Select the most resource-effective design that
satisfies all of the DQOs.
• Document the operational details and
theoretical assumptions of the selected
design in the Sampling and Analysis Plan.
Optirj3iz»-ttTe"TJesSgn for Obtaining Data
7.1 BACKGROUND
The purpose of this step is to identify the most resource-effective sampling design that
generates data which satisfy the DQOs specified in the preceding steps. To develop the optimal
design for this study, it may be necessary to work through this step more than once after revisiting
previous steps of the DQO Process.
This step provides a general description of the activities necessary to generate and select
sampling designs that satisfy the DQOs. In addition, it contains information about how the outputs
from the previous six steps of the DQO Process are used in developing a statistical design. Appendix
I, Section G, discusses the basic principles of developing a statistical design and some basic design
options. This document, however, does not give detailed guidance on the mathematical procedures
involved in developing a statistical sampling design; for this type of guidance, see the references cited
in Appendix I, Section G, or consult with a statistician. Site managers also may want to use EPA's
DQO Decision Error Feasibility Trials software,1 which provides a first-pass rough estimate of sample
1 U.S. EPA. 1993. Data Quality Objectives Decision Error Feasibility Trials Software for Personal Computers.
37
-------
sizes required to satisfy the DQOs. This user-friendly PC software can help speed up the first iteration
through the DQO process.
For most field investigations, a probabilistic sampling approach is necessary for extrapolating
results from a set of samples to the entire site. By combining an efficient probabilistic sampling
design with a statistical hypothesis test, the decision maker will be able to optimize resources such as
funding, personnel, and temporal constraints while still meeting the DQOs. The hypothesis test used
in analyzing the data is an extremely important part of the statistical design, since it provides the
theoretical underpinnings for selecting the number, type, location, and timing of environmental
samples. While it may be true that the hypothesis test may be refined or changed later in the light of
what is discovered when collecting and examining the data, it is essential to have a plan for the
statistical analysis of the data before collecting samples so that the data are more likely to support the
ultimate decision.
For some field investigations, a non-probabilistic (judgmental) sampling approach is
acceptable. A judgmental sampling design consists of directed samples where the decision maker (or
technical expert) selects the specific sampling locations.2 Typically this occurs when the site manager
wants to confirm the existence of contamination at specific locations, based on visual or historical
information. However, when non-probabilistic sampling approaches are used, quantitative statements
about data quality are limited to the measurement error component of total study error. If the site
manager wishes to draw conclusions about areas of the site beyond the exact locations where samples
were taken, then a probabilistic approach should be used. This will allow the site manager to make
quantitative statement about the sampling error component of total study error, and thus determine the
probability of making a decision error regarding larger areas of the site.
Even if a judgmental sampling design is chosen, it is important to implement all applicable
activities of this step. This will ensure that the qualitative data quality objectives, such as budget,
schedule, and the temporal and spatial constraints (boundaries) are met. In addition, this step will help
the scoping team document:
1. the reasons for selecting a non-probabilistic sampling approach;
2. the reasons for selecting specific sampling locations; and
3. the expected performance of the sampling design with respect to the qualitative DQOs.
7.2 ACTIVITIES
Review the DQO Outputs and Existing Environmental Data
The outputs from the previous steps of the DQO Process provide a succinct collection of
information that is used to develop the sampling design in the following way:
• The limits on decision errors provide crucial information for selecting the number of
samples to be collected, the number of analyses per sample, and the hypotheses to be
tested.
samples or transect samples contain an element of randomization because the initial sampling point is chosen
randomly. Therefore they are considered probabilistic designs, not judgmental.
38
-------
• The inputs, boundaries, and decision rule are used in deciding the location and timing of
samples.
Therefore, the scoping team should review the previous DQO outputs and confirm the budget for
sampling and analysis, and the project schedule (especially deadlines). List any logistical or
administrative limitations, such as weather, equipment, and personnel availability identified in Step 4:
DEFINE THE BOUNDARIES. Site characteristics, previous sample locations, quality control data,
and audit reports from earlier field investigations also provide valuable information to the sampling
design team (or statistician).
For probabilistic sampling designs, additional information will be needed regarding the
expected variability of contaminants. Consequently, any existing environmental data from the site (or
from similar sites) should be reviewed. Information about existing environmental data may have been
identified during Step 1: STATE THE PROBLEM and Step 3: IDENTIFY THE INPUTS. If no
existing data are available, it may be necessary to conduct a limited field investigation to develop an
adequate estimate of variability.
Develop General Sampling and Analysis Design Alternatives
The sampling design team will develop alternative sampling and analysis designs that could
generate data needed to test the hypothesis. To generate alternative designs, the statistician may vary
several different aspects of the design, such as the number and locations of samples collected in the
field, the types of samples collected, or the number of replicate analyses performed on samples.
For each sampling design, a statistical model should then be developed that describes the
relationship of the measured value to the "true" value. This mathematical formulation clarifies how
data generated from a design is to be interpreted and processed in testing the hypothesis. A tentative
analytic form for analyzing the resulting data (for example, a student's t-test or a tolerance interval)
should also be specified. Use this information to solve for the minimum sample size that satisfies the
decision maker's limits on decision errors. If the design involves multiple subsample sizes (e.g., for
stratification schemes), then select the optimal mix of subsample sizes.
It is important not to rule out any alternative analytical or field sampling methods due to
preconceptions about whether or not the method is "good enough." It must be remembered that the
objectives of the statistical design are to limit the total error, which is a combination of sampling and
measurement error, to acceptable levels. Traditional laboratory methods tend to minimize
measurement error, but they can be so expensive that only a limited number of samples can be
analyzed within the budget. There often may be advantages to using less precise methods that are
relatively inexpensive, thereby allowing a significantly larger number of samples to be taken. Such a
design would trade off an increase in measurement error for a decrease in sampling error. Given the
large amount of natural variability in many environmental studies, this approach may reduce overall
costs while limiting the total decision error rates to acceptable levels just as well as a design based on
traditional laboratory methods.
39
-------
For Each Design Alternative, Verify that the DQOs are Satisfied
Verify that each design alternative satisfies all of the DQOs, including limits on decision
errors, budget, schedule, and practical constraints. If none of the designs satisfy the DQOs, the
scoping team may need to:
• increase the acceptable decision errors rates;
• increase the width of the gray region;
• relax other project constraints, such as available personnel;
• increase funding for sampling and analysis; or
• change the boundaries; it may be possible to reduce sampling and analysis costs by
changing or eliminating subgroups that will require separate decisions.
Select the Most Resource-Effective Design that Satisfies All of the DQOs
The design team should perform a sensitivity analysis on the alternative designs to see how
each design performs when the assumptions are changed, together with the impact on costs and
resources. Typically, this means changing certain parameters within some reasonable range, and
seeing how each of these changes influences the expected decision error rates. For example, if the
contaminant variability is higher or lower than assumed for the design, what happens to the design
performance? Or, if the final remedial level is more/less stringent than the assumed action level, what
happens to the design performance? A Statistical Power Curve is a useful statistical tool used to
evaluate whether a sampling design has the ability to meet the DQOs.3 An example of a Power
Curve is shown in Figure 7-1.
Evaluate the design options based on cost and ability to meet the DQO constraints and select
the most resource-effective design among the alternatives. The "most resource-effective" may be the
lowest cost alternative that meets the DQOs, or it may be a relatively low-cost design that still
performs well when the design assumptions change.
Document the Operational Details and Theoretical Assumptions of the Selected Design in the
Sampling and Analysis Plan
Once the final design has been selected, it is important to ensure that the design is properly
documented. This will improve efficiency and effectiveness of later stages of the data collection and
analysis process, such as the development of field sampling procedures, quality control procedures, and
statistical procedures for analysis of the data. The key to successful design documentation is in
drawing the link between the statistical assumptions on which the design is based and the practical
activities that ensure that these assumptions generally hold true.
The operational requirements for implementing the sampling design are documented in the
Field Sampling Plan and the Quality Assurance Project Plan, both of which are included in the
Sampling and Analysis Plan. Design elements that must be documented include:
• sample types (e.g., composite vs. grab samples);
3A Power Curve provides a graphical depiction of the sensitivity of a design; the steeper the curve, the more sensitive the
design will be in detecting conditions when the baseline (null) hypothesis should be rejected.
40
-------
Acceptable
False
Negative
POWER CURVE
Acceptable
False
Positive
Decision
Error Rates
(Relatively Large
Deasion Error
Rates are
Considered
Acceptable.)
0.95
50 I 70 I 90 I 110 I 130 I 150 170 I 190 I
60 80 100 120 140 160 180 200
t_
Action Level
True Value of the Parameter (Mean Concentration, ppm)
Figure 7-1. An Example of a Power Curve
• general collection techniques (e.g., split spoon vs. core drill, or activated charcoal media
vs. evacuated canister);
• sample support (i.e., the amount of material to be collected for each sample);
• sample locations (surface coordinates and depth) and how the locations were selected;
• timing issues for sample collection, handling, and analysis;
• analytical methods (or performance standards); and
• quality assurance and quality control needs.
For probabilistic sampling designs, the statistical model and assumptions must also be
documented. This item is often omitted, yet it can be one of the most important aspects of the design
documentation. If the theoretical basis for the design is documented, then the project team has a basis
for handling unexpected problems that inevitably arise in the field. This will help maintain the overall
validity of the study in the face of unavoidable deviations from the original design.
41
-------
7.3 OUTPUTS
The outputs for this step include the optimal (most resource-effective) sampling design for the
field investigation, along with documentation of the key assumptions underlying the design. The data
collected using this design are expected to be "adequate" for the site manager's or other decision
maker's needs.
7.4 SUPERFUND DATA CATEGORIES
During the sampling design step, the design team identified design elements that relate to
QA/QC procedures. As explained later in Chapter 8, these QA/QC-related design elements are
combined with other required QA/QC procedures, and the complete set of QA/QC requirements for the
project are incorporated into the quality assurance project plan (QAPP). The DQC Process provides a
logical basis for linking QA/QC procedures to the intended use of the data, primarily through the
decision maker's acceptable limits on decision errors. The translation of the site manager's acceptable
limits on decision errors into specific QA/QC requirements is done during Step 7: OPTIMIZE THE
DESIGN and completed in the QAPP development process.4
To assist in the interpretation of data, the Superfund program has developed the following two
descriptive data categories:
• Screening data with definitive confirmation;
• Definitive data.
These two data categories are associated with specific quality assurance and quality control
elements, and may be generated using a wide range of analytical methods. The particular type of data
to be generated depends on the qualitative and quantitative DQOs developed during application of the
DQO Process. The decision on the type of data to be collected should not be made prior to
completion of the entire DQO Process.
Screening Data with Definitive Confirmation
Definition of Screening Data
Screening data are generated by rapid, less precise methods of analysis with less rigorous
sample preparation. Sample preparation steps may be restricted to simple procedures such as dilution
with a solvent, instead of elaborate extraction/digestion and cleanup. Screening data provide analyte
identification and quantification, although the quantification may be relatively imprecise. At least 10%
of the screening data are confirmed using analytical methods and QA/QC procedures and criteria
associated with definitive data. Screening data without associated confirmation data are not considered
to be data of known quality.
4 For more information about the QAPP development process, see Guidance for Preparing, Reviewing, and Implementing
Quality Assurance Project Plans for Environmental Programs, EPA/QA/G-5 (Draft).
42
-------
Screening Data QA/QC Elements
• Sample documentation (location, date and time collected, batch, etc.);
• Chain of custody (when appropriate);
• Sampling design approach (systematic, simple or stratified random, judgmental, etc.);
• Initial and continuing calibration;
• Determination and documentation of detection limits;
• Analyte(s) identification;
• Analyte(s) quantification;
• Analytical error determination:5 An appropriate number of replicate aliquots, as specified
in the QAPP, are taken from at least one thoroughly homogenized sample, the replicate
aliquots are analyzed, and standard laboratory QC parameters (such as variance, mean, and
coefficient of variation) are calculated and compared to method-specific performance
requirements specified in the QAPP;
• Definitive confirmation: at least 10% of the screening data must be confirmed with
definitive data as described below. As a minimum, at least three screening samples
reported above the action level (if any) and three screening samples reported below the
action level (or as non-detects, ND) should be randomly selected from the appropriate
group and confirmed.
Definitive Data
Definition of Definitive Data
Definitive data are generated using rigorous analytical methods, such as approved EPA
reference methods. Data are analyte-specific, with confirmation of analyte identity and concentration.
Methods produce tangible raw data (e.g., chromatograms, spectra, digital values) in the form of paper
printouts or computer-generated electronic files. Data may be generated at the site or at an off-site
location, as long as the QA/QC requirements are satisfied. For the data to be definitive, either
analytical or total measurement error must be determined.
Definitive Data QA/QC Elements
• Sample documentation (location, date and time collected, batch, etc.);
• Chain of custody (when appropriate);
• Sampling design approach (systematic, simple or stratified random, judgmental, etc.);
• Initial and continuing calibration;
• Determination and documentation of detection limits;
• Analyte(s) identification;
• Analyte(s) quantification;
• QC blanks (trip, method, rinsate);
• Matrix spike recoveries;
• Performance Evaluation (PE) samples (when specified);
5 The procedures identified here measure the precision of the analytical method, and are required when total measurement
error is not determined under confirmation step.
43
-------
• Analytical error determination (measures precision of analytical method): An appropriate
number of replicate aliquots, as specified in the QAPP, are taken from at least one
thoroughly homogenized sample, the replicate aliquots are analyzed, and standard
laboratory QC parameters (such as variance, mean, and coefficient of variation) are
calculated and compared to method-specific performance requirements defined in the
QAPP;
• Total measurement error determination (measures overall precision of measurement system,
from sample acquisition through analysis): An appropriate number of co-located samples
as determined by the QAPP are independently collected from the same location and
analyzed following standard operating procedures. Based on these analytical results,
standard laboratory QC parameters such as variance, mean, and coefficient of variation
should be calculated and compared to established measurement error goals. This procedure
may be required for each matrix under investigation, and may be repeated for a given
matrix at more than one location at the site.
Impact of Data Categories on Existing Superfund Guidance
These Data Categories replace references to analytical levels, quality assurance objectives, and
data use categories. The major documents impacted by the Data Categories are:
- Data Quality Objective Guidance for Remedial Response Activities: Development Process
and Case Studies: EPA/540/G-87/003 and 004, OSWER Directive 9355.0-7B;
- Quality Assurance/Quality Control Guidance for Removal Activities: Sampling QA/QC
Plan and Data Validation Procedures: EPA/540/G-90/004, OSWER Directive 9360.4-01
April 1990; and
- Guidance for Performing Site Inspections Under CERCLA, OSWER Directive 9:"'
August 1992.
44
-------
CHAPTER 8
BEYOND THE DQO PROCESS:
The Sampling and Analysis Plan and Data Quality Assessment
8.1 OVERVIEW
This chapter explains some important QA management steps that occur after the DQO Process
has been completed. The DQO Process is part of the planning phase of the data collection life cycle,
as illustrated in Figure 8-1. At the completion of the DQO Process, the site manager will have
documented the project objectives and key performance requirements for the data operations in the
DQOs, and will have identified a sampling design that is expected to achieve the DQOs. The
sampling design and DQOs are used to develop the Quality Assurance Project Plan (QAPP) and the
Field Sampling Plan (FSP), both of which are included in the Sampling and Analysis Plan (SAP). The
SAP provides the detailed site-specific objectives, specifications, and procedures needed to conduct a
successful field investigation. During the implementation phase of the data collection life cycle, the
SAP is executed and the samples are collected and analyzed. During the assessment phase, Data
Quality Assessment (DQA) is performed on the data to determine if the DQOs have been satisfied.
The relationships between the DQO Process and these subsequent activities is explained in more detail
below.
8.2 SAMPLING AND ANALYSIS PLAN DEVELOPMENT
The SAP is a formal Superfund project document that specifies the process for obtaining
environmental data of sufficient quantity and quality to satisfy the project objectives. The DQO
Process can be viewed as a preliminary step in the SAP development process, since it logically
precedes the actual development of the SAP document, as shown in the right half of Figure 8-1. The
outputs of the DQO Process feed directly into the development of the QAPP and the FSP, which are
the two main elements of the SAP. Thus, the SAP is a single document that integrates the DQOs,
QAPP, and FSP into a coherent plan for collecting defensible data that are of known quality adequate
for the data's intended use.
The Quality Assurance Project Plan
The QAPP is required for all EPA data collection activities. The QAPP contains information
on project management, measurement and data acquisition, assessment and oversight, and data
validation and useability. DQOs are a formal element of the QAPP, yet information contained in the
DQOs relates indirectly to many other elements of the QAPP. In essence, the DQOs provide
statements about the expectations and requirements of the data user (such as a site manager). In the
QAPP, these requirements are translated into measurement performance specifications and QA/QC
procedures for the data suppliers, to provide them with the information they need to satisfy the data
user's needs.
The Field Sampling Plan
The FSP specifies how to conduct field activities to obtain the environmental data needed for
the project. Whereas the DQO Process generates a sampling design based on the data user's needs,
the FSP provides the operational plan for executing that sampling design. The FSP identifies
45
-------
PLANNING
Data Quality Objectives Process
Sampling and Analysis Plan Development
1
IMPLEMENTATION
Sampling Execution
Sample Analysis
1
ASSESSMENT
Data Quality Assessment
QA PLANNING FOR SUPERFUND
DATA COLLECTION
Data Quality Objectives Process
1
OUTPUTS
Data
Quality
Objectives
I
Sampling
Design
!
INPUTS ,
1
Sampling and Analysis Plan
Development
1
Quality
Assurance
Project
Plan
I
Reid
Sampling
Plan I
ELEMENTS , ,
Sampling and
Analysis Plan
SINGLE
INTEGRATED
DOCUMENT
Figure 8-1. QA Planning and the Data Life Cycle
procedures for collecting samples in a manner that is consistent with the underlying theory and
assumptions upon which the sampling design is based. This, along with the QA/QC procedures
specified in the QAPP, helps ensure that the resulting data will be valid and appropriate for their
intended use.
8.3 DATA QUALITY ASSESSMENT
After the environmental data have been collected and validated in accordance with the SAP,
the data must be evaluated to determine whether the DQOs have been satisfied. EPA has developed
guidance on Data Quality Assessment (DQA) to address this need.1 DQA involves the application of
statistical tools to determine whether the variability and bias in the data are small enough to allow the
site manager to use the data to support the decision with acceptable confidence. The five main steps
of the DQA process are illustrated in Figure 8-2.
1 U. S. Environmental Protection Agency (EPA). 1993. Guidance for Conducting Environmental Data Quality
Assessments. EPA/QA/G-9.
46
-------
1 I DEFINE NULL AND ALTERNATIVE HYPOTHESES
2 DETERMINE ACCEPTABLE DECISION ERROR RATES
3 | IDENTIFY STATISTICAL TEST AND ASSUMPTIONS
I
ASSESS VALIDITY OF STATISTICAL TEST
PERFORM TEST AND ASSESS DESIGN
Figure 8-2. The Data Quality Assessment Process
For DQA to be effective and efficient, the crucial groundwork must have been laid in the
planning phase. The DQOs provide the evaluation criteria by which the data will be assessed, and the
SAP provides the blueprint by which the data will be generated. If the planning has been carried out
thoughtfully, and the plans are executed successfully, then the DQA will provide answers that are
useful for the site manager.
47
-------
APPENDIX I
TECHNICAL SUPPLEMENT TO
THE DATA QUALITY OBJECTIVES
PROCESS
SECTION A: STATE THE PROBLEM
THE CONCEPTUAL SITE MODEL AND THE DQO PROCESS
This discussion focuses on the relationship between the conceptual site model (CSM) and the
DQO process for Phase I of the advanced assessment decision. The DQO process involves a series of
steps that gradually narrows, focuses, and divides a potentially complex problem into manageable
pieces. Site problems can be very complex, especially in cases where contamination is present in
several media or when cross-media contamination exists.
The CSM is developed using readily available (existing) data and illustrates the relationship
between contaminants, retention/transport media, and receptors. The relationship between
contaminants, retention/transport media, potential receptors, and the possibility for exposure to occur is
central to a description of the problem, which is required in the first step of the DQO process.
The CSM also facilitates understanding of why new environmental data may be needed to
resolve the contamination problem. The need for new environmental data may be confirmed by using
the DQO process.
The CSM also serves as a framework for identifying data gaps. Data gaps identified in the
CSM can be addressed by listing them as inputs to the decision in the third step of the DQO process.
Information in the CSM about the location of contamination and potential receptors, as well as
contaminant fate and transport, can be used to establish spatial and temporal boundaries for the field
investigation in the fourth step of the DQO process. In summary, the development of the CSM
directly influences the generation of the outputs of the first four steps of the DQO process.
The following discussion provides more information on developing the CSM and on defining
exposure scenarios.
DEVELOP/REFINE THE CONCEPTUAL SITE MODEL
The following series of tasks are most appropriate for scoping site inspections and Phase I
remedial investigations. In the later phases of the Superfund process, it is most important to confirm
the exposure scenarios and generate a diagram depicting contaminant concentrations superimposed on
a site map.
49
-------
(I) Collect existing site data. Gather all historical site data and other pertinent information
and compile an up-to-date data base on the site. Use this information to prepare written
descriptions and graphic illustrations (diagrams) of contaminant sources, migration and
exposure pathways, and potential physical and environmental targets or receptors. These
illustrations and diagrams condense and document the important elements of exposure,
and facilitate identification of the data needed to assess the potential risks of exposure
associated with the site.
(2) Organize, analyze, and interpret existing site data. Organize site data according to:
• information on sources and source types (e.g., landfills, impoundments, lagoons, or
ditches);
• affected media;
• site's physical and waste characteristics that can influence migration or containment;
and
• potential migration and exposure pathways and receptors.
Summarize the analytical results of previous data collection activities with respect to:
• contaminants of interest;
• contaminant concentrations in each media and the practical concentration ranges of
concern;
• anticipated analytical methods; and
• analytical method performance characteristics such as precision, bias, and method
detection limits.
Perform a site reconnaissance with photographic equipment to document and gather
current information to determine whether observations are consistent with the current
understanding of the site. During the site visit, search for signs of contamination, such as
the appearance of surface water, stressed vegetation, or discolored soil. Use topographic
maps to mark well locations and estimate the extent of source areas or the presence of
sensitive environs. Try to uncover information that will help assess the apparent stability
of the site, such as leaking containment structures or weakening beams. Conduct limited
sampling with portable equipment and gather additional anecdotal information from local
sources that may reveal disposal areas or practices that were previously unknown and
may affect contaminant migration.
(3) Determine if existing data can support the conceptual site model. Assess whether a
limited field investigation is needed to adequately define the conceptual site model. This
assessment helps determine whether or not samples need to be collected and, if so, if they
will be used to supplement or verify existing data.
(4) Define the conceptual site model. The compilation, organization, and interpretation of
historical site data now can be used to develop a diagram that illustrates the conceptual
site model. Representing the linkages among contaminant sources, release mechanisms,
pathways, exposure routes, and receptors in a diagram is a very useful and efficient
technique for summarizing the current understanding of the contamination problem.
50
-------
The written description should be supported with maps and cross-sections depicting
contaminants and contaminant distribution, as appropriate.
DEFINE EXPOSURE SCENARIOS
(1) Identify media of concern. Use historical site data including analytical data to identify
media that is currently contaminated or that can become contaminated through migration.
(2) Identify the contaminants of concern. Develop a broad list of contaminants known or
suspected to be at the site. A comprehensive approach to identifying contaminants
minimizes missing chemicals that may contribute to overall risk at the site or those that
may not contribute to risk significantly, but are present in large quantities.
(3) Define future land use. Currently, a formula for determining the probable future land
use for a site is unavailable. Therefore, begin by considering the current site land use and
determine if factors such as zoning laws, renovation projects, and anticipated population
growth may influence the future land use for a site. The "Risk Assessment Guidance for
Superfund (RAGS) Human Health Evaluation Manual, Part A" (U.S. EPA, July 1989)
provides more detailed support for defining future land use.
(4) Define Applicable or Relevant and Appropriate Requirements (ARARs). Identify the
ARARs for the site. Start with the current list of contaminants and list all the chemical-
specific ARARs from all the environmental statutes. Along with the standard, note the
jurisdictional prerequisites under which the ARAR was established. This information will
be used to determine the applicability, relevancy, and appropriateness of the standard for
CERCLA. The search continues beyond chemical-specific ARARs. It should also
include location- and action-specific ARARs. Further assistance in identifying ARARs
for the site is provided in the "CERCLA Compliance with Other Laws Manual" (U.S.
EPA, August 1988).
(5) Assemble exposure scenarios. Identify all available exposure pathways associated with
the site. An exposure pathway describes a unique mechanism by which a receptor is
exposed to site-related contaminants. Each exposure pathway includes:
• a source and release mechanism;
• a retention and transport medium;
• an exposure point; and
• an exposure route.
For each medium and land-use combination, identify the most appropriate exposure
scenarios.
At this point, several components of an exposure scenario have already been identified
and should be brought forward. One of these components is the potential receptor identified in
the conceptual site model. Use the potential receptors and characterize the exposure setting as
it relates to receptor locations and average daily activity patterns. The scoping team also
considers those physical site characteristics and waste characteristics that influence contaminant
migration. Other components of the conceptual site model that assist this effort are the
51
-------
identified sources and affected or potentially contaminated media. Once these exposure-related
elements have been identified, consider receptor locations and activity patterns and any point
of potential contact with these media. After defining all potential exposure points, identify
probable exposure routes (i.e., ingestion, inhalation, dermal contact).
Next, assemble all of the information collected above into complete exposure pathways
and combine exposure pathways as appropriate.
52
-------
SECTION B: IDENTIFY THE DECISION
RELATIONSHIPS BETWEEN THE DECISION STATEMENTS
AND PRE-SACM SUPERFUND PROCESS
The purpose of the following information is to help users correlate the first three decisions
presented in the guidance to the pre-SACM Superfund process.
Superfund site assessment encompasses identification, evaluation, and response to uncontrolled
releases of hazardous substances and determination of the level of post-cleanup risks to human health
and the environment. To evaluate a site efficiently and minimize unnecessary expenditure of
resources, site assessment activities are performed in stages or tiers.
According to the Office of Solid Waste and Emergency Response Interim Guidance on
"SACM Regional Decision Teams" (Publication 9203.1-0-51, December 1992), site response action
options that are based on information or data generated in the early assessment stage (i.e, site
inspection1) include recommending the initiation of RI activities. Therefore, in general, site inspection
and removal data collection activities and the decisions they support occur in the early assessment
stage timeframe. A statement of the early assessment decision is, "Determine whether the release (or
potential release) poses a threat to human health or the environment." Recognize that a removal action
can occur at any time during site assessment.
The Advanced Assessment Stage activities follow the early assessment. As stated in the
previous paragraph, a remedial investigation2data collection activity and the decision it supports
occurs in the Advanced Assessment Phase I timeframe. A statement of the Advanced Assessment
Phase I decision is, "Determine whether contaminant of concern concentrations exceed ARARs or
contaminant concentrations corresponding to the target risk level for the site."
The Advanced Assessment Phase II data collection activity is conducted only if a
determination is made that contaminant concentrations exceed ARARs or concentrations corresponding
to the target risk level and, as a result, the site warrants a further response action. The Advanced
Assessment Phase E data collection activity occurs in the remedial investigation/feasibility study
timeframe.
SACM Decisions in the Context of the DQO Process
This guidance specifically discusses four site decisions that often require field investigations.
Three are site assessment decisions and the fourth is the cleanup verification decision after the
remedial response action has been completed. This subsection discusses these SACM decisions in the
'The Interim guidance also references Preliminary Assessment/Removal Assessment as part of the Early Assessment Stage activities.
However, this guidance focuses on activities that involve collection of new environmental data. Typically, new environmental data are not
collected during the preliminary assessment. Therefore, this guidance is most concerned with data collection activities in support of site
inspections and removal assessment during the early assessment stage.
2 A combined focused or expanded Sl/RI data collection can also be conducted during the advanced assessment Phase I.
53
-------
context of the DQO process, along with notations that relate the SACM decisions to the corresponding
phase of the pre-remedial and remedial programs.
Early Assessment (Pre-Remedial) Stage
The early assessment (i.e., removal preliminary assessment or remedial preliminary assessment)
allows site managers to screen sites and select those that warrant further assessment and possible
response action using either the removal and/or remedial authorities.3 These preliminary assessments
typically are executed without the collection of waste or environmental samples. Instead, they rely on
the collection of readily available information and therefore are unlikely to realize the full benefit of
DQO application. The assessment may result in a decision to recommend the site evaluation
accomplished (SEA) designation or to recommend further assessment and possible response action for
the site. The further assessment recommendation may involve collection of additional data to perform
a focused site inspection (SI) or an expanded site inspection/remedial investigation (ESI/RI), if the site
has a high likelihood of remedial action. The SI and ESI/RI field investigations usually require the
collection of waste or environmental samples and would benefit from a full application of the DQO
process. A possible response action recommendation may involve an emergency/time-critical removal
action, a non-timecritical early action (removal or early/interim remedial), the initiation of the NPL
listing process concurrent with the early response action or ESI/RI, and/or initiation of enforcement
activities. Generally, it may not be expedient to apply the DQO process to emergency/time-critical
removal action field investigations. On the other hand, DQOs should be developed for non-time-
critical early action field investigations.4
Advanced Assessment Stage (Remedial Investigation Phase I)
The field investigations in the advanced assessment stage field investigations are conducted in
phases. The primary purpose of the first phase is to support the risk assessment, which is an input to
the decision on whether the site warrants an additional response action. In this advanced site
assessment stage, the response action recommendation typically involves a non-time-critical removal or
early and/or long-term remedial action. Sites that require a response action enter the second phase of
the advanced assessment.
Advanced Assessment (Remedial Investigation Phase II)
The purpose of the second phase of the advanced assessment is to determine the extent of
contamination that exceeds ARARs or contaminant concentrations corresponding to the target risk
level. Consistent with SACM and streamlining initiatives, this extent of contamination determination
is performed concurrently with the first phase of the advanced assessment.5 The extent of
contamination determination supports alternative development processes of both removal engineering
evaluation and cost analysis (EE/CA) and remedial feasibility studies (FS).6
3SACM Publication 9203.1-051, September 1992: "SACM Program Management Update, Assessing Sites Under the SACM," page 2.
4SACM Directive 9203.1-051, September 1992: "SACM Program Management Update, Early Action and Long-Term Action Under SACM".
'SACM Publication 9203.1-051, September 1992: "SACM Program Management Update, Assessing Sites Under the SACM," page 3.
'The extent of contamination decision may also support presumptive remedy and lightning ROD streamlining initiatives.
54
-------
Cleanup Attainment Stage
The final SACM decision that will require new data and be the focus of DQO development is
the cleanup attainment decision. This decision addresses whether final response actions achieved final
remediation levels or removal action levels.
55
-------
SECTION C: IDENTIFY THE INPUTS TO THE DECISION
DECISION-SPECIFIC ACTIVITIES
EARLY ASSESSMENT DECISION
The objective of this field investigation is to evaluate the degree to which the site presents a
threat to human health and the environment.
List the Inputs Needed to Support the Decision
Gather the following information during this phase:
• historical waste generation and disposal practices;
• hazardous substances associated with the site;
• potential sources of hazardous substances;
• important migration pathways and affected media;
• a comprehensive survey of targets;
• critical sample locations for the SI;
• contaminants or waste; and
« PA results.
Identify Informational Sources for Each Decision Input
Compile any readily available information about the site and its surroundings. PA
documentation, records that indicate the contaminants at the site, site photographs, and anecdotal
evidence are all potential informational sources. For more involved assessments, documentation of
observed releases, observed contamination, and levels of actual contamination at the site will be
required.
Identify the Inputs that will Require New Environmental Measurements
Some of the information identified in the previous activity may require environmental
measurements. List those inputs requiring environmental measurements that cannot be satisfied by
existing data from previous field investigations.
The following lists summarize the outputs for each decision.
List of Early Assessment Inputs
(1) List of Inputs Needed to Support the Decision:
• contaminant or waste migration pathway
• waste
• contaminants
57
-------
• action level1
(2) List of Inputs That Require New Environmental Measurements:
• contaminant concentrations
• background concentrations'
ADVANCED ASSESSMENT DECISION: PHASE I
List the Inputs Needed to Support the Decision
This stage of the cleanup process will involve determining the nature and magnitude of
contamination. To do so, it is necessary to identify potential contaminants and determine whether or
not their concentrations exceed ARARs or levels that pose an unacceptable risk. Therefore, the
relevant information includes:
• records indicating the contaminants that might be found at the site;
• information that identifies contaminants actually present at the site;
• information about how contaminant concentrations are distributed among media across the
site;
• ARARs (if they exist) or exposure assumptions that will be used in the preliminary
remediation goal (PRG) calculation;
• toxicity information for each contaminant;
• fate and transport information to be used in assessing exposure; and
• a target risk that provides a preliminary definition of the threshold of unacceptable risk.
Determine whether or not contaminant concentrations exceed ARARs or concentrations
corresponding to the target risk level. If ARARs exist, the decision involves determining if the site
complies with explicit regulatory criteria, such as a Maximum Concentration Limit (MCL) for ground
water near a drinking water well. If ARARs do not exist, and the decision will be based on estimates
of the risks posed by the site, then there may be several alternative methods by which site risks can be
estimated. Each method will require different informational inputs. The following suggested activities
apply to this latter, more complicated case.
• Consider each exposure pathway of concern.
• Identify the variables in the risk calculation for each pathway.
• Decide which variables will be estimated using site-specific information and which
variables will be assigned default values.
• For each variable that will be estimated using site-specific information, determine whether
the estimate will be based primarily on modeling or direct measurement, or both.
'This applies when a comparison of site contamination levels to background levels is the basis for decision making.
58
-------
List the sampling and analysis action level.2 If the decision is based on ARARs, then list the
ARARs; if the decision is based on site-specific risk, then list the target risk level.
List all of the decision inputs needed to determine if the site fails to comply with ARARs or
exceeds the acceptable target risk. In both cases, information on concentrations of contaminants will
be required. If the decision is based on site-specific risk, then information on each input to the PRO
calculation for each exposure pathway will be needed (the work done in developing the decision
support strategy should provide a good starting point). This will include the contaminant potency
factors, exposure pathways, fate and transport information, receptor types and activity levels or
patterns, and intake parameters.
Identify Informational Sources for Each Decision Input
For ARARs, identify the specific regulation. For risk-based decisions, identify informational
sources for the target risk and each input to the PRO calculation. Sources may include default values
derived from written guidance, historical records, census data, field measurements or observations, or
professional judgement. If the decision support strategy requires site-specific modeling to estimate any
of the variables in the risk calculation, then identify any key model parameters that need to be
estimated using site-specific information.
Determine if existing data from this site or similar sites exist. If the data do exist, evaluate
them qualitatively to see if they appear to be the type that are appropriate for the decision.
List the Inputs That Will Require New Environmental Measurements
Some of the sources identified in the previous activity will include field measurements. List
those inputs that require environmental measurements and that cannot be satisfied by existing data
from previous field investigations.
List of Advanced Assessment Decision, Phase I, Inputs
(1) List of Inputs Needed to Support the Decision:
• potential contaminants
• concentrations in space and perhaps time
• potency factors or dose/response relationships
• exposure pathways
• media (e.g., soil, surface water, ground water, air)
• rates of migration (within and between media)
• rates of dispersion/accumulation
2This is the contaminant concentration that corresponds to the target risk level, given various assumptions about exposure and contaminant
fate, transport, and dispersion mechanisms.
59
-------
• receptors
• types/subpopulations
• sensitivities
• numbers/densities
• activity levels/patterns
• target risk/ARARs
• site's physical and chemical characteristics that influence technology applicability (e.g.,
presence of organic components, soil permeability, and depth to impervious formation)
(2) List of Inputs That Require New Environmental Measurements:
• contaminant concentrations in space (and perhaps time) for each media of concern
• small- and large-scale variability in potential contaminant concentrations
• other measurements related to risk assessment, such as fate and transport model
parameters
ADVANCED ASSESSMENT DECISION: PHASE U (EXTENT OF CONTAMINATION)
Much of the information developed at this stage of the cleanup process builds on the
foundation laid in the previous stage (if DQOs were not developed for Advanced Assessment Phase I,
then it will be necessary to develop some of that information as part of Phase n). This decision
addresses the extent of contamination that will require remediation. Consequently, the information at
this stage will be similar in character to Phase I, but will be more specific or refined.
List the Inputs Needed to Support the Decision
To calculate the volume of media that will require remediation, information will be needed
about the specific locations where contaminant concentrations exceed ARARs or the sampling and
analysis action levels. Information on remedial alternative effectiveness, efficiency, and cost also will
be needed.
• List the contaminants with concentrations that exceed ARARs or the target risk. If the
decision is based on ARARs, then confirm the list of information required to determine
compliance with the ARARs for each contaminant. If the decision is based on site-
specific risk, then confirm the list of inputs to the PRG calculation that will be required
to determine the extent of contamination that exceeds the PRG.
• List the engineering information required to determine the effectiveness, efficiency, and
cost of each remedial alternative.
• If the removal action level or final remediation level differs from the sampling and
analysis action level,3 then identify the new inputs required to determine the location and
volume of media that exceed the removal action level or final remediation level.
'If decision inputs were not developed for the Phase I advanced assessment decision, then conduct the activities described above for that phase,
except use the final remediation level and the selected remedy in place of the preliminary action level and remedial alternatives, respectively.
60
-------
• List the inputs needed to determine the volume of media that exceeds ARARs or the
sampling and analysis action level.
• This phase focuses on the extent of contamination that will require remediation. The
approach for determining contaminant concentrations usually will follow directly from the
approach taken in Phase n. For decisions based on site-specific risks, the approach to
estimating risk variables also should be consistent with the approach taken in Phase II.
Identify Sources for Each Decision Input
These sources should be similar to those identified in Phase I, unless the removal action level
or final remediation level differs greatly from the sampling and analysis action level.
Identify the Inputs that will Require New Environmental Measurements
Examine the inputs derived from environmental measurements and list those inputs that will
not be satisfied by existing data.
List of Advanced Assessment Decision, Phase n, Inputs
(1) List of Inputs Needed to Support the Decision:
• removal/remedial technologies or alternatives
• contaminants
• refined exposure assumptions or baseline risk assessment assumptions
• sampling and analysis action level or final remediation level
(2) List of Inputs That Require New Environmental Measurements:
• contaminant concentrations
CLEANUP ATTAINMENT DECISION
This stage addresses a question much different than the previous two stages: Do contaminant
concentrations remaining after the remedial action exceed the final remediation level? Nonetheless, the
information required to answer this question closely parallels the information required in the first two
stages.
List the Inputs Needed to Support the Decision
The removal action level or the final remediation level serves as the criterion for deciding if
the response action is complete; hence the scope of information needed at this stage is less than that
61
-------
required in previous stages.4 For the cleanup attainment decision, the primary focus is on the
distribution of contaminant residual concentrations across the site.
• List the removal action level or final remediation level for each contaminant and identify
any other decision criteria that may be specified in the Engineering Evaluation/Cost
Analysis (EE/CA) or the ROD (for example, the ROD may require that a specific
statistical test be performed to determine if the site has attained the final remediation
levels).
• List the inputs required to determine if the contaminant concentrations exceed the
removal action level or final remediation levels.
• Identify any special concerns, such as the desire to ensure that no hot spots above a
certain size and concentration are left behind.
• List the cleanup attainment decision inputs that require field measurements that will not
be satisfied by existing data.
Identify Sources for Each Decision Input
Identify the information sources for each of the cleanup attainment decision inputs. It is
unlikely that any existing data will satisfy this need, unless the data were collected during the remedial
action timeframe (such as monitoring data).
List the Inputs that will Require New Environmental Measurements
List the cleanup attainment decision inputs that require field measurements that will not be
satisfied by existing data.
List of Cleanup Attainment Decision Inputs
(1) List of Inputs Needed to Support the Decision:
• removal action levels or final remediation levels for each contaminant
• distribution of contaminant (or surrogate) concentrations
(2) List of Inputs That Require New Environmental Measurements:
• contaminant (or surrogate) concentrations
4In previous stages, information about the risk calculation may have been included; however, this information is now subsumed within the
removal action level or the final remediation level. Likewise, Advanced Assessment Phase I required information about remedial technologies
and alternatives; after the ROD, the remedy has been selected, which reduces the scope of information required to make subsequent decisions.
62
-------
SECTION D: DEFINE THE STUDY BOUNDARIES
Section D provides the scoping team with relevant information about how to develop risk-
based, technology-based, and other scales of decision making. In addition, this section will focus on
defining spatial boundaries and scales of decision making for four media of concern: surface soil,
subsurface soil, surface water, and ground water.
1. SCALES OF DECISION MAKING
The following section provides relevant information about how to develop risk-based,
technology-based, and other scales of decision making.
RISK-BASED SCALES OF DECISION MAKING
Development of risk-based scales requires substantial input from and relies on the professional
judgement of the risk assessment member of the scoping team. In order to develop risk-based scales
of decision making, the scoping team must evaluate: (1) the daily activity and behavior pattern of the
most sensitive receptor; (2) the exposure pathway and route(s); (3) the current and future media use
designation; and (4) contaminant toxicity values. In some cases, ARARs or a target risk level may be
required to define the scale of decision making.
To make a risk-based decision, the sampling data should be representative of well-defined
areas, volumes, and time periods which the scoping team determines a receptor could be exposed to
given the anticipated use of the site. Since this scale is based on exposure assumptions, they are
referred to as "Exposure Units" (EUs). If possible, the EU should represent a direct correlation
between the area of contamination and the exposure that the receptor is likely to receive. Each media
will have its own unique type of EU. As an example, surface soil has an EU that is defined by length,
width, and depth of the surface soil layer.
TECHNOLOGY-BASED SCALES OF DECISION MAKING
If the Advanced Assessment Decision (Phase I) has already been made, the scoping team may
define a scale of decision making based on the technology that was chosen to remediate the site.
Scales of decision making that correspond to these areas are called Remediation Units (RUs). An RU
is defined as the subset of a medium that can reasonably be remediated with the selected remediation
technology (e.g., the minimum volume of soil that can be efficiently removed with a backhoe). RUs
are defined by the scoping team in order to design the most cost-effective remediation design. The
size of the RU will determine the scale of resolution that will be necessary for the sampling plan and
also the amount of material that will ultimately be remediated. For each medium, the optimal size of
an RU can be determined using a relative cost analysis and an estimate of (or assumptions about) the
variability and distribution of contaminants in the media. When the "relative cost" of remediation is
high compared to sample and analysis costs, and the variability of contaminants is fairly high (e.g., a
patchy distribution), studying each RU and remediating only those that are contributing to risk may
substantially reduce costs without decreasing the level of protection of the public. When the level of
variability is very low, the optimal RU size will most likely be the same as the EU.
63
-------
OTHER SCALES OF DECISION MAKING
In some instances it will be difficult or impossible to directly relate the size or volume of the
media to the exposure of a receptor and there may not be a technological approach that can be
translated into RUs. In these cases, the scoping team must select the scale of decision making that
combines the consideration of risk from exposure with practical considerations about an EU or RU
size. Again, the evaluation of the size or volume of an EU should be based on the future use of the
site (residential, light industrial, recreational, etc.) and the receptors' activity pattern at the site.
EXAMPLES OF SCALES OF DECISION MAKING
In order to explain the process of setting a scale of decision making, three short examples have
been provided. These examples are only meant to illustrate the concept of the scale of decision
making.
Example #1: Risk-Based Scale of Decision Making
Background — The fictitious site is situated in Montana where a lead smelter has operated
over the past 25 years and contaminated a site of approximately 35 acres with lead tailings
and ash from the smelter. The smelter site is surrounded by residential homes and it seems
likely that the site could be used as residential lots in the future. The primary contaminant of
concern on the site is lead in the soil. The exposure pathway is ingestion of soil and the
primary target receptor is small children. One of the primary activities of children that
exposes them to soil is playing in their backyard around areas that are devoid of vegetation.
In this case the risk assessor postulates that the majority of the soil exposure received by a
small child is in an area of the backyard that encompasses the sandbox and swing set.
Given this scenario, it would be reasonable for the scoping team to want to control uncertainty
in the sampling data related to the area or volume where children get the majority of their exposure.
Therefore the scoping team would set the scale of decision making to the 14'-14' area which is equal
to the average size of a backyard play area. This is a risk based scale of decision making because it is
possible to correlate the scale of decision making with the exposure of the most sensitive receptor.
Example #2: Technology-Based Scale of Decision Making
Lagoon Remediation — A Midwestern Coke Plant discharged process waste water into
lagoons on their property. This resulted in the contamination of sediments with organic
chemicals. Solid wastes from the same process were disposed of in several other lagoons and
landfill areas. These contained organic chemicals as well as inorganic contaminants. The
lagoons and landfill areas are surrounded by a wetland area which is the primary concern as
a receptor for the contamination. There are no human receptors nearby. The site manager
recognizes that the cleanup of the lagoons will involve more than one type of remediation
practice and is most likely to involve bio remediation and incineration to reduce the influence
of the organic chemicals.
The scoping team at this site choose to evaluate each lagoon separately based on the
assumption that each lagoon would have homogeneous contamination which could be remediated by a
single, but possibly separate, remediation process. Therefore, each lagoon is considered to be a
distinct RU.
64
-------
Example #3: Other Scales of Decision Making
Carolina Transformer — The soil at an abandoned transformer production and reclamation
facility has been contaminated with PCBs (polychlorinated biphenyls). The expected future
use of the site is light industrial and the major route of exposure is through soil ingestion.
The RPM is most concerned with exposure to children trespassers who play on the site.
In this scenario, the scoping team does not believe that there is a strong correlation between
the size of a soil area and the relative "amount" of exposure that the children will receive. However,
from the anticipated site activities of the children, they can select a size area (scale) that would be
protective under the RME if that area had an average concentration of PCBs below the sampling and
analysis action level. For this site, the scale of 1/2 acre was chosen as the Scale of Decision Making.
While this decision was based on some assumptions or risk and the consideration of the receptor's
activities, the scoping team had to finally make an estimate of the size area that would be protective of
the children rather than rely on a direct correlation between soil area and risk. This is what
differentiates this example from example #1, the risk-based scale of decision making.
2. MEDIA-SPECIFIC BOUNDARY DEVELOPMENT
This section provides specific information or considerations that are useful for the development
of boundaries for specific media. Each medium is treated as a separate chapter. It is useful to have
defined the geographic area of the investigation before using this section.
Surface soil and subsurface soil are treated separately in this guidance. Direct contact
exposure to contaminants in surface soil through ingestion, inhalation of airborne particulate and
dermal absorption exposure routes is the primary focus of the subsequent discussion. Subsurface soil
discussions, on the other hand, primarily focus on indirect exposure routes through other media such
as ground water.
(a) SURFACE SOIL
The media-specific boundary development for surface soil will provide relevant information to
help the scoping team define spatial boundaries and the scale of decision making for surface soil.
DEFINING THE MEDIA
The physical attributes that define surface soil include grain size, depth, relationship to water
(i.e., sand or sediment), organic material content, etc. The scoping team should consider how to
classify objects that appear in surface soil, such as rocks or debris, and whether or not they should be
sampled and/or remediated. The depth of soil that is classified as "surface soil" may be regulated or
standardized in some states or regions. Be sure to check with the proper offices and obtain the
necessary approval before making this decision.
65
-------
DEFINE THE SCALE OF DECISION MAKING FOR SURFACE SOIL
Below are descriptions of how to define the scale of decision making for surface soil.
Risk-Based Scales of Decision Making
(1) Identify the future land-use designation and exposure route and determine if it provides a
basis for defining an exposure area or volume.
(2) Define an area or volume of media within which the receptor is expected to limit his
daily activities or to which the receptor is expected to come into contact during the period
of exposure.
(3) Integrate the information from Steps 1 and 2 with the professional judgement of the risk
assessor in order to define an exposure area or volume. For example, for residential land
use where soil ingestion is determined to be the primary pathway of exposure, young
children may get the majority of their exposure from a typical yard area. A case where a
typical plot size was recommended as such an exposure area can be found in the Risk
Assessment Guidance for Superfund: Human Health Evaluation Manual (EPA July 1989)
in Chapter 6, Section 6.5.3, page 6-28. If the site-specific plot size is 1/3-acre, then the
Va-acre should be considered an estimate for the scale of decision making.
(4) Modify any estimated scales of decision making with information collected during the site
visit and information that may have been collected by the Agency for Toxic Substances
and Disease Registry if human monitoring was conducted. These scales may provide
additional clues about the activity patterns of the receptors.
Where it is difficult to establish a scale of decision making based on land use and receptor
behavior patterns, rely on standard default exposure area values that are available for media-specific
pathways in the Risk Assessment Guidance for Superfund: Human Health Evaluation Manual, Part A.
Contact the Risk Assessment Workgroup in the Toxics Integration Branch of EPA for their current
work on this topic or use a technology-based approach to define the scale of interest.
Technology-Based Scales of Decision Making
There are two types of technology-based scales of decision making. The first relies on
physical features of a site to suggest the scale. These may be features that divide the site into smaller
units, such as roads, buildings, or other physical impediments, or features that suggest the location of
contaminants, such as lagoons, trenches, or waste pits.
The second technological approach for defining the scale of decision making is driven by the
technology used to remove or clean up the contamination. This approach involves the identification of
the most efficient subset of media or minimum volume of contaminated material that can be removed
(i.e., the minimum amount of soil that can be removed with a backhoe) or remediated with the
selected technology during an operation of the equipment or treatment cycle.
66
-------
(b) SUBSURFACE SOIL
This section will describe relevant information to aid the scoping team to develop spatial
boundaries and scale of decision making for subsurface soil.
Because subsurface soil has the potential to distribute contaminants along several exposure
pathways, the development of boundaries must be based on exposure pathways that have been defined
in Step 1: STATE THE PROBLEM. This section will evaluate methods of developing boundaries for
subsurface soil by concentrating on two exposure pathways: 1) Direct Exposure — when the
subsurface soil becomes surface soil through routine building and landscaping operations; and 2)
Indirect Exposure — when the contaminants from the subsurface soil leach into the ground water and
present an exposure through surface or drinking water.
Subsurface soil boundaries must be defined in three dimensions. They should be defined
based on the possible exposure scenario. For example, if exposure to subsurfaces soil is expected to
occur as a result of routine building or landscaping, the scoping team may define the subsurface
boundary as the average depth and width of a building foundation. In other cases, the regional
Superfund office may have a standard definition for subsurface soil that includes dimensions and other
attributes. This definition should be reviewed by the scoping team to determine if it is appropriate for
its circumstances.
DEFINING THE MEDIA
The physical features that describe subsurface soil are similar to those that define surface soil.
Refer to the section on surface soil. The depth of soil that is classified as "subsurface soil" may be
regulated in some states or regions. Be sure to check with the proper offices and to obtain the
necessary approval before making this decision.
DEFINE SCALE OF DECISION MAKING FOR SUBSURFACE SOIL — EVALUATION OF
SURFACE SOIL CONTAMINATION BY SUBSURFACE SOIL
Risk-Based Scales of Decision Making
Currently the Risk Assessment Group of the Toxics Integration Branch of EPA is developing
risk-based approaches for studying subsurface soil. Contact their office for the latest developments in
this area.
Technology-Based Scales of Decision Making
The scale of decision making for subsurface soil brought to the surface during building or
landscaping operations is equal to the volume of subsurface soil that could potentially reach the
surface. In order to determine a scale of decision making for subsurface soil, the scoping team must
understand what potential building and landscaping operations might occur based on the future use of
the site. This information, along with the size and depth of the foundation, basement, or soil removal
will give the scoping team a good estimate of the volume of soil that will be removed. This
subsurface volume becomes the scale of decision making. The scoping team will then evaluate the
potential health risks that this volume of soil presents when it is removed.
67
-------
Once the scale has been set, the scoping team will evaluate how each volume presents
exposure as surface soil based on possible exposure scenarios. For example, the scoping team would
evaluate the possible exposure that the contaminated soil presents by evaluating the range of surface
soil contamination (thickness and extent) and possible contact of receptors spread on the surface.
DEFINE THE SCALE OF DECISION MAKING FOR SUBSURFACE SOIL — EVALUATION
OF GROUND WATER CONTAMINATED BY SUBSURFACE SOIL
Risk-Based Scales of Decision Making
Currently the Risk Assessment Group of the Toxics Integration Branch of EPA is developing
risk-based approaches for studying subsurface soil. Contact their office for the latest developments in
this area.
Technology-Based Scales of Decision Making
A technology-based scale of decision making would be one that is defined as the smallest unit
of subsurface soil that could efficiently be remediated to limit the contamination of ground water using
current technology.
(c) SURFACE WATER AND ASSOCIATED MEDIA
Developing boundaries for surface water is particularly difficult because a surface water body
may be either static or dynamic. The dynamic systems can have inputs from non-contaminated and
contaminated sources. Under dynamic or static conditions, the concentration of contaminant of the
water body can be reduced due to dilution or increase through contaminant inputs from other media
such as surface soil, sediment, and ground water. Defining the boundaries of surface water will not
only involve defining the bodies that are contaminated, but also defining the media that have the
potential to contaminate surface water in the future.
This section will describe relevant information to aid the scoping team to develop spatial and
temporal boundaries and scales of decision making for surface water bodies.
DEFINE THE MEDIA
Some of the physical features that describe surface water are depth, breadth, width, and
volume. In the case where a flowing body of water is being evaluated, the scoping team should
determine the extent (run) where they feel contamination is possible. Use historical information and
existing analytical data to divide the surface water into areas that are relatively homogeneous within
the geographic area of the investigation. Consider making separate decisions about surface water
based on the sources of contamination or concentration of contamination. Surface water such as lakes
and ponds may be stratified based on depth where contaminants may concentrate. Alternatively,
flowing bodies such as rivers and streams may be stratified based on their proximity to contaminant
sources.
68
-------
DEFINE THE SCALE OF DECISION MAKING FOR SURFACE WATER
The scale of decision making for surface water is defined as the smallest unit (volume, depth,
etc.) of surface water or associted media for which the scoping team wishes to limit the probability of
a decision error. For surface water, there are many potential sources of contamination from associated
media. Theref9re, this section will help the scoping team define the scale of decision making for the
associated media as well as the surface water.
Risk-Based Scales of Decision Making
Currently the Risk Assessment Group of the Toxics Integration Branch of EPA is developing
risk-based approaches for studying surface water. Contact their office for the latest developments on
this topic.
Technology-Based Scales of Decision Making
The technology scale of decision making for surface soil is defined as the smallest unit of
surface water or other contaminated media that could efficiently be remediated to limit contaminant
exposure to the receptor.
Scales of Decision Making for Surface Water By Source of Contamination
Surface Soil Contamination of Surface Water
It may be useful to delineate watershed areas within the site in order to define areas where soil
contamination may impact the surface water quality. Evaluate both the dissolved and suspended
portions of soil (runoff as well as leachate). In order to evaluate contaminant leaching, it is essential
to have a good understanding of the physical and chemical properties of both the soil and the
contaminant(s). In addition, the scoping team should evaluate the normal and the extreme conditions
on the site such as extreme rain events, flooding, spring runoff, etc.
Ground-Water Contamination of Surface Water
Ground-water contamination of surface water is particularly difficult to study because
contaminant concentration and flow volume are difficult to measure or model with accuracy. In
addition, these parameters may vary over time. It may not be possible in this case to develop a scale
of decision making. In this event, the goal of the scoping team will be to locate the sources of
contamination and to estimate the extent of ground-water contamination.
Sediment Contamination of Surface Water
In evaluating sediment contamination of ground water, the goal of the scoping team is to
determine the quantity of sediment that already exists in the river or lake that could possibly
contaminate the surface water through leaching, or the mobilization of the sediment into the surface
water.
-------
(d) GROUND WATER
Ground water is the most difficult media to evaluate primarily because it exists within a soil
matrix which is difficult to sample and evaluate. In addition, many of the techniques that are used in
the boundary section such as exposure units do not apply well to the ground-water system.
DEFINE THE MEDIA
This guidance defines boundaries of ground water to include the overall spatial features of
ground-water depth and range, and the temporal aspects of flow, including rate, water table height, and
variation.
DEFINE THE SCALE OF DECISION MAKING
Consult the hydrogeologist and ground-water specialist when considering scales of decision
making for ground water.
70
-------
SECTION E: DEVELOP A DECISION RULE
CHOOSING A POPULATION PARAMETER
The first activity in developing a decision rule is choosing the parameter to characterize the
population of interest. Choosing the parameter of interest involves several considerations that are
discussed below.
AVOIDING PREMATURE CONCLUSIONS ABOUT THE STATISTICAL DESIGN
It is important to remember in the discussion that follows that the decision rule is not intended
to constrain the statistical design. Therefore, the decision maker need only specify the population
parameter that corresponds to the decision, instead of specifying a summary statistic. For instance,
instead of specifying "a geometric average", the decision maker should only specify "a mean". This
will allow the statistician to choose a summary statistic, either to conform to the assumptions of the
statistical model that underlies the design, or in response to an analysis of the actual data if the design
assumptions are not supported by the data.
CLARIFYING WHAT THE DECISION MAKER REALLY WOULD LIKE TO KNOW
When specifying an appropriate population parameter, the best guideline to follow is to ask the
question, "What would the decision maker really like to know?" If it is an 'average* condition across
an area or time interval at the site, then this will be important information in developing the sampling
design. If it is a peak value at the site, then the sampling strategy may be quite different. If the
decision maker wants to know where the "hot spots" exist, then yet another sampling design may be
appropriate. Clarifying what the decision maker would like to know if the true conditions at the site
could be known will help focus the discussion on matters most relevant to the decision rule.
UNDERSTANDING THE IMPLICATIONS OF DIFFERENT STATISTICAL PARAMETERS
Data may be summarized in a variety of ways, and each statistical parameter will have certain
implications regarding the site. Consequently, it is important to specify a parameter that logically
corresponds to the decision at hand. The following examples illustrate this point.
Mean
The mean is a measure of central tendency of a distribution. The mean concentration of a
contaminant often is used by risk assessors as a mathematical model of long-term exposure. It usually
requires fewer samples than other parameters to achieve a similar level of confidence, and is useful
when the contaminated medium is relatively uniform with a small variance. The mean may be
sensitive to extreme values; hence a few high concentrations can significantly raise a mean, while a
number of low values (such as "non-detects") can reduce the mean. This sometimes gives rise to
concerns about "averaging away" a contamination problem at a site. In addition, the mean is not
representative of a site when there are a large proportion of non-detects.
71
-------
Median
The median is another measure of central tendency that is used to estimate the 50th percentile
of a distribution. The median is less sensitive to extreme values, and may be appropriate to use when
the contaminants are distributed in a manner that violates the usual assumptions of a bell-shaped
(normal) or lognormal curve.
Percentiles
Percentiles describe conditions where x percent of the distribution is less than or equal to the
percentile value. For example, if a 95th percentile of a contaminant distribution is equal to 400 parts
per million, then 95% of the concentration levels are less than or equal to 400 ppm. Percentiles may
be used to ensure that the "tails" of a distribution are factored into a decision so that, for instance,
"almost all" of the contamination falls below a certain threshold value.
72
-------
SECTION F: SPECIFY LIMITS ON DECISION ERRORS
ESTABLISHING PROBABILITY LIMITS ON DECISION ERRORS
After defining the gray region, the decision maker will need to determine the acceptable
probabilities of each decision error. In some non-Supetfund applications, one or more of these
probabilities will be established by regulation. For example, the RCRA rule for determining whether a
waste is hazardous because of lead contamination specifies that an upper 90% confidence limit on the
mean lead concentration be compared to the standard; this is comparable to specifying a 0.10
probability limit for the false positive decision error. In the Superfund program, however, these types
of explicit standards usually are not pre-set.
If the acceptable probabilities for decision errors are not established by regulation, the decision
maker will need to set them. Setting the probability limits on decision errors will depend on two main
factors: the relative consequences of each decision error, and the cost of attaining the decision error
rates. When setting the decision error rates, the decision maker must keep in mind that the cost of
attaining the decision error rates should not exceed the consequences of the decision error. Usually
this will require professional judgments about the likelihood of different consequences and the
magnitudes of their corresponding costs and benefits. By using judgment to balance the costs and
benefits of reducing the probability of decision errors versus the costs and benefits of their potential
consequences, the decision maker establishes how definitive or conclusive the data must be in
supporting the decision.
By defining the limits on decision errors for both the null hypothesis and alternative
hypothesis, the decision maker is actually setting limits on two different aspects of the problem. One
of the limits will restrict the decision errors that could cause risk of exposure to inhabitants and the
environment. The other limit will restrict the decision error that would cause unnecessary cleanup of
the site when the actual risks are below regulated standards.
73
-------
SECTION G: OPTIMIZE THE DESIGN
This appendix discusses some basic concepts involved in creating a sampling design.
Probability sampling designs and statistical models are discussed and examples of these concepts are
included in the DQO applications at Superfund sites contained in Appendix II. In addition, a
discussion on confidence intervals and hypothesis tests is also included to demonstrate the difference
in these techniques. However, methods for creating and analyzing sampling designs and building
statistical models are beyond the scope of this guidance. The reader is referred to Cochran (1977),
Gilbert (1987), and U.S. EPA (1989) for more information. It is recommended that those unfamiliar
with statistical sampling techniques consult a statistician or someone familiar with statistical sampling
designs. If certain critical statistical design assumptions are violated, the data may become unusable for
the specified purpose.
1. SAMPLING DESIGNS
NON-PROBABILISTIC SAMPLING
Non-probabilistic sampling (judgmental sampling) involves an expert selecting sample
locations based on experience and knowledge of the site. The results from these samples cannot be
extrapolated to the entire site, and it is difficult to measure the accuracy of any estimates using the
data. However, judgmental samples can be used subjectively to provide information about specific
areas of the site, which is generally useful during the preliminary assessment and site investigation
stages if there is substantial information on the contamination sources and history. For instance,
judgmental sampling is useful when the sampling objective is to confirm specific locations of
contamination that have already been identified through visual or historical information. If any
statistical conclusions are desired, however, judgmental sampling is not applicable.
PROBABILISTIC SAMPLING
Probability sampling designs allow the results from a set of samples to be generalized to the
entire site. All probability sampling designs have an element of randomization which allows
probability statements to be made about the quality of estimates derived from the data. Every
potential sampling point within the sampling unit has a positive probability of being sampled.
Therefore, probability samples are useful for testing hypotheses about whether a site is contaminated,
the level of contamination, and other common problems that occur with Superfund sites.
There are many different probability sampling designs, each with advantages and
disadvantages. A few of the most basic designs include simple random sampling, sequential sampling,
systematic sampling, and stratified sampling. Other probability designs, such as multistage probability
sampling and search sampling, are too complicated to be explained in this guidance. It is
recommended that a statistician be consulted to determine the best design and the most appropriate
analysis.
Simple Random Sampling
The simplest probability sample is the simple random sample. With a random sample, every
possible sampling point has an equal probability of being selected and each sample point is selected
independently from all other sample points. Random sample locations are usually generated using a
random number table or through computer generation of pseudo-random numbers. Simple random
75
-------
sampling is appropriate when little or no information is available for a site, and the population does
not contain any trends. If some information is available, simple random sampling may not be the
most cost-effective sampling design available.
Sequential Random Sampling
Sequential random sampling is a variation of simple random sampling. As before, every
possible sampling point has an equal probability of being selected, and sample locations are selected
randomly. However, instead of conducting a hypothesis test with all the data, a decision is made after
each sampling round is collected and measured. This decision can have three possible results: reject
the hypothesis, accept the hypothesis, or continue collecting data. Therefore, it may not be necessary
to collect and analyze all the samples required for a simple random sample.
Sequential sampling designs are useful when analyses are very expensive and not much
information is known about sampling and/or measurement variability. However, this method can only
be used when the contaminant distribution is stable over the sampling time frame.
Systematic Sampling
Systematic sampling achieves a more uniform spread of sampling points than simple random
sampling by selecting sample locations using a spatial grid, such as a square, rectangle, or triangle, in
two or three dimensions. To determine sample locations, a random starting point is chosen, the grid
is laid out using this starting point as a guide, then all points on the grid (grid nodes) are sampled.
Since sampling locations are located at equally spaced points, they may be easier to locate in
the field than simple random samples or other probability samples. However, a systematic sampling
design should not be used if the contamination exhibits any cyclical patterns.
Stratification
Stratified random sampling is used to improve the precision of a sampling design. To create a
stratified sample, divide the study area into two or more non-overlapping subsets (strata) that cover the
entire site. Strata should be defined so that physical samples within a stratum are more similar to each
other than to samples from other strata. Sampling depth, concentration level, previous cleanup
attempts, and confounding contaminants can be used as the basis for creating strata. Once the strata
have been defined, each stratum is then sampled separately using one of the above methods.
A stratified sample can control the variability due to media, terrain characteristics, etc., if the
strata are homogenous. Therefore, a stratified random sample may provide more precise estimates of
contaminant levels than those obtained from a simple random sample. Even with imperfect
information, a stratified sample can be more cost-effective. In addition, stratification can be used to
ensure that important areas of the site are represented in the sample. However, analysis of the data is
more complicated than for other sampling designs.
The purpose of defining strata for a stratified random sample is different from the purpose of
defining strata for a scale of decision making. The strata in a stratified random sample are sampled
separately, then the data are combined to create estimates for the entire site or scale of decision
making. Stratum estimates are also available; however, decisions based on individual stratum
estimates will not have the same decision error rates as those defined in Step 6: SPECIFY LIMITS
76
-------
ON DECISION ERRORS.
Composite Sampling
If analysis costs are high compared to sampling costs and the parameter of interest is the
mean, then the use of composite samples should be considered. Composite sampling involves
physically mixing two or more samples before analysis. This method must be used in conjunction
with a sample design in order to determine sample locations (for instance, random composite
sampling). Compositing samples can be a cost-effective way to select a large number of sampling
units and provides better coverage of the site without analyzing each unit.
Composite sampling is useful for estimating or testing the mean when information about
variability is not necessary. It is also useful if the samples are to be used as a screening device.
Additionally, since the amount of contamination in a composite sample should be larger than in an
individual sample, there are times when a contaminant may be more easily detected in a composite
sample. However, information on extreme values and variability is lost with composite data. The
population of interest must be relatively homogeneous for compositing to be feasible. Sometimes
individual samples are changed by the mixing process; for instance, volatile chemicals may evaporate.
In addition, when the action level is close to the limit of detection, the potential dilution caused by
compositing makes the use of composite sampling infeasible. Therefore, composite sampling designs
should be considered with caution.
2. STATISTICAL MODELS
Statistical models describe how the observed responses are expected to behave by relating a
measured value to the true parameter of interest and any sources of uncontrolled variation. Estimates
can then be derived for the parameter of interest and these sources of variation using the model. The
model is very important for understanding the assumptions underlying a proposed test statistic and
sampling design. Thus, it will later serve as the basis for the data quality assessment.
A statistical model consists of fixed components and random components. What is regarded
as fixed or random will be determined by the test of interest and by the inherent structure of the
survey design. Usually, the parameter of interest (for instance, a mean) is considered fixed while the
sources of uncontrolled variation are considered random. These sources include analytic/measurement
errors, temporal and spatial components, and any other factors that may affect the data collection.
The model should:
1. Specify distributional characteristics of the random components; for instance, their means
are usually assumed to be zero and the variances are assumed to be stable.
2. Identify which components are independent of one another. This information is usually
based on historical information, pilot data, or professional judgement.
3. Specify the relationship between the various components; for instance, if they behave in
an additive or multiplicative fashion (or some combination).
4. Identify any correlation structure if temporal or spatial autocorrelations are considered
present.
77
-------
3. CONFIDENCE INTERVALS AND HYPOTHESIS TESTS
Confidence intervals and formal hypothesis tests are two statistical methods that can be used
for decision making. A hypothesis test controls both the false positive decision error rate (a) and false
negative decision error rate (p). A confidence interval only controls the probability of making a false
positive decision error (a) (for example, concluding that a site is clean when it is truly dirty).
However, the probability of making a false negative decision error (p) is fixed at 50% for confidence
intervals (i.e., (3 = .5).
A confidence interval and a hypothesis test can be very similar. Consider the problem of
determining whether the mean concentration (u) of a site exceeds a cleanup standard (CS), where the
contaminant is normally distributed. A confidence interval could be constructed for the mean, or a t-
test could be used to test the statistical hypothesis:
HQ: u > CS vs. Ha: u < CS.
If the site manager's false negative decision error rate is .5 (i.e., P=.5) then these methods are
the same. Additionally, with a fixed a, the sample size of a confidence interval only influences the
width of the interval (since P=.5). Similarly, the sample size of a t-test influences p and 5 (where 8 =
upper value of the gray region minus the lower value of the gray region). However, by solving for the
sample size using a t-test, one can substitute back into the sample size equation for a confidence
interval and compute a width corresponding to this sample size. Then the results of the two methods
will be identical.
Although the results of the hypothesis test and the confidence interval may be identical, the
hypothesis test has the added advantage of a power curve. The power curve is defined as the
probability of rejecting the null hypothesis. An ideal power curve is 1 for those values corresponding
to the alternative hypothesis (all u < CS, in the example above) and 0 for those values corresponding
to the null hypothesis (all u > CS, in the example above). The power curve is thus a way to tell how
well a given test performs, and can be used to compare two or more tests. Additionally, if the null
hypothesis is not rejected, the power curve gives the decision maker some idea of whether or not the
design could actually reject the null hypothesis for a given level (u).
There is no corresponding idea of a power curve in terms of confidence intervals. To derive a
power curve, one would need to translate the confidence interval into the corresponding test (i.e., a t-
test) and then compute the power curve. Additionally, whereas a statistical test accounts directly for
the false negative decision error, a confidence interval does not (P = .5). Finally, a confidence interval
and a statistical test almost always are based on distributional assumptions, independence assumptions,
etc. If these assumptions are violated, it may be easier to select an alternative test (for example, a
non-parametric test) than it is to derive an alternative confidence interval. For these reasons, this
document concentrates its discussion on hypothesis testing.
78
-------
SECTION H: THE DQO PROCESS AND THE SUPERFUND
ACCELERATED CLEANUP MODEL
OVERVIEW OF THE SUPERFUND ACCELERATED CLEANUP MODEL
The Office of Solid Waste and Emergency Response has introduced an initiative that is
designed to streamline and accelerate Superfund cleanups. This initiative is called the Superfund
Accelerated Cleanup Model (SACM). The goals of SACM are to make hazardous waste cleanups
more timely and efficient through better planning and integration of all Superfund programs (within
existing statutory and regulatory requirements). The DQO process provides a framework for planning
field investigations under SACM.
SACM eliminates certain distinctions between the remedial and removal programs and views
them as separate legal authorities under one program: the Superfund program.1 Response actions are
divided into early actions and long-term actions based primarily on the length of time the response
action will take. Early actions can be taken under either removal or remedial authorities. Long-term
actions will be taken under remedial authority. SACM provides a streamlined approach for non-
timecritical removals and all remedial actions. This approach has six aspects:
• a continuous process for assessing site-specific conditions and the need for action;
• cross-program coordination of response planning;
• prompt risk reduction through early action (removal or remedial);
• appropriate cleanup of long-term environmental problems;
• early public notification and participation; and
• early initiation of enforcement activities.2
THE ROLE OF THE DQO PROCESS IN IMPLEMENTING SACM
To produce data that can be used for multiple purposes, careful planning is required. Site
managers need to define the objectives of their field investigations and coordinate among different
existing programs (e.g., the removal, site assessment, and remedial programs). They also will need to
document planning activities well so that if the site manager or Regional Decision Team (RDT)
determines later that a further assessment or different response action is appropriate, the planning
information and data collected in the earlier field investigation can be used by others within
Superfund.
The DQO process provides a framework for planning multiple field investigations and
documenting those planning activities. The DQO process encourages the participation of all those
people involved in generating or using site data. If there is a reasonable chance that the site could
require response actions under different legal authorities (removal/remedial) or different programs
under the same authority (site assessment/remedial), then representatives from these programs are
encouraged to participate on the DQO planning team. The DQO process provides a logical, step-by-
step procedure for organizing the complex issues that cut across different programs and project phases
and for keeping the team focused on the issues most relevant to planning the field investigation.
'U.S. EPA, "Superfund Accelerated Cleanup Model (SACM)," Publication No. 9203.1-01, Memo from Don R. Clay, April 7, 1992, p. 3.
2OSWER Publication 9203.1-051, Status of Key SACM Program Management Issues — Interim Guidance, December 1992, p. 1.
79
-------
APPENDIX II
APPLICATION OF
DATA QUALITY OBJECTIVES
TO SUPERFUND SITES
EXAMPLES
SECTION A
GROUND-WATER EXAMPLE
THE WATERVILLE MUNICIPAL LANDFILL SUPERFUND SITE
1.0 BACKGROUND
The Waterville Municipal Landfill was in operation from 1967 to 1985. During this time, the
facility accepted residential and commercial waste. Historical information indicates that waste solvent
was disposed of at the Waterville Municipal Landfill. One chemical in particular, perchloroethylene
(PCE), was disposed of in large quantities. PCE is a class C, possible human carcinogen which
mainly targets the kidney. Ingestion and inhalation of drinking water from contaminated ground water
are considered viable exposure routes.
The Waterville Municipal Landfill is situated in the Atlantic coastal plain overlying an
unconfined aquifer that serves as a drinking water source for nearby residents via domestic wells (see
Figure A-l). Local residents are concerned that the landfill may be releasing contaminants into the
ground water. EPA has initiated an Expanded Site Investigation (ESI) because of the potential for
exposure to PCE through drinking water.
The aquifer underlying the landfill site was previously contaminated by PCE from a leaking
tank at a dry cleaning facility, which is hydraulically upgradient from the landfill site. The leaking
tank was removed in 1990. PCE was detected during quarterly sampling in 1991 and 1992, but was
detected below levels of concern. Well A is hydraulically upgradient from the landfill and is located
at the site boundary. Two drinking water wells — wells B and C — are within 1A mile and are
hydraulically downgradient from the site (see Figure A-2). Any leakage from the landfill will affect
only the downgradient wells.
2.0 DQO DEVELOPMENT
The following is an example of the output from each step of the DQO process.
Step 1: State the Problem — a description of the problem and specifications of available resources
and relevant deadlines for the study.
81
-------
(1) Identify the members of the DQO scoping team — The members of the scoping team
will include the Site Assessment Manager (SAM), a field sampling expert, a chemist, a
hydrogeologist, a QA Officer, and a statistician. The SAM is the decision maker.
(2) Define/refine the conceptual site model — Figure A-l illustrates some of the main
elements of the conceptual site model, such as the source of contamination, routes of
migration, and potential receptors (humans living in households connected to the
domestic water supply fed by wells B and C). Additional information needed to
82
-------
Upgradlent
Downgradtent
Drinking Water
Wells B and C
We
Vac!
Zc
\
i
Sati
Zc
1
i
lose
>rw
rated
>ne
HA
l ^^^ , . • • . . i "^^ „ , AT
fr Municipal /
V Landl« /|
1 i
1 Leachate .
| Plume '
Water |
Table n ' i
' ^
\ ^_ ^
N "" ~~ ^ __
V
: x
: \.
: \
1
> i
•\.
. — " —
Tb Domestic
Water D/ttritxjfon
SytHm
Figure A-l. Cross-section View of Waterville Site
Upgradient Well
Well A
Property Boundary
Downgradient Wells
Well B well C
Figure A-2. Plan View of Waterville Site
83
-------
complete the conceptual site model includes the type of contaminant (PCE) and a
range of expected concentrations.
(3) Define exposure scenario — PCE located in the landfill can be released from decaying
containers, escape from the unlined landfill, and migrate into the ground-water aquifer
which is the drinking water supply for the town. Residents may be exposed to PCE
contamination through dermal contact, inhalation, and ingestion of drinking water
during routine daily activities in their homes, such as cooking and showering.
(4) Specify the available resources — EPA would like to take the minimum samples
necessary that would still provide adequate data quality to support a defensible
decision. There are adequate resources to collect and analyze a few samples from each
of the three wells.
(A) Time — Residents with wells near the site are concerned about the safety of
their drinking water. Local representatives would like this problem addressed
within 6 months.
(B) Identify project constraints — In the pre-remedial phase of the Superfund
process, financial resources are limited.
(5) Write a brief summary of the contamination problem — The Waterville Municipal
Landfill is known to have accepted large quantities of PCE, and now residents of the
town are concerned that the PCE may be leaking and contaminating their domestic
water supply via two drinking water wells located near the landfill.
Step 2: Identify the Decision — a statement of the decision that will use environmental data and the
actions that could result from this decision.
(1) State the decision — Determine whether there has been a release of PCE from the
Waterville Municipal Landfill into the drinking water aquifer of Waterville.
(2) State the actions that could result from the decision —
(a) Recommend Site Evaluation Accomplished (SEA); or
(b) Recommend further assessment or a response action.
Step 3: Identify the Inputs to the Decision — a list of the environmental variables or characteristics
that will be measured and other information needed to make the decision.
(1) Identify the informational inputs needed to resolve the decision — Concentrations of
PCE in ground water are needed from at least one upgradient location and at least one
downgradient location near the landfill.
(2) Identify sources for each informational input — The information on PCE
concentrations in ground water can be obtained through analytical measurements
performed on water samples drawn from upgradient well A and downgradient wells B
and C. There are existing data for well A gathered during 1991 and 1992.
84
-------
During 1991 and 1992, quarterly PCE data were collected from well A, the upgradient
well. The SAM is concerned that the upgradient level of PCE contamination may
have changed over the course of the sampling which began two years ago. If the
contamination problem has changed during the two years, the previously collected data
may not be appropriate and new data may need to be collected. Therefore, the SAM
needs to verify that there are no temporal trends in the data for well A. A plot of the
eight observations shows no visible trends. The SAM, however, has decided to
compare the data from 1991 and 1992 to verify that the distribution of PCE
contamination has not changed.
Year
1991
1992
Differences
(1991 minus 1992)
Observations of PCE Concentrations (ppb)
Jan. 1
0.406
0.434
-0.028
April 1
0.399
0.347
0.052
July 1
0.340
0.422
-0.082
Oct. 1
0.383
0.383
0.0
Mean
0.382
0.397
-0.0145
Std. Dev.
0.0296
0.0395
0.0559
Variance
8.767E-04
1.563E-03
3.124E-03
Evaluation of changes in the PCE concentration over the sampling period 1991-1992
Comparison of Sample Variance: An F-test can be used to test the uniformity of two
variances by comparing the ratio of the two variances with critical values from an F-
distribution. The ratio of 1991 and 1992 variances is:
F - L563£-°3 - 1.783
8.767E-04
Since the SAM wishes to test HQ : a2!,,, = cr2,^ versus H! : a2,,,, * o2,^, the
critical region (with a = .1) is given by:
F>F,r
= 0.1078
= 9.28
Since 1.783 * 0.1078 and 1.783 > 9.28, the SAM cannot conclude that the variance in
1991 is different from the variance in 1992. Therefore, the SAM may assume these
variances are equal.
Comparison of Sample Means: A t-test can be used to test the equivalence of two
sample means. Since it has already been concluded that the variances are not
different, a pooled t-test of the form:
85
-------
.0145
5,
.0346
4 4
5
may be used. This value will be compared to the critical value of a t-distribution with
6 degrees of freedom. Since 0.593 is less than the critical value, 1.943, the SAM
cannot conclude that the yearly means are different. As a result, the SAM has
determined that the sampling data from 1991 and 1992 are adequate for use in the
comparison with downgradient wells.
(3) Define the basis for establishing contaminant-specific action levels — The action level
for this problem is the lowest possible PCE concentration that demonstrates a
significant increase in comparison to the upgradient concentration.
(4) Identify potential sampling techniques and appropriate analytic methods — The bottom
valve bailer (teflon or stainless steel 316) has been identified as a potential sampling
technique. A dedicated sampler will be used for each well. GC/MS is the proposed
analytical technique.
Step 4: Define the Boundaries of the Study — a detailed description of the spatial and temporal
boundaries of the decision; characteristics that define the environmental media, objects, or
people of interests; and any practical considerations for the study.
(1) Define the spatial boundaries —
(A) Define the domain within which all decisions must apply. The study will focus
on ground water within the unconfined aquifer below the landfill.
(B) Specify the characteristics that define the population of interest. PCE
concentrations in ground-water monitoring wells B and C. For the purposes of this
study, these wells are assumed to be representative of the aquifer below the landfill.
(C) Define the scale of decision making. Samples will be taken from the two
downgradient ground-water monitoring wells (B and C). A separate decision will be
made for each drinking water well.
(2) Define the temporal boundaries —
(A) Determine what timeframe the sampling data must represent. Because the study
is not intended to determine health risks posed by PCE, there is no specific timeframe
to which the results will apply.
86
-------
(B) Determine when to collect data. EPA is interested in characterizing the
contamination at this site quickly because of the potential adverse health effects of
exposure to PCE in drinking water. Because the data from the three wells will be
compared, samples will be collected on the same day. Past experience at similar sites
indicates that there are no systematic variations in PCE concentration over time, so
samples may be taken at any time of day.
(3) Identify practical considerations that may interfere with the study — EPA does not
expect to encounter any practical constraints while sampling.
Step 5: Develop a Decision Rule — an "if...then..." statement that defines the conditions that would
cause the decision maker to choose among alternative actions.
(1) Specify the parameter of interest — The study is trying to quickly determine whether
the downgradient concentration of PCE is significantly greater than the upgradient
concentration, so the SAM has decided to specify the parameter as an observation of
PCE concentration in each of the downgradient wells.
(2) Specify the action level for the study — The action level for this problem is the lowest
possible PCE concentration that demonstrates a significant increase when compared
with the upgradient concentration. The specific concentration will be identified during
the Optimize the Design step.
(3) Develop a decision rule (an "if...then..." statement) — If any downgradient sample
yields a PCE value significantly greater than the upgradient well, then there is actual
contamination of the ground water and further assessment or response is required;
otherwise recommend SEA.
Step 6: Specify Limits on Decision Errors — the SAM's acceptable decision error rates based on a
consideration of the consequences of making an incorrect decision.
(1) Determine the possible range of the parameter of interest — The scoping team has
estimated the range of the parameter of interest to be 0-10 ppb PCE in the ground
water, based on the evaluation of similar PCE releases from other sites.
(2) Define both types of decision errors and identify the potential consequences of each —
(A) Define both types of decision errors and establish which decision error has the
more severe consequences. The two decision errors are:
Decision Error 'a': Deciding that the downgradient well PCE concentration is greater
than the upgradient well when it is not. The consequences of this decision error
include the unnecessary costs of further study, and the possibility of unnecessary
remedial or emergency removal action. Treating ground water is usually a lengthy and
resource-intensive process. Other remedial options such as providing an alternate
drinking water supply can be very costly also. A positive consequence of taking
unnecessary action is that some environmental improvement may occur (e.g., through
87
-------
removing very low levels of PCE and other contaminants), even though the
improvement may be of little value when compared to the costs.
Decision Error 'b': Deciding that the downgradient well PCE concentration is not
greater that the upgradient well when it is. Some consequences of this decision error
include environmental damage, increased future health costs, and increased cancer
illness and deaths. A positive consequence is that resources are conserved. While the
resource savings may be of small consequence when weighed against the negative
consequences, it is important to consider them here. A complete, balanced picture of
the problem can only be developed if both positive and negative consequences of the
decision error are considered. Decision Error 'b' is the more severe decision error.
\
(B) Establish the true state of nature for each decision error. The true state of nature
for decision error 'a' is that the downgradient well does not have a higher
concentration of PCE than the upgradient well. The true state of nature for decision
error 'b' is that the downgradient well has a higher concentration of PCE than the
upgradient well.
(C) Define the true state of nature for the more severe decision error as the baseline
condition (null hypothesis) and define the true state of nature for the less severe
decision error as the alternative hypothesis.
Null hypothesis, H0 = The downgradient well has a higher concentration of PCE than
the upgradient well.
Alternative hypothesis, H, = The downgradient well does not have a higher
concentration of PCE than the upgradient well.
(D) Assign the terms "false positive" and "false negative" to the proper errors.
False positive error = decision error 'b'
False negative error = decision error 'a'
(3) Identify Acceptable Decision Error Rates —
False Positive Error: If the downgradient concentration of PCE is greater than the
upgradient concentration due to a release, the SAM desires at least a 95 percent
probability of finding that a release has occurred (5% probability of a false positive
error). In this example, the SAM becomes increasingly concerned the higher the
downgradient PCE concentration is hi comparison to the upgradient well.
False Negative Error: If there truly has been no release, the SAM wants at most a 5
percent probability that the data indicate a release.
(4) Specify the Gray Region — There will be no gray region for this problem since the
decision is to determine a "significant difference" between the concentration of the
downgradient wells and background concentrations rather than a fixed point (action
level).
88
-------
Step 7: Optimize the Design — the decision maker will analyze existing data and select the lowest
cost sampling design that is expected to achieve the DQOs.
(1) Develop general sampling and analysis design alternatives — Existing data from well
A were found to be useful in determining the contamination level upgradient of the
site. New data will be generated for the downgradient wells and tested to determine
whether they belong to the same population as the upgradient data. If the
downgradient values are significantly higher, then it will be concluded that the
upgradient and downgradient concentration levels come from different populations.
An upper 95% tolerance limit on the population (with 95% probability that at least
95% of the distribution will be less than the limit) will be used to make this
determination.
A tolerance interval may be used to prove that a well is contaminated; however, it
cannot conclusively determine that a well is not contaminated; The scoping team
believes, based on the past history of the site, that wells B and C are contaminated.
Thus, a tolerance interval will be used to quickly verify that the wells are
contaminated. If data from wells B and C fail to exceed the upper tolerance limit,
then this method is inconclusive and an alternative sampling design should be
developed.
The tolerance interval used will be based on a normal distribution. Hence, the
assumption that the eight observations from well A follow a normal distribution should
be tested. Due to the small sample size, Geary's Test for Normality will be used to
test this assumption. The test statistic will be
a = l±
and an approximate test for normality will be
z = (a - 0.7979)
|
0.2123
If Z > 1.96, the assumption of normality at a 5% level of significance will be rejected.
89
-------
For the data from well A,
0.248829
0.835914
0.007739
(0-835914 - 0.7979) =
Since Z < 1.96, the idea that the data are normally distributed cannot be rejected.
Therefore, it will be assumed that the upgradient data are normally distributed and can
be used to construct a tolerance interval.
Using the eight observations from well A, an upper tolerance interval (TL) can be
constructed by:
TL = mean + K * Std. Dev.
where K is a one-sided normal tolerance factor. A table of tolerance factors can be
found in the Guidance Document on the Statistical Analysis of Ground-water
Monitoring Data at RCRA Facilities, EPA, 1993. In this case, K(0.95, 0.95, 8) =
3.188, and
TL = 0.389 + 3.188 * 0.03325 = 0.495
Any one observation over 0.495 will cause the SAM to conclude that additional
contamination above the upgradient level has been observed. In other words, any one
observation from either downgradient well that exceeds 0.495 will be cause for
deciding that there has been a release from the landfill.
Statistical Models
For each observation YJ from the upgradient well A,
y, = " + e,
where u represents the mean PCE concentration for the upgradient well and the Cj's
represent sampling and measurement error which are assumed to be distributed with a
mean of 0 and a variance equal to o2. Unless the data demonstrate otherwise, the
observations from the downgradient wells B and C should also follow this model.
90
-------
Sample Size
Ideally the SAM would like to collect just one sample from each of the two
downgradient wells. Collection of one additional sample from the upgradient well is
recommended to ensure that the direction of the plume from the dry cleaning facility
has not changed.
(2) Select the most resource-effective design that satisfies all of the DQOs — This design
is resource-effective because it requires a small number of samples (one from each
well). However, if neither sample exceeds 0.495, then an alternative sampling design
will be developed which would satisfy the scoping team's limits on decision errors.
(A tolerance interval will only satisfy the limits of a false-positive error.)
(3) Document the details and assumptions of the selected design — This design assumes
that the purpose of sampling is to verify that a release has occurred. If the data do not
demonstrate that a release has occurred, the decision maker cannot conclude that the
wells are not contaminated and an alternative sampling design will be developed.
91
-------
SECTION B
REMOVAL PROGRAM EXAMPLE
THE LEADBURY SUPERFUND SITE
1.0 BACKGROUND
The Leadbury Superfund Site covers a large area in two counties within the State of
Oklahoma. The soil within this area has elevated levels of lead. The site surrounds the town of
Leadbury where the Lead Smelter Co. has been mining and smelting lead since 1933. Currently, the
area of surface soil contamination extends for approximately 36 square miles surrounding the town.
The lead has allegedly originated from stack emissions or possibly from improper disposal of waste
materials from the smelting and mining processes. Lead concentrations exceed 500 ppm at some
portions of the site.
The Environmental Protection Agency (EPA) has decided to conduct the Remedial
Investigation/Feasibility Study (RI/FS) and the remedial design for this site concurrently with the
removal action in observance of the Superfund Accelerated Cleanup Model (SACM) guidance.
Therefore, all data collected during the removal phase will be used in later phases of the study.
The predominant threat to the public from this site comes from the inhalation and/or ingestion
of lead-contaminated soil particles. Lead is known to produce many adverse health effects in humans
ranging from reproductive system disorders, delays in neurological and physical development,
cognitive and behavioral changes, and increased blood pressure. The main exposure pathway for lead
is inhalation. Inhalation exposure is most likely to occur during dry and windy conditions that are
prevalent during the summer months. Children are at special risk from lead exposure because their
behavior traits result in greater intake of soil per body weight. In addition, children are more likely
than adults to have nutrient deficiencies which increase the metal absorption and retention. It has also
been indicated that adverse neurological effects occur at lower blood lead level thresholds in children.
An Emergency Removal Branch (ERB) assessment of the site was conducted in two phases.
During Phase I, an area of 36 miles surrounding the town was sampled to determine the contaminants
of concern. The samples were analyzed for 24 target compound metals and the results identified lead
as the contaminant that should be addressed in more extensive sampling. In Phase n, additional
surface soil locations were sampled within the Phase I area from 53 locations that were determined to
be "high-access" areas for children, the target population at risk. These included school yards,
playgrounds, day care centers, and church yards. Twenty-six of the high-access areas were determined
to have concentrations of lead in excess of the removal program's action level of 500 ppm. These 26
areas were considered to present imminent and substantial endangerment to the public.
As part of the sampling done in Phase n, the removal program determined that the lead
contamination was distributed bimodally (i.e., a graph of the distribution of lead concentrations shows
two distinct peaks). The concentration of the low mode is 30 ppm while the concentration of the high
mode is 700 ppm. The lower concentration of lead is thought to have come from aerial deposition
93
-------
associated with the lead smelter and other mining operations. The higher concentrations are thought to
be due to the use of contaminated fill material. The fill most likely came from mining tailings. It was
therefore decided that a sampling plan should be initiated to locate the portions of the high-access
areas that had lead contamination in excess of 500 ppm. The contaminated soils would then be
removed and clean fill would replace it. The removal program has decided to use the DQO Process to
help them develop the sampling plan to locate areas of excess lead contamination.
As a precursor to the DQO Process, the ERB estimated the cost of disposal for the
contaminated soil. They subjected soil samples to the Toxicity Characteristic Leaching Procedure
(TCLP) to determine if the contaminated soil was considered a "hazardous substance" under RCRA
regulations and would therefore need to be disposed of at a more expensive hazardous waste facility.
The tests showed that the contaminated soil was considered non-hazardous and could therefore be
disposed of at a less costly municipal landfill.
2.0 DQO DEVELOPMENT
Step 1: State the Problem — a description of the problem(s) and specifications of available
resources and relevant deadlines for the study.
(1) Identify the members of the DQO scoping team — The members of the scoping team
will include the On-Scene Coordinator (OSC), the manager of the Lead Smelter Co., a
Quality Assurance Officer, a representative of the Leadbury town council, a statistician
who has experience with sampling design, and a chemist with field experience. The
decision maker will be the OSC of the removal program.
(2) Define/refine the conceptual site model — The source of contamination is
from lead found in surface soil at 26 "high-access" areas around the city. The
lead has been deposited through air deposition at the high-access areas from
lead smelter operations in the region over a period of 60 years. The
concentration of lead is expected to be from 0 - 1000 ppm based on site
preliminary site investigations. The receptors are children between the ages of
1-12 years.
(3) Define the exposure scenario — EPA is concerned about the secondary source of lead
contamination existing in the surface soil at 26 high-access areas throughout the city,
so the original release mechanism from the smelter is not directly relevant. However,
lead will be released from the surface soil in the form of dust. The lead will be bound
to soil particles. Children will be exposed through inhalation of the dust particles and
through ingestion of contaminated soil at each site. The future land use is assumed to
be the same as the current mixed uses.
(4) Specify available resources — The total budget for sampling, removal, and disposal is
$5,560,000. Therefore approximately $200,000 is available for each of the 26 high-
access areas.
94
-------
(A) Time. All removals should be completed within 6 to 8 months.
(B) Identify project constraints. The OSC has requested that all stages of the operation
be performed in a manner that minimizes the time and cost of sampling, analysis, and
disposal.
(5) Write a brief summary of the contamination problem — Surface soil in high-access
areas of Leadbury arc contaminated with relatively high concentrations of lead. EPA
needs to determine what portions of soil within the high-access areas need to be
removed.
Step 2: Identify the Decision — a statement of the decision that will use environmental data and the
actions that could result from this decision.
(1) State the decision(s) — Determine what areas within the 26 high-access areas have
concentrations of lead in the soil that exceed the removal program's regulated
standard.
(2) State the actions that could result from the decision —
(a) Further study will take place to delineate contamination, the surface soil will
be removed, and clean fill will replace it.
(b) The surface soil will be left intact.
Step 3: Identify the Inputs to the Decision — a list of the environmental variables or characteristics
that will be measured and other information needed to make the decision.
(1) Identify the informational inputs needed to resolve the decision — Concentration of
lead in the soil within the 26 high-access areas.
(2) Identify sources for each informational input — The concentration of lead can be
measured from soil samples.
(3) Define the basis for establishing contaminant-specific action levels — The action level
for lead in soil has been set for the removal program by the Agency for Toxic
Substance Disease Registry (ATSDR), based on the risk of exposure and the
possibility of adverse health consequences. The action level is 500 ppm.
(4) Identify potential sampling techniques and appropriate analytic methods — The
analytical method will be atomic absorption. The tulip bulb planter has been identified
as a potential sample collection device.
Step 4: Define the Boundaries of the Study — a detailed description of the spatial and temporal
boundaries of the decision; characteristics that define the environmental media, objects, or
people of interest; and any practical considerations for the study.
95
-------
(1) Define the spatial boundaries —
(A) Define the domain within which all decisions must apply. The boundaries of the
study will be limited to the property boundaries of each separate high-access area that
has been identified as having soil contamination that exceeds the removal program
standard of 500 ppm for lead. Each of the 26 high-access areas will be evaluated and
sampled separately.
(B) Specify the characteristics that define the population of interest. Surface soil (0-6
inches) associated with the site. Each of the 26 high-access areas will be considered
subpopulations.
(C) Define the scale of decision making. Because the contaminated soil is thought to
come from fill material, the sampling plan should be adequate to detect the smallest
area that would reasonably have been filled within the high-access areas. The scoping
team has chosen a circle with a diameter of 40 feet to a depth of 6 inches to represent
the area that corresponds to the smallest area that could reasonably have been filled.
This is the area that corresponds to four dump truck loads (8 tons) of fill material,
spread 6 inches thick. Therefore the sampling plan must adequately detect
contaminated circular areas of contaminated soil that have a diameter of 40 feet.
(2) Identify temporal boundaries — The EPA is facing public pressure to reduce the
exposure risks from the site quickly.
(A) Determine what timeframe the sampling data must represent. Because the study
is not intended to determine risk, there is no specific timeframe to which the results
will apply.
(B) Determine when to sample. Lead in soil is stable. It will not degrade or migrate
from the "high-access areas". Therefore lead can be sampled at any time. For best
results, soil samples should be taken when the soil moisture is relatively low (less than
30%) so that the core samples will hold their form.
(3) Identify practical considerations that may interfere with the study — Two of the high-
access areas provide a passageway between elementary school buildings. For students
to avoid possible exposure, a walkway built of plywood will be installed.
Additionally, it will not be possible to perform removals on these areas during regular
school hours (8:00 am - 2:30 pm).
Step 5: Develop a Decision Rule — an "if...then..." statement that defines the conditions that would
cause the decision maker to choose among alternative actions.
(1) Specify the parameter of interest — A hot spot can be considered as a maximum
concentration. Therefore the parameter of interest is the maximum concentration.
(2) Specify the action level for the study — The removal program's action level for lead
in soil is 500 ppm. The action level has been set by the ATSDR.
96
-------
(3) Develop a decision rule (an "if...then..." statement) — If the maximum concentration
of lead in any high-access area is greater than 500 ppm, then a second round of
sampling will be implemented to delineate the extent of soil contamination.
Otherwise, no action will take place.
Step 6: Specify Limits on Decision Errors — the decision maker's acceptable decision error rates
based on a consideration of the consequences of making an incorrect decision.
(1) Determine the possible range of the parameter of interest — The possible range of lead
concentrations is expected to be from 0-1000 ppm.
(2) Define both types of decision errors and identify the potential consequences of each —
(A) Define both types of decision errors and determine which decision error has the
more severe consequences. The two decision errors are:
Decision Error 'a': Determining that circular areas of contaminated soil with a radius
of 40 feet or greater do not exist when they actually do; i.e., determining there are no
hot spots when a hot spot actually exists. The consequence of this error is that
contaminated soil will not be removed and human health will be endangered. Decision
Error 'a' is the more severe decision error.
Decision Error 'b': Determining that the soil is contaminated when in reality it is not;
i.e., determining that a hot spot exists when in reality there are no hot spots. The
consequence of this error is that time and energy will be spent on additional sampling.
The public will view this error positively in that it shows that the overriding concern is
for protecting human health. The consequences, therefore, are far less severe than the
consequences of the other decision error.
(B) Establish the true state of nature for each decision error. The true state of nature
for decision error 'a' is that a hot spot exists. The true state of nature for decision
error 'b' is that there are no hot spots.
(C) Define the true state of nature for the more severe decision error as the baseline
condition or null hypothesis and define the true state of nature for the less severe
decision error as the alternative hypothesis.
Null Hypothesis, HQ = A hot spot exists. (The concentration of an individual sample is
above 500 ppm.)
Alternative Hypothesis, H, = A hot spot does not exist. (The concentration of an
individual sample is less than 500 ppm.)
(D) Assign the terms "false positive" and "false negative" to the proper errors.
False positive error = decision error 'a'
False negative error = decision error 'b'
97
-------
Specify the Gray Region — The scoping team has set the gray region, which spans 100 ppm, to the
left of the action level.
(4) Identify Acceptable Decision Error Rates —
(a) False Positive Error: The scoping team can accept a rate of 20% for the
probability of a false positive (see Figure B-l).
(b) False Negative Error. The scoping team has set the acceptable rate of making
a false negative error at 30% (see Figure B-l).
Acceptable Probability of Declaring That
EU Contains a Hot Spot
ooooooooo
o'-thou'-kinot-^CBO-*
Acceptable
False Negative
Decision Errors
Acceptable
False Positive
Decision Errors
-
-
1 200 1 400 600 1 800
100 300 500 700
* Action Level
True Concentration of Lead (ppm)
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Figure B-l. Design Performance for Soil Lead Testing
Step 7: Optimize the Design — the decision maker will select the lowest cost sampling design that
is expected to achieve the DQOs.
(1) Develop general sampling and analysis design alternatives — For each design
alternative, the statistician must formulate a statistical model (i.e., a mathematical
expression) that tests the hypothesis and select the optimal sample size that satisfies
the decision maker's limits on decision errors.
98
-------
A search sampling method using systematic (or grid) samples will be used to
determine whether or not a "hot spot" of contamination exists. If the concentration ot
lead in any sample within the boundaries is significantly greater than 500 ppm, then a
second round of sampling will be implemented to determine the extent of soil
contamination. Otherwise, no action will take place.
The second round of sampling, sequential sampling, will characterize the extent of the
area that requires removal. Additional soil samples will be taken at a point one-half
the distance to the next non-contaminated sampling point. If any sample in the second
round is contaminated, additional samples will continue to be collected one-half the
distance to the nearest non-contaminated sampling point until a sample shows no
contamination. Once this occurs, contaminated soil will be removed up to and
including the last clean sample. The soil will be removed to a depth of 8 inches
because this is the maximum depth that children are expected to receive exposure from
soil during normal activity. Clean fill will be used to fill the depressions made during
removal activity.
Samples will be taken in a triangular-shaped grid pattern. The distance between
samples will be 42.5 feet (see Figure B-2). Six-inch core samples will be taken at the
grid nodes, homogenized, and analyzed at each sampling location.
Because of the extreme bimodal distribution of the lead concentration, the design
assumes that when a hot spot is sampled, it will not be mistaken for background and
vice versa.
Statistical Models
For each observation y(:
Xi = v, + e(
where v( = true value of the i^ observation and
Cj = sampling error for the !„, observation.
The e/s are independently and identically distributed with the mean equal to 0 and
variance equal to o2,.
Sample Size
Below is an explanation of a procedure that is used to determine the number of
samples needed to detect hot spots of contamination within a pre-specified confidence
limit. The procedure employs three common sampling patterns (square, rectangular,
and triangular) to determine the optimal sample spacing and distance between samples.
To determine the minimum spacing between samples that will detect an elliptical hot
spot of a pre-specified size and shape with a specified confidence, the following
procedure is used:
99
-------
Sampling Plan for Representative High Access Area - School Playground
I 1 300'I 1
300"
Grid Notes Triangular Sampling
Grid
Samples are collected
at grid nodes
G = Spacing between grid
lines. For this example
G=42.5 ft.
Figure B-2. Triangular Sampling Grid Used to Detect Soil Lead Contamination in a
300' x 300' School Playground
100
-------
(A) Specify the length (L) of the long axis of the hot spot ellipse: L = 20 ft.
(B) Specify the length (R) of the short axis of the hot-spot ellipse: R = 20 ft.
(C) Divide the length of the short axis by the length of the long axis. The solution,
S, is called the shape:
Length of the short axis of the hot-spot ellipse
s = _ =1
Length of the long axis of the hot-spot ellipse
(D) Specify the acceptable probability of not finding the hot spot. In our example the
probability of not finding the hot spot corresponds to p = .2. (In this case, a false
positive error.)
(E) Determine the distance between samples (G) using the nomograph (see Figures 2-
3 and 2-4) to meet the constraints specified in the first four steps. For a square
playground area with a size of 300 ft. x 300 ft., the distance between samples and the
number of samples needed to meet the DQOs will be:
Using a square sampling pattern, G = 39.2 feet: 64 samples.
Using a triangular sampling pattern, G = 42.5 feet : 49 samples.
(2) Select the most resource-effective design that satisfies all of the DQOs — Sampling
costs include both the cost of collecting and analyzing samples. Each soil sample
tested for lead will cost $75.00. The total cost of sampling will depend on the total
number of samples.
(3) Document the details and assumptions of the selected design —
• The target (hot spot) is circular. For subsurface targets, this applies to the
projection of the target to the surface.
• Samples or measurements are taken on a triangular grid.
• The distance between grid points is much larger than the area sampled, measured,
or cored at grid points — that is, a very small proportion of the area being studied
can actually be measured.
• The definition of "hot spot" is clear and unambiguous. This definition implies that
the types of measurement and the levels of contamination that constitute a hot spot
are clearly defined.
• There are no measurement misclassification errors — that is, no errors are made in
deciding when a hot spot has been hit.
The most efficient sampling plan is one that uses a triangular sampling grid (see
Figure B-2) because it meets the constraints of the DQOs with the fewest number of
samples and therefore has the lowest total cost.
101
-------
1.00
0.80 -
0.60 -
0.40 -
0.20 -
0.00
Square sampling and
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00
Figure B-3. Curves relating L/G to consumer's uncertainty, J3, for different target shapes
using a square grid (from Zirschky and Gilbert 1984, with permission)
i.oo
o.ao -
0.60 -
0.40 -
0.20 -
0.00
0.00 0.10
0.80 0.90 1.00
Figure B-4. Curves relating L/G to consumer's uncertainty, p, for different target shapes
using a triangular grid (from Zirschky and Gilbert 1984, with permission)
102
-------
SECTION C
REMEDIAL PROGRAM EXAMPLE
THE RAWHIDE SUPERFUND SITE
1.0 BACKGROUND
The Rawhide Superfund Site is a former leather tannery. Between 1982 and 1985, tannery
waste sludge was landfarmed over part or all of a 29-acre pasture (see Figure C-l). "Landfarming"
refers to a process of waste disposal that involves spraying or pouring waste onto the soil and then
disking the waste into the soil. At this site, the sludge containing high levels of chromium compounds
was disked into the soil to a depth of approximately 8 inches. Historical site information indicates that
several portions of the landfarm area have received little or no waste.
High concentrations of chromium IH and VI have been detected in surface soil samples at the
landfarm. This may indicate that wastes were dumped on the ground, but not disked into the soil.
Ground-water sampling in wells and springs within three miles of site have shown the presence of
chromium and lead at levels below maximum contaminant levels (MCLs). Due to the high levels of
chromium in the surface soil, the site has been placed on the National Priorities List (NPL).
The site is currently used to graze cattle. Several residences are located adjacent to the site.
Potential human exposure routes identified by the site risk assessor include ingestion and inhalation of
soil particulates and ingestion of ground water. Chromium VI compounds are suspected human
carcinogens through the inhalation pathway only. Chromium ffl compounds are not considered
carcinogenic. Direct contact with chromium compounds can cause a hypersensitivity reaction.
The scoping team has decided to employ the DQO process to help them determine if there are
any areas of the landfarm that pose an unacceptable risk to human health and the environment and
thus require further assessment or a response action. By using the DQO process, the team plans to
generate a statistically valid sampling design, generate results of known confidence, make defensible
decisions, and save time and resources.
2.0 DQO DEVELOPMENT
Following is an example of the output from each step of the DQO process.
Step 1: State the Problem — a description of the problem(s) and specifications of available
resources and relevant deadlines for the study.
(1) Identify the members of the DQO scoping team — The members of the DQO scoping
team include the RPM, a field sampling expert, a chemist, an engineer, a risk assessor,
a QA Officer, a hydrogeologist, a DQO facilitator, and a statistician. The RPM is the
decision maker.
103
-------
Site
boundai
Figure C-l. Site Map of Rawhide Superfund Site
(2) Define/refine the conceptual site model — The source of the contamination is from
landfarming waste disposal operations at a former leather tannery. High concentrations
of chromium have been observed in soil associated with the site. Chromium and lead
were detected in ground-water samples at levels below the MCLs. Contaminants are
migrating from surface and subsurface soils to ground water. Contaminants may also
become airborne primarily due to wind. The receptors are humans of all ages who live
within a 2-mile radius and who derive their drinking water from ground-water wells
which are connected to the ground-water aquifer below the site. Cattle who graze on
the site are also potential receptors.
(3) Define exposure scenarios — The source of the contamination is the chromium-
contaminated soil and the ground water associated with the site. Contaminants will be
released through aerial transport and migration to ground water. Contaminants may
also migrate through ground water to drinking water wells. The chromium will be
bound to soil dust particles or dissolved in ground water. The exposure routes include
ingestion of soil, inhalation of dust particles, and ingestion of ground water. The
potential exposure points are the contaminated soils on-site and houses connected to
drinking water supply. The land use for the site is residential.
104
-------
(4) Specify the available resources — EPA is concerned about the cost of extensive
sampling and analysis, but adequate data quality is a priority. EPA has allocated the
funds necessary for a sampling crew of four people for only one week. All sampling
must be done within that week.
(A) Time. The RPM wants this site addressed in a "reasonable timeframe." The
RPM expects data validation to be the most time-consuming aspect of data generation.
It may take up to three months after samples are collected before the data are
available.
(B) Identify project constraints. The sampling team has a limited amount of time to
collect samples due to budget constraints. This will be a major consideration during
the development of the sampling and analysis design.
(5) Write a brief summary of the contamination problem — This site was placed on the
NPL due to the discovery of chromium contaminated soil. Chromium was also
detected in ground water associated with the site which is hydraulically connected to
drinking water wells. Residents in the area can be exposed to contaminants in soil and
ground water via ingestion. Residents can also be exposed to contaminated
particulates via inhalation. The site manager has designated the soils associated with
the site as an operable unit. Since the site is on the NPL, a remedial investigation will
be performed to determine which areas of the soil pose an unacceptable risk to human
health or the environment and require further assessment or a response action.
Step 2: Identify the Decision — a statement of the decision that will use environmental data and the
actions that could result from this decision.
(1) State the decision(s) — Determine whether sections of the landfarm (soil) pose an
unacceptable risk to human health or the environment or whether they exceed ARARs.
(2) State the actions that could result from the decision —
(a) No action.
(b) Recommend further assessment or a response action.
Step 3: Identify the Inputs to the Decision — a list of the environmental variables or characteristics
that will be measured and other information needed to make the decision.
(1) Identify the informational inputs needed to resolve the decision — Surface soil
samples need to be taken within the site boundaries.
(2) Identify sources for each information input — Total chromium will be measured in
soil samples.
(3) Define the basis for establishing contaminant-specific action levels — Since a health-
based non-carcinogenic value (600 ppm of total chromium) is lower than the risk-
based carcinogenic PRO of 700 ppm for hexavalent chromium, the total chromium
concentration value is considered more protective.
105
-------
(4) Identify potential sampling techniques and appropriate analytic methods — A soil
coring device has been identified as the potential sampling technique. Atomic
absorption is the proposed analytical methodology.
Step 4: Define the Boundaries of the Study — a detailed description of the spatial and temporal
boundaries of the decision; characteristics that define the environmental media, objects, or
people of interests; and any practical considerations for the study.
(1) Define spatial boundaries —
(A) Define the domain within which all decisions must apply. Surface soil is defined
as the top 12 inches of soil within the geographic boundaries of the 29-acre landfarm
area, excluding forested areas where landfarming and disposal could not have taken
place.
(B) Specify the characteristics that define the population of interest. Chromium
concentrations in soil samples.
(C) Define the scale of decision making. Although the area is rural, future residential
development is possible. Residential land use represents a reasonable worst-case
scenario. The entire site has been divided into square areas that are approximately 200
x 200 feet. These areas are approximately one acre in size and correspond to the
expected residential lot size. These areas are referred to as "exposure units" (EUs).
EUs which overlapped the site boundaries were combined with EUs having forested
areas so that 20 EUs of approximately one acre would result. A separate decision will
be made for each EU.
(2) Identify temporal boundaries — EPA is facing public pressure to reduce the exposure
risk from the site quickly.
(A) Determine what time frame the sampling data must represent. Because chromium
is not migrating or degrading to any significant degree, the sampling results will apply
to lifetime exposure.
(B) Determine when to collect data. Sampling must occur within a one-week period
when EPA has made funds available.
(3) Identify practical considerations that may interfere with the study — The center of
each EU will be marked with a wire flag. Because the site is currently used for
grazing, there is considerable concern that the cows will ingest the wire flags. This
would injure the cows and impede timely sample collection. Some background
investigation has indicated that it is not likely the cows will eat the wire flags. As a
precaution, the farmers will be informed of the sampling activities in order to protect
the welfare of the cows.
106
-------
Step 5: Develop a Decision Rule — an "if...then..." statement that defines the conditions that would
cause the decision maker to choose among alternative actions.
(1) Specify the parameter of interest — The mean concentration of total chromium within
each EU will be compared to the action level.
(2) Specify the action level for the study — The action level for this problem will be 600
ppm of total chromium.
(3) Develop a decision rule (an "if...then" statement) — If the average total chromium
concentration in the surface soil of an EU exceeds 600 ppm, then recommend further
assessment or a response action will be taken. Otherwise, no action will be taken.
Step 6: Specify Limits on Decision Errors — the decision maker's acceptable decision error rates
based on a consideration of the consequences of making an incorrect decision.
(1) Determine the possible range of the parameter of interest — The possible range of
chromium concentrations is 0-1000 ppm.
(2) Define both types of decision errors and identify the potential consequences of each —
(A) Define both types of decision errors and establish which decision error has the
more severe consequences.
The two decision errors are:
Decision Error 'a': One decision error occurs when the decision maker decides an EU
is not contaminated when, in truth, the mean concentration of chromium is greater than
or equal to 600 ppm. If an EU that poses an unacceptable risk is not remediated,
some resources may be saved, but this would be at the cost of increased human health
and/or environmental risk. Increased future health costs or cancer deaths may also
result. This decision error is more severe.
Decision Error 'b': The other decision error occurs when the decision maker decides,
based on the data, to take action when, in truth, the mean concentration of chromium
is less than 600 ppm. One possible consequence of this decision error is unnecessary
further study in the EU. This would result in wasted resources and time. Offsetting
this to some degree would be the marginal reduction in health risk if a response action
is taken.
(B) Establish the true state of nature for each decision error. The true state of nature
for decision error 'a' is that the mean concentration of chromium is greater than 600
ppm. The true state of nature for decision error 'b' is that the mean concentration of
chromium is less than 600 ppm.
(C) Define the true state of nature for the more severe decision error as the baseline
condition or null hypothesis and define the true state of nature for the less severe
decision error as the alternative hypothesis. The hypothesis test is stated as:
107
-------
(3)
(4)
Null Hypothesis (HJ: Mean concentration in the EU £ 600 ppm
Alternate Hypothesis (HJ: Mean concentration in the EU < 600 ppm
(D) Assign the terms "false positive" and "false negative" to the proper errors.
false positive error = decision error 'a'
false negative error = decision error 'b'
Specify the Gray Region — The gray region corresponds to the area where the
decision maker considers the consequences of making a false negative decision error to
be relatively minor. In this example, the gray region is set to the left of the action
level between 500 ppm and 600 ppm (see Figure C-2).
Identify Acceptable Decision Error Rates — The decision maker specified the
probability of deciding to take action at four different total chromium concentrations.
True Concentration of Total Chromium
100 ppm
250 ppm
500 ppm
600 ppm
Acceptable Probability of Taking Action
less than or equal to 1%
less than or equal to 10%
less than or equal to 25%
greater than or equal to 95%
Based on the above table, at a true mean of 100 ppm, the decision maker can tolerate
making a false negative decision error 1 % of the time. At 600 ppm (the action level),
the decision maker wants to be confident of taking action 95% of the time (i.e., can
tolerate making a false positive decision error 5% of the time).
Step 7: Optimize the Design — ihe decision makerts) will select the lowest cost sampling design
that is expected to achieve (he DQOs.
(1) Develop general sampling and analysis design alternatives — For each design
alternative, the statistician must formulate a statistical model (i.e., a mathematical
expression) that tests the hypothesis and select the optimal sample size that satisfies
the decision maker's limits on decision errors.
Several alternate designs were discussed and subsequently deemed impractical by the
decision maker. One design was considered possible, however. A spatially intensive
design was developed which would gather composite soil samples from each EU.
Samples will be taken using a systematic grid. The sampling crew is more
comfortable with this type of design than with a random sampling plan. An
approximate t-test is suggested for each EU by calculating
t =
600 - M.
108
-------
where Mh is the mean of the h* EU and v is the pooled within-EU variance. This will
be compared with the critical value of a t-distribution for a = 0.05 and 20 degrees of
freedom. If the computed value exceeds the critical value, the null hypothesis will be
rejected.
1
*- _ 0.9
0)* °8
•i§
% '•§ °-7
»--0 0.6
O 0>
— X OS
5 UJ
So
2 § °-4
Q-0
|6 °-3
§" 0> 0.2
0.1
0.01
0
(
„ „._,
_
-
-
$$&
m
m
xXv
x^X/1
kgg>
jAAA,
x^rx
vsey
vOC*
xx5>
wy
vCv
XXX
£x?
?885
§66
Vxx
,/^>$8|
^^ False XXX>
.^ Negative XX>Q<
^^ Decision Errors >yyy*
) I 200 I 400
6(
Acceptable*
False
Positive _
Decision
Errors
-
-
«... A QfT
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
n
X \ 800
100 300 500 A 700
' Action Level
True Mean Concentration of Cr in EU (ppm)
Figure C-2. Design Performance Goal for Rawhide Site
Estimate of Variance
A limited field investigation was conducted in order to develop an estimate of the
expected variability of the contaminant. A preliminary estimate of the total standard
deviation of the chromium is 65.70 ppm.
Statistical Model
The model proposed for the observed composite sample concentrations is
where: Xy = j* composite sample of the 1th EU
= mean concentration of the i* EU
= deviation from Uj for j* composite sample of the i"1 EU
and the e's are distributed normally with mean zero.
Mi
109
-------
Sample Size
A maximum of nine samples per composite can be realistically handled. Using this
information and the prior estimate of the standard deviation, two composite samples of
nine scoops each will be randomly selected from each of the 20 EUs. This sample
size will provide 20 degrees of freedom, provided that the within-EU variances can be
pooled.
(2) Select the most resource-effective design that satisfies all of the DQOs — Composite
samples save money by reducing analysis costs, which is important for the initial study
as well as for the next phase of study.
This design meets the decision maker's objectives for adequately identifying which
EUs require further study or a response action. This is critical given the expected high
cost of remediation.
(3) Document the details and assumptions of the selected design — Two composite
samples of nine scoops each will be selected within each EU. A systematic grid with
nine nodes will be used to collect the first composite sample. The second composite
sample will consist of nine samples that are offset from the original grid nodes.
Within each EU it is assumed that the variance is the same, regardless of the level of
contamination. This assumption can be tested after the data are collected.
110
-------
APPENDIX III
GLOSSARY
GLOSSARY OF TERMS
action level: the numerical value that causes the decision maker to choose one of the alternative actions
(e.g., compliance or noncompliance). It may be a regulatory threshold standard, such as a
Maximum Contaminant Level for drinking water, a risk-based concentration level, a technological
limitation, or reference-based standard.
bias: the systematic or persistent distortion of a measurement process which causes errors in one direction
(i.e., the expected sample measurement is different than the sample's true value).
boundaries: the area or volume (spatial boundary) and the time period (temporal boundary) to which the
decision will apply. Samples are collected within these boundaries to be representative of the
population of interest for the decision.
Data Quality Assessment (DQA): a process of statistical and scientific evaluation that is used to assess
the validity and performance of the data collection design and statistical test, and to establish
whether a data set is adequate for its intended use.
Data Quality Objectives (DQOs): qualitative and quantitative statements derived from the outputs of
each step of the DQO Process which specify the study objectives, domain, limitations, the most
appropriate type of data to collect, and specify the levels of decision error that will be acceptable
for the decision.
Data Quality Objectives Process: a Quality Management tool based on the Scientific Method and
developed by the U.S. Environmental Protection Agency to facilitate the planning of
environmental data collection activities. The DQO Process enables planners to focus their
planning efforts by specifying the use of the data (the decision), the decision criteria (action level),
and the decision maker's acceptable decision error rates. The products of the DQO Process are
the DQOs.
decision errors:
false positive error — The false positive error occurs when data mislead a decision maker into
believing that the burden of proof in a hypothesis test has been satisfied, so that the null
hypothesis is erroneously rejected. A statistician usually refers to the false positive error as alpha
(a), the level of significance, the size of the critical region, or a Type I error.
false negative error — The false negative error occurs when data mislead the decision maker into
wrongly concluding that the burden of proof in a hypothesis test has not been satisfied so that the
null hypothesis is accepted. A statistician usually refers to this as beta (P), or a Type n error. It
is also known as the complement of Power.
Ill
-------
defensible: the ability to withstand any reasonable challenge related to the veracity or integrity of
laboratory documents and derived data.
directed sampling: see judgmental sampling.
gray region: an area that is adjacent to or contains the action level, and where the consequences of
making a decision error are relatively small.
judgmental sampling: a subjective selection of sampling locations based on experience and knowledge
of the site by an expert.
limits on decision errors: the acceptable decision error rates established by the decision maker.
Economic, health, ecological, political, and social consequences should be considered when setting
limits on decision errors.
mean: the arithmetic average of a set of values.
measurement error: the difference between the true or actual state and that which is reported from
measurements.
median: the middle value for an ordered set of n values; represented by the central value when n is odd
or by the average of the two most central values when n is even.
medium: a substance (e.g., air, water, soil) which serves as a carrier of the analytes of interest.
natural variability: the variability that is inherent or natural to the media, objects, or people being
studied.
parameter: a numerical descriptive measure of a population.
percentile: a value on a scale of 100 that indicates the percentage of a distribution that is equal to or
below it.
population: the total collection of objects or people to be studied and from which a sample is to be
drawn.
power curve: the probability of rejecting the null hypothesis (H0) over the range of the population. The
power function is used to assess the goodness of a test or to compare two competing tests.
probabilistic sampling: a random selection of sampling locations that allows the sampling results to be
extrapolated to an entire site (or portion of the site).
quality assurance (QA): an integrated system of management activities involving planning, quality
control, quality assessment, reporting, and quality improvement to ensure that a product or service
(e.g., environmental data) meets defined standards of quality with a stated level of confidence.
112
-------
Quality Assurance Project Plan (QAPP): a formal technical document containing the detailed
procedures for assuring the quality of environmental data prepared for each EPA environmental
data collection activity and approved prior to collecting the data.
quality control (QC): the overall system of technical activities whose purpose is to measure and control
the quality of a product or service so that it meets the needs of users. The aim is to provide
quality that is satisfactory, adequate, dependable, and economical.
Quality Management Plan (QMP): a formal document describing the management policies, objectives,
principles, organizational authority, responsibilities, accountability, and implementation protocols
of an agency, organization, or laboratory for ensuring quality in its products and utility to its
users. In EPA, QMPs are submitted to QAMS for approval.
range: the numerical difference between the minimum and maximum of a set of values.
'sample: a single item or specimen from a larger whole or group, such as any single sample of any
medium (air, water, soil, etc.).
2sample: a group of samples from a statistical population whose properties are studied to gain information
about the whole.
sample variance: a measure of the dispersion of a set of values.
sampling: the process of obtaining a subset of measurements from a population.
sampling error: the error due to observing only a limited number of the total possible values that make
up the population being studied. It should be distinguished from errors due to imperfect selection,
bias in response, and errors of observation, measurement, or recording, etc.
scoping team: the group of people that will carry out the DQO Process. Members include the decision
maker (senior manager), representatives of other data users, senior program and technical staff,
senior managers (decision makers), someone with statistical expertise, and a QA/QC advisor (such
as a QA Manager).
standard deviation: the square root of the variance.
statistic: a function of the sample measurements; e.g., the sample mean or standard deviation.
study design: a study design specifies the final configuration of the environmental monitoring effort to
satisfy the DQOs. It includes the types of samples or monitoring information to be collected;
where, when, and under what conditions they should be collected; what variables are to be
measured; and the Quality Assurance and Quality Control (QA/QC) components that ensure
acceptable sampling error and measurement error to meet the decision error rates specified in the
DQOs. The study design is the principal part of the QAPP.
total study error: the sum of all the errors that are incurred during the process of sample design through
data reporting. Total study error is related to decision error.
113
-------
true: being in accord with the actual state of affairs.
Type I error: an error that can occur during a statistical hypothesis test. A Type i error occurs when
a decision maker rejects the null hypothesis (decides that the null hypothesis is false) when it is
actually true.
Type II error: an error that can occur during a statistical hypothesis test. A Type II error occurs when
the decision maker accepts the null hypothesis (decides that the null hypothesis is true) when it
is actually false.
uncertainty: a measure of the total variability associated with sampling and measurement that includes
the two major error components: systematic error (bias) and random error (imprecision)
114
-------
APPENDIX IV
BIBLIOGRAPHY
General
Environmental Protection Agency (EPA). 1987. Data Quality Objectives for Remedial Response
Activities: Development Process. EPA/540/G-87/003.
Environmental Protection Agency (EPA). 1987. Data Quality Objectives for Remedial Response
Activities, Example Scenario: RI/FS Activities at a Site with Contaminated Soil and Ground
Water. Office of Emergency and Remedial Response. EPA/540/G-87/004.
Environmental Protection Agency (EPA). 1991. Role of the Baseline Risk Assessment in Superfund
Remedy Selecting Decision. Office of Solid Waste and Emergency Response. OSWER
Directive 9355.0-30.
Environmental Protection Agency (EPA). 1992. Guidance for Data Useability in Risk Assessment:
Final. Office of Emergency and Remedial Response. Part A: 9285.7-09A.
Environmental Protection Agency (EPA). 1992. Guidance on Implementation of the Superfund
Accelerated Cleanup Model under CERCLA and the NCP. OSWER Directive No. 9203.1-03.
Environmental Protection Agency (EPA). 1992. Interim Draft EPA Requirements for Quality
Management Plans. EPA/QA/R-2.
Environmental Protection Agency (EPA). 1992. Piloting the New Superfund Accelerated Cleanup
Model "The New Superfund Paradigm." Office of Solid Waste and Emergency Response.
Environmental Protection Agency (EPA). 1992. Review of Draft Superfund Quality
Assurance/Quality Control (QA/QC) Fact Sheet. Office of Solid Waste and Emergency
Response.
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Assessing
sites Under the Superfund Accelerated Cleanup Model. Office of Solid Waste and Emergency
Response. 9203.1-051.
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Early Action
and Long-Term Action Under the Superfund Accelerated Cleanup Model. Office of Solid
Waste and Emergency Response. 9203.1-051.
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Enforcement
Under the Superfund Accelerated Cleanup Model. Office of Solid Waste and Emergency
Response. 9203.1-051.
115
-------
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Identifying
SACM Program Management Issues. Office of Solid Waste and Emergency Response.
9203.1-051.
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Regional
Decision Teams. Office of Solid Waste and Emergency Response. 9203.1-051.
Environmental Protection Agency (EPA). 1992. Superfund Accelerated Cleanup Model. Office of
Solid Waste and Emergency Response. Publication number 9203.1-01.
Environmental Protection Agency (EPA). 1993. EPA Quality System Requirements for Environmental
Programs (Draft). EPA7QA/R-1.
Environmental Protection Agency (EPA). 1993. EPA Requirements for Quality Assurance Project
Plans for Environmental Data Operations (Draft Final). EPA/QA/R-5.
Environmental Protection Agency (EPA). 1993. Guidance for Planning for Data Collection in
Support of Environmental Decision Making Using the Data Quality Objectives Process.
EPA/QA/G-4.
Environmental Protection Agency (EPA). 1993. Guidance for Conducting Environmental Data
Quality Assessments. EPA/QA/G-9.
Zirschky and Gilbert, July 9, 1989. "Detecting hot spots at hazardous waste sites." Chemical
Engineering, pp. 97-100.
State the Problem
Environmental Protection Agency (EPA). January 1987. Technology Briefs, Data Requirements for
Selecting Remedial Action Technology. Hazardous Waste Engineering Research Laboratory.
EPA/600/2-87/001.
Environmental Protection Agency (EPA). August 1988. CERCLA Compliance with Other Laws
Manual. Office of Emergency and Remedial Response. EPA/540/G-89/006.
Environmental Protection Agency (EPA). October 1988. Guidance for Conducting Remedial
Investigations and Feasibility Studies Under CERCLA. Office of Emergency and Remedial
Response. EPA/540/G-89/004.
Environmental Protection Agency (EPA). March 1989. Soil Sampling Quality Assurance User's
Guide (Second Edition). Environmental Monitoring Systems Laboratory. EPA/600/8-89/046.
Environmental Protection Agency (EPA). July 1989. Risk Assessment Guidance for Superfund,
Human Health Evaluation Manual: Part A. Office of Solid Waste and Emergency Response.
9285.701A.
116
-------
Environmental Protection Agency (EPA). August 1990. Guidance on Expediting Remedial Design
and Remedial Action. Office of Emergency and Remedial Response. EPA/540/G-90/006.
Environmental Protection Agency (EPA). October 1990. Guidance for Data Useability in Risk
Assessment. Office of Emergency and Remedial Response. EPA/540/G-90/008.
Environmental Protection Agency (EPA). October 1990. Subsurface Contamination Reference Guide.
Office of Emergency and Remedial Response. EPA/540/2-90/011.
Environmental Protection Agency (EPA). December 1991. Risk Assessment Guidance for Superfund:
Volume 1- Human Health Evaluation Manual: Part C. (Risk Evaluation of Remedial
Alternatives). Office of Research and Development. EPA/540/R-92/004.
Environmental Protection Agency (EPA). December 1991. Risk Assessment Guidance for Superfund:
Volume 1- Human Health Evaluation Manual: Part B. (Development of Risk-based
Preliminary Remediation Goals). Office of Research and Development. EPA/540/R-92/003.
Environmental Protection Agency (EPA). December 1991. Risk Assessment Guidance for Superfund:
Volume II Environmental Evaluation Manual. Office of Emergency and Remedial Response.
EPA 540/1-89 001.
Environmental Protection Agency (EPA). September 1991. Guidance for Performing Preliminary
Assessments Under CERCLA. Office of Emergency and Remedial Response. EPA/540/G-
91/013.
Environmental Protection Agency (EPA). September 1992. Guidance for Performing Site Inspections
Under CERCLA. Office of Emergency and Remedial Response. EPA/540/R-92/021.
Environmental Protection Agency (EPA). Handbook, Ground Water. Robert S. Kerr Environmental
Research Laboratory. EPA/625/6-87/016.
Identify the Decision
Environmental Protection Agency (EPA). October 1988. Guidance for Conducting Remedial
Investigations and Feasibility Studies Under CERCLA. Office of Emergency and Remedial
Response. EPA/540/G-89/004.
Environmental Protection Agency (EPA). August 1990. Guidance on Expediting Remedial Design
and Remedial Action. Office of Emergency and Remedial Response. EPA/540/G-90/006.
Environmental Protection Agency (EPA). September 1991. Guidance for Performing Preliminary
Assessments Under CERCLA. Office of Emergency and Remedial Response. EPA/540/G-
91/013.
Environmental Protection Agency (EPA). September 1992. Guidance for Performing Site Inspections
Under CERCLA. Office of Emergency and Remedial Response. EPA/540/R-92/021.
117
-------
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Assessing
Sites Under the Superfund Accelerated Cleanup Model. Office of Solid Waste and Emergency
Response. 9203.1-051.
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Early Action
and Long-Term Action Under the Superfund Accelerated Cleanup Model. Office of Solid
Waste and Emergency Response. 9203.1-051.
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Enforcement
Under the Superfund Accelerated Cleanup Model. Office of Solid Waste and Emergency
Response. 9203.1-051.
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Identifying
SACM Program Management Issues. Office of Solid Waste and Emergency Response.
9203.1-051.
Environmental Protection Agency (EPA). 1992. SACM Program Management Update: Regional
Decision Teams. Office of Solid Waste and Emergency Response. 9203.1-051.
Identify the Inputs
Environmental Protection Agency (EPA). August 1988. CERCLA Compliance with Other Laws
Manual. Office of Emergency and Remedial Response. EPA/540/G-89/006.
Environmental Protection Agency (EPA). October 1991. Guidance for Data Useability in Site
Assessment (Draft). Office of Emergency and Remedial Response.
Environmental Protection Agency (EPA). December 1991. Risk Assessment Guidance for Superfund:
Volume 1- Human Health Evaluation Manual: Part B. (Development of Risk-based
Preliminary Remediation Goals) Office of Research and Development. EPA/540/R-92/003.
Environmental Protection Agency (EPA). April 1992. Guidance for Data Useability in Risk
Assessment: Part A. Office of Emergency and Remedial Response. Publication 9285.7-09A.
Define the Study Boundaries
California Department of Health Services. 1986. The California State Mitigation Decision Tree
Manual. Sacramento, CA.
California Department of Transportation. 1988. Standard Specifications. Sacramento, CA.
Cross, F.L. Jr., Cameron, W.W. Jr. 1974. Handbook of Swimming Pool Construction, Maintenance
and Sanitation. Technomic, Westport, CT.
118
-------
Environmental Protection Agency (EPA). 1988. Ground Water Modeling: An Overview and Status
Report. EPA/600/2-89/028.
Environmental Protection Agency (EPA). 1989. Determining Soil Response Action Levels Based on
Potential Contaminant Migration to Ground Water: A Compendium of Examples. EPA/540/2-
89/057.
Environmental Protection Agency (EPA). 1989. Guidance on Applying the Data Quality Objectives
Process for Ambient Air Monitoring Around Superfund Sites (Stages 1 and 2). Office of Air
Quality and Planning and Standards. EPA/450/4-89/015.
Environmental Protection Agency (EPA). 1989. Soil Sampling Quality Assurance User's Guide.
Environmental Monitoring Systems Laboratory. Las Vegas, NV. EPA/600/8-89/046.
Environmental Protection Agency (EPA). 1991. Representative Sampling Guidance, Vol I, Soil.
OSWER Directive 9360.4-10.
Gibbons, R.D. March-April 1990. "A General Statistical Procedure for Ground-Water Detection
Monitoring at Waste Disposal Facilities." Ground Water 28:2:235-43.
Hydak, P.P. and H.A. Loaciga. March 1992. "A Location Modeling Approach for Groundwater
Monitoring Networking Augmentation." Water Resources Research 28:3:643-649.
International Association of Plumbing and Mechanical Officials. 1985. Uniform Plumbing Code. Los
Angeles, CA.
International Conference of Building Officials. 1985. Uniform Building Code. Whittier, CA.
Kimbrough, R.D., H. Falk, P. Stehr and G. Fries. 1984. "Health implications of 2,3,7,8-
tetra;chlorodibenzodioxin (TCDD) contamination of residential soil." /. Toxicology, Environ.
Health, 14:47.
Koeford, K., R. Fisher, and R. Johnson. December 12, 1988. Personal Communications.
Krebs, R.D. and R.D. Wallace. 1971. Highway Materials. McGraw Hill, New York.
McBear, E.A. and F. A. Rovers. 1992. "Estimation of the probability of Exceedance of Contaminant
Concentrations." Ground Water: Monitoring Review. Winter 1992: 115-19.
Monahan, E.J. 1986. Construction of and on Compacted Fills. J. Wiley and Sons, New York.
National Fire Protection Association. 1984. National Electrical Code. Quincy, Massachusetts.
Nichols, H.L. 1974. Moving the Earth, Workbook of Excavation. 3rd ed. North Castle Books,
Greenwich, CT.
119
-------
Peck, R.B., W.E. Hansen and T.H. Thornburn. 1979. Foundation Engineering. 2nd ed. J. Wiley and
Sons, New York.
Reynolds, S.D., P.W. Hadleyand and R.M. Sedman. 1990. "A health based approach for evaluating
soils at hazardous waste sites using the AAL soil contact criterion." Risk Analysis 10:571.
Sedman, R.M. 1990. "The development of applied action levels for soil contact: A scenario for the
exposure of humans to soil in a residential setting." Environ. Health Perspect. 84-203.
Sowers, G.F. 1979. Introductory Soil Mechanics and Foundations: Geotechnical Engineering. 4th
ed. MacMillan, New York.
Spruill, T.B. and L. Candale. 1990. "Two Approaches to Design of Monitoring Networks." Ground
Water GRWAAP May/June 1990 28:3:430-42.
Specify Limits on Decision Errors
Environmental Protection Agency (EPA). 1989. Methods for Evaluating the Attainment of Cleanup
Standards: Volume 1: Soils and Solid Media. EPA/230/02-89/042. Office of Policy, Planning,
and Evaluation.
Environmental Protection Agency (EPA). 1989. Methods for Evaluating the Attainment of Cleanup
Standards: Volume 2: Ground Water. EPA/230/R-92/014. Office of Policy, Planning, and
Evaluation.
Gilbert, Richard O. 1987. Statistical Methods for Environmental Pollution Monitoring. New York:
Van Nostrand Reinhold Company.
Natrella, Mary Gibbons. 1963. Experimental Statistics. US Department of Commerce, National
Bureau of Standards Handbook 91.
Snedecor, G.W. and W.G. Cochran. 1980. Statistical Methods, 7th Edition. Ames, Iowa: Iowa State
University Press.
Steel, R.G.D., and J. H. Torrie. 1980. Principles and Procedures of Statistics. New York: McGraw-
Hill Book Company.
Optimize the Design
Barnett, V. and T. Lewis. 1984. Outliers in Statistical Data. New York: John Wiley & Sons.
Cochran W. 1977. Sampling Techniques. New York: John Wiley & Sons.
120
-------
Environmental Protection Agency (EPA). December 29, 1980. Interim Guidelines and Specifications
for Preparing Quality Assurance Project Plans. QAMS-005/80. Office of Monitoring
Systems and Quality Assurance, Office of Research and Development.
Environmental Protection Agency (EPA). 1986. Test Methods for Evaluating Solid Waste (SW-846):
Physical/Chemical Methods, Third Edition. Office of Solid Waste.
Environmental Protection Agency (EPA). 1989. Removal Program's Representative Sampling
Guidance Document. OSWER Directive: 9360.4-10.
Environmental Protection Agency (EPA). 1993. Guidance Document on the Statistical Analysis of
Ground-Water Monitoring Data at RCRA Facilities. Office of Solid Waste.
Environmental Protection Agency (EPA). 1989. Methods for Evaluating the Attainment of Cleanup
Standards: Volume 1: Soils and Solid Media. EPA 230/02-89-042. Office of Policy,
Planning, and Evaluation.
Environmental Protection Agency (EPA). 1993. Methods for Evaluating the Attainment of Cleanup
Standards: Volume 2: Ground Water. EPA/230/R-92/014. Office of Policy, Planning, and
Evaluation.
Environmental Protection Agency (EPA). 1990. A Rationale for the Assessment of Errors in the
Sampling of Soils. Office of Research and Development. EPA/600/4-90/013.
Environmental Protection Agency (EPA). April, 1992. Guidance for Data Useability in Risk
Assessment (Part A). Publication 9285.7-09A. Office of Emergency and Remedial Response.
Gilbert, Richard O. 1987. Statistical Methods for Environmental Pollution Monitoring. New York:
Van Nostrand Reinhold Company.
• U S. GOVERNMENT PRINTING OFFICE- 1994— 300-574 / 0 i» 0 0 0
121
------- |