United States         Off ice of          Publication 9355.9-01 A
Environmental Protection      Solid Waste and       EPA540-R-93-078
Agency            Emergency Response     PB94-963204
                          September 1993

Superfund



DATA QUALITY OBJECTIVES


PROCESS FOR SUPERFUND
Workbook

-------
                                         9355.9-01 A
                                     EPA540-R-93-078
                                       PB94-963204
                                      September 1993
DATA QUALITY OBJECTIVES PROCESS
             FOR SUPERFUND

                  Workbook
         Office of Emergency and Remedial Response
           U.S. Environmental Protection Agency
                Washington,  DC 20460
                    Printed on Recycled Paper
                           U.S. Environment! Protection Agency
                           Rc-gion 5, Librp'v (P1 -]"';)

-------
                        TABLE OF CONTENTS




                                                                  Page




INTRODUCTION	  1




STEPS IN THE DQO PLANNING PROCESS 	  3




STEP 1.  STATE THE PROBLEM	  5




STEP 2.  IDENTIFY THE DECISION	 11




STEP 3.  IDENTIFY THE INPUTS TO THE DECISION  	 15




STEP 4.  DEFINE THE BOUNDARIES OF THE STUDY  	 19




STEP 5.  DEVELOP A DECISION RULE	 23




STEP 6.  SPECIFY LIMITS ON DECISION ERRORS	 25




STEP 7.  OPTIMIZE THE DESIGN 	 31
                                  iii

-------
                                    INTRODUCTION
Overview and Purpose
This workbook assists the user in implementing the Data Quality Objective (DQO) Process for
Superfund by providing concise descriptions of each of the seven steps as well as  space to document
the outputs of the current study. This workbook is organized in a sequential, step-by-step fashion to
help the user consider all aspects in the DQO planning Process.  The steps described are as follows:

       STEP 1.       State the Problem
       STEP 2.       Identify the Decision
       STEP 3.       Identify the Inputs to the Decision
       STEP 4.       Define the Boundaries of the Study
       STEP 5.       Develop a  Decision Rule
       STEP 6.       Specify Limits  on Decision Errors
       STEP 7.       Optimize the Design
Each step is one component in the DQO Process.  The first section of this workbook provides a
summary of the purpose of each step. If more detailed information is required, the reader should refer
to "Data Quality Objectives Process for Superfund: Interim Final Guidance," EPA540-R-93-071.
Following the summaries, each step is discussed in detail, listing all the activities that should be
considered to complete that step.

The user is urged to be as detailed as possible in documenting the DQOs  in order to make the process
effective.

-------
                     STEPS IN THE DQO PLANNING PROCESS
Step 1.        State the Problem

       The purpose of this step is to describe the problem, specify the available resources and
       relevant deadlines for the study, and organize the scoping team. Stating the problem is one of
       the most important steps of the DQO Process.  The information collected in this step should
       provide a solid foundation upon which the rest of the process will be built. The problem
       statement should be as detailed as possible and address both the chemical and physical issues
       present.  The scoping team identified in this step will use this information later in the DQO
       Process to determine what new environmental data and information will be needed to resolve
       the problem.
Step 2.        Identify the Decision

       The purpose of this step is to define the decision that will use environmental data to address or
       solve the problem and to identify alternative actions to be taken based on the findings of the
       field investigation.  The combination of the decision and alternative actions will define the
       objectives of the study.  This step will help focus the efforts of the scoping team toward a
       common goal.
Step 3.        Identify the Inputs to the Decision

       The purpose of this step is to develop a detailed description of the data or informational inputs
       required to support the decision and to specify which inputs will require new environmental
       measurements.  Participation of the entire scoping team will allow access to  a broader base of
       knowledge and will assure that a thorough description of the data requirements is prepared.
Step 4.        Define the Boundaries of the Study

       The purpose of this step is to define the spatial and temporal boundaries of the study to clarify
       the domain of what the sample data are intended to represent.  In this step the scoping team
       will define the boundaries of the entire site, each operable unit of concern, and the scale of
       decision making.
Step 5.         Develop a Decision Rule

       The purpose of this step is to integrate information from the previous steps into a statement
       that describes the logical basis for choosing among alternative actions.  The Decision Rule is a
       summary statement that specifies how the decision maker expects to use data to make the
       decision.

-------
Step 6.        Specify Acceptable Limits on Decision Error

       The purpose of this step is to specify the site manager's acceptable limits on decision errors.
       These limits will be used in Step 7 to develop a sampling design that is adequate for the data's
       intended use.
Step 7.        Optimize the Design

        The purpose of this step is to develop a sampling strategy that satisfies all of the previously
        defined data quality objectives, and falls within the project's budget constraints. Specific
        operational details of the sampling plan are outlined by the scoping team, as well as the
        limitations and assumptions of the sampling design.

-------
                           STEP  1.  STATE THE PROBLEM

Background

       The purpose of this step is to describe the problem, specify the available resources and
relevant deadlines for the study, and organize the scoping team. Stating the problem is one of the
most important steps of the DQO  Process.  The information collected in this step should provide a
solid foundation upon which the rest  of the Process will be built.  The problem statement should be as
detailed as possible and address both  the chemical  and physical issues present.  The scoping team
identified in this step will use this information later in the DQO Process to determine what new
environmental data and information  will be needed to resolve  the problem.

Activities

1.     Identify the Members of the Scoping Team.

       •      Identify members  of the scoping team. These may include but are not limited to:
              samplers, chemists, modelers, technical experts, data users, QA specialists, risk
              assessors and a statistician.
       •      Identify the team/project leader who will be the primary decision maker.

2.     Develop/Refine the Conceptual Site Model.

       •      Collect historical site  data associated with previous data collection activities.
       •      List known or suspected sources of contamination.
       •      List types of contaminants  and affected media.
       •      List known or potential routes of migration.
       •      List known or potential human and environmental  receptors.

3.     Define the Exposure Pathways and Exposure Scenarios.

       •      Define the exposure pathways.
       •      Define the current and future land uses.
       •      Define applicable  or relevant and appropriate requirements (ARARs) or preliminary
              remediation goals  (PRGs).
       •      Develop the Exposure Scenario.

4.     Specify the Available Resources.

       •      Specify the monetary  budget for the field investigation.
       •      Define time constraints relative to health risks,  regulatory time limits, and political
              factors such as public concern.

5.     Summarize the  Contamination Problem.

       •      Combine the relevant background information into a concise description of the
              problem to be resolved.

-------
Worksheets for STEP 1 - STATE THE PROBLEM

State the Problem - Describe the problem and specify available resources and relevant
deadlines for the study.
(1)
Identify the Members of the Scoping Team.

(a)     Identify members of the scoping team.
        (b)     Identify the primary decision maker.
(2)     Develop/Refine the Conceptual Site Model.

        (a)     List sources of historic data associated with previous data collection activities.

-------
Worksheets for STEP 1 - STATE THE PROBLEM




       (b)     List known or suspected sources of contamination.
       (c)     List types of contaminants and affected media.
       (d)     List known or potential routes of migration.

-------
Worksheets for STEP 1 - STATE THE PROBLEM
       (e)     List known or human and environmental receptors.
(3)    Define Exposure Pathways and Exposure Scenarios.




       (a)     Define the exposure path way (s).
       (b)     Define the current and future land use.

-------
Worksheets for STEP 1 • STATE THE PROBLEM
       (c)     Define applicable or relevant and appropriate requirements (ARARs) or
              preliminary remediation goals (PRGs).
       (d)    Develop the exposure scenario.
(4)    Specify the Available Resources

       (a)     Specify the monetary budget for the field investigation.
       (b)     Define relevant time constraints.

-------
      Worksheets for STEP 1 • STATE THE PROBLEM
       (5)    Summarize the Contamination Problem

             (a)     Combine the relevant background information into a concise description of the
                    problem to be resolved.
NOTES:
                                          10

-------
                         STEP 2. IDENTIFY THE DECISION

Background
       The purpose of this step is to define the decision that will use environmental data to address or
solve the problem and to identify  alternative actions to be taken based on the findings of the field
investigation. The combination of the decision and alternative actions will define the objectives of the
study.  This step will help focus the efforts of the scoping team toward a common goal.
Activities
1.      Select the appropriate decision for the current phase of the site assessment process from the
       list below.

       Early Assessment Decision: Determine whether the release (or potential release) poses a
       threat to human health or the environment.

       Advanced Assessment Decision, Phase I: Determine whether the concentration of
       contaminants of concern  exceed ARARs or exceed contaminant concentrations corresponding
       to the preliminary remediation goal for the site.

       Advanced Assessment Decision, Phase n (Extent of Contamination): Determine the
       volume of media that exceeds action levels (i.e., ARARs, concentrations corresponding to the
       preliminary remediation goal, removal action  levels, or final remediation levels).

       Cleanup Attainment Decision:  Determine whether the final remediation level(s) or removal
       action level(s) have been achieved.
                                            11

-------
2.     Identify Alternative Actions that May Be Taken Based on the Findings of the Field
       Investigation.

       •       Select the actions that will be taken based on the outcome of the field investigation
               that corresponds with the selected decision.

               Early Assessment:
                      recommend the site evaluation accomplished (SEA) designation for the site;
                      recommend further assessment such as a focused site inspection or an
                      expanded site inspection/remedial investigation; or
                      recommend a response action such as an emergency/time-critical removal
                      action, a non-time-critical early action, the initiation of the NPL listing
                      process, and/or the initiation of enforcement activities.
               Advanced Assessment, Phase I:
                      recommend the SEA designation for the site; or
                      recommend a response action such as a non-time-critical removal or early
                      and/or long-term remedial action.
               Advanced Assessment, Phase II:
                      designate the area/volume of contaminated material for remediation;
                      do not designate the area/volume for remediation; or
                      recommend the development of removal engineering evaluation and cost
                      analysis (EE/CA) and remedial feasibility studies (FS).
               Cleanup Attainment:
                      recommend SEA designation and proceed with delisting procedures; or
                      recommend that further response is appropriate for  the site.
3.     Identify Relationships Among Decisions.

       •       Prioritize decisions.
       •       Determine the logical sequence of actions.
                                               12

-------
Worksheets for STEP 2 - IDENTIFY THE DECISION

Identify the Decision - Define the decision that will use environmental data to address the
problem and identify alternative actions to be taken based on the findings of the field
investigation.

(1)     Select the appropriate decision for the current phase of the site assessment process.
(2)
Identify Alternative Actions That May be Taken Based on the Findings of the Field
Investigation.

(a)     Select the actions that will be taken based on the outcome of the field
       investigation that corresponds with the selected decision.
(3)     Identify Relationships Among Decisions.

        (a)     Prioritize decisions.
                                        13

-------
      Worksheets for STEP 2 - IDENTIFY THE DECISION




      (b)     Determine the logical sequence of actions.
NOTES:
                                         14

-------
              STEP 3. IDENTIFY  THE INPUTS  TO THE DECISION


Background
        The purpose of this step is to develop a detailed description of the data or informational inputs
required to support the decision and to specify which inputs will require new environmental
measurements.  Participation of the entire scoping team will allow access  to a broader base of
knowledge and will assure that a thorough description of the data requirements is prepared.
Activities
1.      Identify the Informational Inputs Needed to Resolve the Decision.

        •       Prepare a list of all of the data needed to accomplish the objectives, including data that
               may already exist and data that must be collected. Brainstorming techniques may be
               useful during this step.  Diagramming techniques may also be useful in organizing data
               needs.

        •       Indicate how to generate the necessary data (e.g., sampling, modeling, etc.).
2.      Identify Sources for Each Environmental Input and List the Inputs That are Obtained Through
        Environmental Measurements.

        •       Identify existing sources of information that can support the decision.
3.     Determine the Basis for Establishing Contaminant-Specific Action Level(s).

       •       Identify the possible basis for establishing the action level (e.g., regulatory threshold,
               risk or exposure assessment, technological limits, reference-based standards, etc.).

4.     Identify Potential Sampling Techniques and Appropriate Analytical Methods
                                              15

-------
Worksheets for STEP 3 - IDENTIFY THE INPUTS TO THE DECISION

Identify the Inputs to the Decision - Identify the informational inputs needed to support the
decision and specify which inputs will require new environmental measurement.

(1)    Identify the Informational Inputs Needed to Resolve the Decision.

       (a)     Prepare a list of all of the data needed to resolve the decision.
       (b)    Indicate how to generate the necessary data (e.g., sampling, modeling etc.).
(2)    Identify Sources for Each Environmental Input and List Those Inputs That are
       Obtained Through Environmental Measurements.

       (a)    Identify existing sources of information that can support the decision.
                                      16

-------
Worksheets for STEP 3 - IDENTIFY THE INPUTS TO THE DECISION

(3)     Determine the Basis for Establishing Contaminant-Specific Action Level(s)

       (a)     List the possible basis for establishing the action level (e.g., regulatory
              threshold, risk or exposure assessment, technological limits, reference based
              standards, etc.).
(4)     Identify Potential Sampling Approaches and Appropriate Analytical Methods.
                                      17

-------
     Worksheets for STEP 3 - IDENTIFY THE INPUTS TO THE DECISION




NOTES:
                                   18

-------
             STEP 4.  DEFINE THE BOUNDARIES OF THE STUDY
Background
       The purpose of this step is to define the spatial and temporal boundaries of the study to clarify
the domain of what the sample data are intended to represent.  In this step the scoping team will
define the boundaries of the entire site, each operable unit of concern, and the scale of decision
making.
Activities
1.      Define the Geographic Areas of the Field Investigation.

       •      Define the domain or geographic area within which all decisions must apply (in some
              cases this may be defined by the Operable Unit).

       •      Specify the characteristics that define the population of interest.

       •      When appropriate, divide the population into strata that have relatively homogeneous
              characteristics.

       •      Define the Scale of Decision Making.
2.      Define the Temporal Boundaries of the Decision.

       •      Determine the time frame to which the study data apply.

       •      Determine when to collect data.


3.      Identify Any Practical Constraints on Data Collection.
                                            19

-------
Worksheets for STEP 4 - DEFINE THE BOUNDARIES OF THE STUDY

Define the Boundaries of the Study - Define the spatial and temporal boundaries of the study
in order to clarify what the samples are intended to represent

(1)     Define the Geographic Areas of the Field Investigation.

       (a)     Define the domain or geographic area within which all decisions must apply
               (in some cases this may be defined by the Operable Unit).
       (b)     Specify the characteristics that define the population of interest.
       (c)
When appropriate, divide the population into strata that have relatively
homogeneous characteristics.
       (c)     Define the Scale of Decision Making.
                                      20

-------
Worksheets for STEP 4 - DEFINE THE BOUNDARIES OF THE STUDY




(2)     Define the Temporal Boundaries of the Decision.




       (a)    Determine the time frame to which the study data apply.
       (b)
Determine when to collect data.
(3)     Identify Any Practical Constraints on Data Collection.
                                   21

-------
     Worksheets for STEP 4 - DEFINE THE BOUNDARIES OF THE STUDY
NOTES:
                                 22

-------
                      STEP 5.  DEVELOP A DECISION RULE
Background
       The purpose of this step is to integrate information from the previous steps into a statement
that describes the logical basis for choosing among alternative actions. The Decision Rule is a
summary statement that specifies how the decision maker expects to use data to make the decision.
Activities


1.      Specify the Parameter that Characterizes the Population of Interest.


2.      Specify the Action Level or preliminary action level for the Decision.


3.      Develop the Decision Rule.

       •       Combine the outputs of the previous DQO steps into an "if ... then ..." decision rule
               that includes the parameter of interest, the action level, and the alternative actions.
                                            23

-------
Worksheets for STEP 5 - DEVELOP A DECISION RULE

Develop a Decision Rule - Define the logical basis for choosing among alternative actions.

(1)     Specify the Parameter that Characterizes the Population of Interest.
(2)    Specify the Action Level or Preliminary Action Level for the Decision.
(3)    Develop the Decision Rule.

       (a)    Combine the outputs of the previous DQO steps into an "if ... then ..." decision
              rule that includes the parameter of interest, the action level, and the alternative
              actions.
                                      24

-------
                STEP 6.  SPECIFY LIMITS ON DECISION ERRORS
Background
        The purpose of this step is to specify the site manager's acceptable limits on decision errors.
These limits will be used in Step 7 to develop a sampling design that is adequate for the data's
intended use.
Activities
1.       Determine the Possible Range of the Parameter of Interest.

        •       Determine the upper and lower bounds for the parameter of interest using relevant
               historical  site data.
2.      Define Both Types of Decision Errors and Identify the Potential Consequences of Each.

        •       Using the actions, action level, and decision rule, define both types of decision errors.
        •       Establish the true state of nature for each decision error.
        •       Define the true state of nature for the more severe decision error as the baseline
               condition or the null hypothesis (H0) and define the true state of nature for the less
               severe decision error as the alternative hypothesis (Ha).
        •       Assign the terms  "false positive" and"false negative" to the proper decision errors.
3.      Specify a Range of Possible Parameter Values Where the Consequences of Decision Errors are
        Relatively Minor (the Gray Region).

        •       Identify the range of points on the false negative side of the action level where the
               consequences of making a decision error are relatively minor. This range establishes
               the gray region.
4.      Assign Probability Values to Points Above and Below the Action Level That Reflect the
        Acceptable Probability for the Occurrences of Decision Errors.
5.     Check for Consistency.

       •       Check the limits on decision errors to ensure that they accurately reflect the decision
               maker's concerns about the relative consequences for each type of decision error.
                                               25

-------
Worksheets for STEP 6 - SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS

Specify Acceptable Limits on Decision Errors - Specify acceptable decision error limits
based on the consideration of the consequences of making an incorrect decision.

(1)    Determine the Possible Range of the Parameter of Interest.

       (a)     Determine the upper and lower bounds for the parameter of interest using
               relevant historical site data.
(2)    Define Both Types of Decision Errors and Identify the Potential Consequences of
       Each.

       (a)     Using the actions, action level, and decision rule, define both types of decision
               errors.
       (b)
Establish the true state of nature for each decision error.
                                       26

-------
Worksheets for STEP 6 - SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS
       (c)
Define the true state of nature for the more severe decision error as the
baseline condition or the null hypothesis (HJ and define the true state of
nature for the less severe decision error as the alternative hypothesis (HJ.
        (d)     Assign the terms "false positive" and "false negative" to the proper decision
               errors.
(3)     Specify a range of possible parameter values where the consequences of decision
        errors are relatively minor (the gray region).

        (a)     Identify the range of points on the false negative side of the action level where
               the consequences of making a decision error are relatively minor. This range
               establishes the gray region.
                                       27

-------
Worksheets for STEP 6 - SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS
(4)     Assign Probability Values to Points Above and Below the Action Level That Reflect
       the Acceptable Probability for the Occurrences of Decision Errors.
(5)    Check for Consistency.

       (a)    Check the limits on decision errors to ensure that they accurately reflect the
              decision maker's concerns about the relative consequences for each type of
              decision error.
                                      28

-------
     Worksheets for STEP 6 • SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS




NOTES:
                                  29

-------
     Worksheets for STEP 6 - SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS




NOTES:
                                   30

-------
                          STEP 7.  OPTIMIZE THE DESIGN
Background
       The purpose of this step is to develop a sampling strategy that satisfies all of the previously
defined data quality objectives,  and falls within the project's budget constraints.  Specific operational
details of the sampling plan are outlined by the scoping team, as well as the limitations and
assumptions of the sampling design.

Activities


1.     Review the DQO outputs and existing environmental data.

       •       Compile outputs from previous steps of the DQO Process.


2.     Develop General Sampling and Analysis Design Alternatives.

       •       Develop a preliminary estimate of variability.

       •       Develop a sampling strategy for the site that specifies all of the previously defined
               data quality objectives.


3.     For Each Design Alternative, Verify That the DQOs are Satisfied.


4.     Select the Most Resource-Effective Design That Satisfies All of the DQOs.
5.     Document the Operational Details and Theoretical Assumptions of the Selected Design in the
       Sampling and Analysis Plan.

       •       Sample types (e.g., composite v. grab samples).
       •       General collection techniques (e.g., split spoon v. core drill, or activated charcoal
               media v. evacuated canister).
       •       Sample support (i.e., the amount of materials to be collected for each sample).
       •       Sample locations (surface coordinates and depth) and how the locations were selected.
       •       Timing issues for sample collection, handling, and analysis.
       •       Analytical methods (or performance standards).
       •       Quality assurance and quality control needs.
       •       The assumptions  of the  statistical model.
                                              31

-------
Worksheets for STEP 7 - OPTIMIZE THE DESIGN

Optimize the Design - Outline a sampling design, specifying the operational details of the
sampling plan which falls within the projects constraints.
(1)    Review the DQO Outputs and Existing Environmental Data.

       (a)     Compile outputs from previous steps of the DQO Process.
(2)    Develop General Sampling and Analysis Design Alternatives.

       (a)     Develop a preliminary estimate of variability.
       (b)     Develop a sampling strategy for the site that specifies all of the previously
               defined data quality objectives.
                                      32

-------
       Worksheets for STEP 7 - OPTIMIZE THE DESIGN
       (3)    For Each Design Alternative, Verify That the DQOs are Satisfied.
       (4)     Select the Most Resource-Effective Design That Satisfies All of the DQOs.
       (5)     Document the operational details and theoretical assumptions of the selected design in
              the Sampling and Analysis Plan.
NOTES:
                                           33

-------
Worksheets for STEP 7 • OPTIMIZE THE DESIGN
                              34

-------
ADDITIONAL WORKSHEETS
          35

-------
ADDITIONAL WORKSHEETS
          36

-------
ADDITIONAL WORKSHEETS
          37

-------