United States Off ice of Publication 9355.9-01 A Environmental Protection Solid Waste and EPA540-R-93-078 Agency Emergency Response PB94-963204 September 1993 Superfund DATA QUALITY OBJECTIVES PROCESS FOR SUPERFUND Workbook ------- 9355.9-01 A EPA540-R-93-078 PB94-963204 September 1993 DATA QUALITY OBJECTIVES PROCESS FOR SUPERFUND Workbook Office of Emergency and Remedial Response U.S. Environmental Protection Agency Washington, DC 20460 Printed on Recycled Paper U.S. Environment! Protection Agency Rc-gion 5, Librp'v (P1 -]"';) ------- TABLE OF CONTENTS Page INTRODUCTION 1 STEPS IN THE DQO PLANNING PROCESS 3 STEP 1. STATE THE PROBLEM 5 STEP 2. IDENTIFY THE DECISION 11 STEP 3. IDENTIFY THE INPUTS TO THE DECISION 15 STEP 4. DEFINE THE BOUNDARIES OF THE STUDY 19 STEP 5. DEVELOP A DECISION RULE 23 STEP 6. SPECIFY LIMITS ON DECISION ERRORS 25 STEP 7. OPTIMIZE THE DESIGN 31 iii ------- INTRODUCTION Overview and Purpose This workbook assists the user in implementing the Data Quality Objective (DQO) Process for Superfund by providing concise descriptions of each of the seven steps as well as space to document the outputs of the current study. This workbook is organized in a sequential, step-by-step fashion to help the user consider all aspects in the DQO planning Process. The steps described are as follows: STEP 1. State the Problem STEP 2. Identify the Decision STEP 3. Identify the Inputs to the Decision STEP 4. Define the Boundaries of the Study STEP 5. Develop a Decision Rule STEP 6. Specify Limits on Decision Errors STEP 7. Optimize the Design Each step is one component in the DQO Process. The first section of this workbook provides a summary of the purpose of each step. If more detailed information is required, the reader should refer to "Data Quality Objectives Process for Superfund: Interim Final Guidance," EPA540-R-93-071. Following the summaries, each step is discussed in detail, listing all the activities that should be considered to complete that step. The user is urged to be as detailed as possible in documenting the DQOs in order to make the process effective. ------- STEPS IN THE DQO PLANNING PROCESS Step 1. State the Problem The purpose of this step is to describe the problem, specify the available resources and relevant deadlines for the study, and organize the scoping team. Stating the problem is one of the most important steps of the DQO Process. The information collected in this step should provide a solid foundation upon which the rest of the process will be built. The problem statement should be as detailed as possible and address both the chemical and physical issues present. The scoping team identified in this step will use this information later in the DQO Process to determine what new environmental data and information will be needed to resolve the problem. Step 2. Identify the Decision The purpose of this step is to define the decision that will use environmental data to address or solve the problem and to identify alternative actions to be taken based on the findings of the field investigation. The combination of the decision and alternative actions will define the objectives of the study. This step will help focus the efforts of the scoping team toward a common goal. Step 3. Identify the Inputs to the Decision The purpose of this step is to develop a detailed description of the data or informational inputs required to support the decision and to specify which inputs will require new environmental measurements. Participation of the entire scoping team will allow access to a broader base of knowledge and will assure that a thorough description of the data requirements is prepared. Step 4. Define the Boundaries of the Study The purpose of this step is to define the spatial and temporal boundaries of the study to clarify the domain of what the sample data are intended to represent. In this step the scoping team will define the boundaries of the entire site, each operable unit of concern, and the scale of decision making. Step 5. Develop a Decision Rule The purpose of this step is to integrate information from the previous steps into a statement that describes the logical basis for choosing among alternative actions. The Decision Rule is a summary statement that specifies how the decision maker expects to use data to make the decision. ------- Step 6. Specify Acceptable Limits on Decision Error The purpose of this step is to specify the site manager's acceptable limits on decision errors. These limits will be used in Step 7 to develop a sampling design that is adequate for the data's intended use. Step 7. Optimize the Design The purpose of this step is to develop a sampling strategy that satisfies all of the previously defined data quality objectives, and falls within the project's budget constraints. Specific operational details of the sampling plan are outlined by the scoping team, as well as the limitations and assumptions of the sampling design. ------- STEP 1. STATE THE PROBLEM Background The purpose of this step is to describe the problem, specify the available resources and relevant deadlines for the study, and organize the scoping team. Stating the problem is one of the most important steps of the DQO Process. The information collected in this step should provide a solid foundation upon which the rest of the Process will be built. The problem statement should be as detailed as possible and address both the chemical and physical issues present. The scoping team identified in this step will use this information later in the DQO Process to determine what new environmental data and information will be needed to resolve the problem. Activities 1. Identify the Members of the Scoping Team. • Identify members of the scoping team. These may include but are not limited to: samplers, chemists, modelers, technical experts, data users, QA specialists, risk assessors and a statistician. • Identify the team/project leader who will be the primary decision maker. 2. Develop/Refine the Conceptual Site Model. • Collect historical site data associated with previous data collection activities. • List known or suspected sources of contamination. • List types of contaminants and affected media. • List known or potential routes of migration. • List known or potential human and environmental receptors. 3. Define the Exposure Pathways and Exposure Scenarios. • Define the exposure pathways. • Define the current and future land uses. • Define applicable or relevant and appropriate requirements (ARARs) or preliminary remediation goals (PRGs). • Develop the Exposure Scenario. 4. Specify the Available Resources. • Specify the monetary budget for the field investigation. • Define time constraints relative to health risks, regulatory time limits, and political factors such as public concern. 5. Summarize the Contamination Problem. • Combine the relevant background information into a concise description of the problem to be resolved. ------- Worksheets for STEP 1 - STATE THE PROBLEM State the Problem - Describe the problem and specify available resources and relevant deadlines for the study. (1) Identify the Members of the Scoping Team. (a) Identify members of the scoping team. (b) Identify the primary decision maker. (2) Develop/Refine the Conceptual Site Model. (a) List sources of historic data associated with previous data collection activities. ------- Worksheets for STEP 1 - STATE THE PROBLEM (b) List known or suspected sources of contamination. (c) List types of contaminants and affected media. (d) List known or potential routes of migration. ------- Worksheets for STEP 1 - STATE THE PROBLEM (e) List known or human and environmental receptors. (3) Define Exposure Pathways and Exposure Scenarios. (a) Define the exposure path way (s). (b) Define the current and future land use. ------- Worksheets for STEP 1 • STATE THE PROBLEM (c) Define applicable or relevant and appropriate requirements (ARARs) or preliminary remediation goals (PRGs). (d) Develop the exposure scenario. (4) Specify the Available Resources (a) Specify the monetary budget for the field investigation. (b) Define relevant time constraints. ------- Worksheets for STEP 1 • STATE THE PROBLEM (5) Summarize the Contamination Problem (a) Combine the relevant background information into a concise description of the problem to be resolved. NOTES: 10 ------- STEP 2. IDENTIFY THE DECISION Background The purpose of this step is to define the decision that will use environmental data to address or solve the problem and to identify alternative actions to be taken based on the findings of the field investigation. The combination of the decision and alternative actions will define the objectives of the study. This step will help focus the efforts of the scoping team toward a common goal. Activities 1. Select the appropriate decision for the current phase of the site assessment process from the list below. Early Assessment Decision: Determine whether the release (or potential release) poses a threat to human health or the environment. Advanced Assessment Decision, Phase I: Determine whether the concentration of contaminants of concern exceed ARARs or exceed contaminant concentrations corresponding to the preliminary remediation goal for the site. Advanced Assessment Decision, Phase n (Extent of Contamination): Determine the volume of media that exceeds action levels (i.e., ARARs, concentrations corresponding to the preliminary remediation goal, removal action levels, or final remediation levels). Cleanup Attainment Decision: Determine whether the final remediation level(s) or removal action level(s) have been achieved. 11 ------- 2. Identify Alternative Actions that May Be Taken Based on the Findings of the Field Investigation. • Select the actions that will be taken based on the outcome of the field investigation that corresponds with the selected decision. Early Assessment: recommend the site evaluation accomplished (SEA) designation for the site; recommend further assessment such as a focused site inspection or an expanded site inspection/remedial investigation; or recommend a response action such as an emergency/time-critical removal action, a non-time-critical early action, the initiation of the NPL listing process, and/or the initiation of enforcement activities. Advanced Assessment, Phase I: recommend the SEA designation for the site; or recommend a response action such as a non-time-critical removal or early and/or long-term remedial action. Advanced Assessment, Phase II: designate the area/volume of contaminated material for remediation; do not designate the area/volume for remediation; or recommend the development of removal engineering evaluation and cost analysis (EE/CA) and remedial feasibility studies (FS). Cleanup Attainment: recommend SEA designation and proceed with delisting procedures; or recommend that further response is appropriate for the site. 3. Identify Relationships Among Decisions. • Prioritize decisions. • Determine the logical sequence of actions. 12 ------- Worksheets for STEP 2 - IDENTIFY THE DECISION Identify the Decision - Define the decision that will use environmental data to address the problem and identify alternative actions to be taken based on the findings of the field investigation. (1) Select the appropriate decision for the current phase of the site assessment process. (2) Identify Alternative Actions That May be Taken Based on the Findings of the Field Investigation. (a) Select the actions that will be taken based on the outcome of the field investigation that corresponds with the selected decision. (3) Identify Relationships Among Decisions. (a) Prioritize decisions. 13 ------- Worksheets for STEP 2 - IDENTIFY THE DECISION (b) Determine the logical sequence of actions. NOTES: 14 ------- STEP 3. IDENTIFY THE INPUTS TO THE DECISION Background The purpose of this step is to develop a detailed description of the data or informational inputs required to support the decision and to specify which inputs will require new environmental measurements. Participation of the entire scoping team will allow access to a broader base of knowledge and will assure that a thorough description of the data requirements is prepared. Activities 1. Identify the Informational Inputs Needed to Resolve the Decision. • Prepare a list of all of the data needed to accomplish the objectives, including data that may already exist and data that must be collected. Brainstorming techniques may be useful during this step. Diagramming techniques may also be useful in organizing data needs. • Indicate how to generate the necessary data (e.g., sampling, modeling, etc.). 2. Identify Sources for Each Environmental Input and List the Inputs That are Obtained Through Environmental Measurements. • Identify existing sources of information that can support the decision. 3. Determine the Basis for Establishing Contaminant-Specific Action Level(s). • Identify the possible basis for establishing the action level (e.g., regulatory threshold, risk or exposure assessment, technological limits, reference-based standards, etc.). 4. Identify Potential Sampling Techniques and Appropriate Analytical Methods 15 ------- Worksheets for STEP 3 - IDENTIFY THE INPUTS TO THE DECISION Identify the Inputs to the Decision - Identify the informational inputs needed to support the decision and specify which inputs will require new environmental measurement. (1) Identify the Informational Inputs Needed to Resolve the Decision. (a) Prepare a list of all of the data needed to resolve the decision. (b) Indicate how to generate the necessary data (e.g., sampling, modeling etc.). (2) Identify Sources for Each Environmental Input and List Those Inputs That are Obtained Through Environmental Measurements. (a) Identify existing sources of information that can support the decision. 16 ------- Worksheets for STEP 3 - IDENTIFY THE INPUTS TO THE DECISION (3) Determine the Basis for Establishing Contaminant-Specific Action Level(s) (a) List the possible basis for establishing the action level (e.g., regulatory threshold, risk or exposure assessment, technological limits, reference based standards, etc.). (4) Identify Potential Sampling Approaches and Appropriate Analytical Methods. 17 ------- Worksheets for STEP 3 - IDENTIFY THE INPUTS TO THE DECISION NOTES: 18 ------- STEP 4. DEFINE THE BOUNDARIES OF THE STUDY Background The purpose of this step is to define the spatial and temporal boundaries of the study to clarify the domain of what the sample data are intended to represent. In this step the scoping team will define the boundaries of the entire site, each operable unit of concern, and the scale of decision making. Activities 1. Define the Geographic Areas of the Field Investigation. • Define the domain or geographic area within which all decisions must apply (in some cases this may be defined by the Operable Unit). • Specify the characteristics that define the population of interest. • When appropriate, divide the population into strata that have relatively homogeneous characteristics. • Define the Scale of Decision Making. 2. Define the Temporal Boundaries of the Decision. • Determine the time frame to which the study data apply. • Determine when to collect data. 3. Identify Any Practical Constraints on Data Collection. 19 ------- Worksheets for STEP 4 - DEFINE THE BOUNDARIES OF THE STUDY Define the Boundaries of the Study - Define the spatial and temporal boundaries of the study in order to clarify what the samples are intended to represent (1) Define the Geographic Areas of the Field Investigation. (a) Define the domain or geographic area within which all decisions must apply (in some cases this may be defined by the Operable Unit). (b) Specify the characteristics that define the population of interest. (c) When appropriate, divide the population into strata that have relatively homogeneous characteristics. (c) Define the Scale of Decision Making. 20 ------- Worksheets for STEP 4 - DEFINE THE BOUNDARIES OF THE STUDY (2) Define the Temporal Boundaries of the Decision. (a) Determine the time frame to which the study data apply. (b) Determine when to collect data. (3) Identify Any Practical Constraints on Data Collection. 21 ------- Worksheets for STEP 4 - DEFINE THE BOUNDARIES OF THE STUDY NOTES: 22 ------- STEP 5. DEVELOP A DECISION RULE Background The purpose of this step is to integrate information from the previous steps into a statement that describes the logical basis for choosing among alternative actions. The Decision Rule is a summary statement that specifies how the decision maker expects to use data to make the decision. Activities 1. Specify the Parameter that Characterizes the Population of Interest. 2. Specify the Action Level or preliminary action level for the Decision. 3. Develop the Decision Rule. • Combine the outputs of the previous DQO steps into an "if ... then ..." decision rule that includes the parameter of interest, the action level, and the alternative actions. 23 ------- Worksheets for STEP 5 - DEVELOP A DECISION RULE Develop a Decision Rule - Define the logical basis for choosing among alternative actions. (1) Specify the Parameter that Characterizes the Population of Interest. (2) Specify the Action Level or Preliminary Action Level for the Decision. (3) Develop the Decision Rule. (a) Combine the outputs of the previous DQO steps into an "if ... then ..." decision rule that includes the parameter of interest, the action level, and the alternative actions. 24 ------- STEP 6. SPECIFY LIMITS ON DECISION ERRORS Background The purpose of this step is to specify the site manager's acceptable limits on decision errors. These limits will be used in Step 7 to develop a sampling design that is adequate for the data's intended use. Activities 1. Determine the Possible Range of the Parameter of Interest. • Determine the upper and lower bounds for the parameter of interest using relevant historical site data. 2. Define Both Types of Decision Errors and Identify the Potential Consequences of Each. • Using the actions, action level, and decision rule, define both types of decision errors. • Establish the true state of nature for each decision error. • Define the true state of nature for the more severe decision error as the baseline condition or the null hypothesis (H0) and define the true state of nature for the less severe decision error as the alternative hypothesis (Ha). • Assign the terms "false positive" and"false negative" to the proper decision errors. 3. Specify a Range of Possible Parameter Values Where the Consequences of Decision Errors are Relatively Minor (the Gray Region). • Identify the range of points on the false negative side of the action level where the consequences of making a decision error are relatively minor. This range establishes the gray region. 4. Assign Probability Values to Points Above and Below the Action Level That Reflect the Acceptable Probability for the Occurrences of Decision Errors. 5. Check for Consistency. • Check the limits on decision errors to ensure that they accurately reflect the decision maker's concerns about the relative consequences for each type of decision error. 25 ------- Worksheets for STEP 6 - SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS Specify Acceptable Limits on Decision Errors - Specify acceptable decision error limits based on the consideration of the consequences of making an incorrect decision. (1) Determine the Possible Range of the Parameter of Interest. (a) Determine the upper and lower bounds for the parameter of interest using relevant historical site data. (2) Define Both Types of Decision Errors and Identify the Potential Consequences of Each. (a) Using the actions, action level, and decision rule, define both types of decision errors. (b) Establish the true state of nature for each decision error. 26 ------- Worksheets for STEP 6 - SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS (c) Define the true state of nature for the more severe decision error as the baseline condition or the null hypothesis (HJ and define the true state of nature for the less severe decision error as the alternative hypothesis (HJ. (d) Assign the terms "false positive" and "false negative" to the proper decision errors. (3) Specify a range of possible parameter values where the consequences of decision errors are relatively minor (the gray region). (a) Identify the range of points on the false negative side of the action level where the consequences of making a decision error are relatively minor. This range establishes the gray region. 27 ------- Worksheets for STEP 6 - SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS (4) Assign Probability Values to Points Above and Below the Action Level That Reflect the Acceptable Probability for the Occurrences of Decision Errors. (5) Check for Consistency. (a) Check the limits on decision errors to ensure that they accurately reflect the decision maker's concerns about the relative consequences for each type of decision error. 28 ------- Worksheets for STEP 6 • SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS NOTES: 29 ------- Worksheets for STEP 6 - SPECIFY ACCEPTABLE LIMITS ON DECISION ERRORS NOTES: 30 ------- STEP 7. OPTIMIZE THE DESIGN Background The purpose of this step is to develop a sampling strategy that satisfies all of the previously defined data quality objectives, and falls within the project's budget constraints. Specific operational details of the sampling plan are outlined by the scoping team, as well as the limitations and assumptions of the sampling design. Activities 1. Review the DQO outputs and existing environmental data. • Compile outputs from previous steps of the DQO Process. 2. Develop General Sampling and Analysis Design Alternatives. • Develop a preliminary estimate of variability. • Develop a sampling strategy for the site that specifies all of the previously defined data quality objectives. 3. For Each Design Alternative, Verify That the DQOs are Satisfied. 4. Select the Most Resource-Effective Design That Satisfies All of the DQOs. 5. Document the Operational Details and Theoretical Assumptions of the Selected Design in the Sampling and Analysis Plan. • Sample types (e.g., composite v. grab samples). • General collection techniques (e.g., split spoon v. core drill, or activated charcoal media v. evacuated canister). • Sample support (i.e., the amount of materials to be collected for each sample). • Sample locations (surface coordinates and depth) and how the locations were selected. • Timing issues for sample collection, handling, and analysis. • Analytical methods (or performance standards). • Quality assurance and quality control needs. • The assumptions of the statistical model. 31 ------- Worksheets for STEP 7 - OPTIMIZE THE DESIGN Optimize the Design - Outline a sampling design, specifying the operational details of the sampling plan which falls within the projects constraints. (1) Review the DQO Outputs and Existing Environmental Data. (a) Compile outputs from previous steps of the DQO Process. (2) Develop General Sampling and Analysis Design Alternatives. (a) Develop a preliminary estimate of variability. (b) Develop a sampling strategy for the site that specifies all of the previously defined data quality objectives. 32 ------- Worksheets for STEP 7 - OPTIMIZE THE DESIGN (3) For Each Design Alternative, Verify That the DQOs are Satisfied. (4) Select the Most Resource-Effective Design That Satisfies All of the DQOs. (5) Document the operational details and theoretical assumptions of the selected design in the Sampling and Analysis Plan. NOTES: 33 ------- Worksheets for STEP 7 • OPTIMIZE THE DESIGN 34 ------- ADDITIONAL WORKSHEETS 35 ------- ADDITIONAL WORKSHEETS 36 ------- ADDITIONAL WORKSHEETS 37 ------- |