United States
Environmental Protection
Agency
Office of Environmental
Information
Washington, DC 20460
EPA/240/B-01/007
September 2001
Data Quality Objectives
Decision Error Feasibility Trials
Software (DEFT) - USER'S GUIDE
EPA QA/G-4D

-------
                                       DISCLAIMER

       The Data Quality Objectives Decision Error Feasibility Trials Software and documentation are
provided "as is," without guarantee or warranty of any kind, expressed, or implied. The Quality Staff,
U.S. Environmental Protection Agency, or the United States Government will not be liable for any
damages, losses, or claims consequent to use of the software or documentation.

       Reference herein to any specific commercial product, process, or service by trade name,
trademark, manufacturer, or otherwise does not constitute or imply its endorsement, recommendation,
or favoring by the U.S. Environmental Protection Agency or the United States Government.

-------
                                      FOREWORD

       The U.S. Environmental Protection Agency (EPA) has developed the Data Quality Objectives
Decision Error Feasibility Trials (DEFT) software (Windows Version 1.0) to support the application of
the Data Quality Objectives (DQO) Process, a systematic planning process developed by EPA. The
DQO Process is the Agency's preferred planning process when making decisions that involve selecting
between opposing conditions. The DQO Process is an important tool for project managers and
planners to define the type, quality, and quantity of data needed to make defensible decisions.

        This document provides guidance to EPA program managers and planning teams.  It does not
impose legally binding requirements and may not apply to a particular situation based on the
circumstances. EPA retains the discretion to adopt approaches on a case-by-case basis that differ
from this guidance where appropriate.  EPA may periodically revise this guidance without public notice.

       This document is one of the U.S. Environmental Protection Agency Quality System Series
documents. These documents describe the EPA policies and procedures for planning, implementing,
and assessing the effectiveness of a quality system. Questions regarding this document or other Quality
System Series documents should be directed to the Quality Staff at:

                            U.S. EPA
                            Quality Staff (2811R)
                            1200 Pennsylvania Avenue, NW
                            Washington, DC 20460
                            Phone: (202)564-6830
                            Fax: (202)565-2441
                            e-mail: quality@epa.gov

Copies of the EPA Quality System Series documents may be obtained from the Quality Staff or by
downloading them from the Quality Staff Home Page:

                            http://www.epa.gov/quality
DEFT                                                                      Windows Version 1.0
EPAQA/G-4D                                  i                                  September 2001

-------
DEFT                                                                        Windows Version 1.0
EPAQA/G-4D                                  ii                                  September 2001

-------
                           TABLE OF CONTENTS

                                                                     Page
FOREWORD                                                              i

CHAPTER 1. GETTING STARTED	1
      1.1   INTRODUCTION	1
      1.2   CONSIDERATIONS FOR DECIDING WHEN TO USE DEFT	3
      1.3   INSTALLATION AND USE	6
      1.4   RELATED SOFTWARE PRODUCTS	7

CHAPTER 2. USING THE SOFTWARE 	9
      2.1   ENTRY SCREENS 	9
      2.2   THE INPUT VERIFICATION SCREEN	16
      2.3   THEDESIGN/DQO SUMMARY SCREEN  	17
           2.3.1  Modifying the DQOs	18
           2.3.2  Selecting aNew Sampling Design	19
           2.3.3  Modifying Design-Specific Information	21
           2.3.4  Specifying a Sample Size or Budget	21
           2.3.5  Displaying the Decision Performance Goal Diagram 	22
           2.3.6  Saving the Current Information	22
           2.3.7  Restoring the Original DQOs	22
      2.4   THE DECISION PERFORMANCE GOAL DIAGRAM SCREEN 	23
           2.4.1  The Decision Performance Goal Diagram	24
           2.4.2  Copying and Saving the Diagram	24

CHAPTERS. EXAMPLES OF DEFT APPLICATIONS	25
      3.1   TESTINGAMEAN AGAINST A FIXED STANDARD  	25
      3.2   TESTING A PERCENTILE AGAINST A FIXED STANDARD 	29
      3.3   TESTING THE DIFFERENCE BETWEEN TWO MEANS  	31
      3.4   TESTING THE DIFFERENCE BETWEEN TWO PROPORTIONS	34

CHAPTER 4. EXTENDED APPLICATIONS OF DEFT	37
      4.1   USING DEFT TO DETERMINE SAMPLE SIZES FOR ESTIMATION	37
      4.2   USING DEFT TO RECONCILE SAMPLE DATA WITH PROJECT DQOS ... 38
           4.2.1  Estimation Problems	38
           4.2.2  Hypothesis Testing	38
      4.3   USING DEFT FOR GRID SAMPLING DESIGNS	39
      4.4   TESTING A PERCENTILE AGAINST A FIXED STANDARD 	40
DEFT                                                        Windows Version 1.0
EPAQA/G-4D                           iii                          September 2001

-------
                                                                             Page
CHAPTER 5. ALGORITHMS USED IN DEFT	43
      5.1    TESTING A MEAN AGAINST A FIXED STANDARD	43
             5.1.1   Simple Random Sampling	43
             5.1.2   Composite Sampling 	44
             5.1.3   Stratified Sampling	45
      5.2    TESTING A PERCENTILE AGAINST A FIXED STANDARD  	47
             5.2.1   Simple Random Sampling	47
             5.2.2   Stratified Sampling	48
      5.3    TESTING THE DIFFERENCE BETWEEN TWO MEANS	50
      5.4    TESTING THE DIFFERENCE BETWEEN TWO PROPORTIONS	51
      5.5    ESTIMATING A POPULATION MEAN 	52
      5.6    ESTIMATING A POPULATION PROPORTION	52

REFERENCES	53

                                LIST OF FIGURES

                                                                             Page
Figure 1. DEFT and the DQO Process 	2
Figure 2. Input Verification Screen	17
Figure 3. Example Design/DQO Summary Screen	18
Figure 4. Example Design Performance Goal Diagram Screen	23
Figure 5. Input Verification Screen for Example 1 	27
Figure 6. Design/DQO Summary Screen for Example 1	27
Figure 7. Decision Performance Goal Diagram for Example 1	28
Figure 8. Input Verification Screen for Example 2 	31
Figure 9. Design/DQO Summary Screen for Example 3	34
Figure 10. Decision Performance Goal Diagram for Example 4	36

                                LIST OF TABLES
                                                                             Page
Table 1.  DQOs to Enter Into DEFT 	10
Table 2.  Sampling Designs Available in DEFT	19
Table 3.  Summary of Design Information	21
Table 4.  Using DEFT for Estimation 	37
Table 5.  Translating DQOs for Percentiles into DQOs for Proportions	41
DEFT                                                              Windows Version 1.0
EPAQA/G-4D                              iv                             September 2001

-------
                                        CHAPTER 1
                                   GETTING STARTED

1.1    INTRODUCTION

What is the DEFT software and this User's Guide?

       The Decision Error Feasibility Trials (DEFT) software (Windows Version 1.0) was developed
to assist in determining the feasibility of data quality objectives (DQOs) developed using the Data
Quality Objectives Process.  DEFT allows decision makers and members of a planning team1 to
quickly generate cost information about several simple sampling designs based on the DQOs. If
necessary, the planning team can change the DQOs and evaluate the effect of these changes.

       This user's guide contains detailed instructions on how to use DEFT. It is designed to
supplement the EPA Guidance on the Data Quality Objectives Process (QA/G-4) (EPA, 2000c)
which describes the DQO Process in detail.  Therefore, this user's guide does not provide instructions
on implementing the DQO Process, but instead contains information on how to use the DQOs
generated through the DQO Process  in DEFT.

How does DEFT determine feasibility?

       DEFT uses the DQOs developed by a planning team to provide an estimate of sample number
and cost.  It determines feasibility based on economic considerations, not policy or other qualitative
criteria.

How does this version of the software differ from the last [Version 4.0 (DOS)]?

       This is the Windows Version 1.0 of DEFT. Major changes from the previous release of DEFT,
which was DOS version 4.0, include:

       •       added capabilities for addressing hypotheses concerning population proportions and
              population percentiles;
              new routines for determining false acceptance error rates when a sample size is
              specified;
       •       the ability to save, print, and copy the decision performance goal diagram; and
  1 The DQO Process emphasizes using a multi-disciplinary team approach to offer different kinds of perspectives
for reaching consensus about critical elements of the planning process, such as decision statements and decision
error limits that are acceptable. An example DQO planning team might include a chemist, engineer, geologist, and
toxicologist to support the project manager and QA officer.

DEFT                                                                        Windows Version 1.0
EPAQA/G-4D                                   1                                  September 2001

-------
              a Windows platform design (i.e., the software is now designed to run in a Windows
              environment instead of DOS).

Other minor changes have also been implemented, such as the ability to consider a fixed sampling cost
or the use of a coefficient of variation instead of a standard deviation.

What is the DQO Process and how does DEFT assist in its implementation?

       The DQO Process (Figure 1) is a 7-step systematic planning process developed by EPA (EPA
2000c). It provides a systematic procedure for defining the criteria that a data collection design should
satisfy, including when to collect samples, where to collect samples, the tolerable level of decision
errors for the study, and how many samples to collect.
The DQO Process usually is conducted using a multi-
disciplinary team approach.

       Two difficult steps in the DQO Process are Step
6: Specify Tolerable Limits on Decision Errors, and Step
7: Optimize the Design. During Step 7, the DQOs  are
incorporated into a sampling design.  If the DQOs are not
feasible, it is necessary to iterate through one or more of
the earlier steps of the DQO Process to revise or relax the
criteria until the planning team is able to identify a sampling
design that will meet the budget and generate data that are
adequate for the decision. This iteration can be time-
consuming and costly. DEFT reduces the need for this
iteration by determining the feasibility of the DQOs before
the final step of the DQO Process is implemented.

What are DQOs and how do they related to DEFT?

       DQOs are qualitative and quantitative statements
derived from the outputs of the first six steps of the DQO
Process that:

       •      Clarify the study objective;
       •      Define the most appropriate type of data
              to collect;
              Determine the most appropriate conditions
              from which to collect the data; and

                                                      Figure 1. DEFT and the DQO Process






Step 1 :
State the Problem
*
Step 2:
Identify the Decision
*
Step 3:
Identify Inputs to the Decision
*
Step 4:
Define the Study Boundaries
*
Step 5:
Develop a Decision Rule
*
Step 6:
Specify Limits on Decision Errors


A

_P"™^| DEFT
T
Step 7:
Optimize the Design for Obtaining Data

DEFT
EPA QA/G-4D
Windows Version 1.0
     September 2001

-------
       •      Specify tolerable limits on decision errors which will be used as the basis for
              establishing the quantity and quality of data needed to support the decision.

The DQOs are then used to develop a scientific and resource-effective data collection design. DEFT
helps determine the feasibility of the DQOs before a data collection design is developed.

Where can I find information on the DQO Process?

       The DQO Process is described in the following two documents:

       •      Guidance for the Data Quality Objectives Process (QA/G-4)  (EPA, 2000c)

       •      The Data Quality Objectives Process for Hazardous Waste Site Investigations
              (QA/G-4HW) (EPA, 2000a)

The first document provides general guidance; the second provides guidance for Superfund and
Resource Conservation and Recovery Act applications.

       This User's Guide does not describe the DQO Process or its outputs (DQOs) in detail because
this information is contained in the documents listed above. It is strongly recommended that those who
are unfamiliar with the DQO Process use the above documents and the help screens in DEFT to obtain
more information on the DQO Process.

1.2    CONSIDERATIONS FOR DECIDING WHEN TO USE DEFT

When should I use DEFT during the DQO Process?

       DEFT was developed primarily as  a tool for the program manager and planning team to use
before consulting with a statistician to develop a sampling design.  The software is mostly used between
Step  6:  Specify Limits and Decision Errors and Step 7:  Optimize the Design of the DQO Process.
DEFT generates cost information about several simple sampling designs based on the outputs from the
first six steps of the DQO Process.  The planning team can use this information to evaluate whether the
DQOs generate cost-feasible sample sizes before the sampling and analysis design team begins
developing a final sampling design in the last step of the DQO process.

What are some additional applications of DEFT?

       In addition to the standard application of the DQO process described above, DEFT can
address alternative situations including:
DEFT                                                                      Windows Version 1.0
EPAQA/G-4D                                  3                                September 2001

-------
              Estimation of population parameters;
              Reconciling proj ect results with the DQOs;
       •      Testing hypothesis for percentiles; and
       •      Estimating sample sizes for grid sampling.

These topics are discussed in Chapter 4.

What planning should I do before using DEFT?

       Before using DEFT, the planning team should complete Steps 1 through 6 of the DQO Process
to define the DQOs required to achieve data of appropriate quality for its intended use.  For example,
the planning team should carefully define the decision rule to be tested (Step 5 of the DQO Process) in
order to properly frame the use of the outputs from DEFT.  The team should also carefully consider the
consequences of decision errors and use this analysis to set the limits on decision errors (Step 6 of the
DQO Process). Note that there are no rules for setting the limits on  decision errors, and there is no
easy way to select limits. EPA recommends setting the limits based on an analysis of the consequences.
A lack of serious consideration about the consequences of making a  false rejection or a false
acceptance decision undermines the effectiveness of using DEFT to calculate sample size.

For what problems can DEFT generate sample sizes?

       This version of DEFT will generate  sample sizes using different sampling designs for the
following questions:

       •      Is the population mean greater/less than a fixed standard? For example, does the
              mean concentration of hazardous waste in a drum  exceed the regulatory threshold?

       •      Is the population proportion/percentile greater/less than a fixed standard? For
              example, in a storage yard where waste drums of many types have been placed, does
              the proportion of drums containing hazardous waste exceed 50%?

       •      Is the difference between  two population means significant? For example, does
              the mean concentration of radioactive soil contaminants at the former fuel processing
              facility exceed the mean concentration of radioactive  soil contaminants in the downtown
              city park?

       •      Is the difference between two population proportions/per centiles significant? For
              example, does  the 98th percentile of daily PM10 particulate concentration measurements
              taken during 1998 in St. Louis differ significantly from the same measurements taken
              during 1999?
DEFT                                                                       Windows Version 1.0
EPAQA/G-4D                                  4                                 September 2001

-------
Chapter 2 describes the sampling designs available in DEFT that can be used in designing a study to
answer these questions and Chapter 3 provides examples of these applications using DEFT.

When shouldn 't I use DEFT?

       DEFT is not an expert system that considers the appropriateness of the DQOs or ensures an
optimal  (or even feasible) sampling design. Therefore, the software should not be used to validate the
DQOs or to select a final sample size.  DEFT should be used only to evaluate the feasibility of the
DQOs generated through Step 6 of the DQO Process. In Step 7 of the DQO process, more
sophisticated tools may be used to aid in design optimization, which may yield a lower-cost design.

       There is no easy method for developing an optimal sampling design.  Factors such  as
environmental medium, parameter of interest, contaminant of interest, and sampling boundaries as well
as components of cost and variance all affect the choice of a sampling design.  The application of DEFT
for calculating a number of samples is straightforward for random sampling across space when the
population remains relatively static over time. For example, DEFT is particularly applicable for
calculating sample sizes when investigating slow-moving contaminants in surface soil because the
samples can be collected randomly across space, and the concentrations do not change much over
time.  On the other hand, when investigating contaminants in ground water, sampling locations may need
to be restricted to locations where wells currently exist, and the concentrations at any given location
may vary greatly over relatively short periods of time, making the problem much more dynamic.  DEFT
is not designed to handle problems that involve streams of data over time, which require careful
consideration of how  correlations affect the analysis.

       Volatile contaminants may present complex challenges because they may move quickly through
an environmental medium, thereby creating a dynamic sampling problem in the field, while also posing
difficulties in implementing analytical methods, thereby creating measurement problems in both the field
and the laboratory. DEFT does not address these types of problems involving dynamic fate and
transport for processes such as volatilization, retardation, or decay.

       DEFT has capabilities that can be  misused as well.  A composite sampling design is applicable
for testing hypotheses concerning the mean; however, it is not applicable for testing hypotheses
concerning percentiles. An optimal sampling design accounts for all factors relevant to the  problem at
hand, and is practical, feasible, and satisfies the DQOs. DEFT cannot take all of these factors into
account, hence it should not be used to determine the  sampling design or final sample size.

What statistical assumptions does DEFT make?

       For the one-population cases, it is  assumed that the action level is fixed (i.e., the action level is a
known quantity) and that there is only one infinite (or extremely large) population. For the  two-
DEFT                                                                       Windows Version 1.0
EPAQA/G-4D                                   5                                 September 2001

-------
population cases, it is assumed that both sample sizes are large and that the variability of the two
populations are approximately equal. For example, the one-population case tests whether the mean
concentration of a contaminant at a site exceeds a health-based standard.  A two-population case tests
whether or not the mean concentration of a contaminant at an industrial site exceeds the concentration
at a nearby residential site (each site is considered to have a separate population of interest, and
samples from both sites are required for the calculation). DEFT also assumes that a design comparable
to either a simple random sampling design or a stratified simple random sampling design is feasible.  For
example, DEFT is not designed to be used to determine the number of drinking water wells to be
selected when the sample wells will be selected on the basis of hydrogeology instead of selected
randomly.

What quality control procedures has DEFT been subjected to?

       This software was peer reviewed and incorporates revisions recommended by the reviewers.
It has been tested extensively including an analysis of the inputs, processes and expected outputs for
each routine.  This testing is documented in the Test Plan for the Data Quality Objectives Decision
Error Feasibility Trials (DQO/DEFT) Software (Flanagan and Aanstoos, 2001).

1.3    INSTALLATION AND USE

What computers will run DEFT?

       Any computer running Windows 95 or Windows NT or their successors should be able to run
DEFT. Its memory and disk requirements are negligible compared with other Windows applications.
The minimum graphic resolution required  is 640x480, with 800x600 recommended.

How do I install DEFT?

       To install DEFT, save the file g4d-final.exe to your computer. Then

       1.     Select "Run" from the Taskbar Start menu
       2.     Enter "x:\g4d-final.exe" substituting the location where you saved the g4d-final.exe file
              for x. If you received the DEFT software on a floppy disk, enter the drive letter for x.

The DEFT software is then installed in the in the default folder c:\deft.

How do I start DEFT?

       After DEFT has been installed:
DEFT                                                                      Windows Version 1.0
EPAQA/G-4D                                  6                                 September 2001

-------
       1.      Click on the Start button found on the Windows task bar
       2.      Select "Run"
       3.      Enter "c:\deft\deft.exe" by the 'Open' prompt and then press "OK".

Alternatively, you can run DEFT directly from a floppy disk by entering "x:\deft.exe" (where x is the
letter of your disk drive) in Step 3. Also, you can manually create an icon from which to launch DEFT
and place it on your desktop and/or Start menu. See your Windows documentation for directions.

       When DEFT is launched it displays an opening screen with general information about the
program's purpose and proper use.  After you click the OK button,  DEFT then prompts you for your
initial DQO inputs which are described in Chapter 2.

How do I skip the entry screens?

       DEFT prompts the user to enter the information from the DQO Steps 1 through 6 based on a
series of five entry screens.  The first entry screen determines the parameter of interest (see Section
2.1). To skip the remaining entry screens, click on the Summary button in the bottom right corner of
the second entry screen. This will take you directly to the Design/DQO Summary Screen (Section 2.3)
using the default values contained in DEFT.

How do I start a new analysis?

       To exit DEFT through the Design/DQO Summary Screen, press the Exit button on the bottom
right-hand corner of the Design/DQO Summary Screen.  DEFT will then ask if you want to "Start a
new DQO analysisT Pressing the No button will exit the program.

Where can I get help?

       An electronic copy of this User's Guide is accessible by clicking on the Help button contained
on each DEFT window or dialog box.

How do I exit DEFT?

       To exit the software at anytime, click on the close button (the X in the upper right hand button)
of an DEFT  window or dialog box.

1.4    RELATED SOFTWARE PRODUCTS

       In addition to DEFT, there are several free computer-based programs available that are related
to the DQO Process. Each of the programs listed below operate on an IBM PC with a VGA monitor.
DEFT                                                                     Windows Version 1.0
EPAQA/G-4D                                  7                                 September 2001

-------
       •      Visual Sample Plan (VSP) (dqo.piJ.gov/VSP/Index.htm) — VSP is designed to
              select the number of samples and provide random or gridded sampling locations, based
              on various sampling schemes, overlaid on a site map.  VSP includes SampTOOL, an
              Internet tool to guide the user in selecting an appropriate sampling design given the type
              of problem and environmental medium (surface soil, subsurface soil, sediment, surface
              water, groundwater, air, biota, contaminated material,  and surface).

       •      DQO-PRO (www.acs-envchem.duq.edu/dqopro.htiTO — DQO-PRO helps a user
              understand the significance of the DQOs by showing the relationship between the
              numbers of samples and DQO parameters such as confidence levels for false
              acceptance and false rejection decision errors; tolerable error versus analyte
              concentration, standard deviation, etc.; and confidence levels versus sampling area grid
              size. DQO-PRO can be used to calculate the number of samples required to meet the
              DQOs or satisfy the desired confidence interval widths.

       •      GEOPACK (www.epa.gov/ada/csmos/models/geopack.htmn — GEOPACK is a
              comprehensive geostatistical software system for conducting analysis of the spatial
              variability of one or more random functions. GEOPACK is menu-driven,  user-friendly,
              requires a minimum number of input data, and includes on-line help.

In addition to the programs listed above, there are numerous statistical packages that are useful in
implementing the DQO process.
DEFT                                                                      Windows Version 1.0
EPAQA/G-4D                                  8                                 September 2001

-------
                                       CHAPTER 2
                                 USING THE SOFTWARE

       DEFT uses the DQOs defined in Steps 1 - 6 of the DQO Process to determine their feasibility
based on several simple sampling designs. This is done in three steps:

       1)     the information from DQO Steps 1-6 are entered into DEFT (Entry Screens),
       2)     this information is then verified and saved (the Input Verification Screen), and
       3)     finally DEFT uses this information with different sampling designs to estimate sample
              size and costs (the Design/DQO Summary Screen).

These steps are described in detail in this Chapter and examples are provided in Chapter 3.

              Note:  The information below describes information required for
              the software and constraints related to the software.  It does not
              describe the DQO Process or its outputs in any detail. For this
              information, consult the EPA Guidance for the Data Quality
              Objectives Process (EPA, 2000c).

2.1    ENTRY SCREENS

       DEFT prompts the user to enter the information from the DQO  Steps 1 through 6 based on a
series of five entry screens. This information is described below and summarized in Table  1. For each
item, the relevant step of the DQO Process is provided.

              Note:  To skip all entry screens after the first, click on the
              Summary button in the bottom right corner of the second entry
              screen.  This will take you directly to the Design/DQO Summary
              Screen (see Section 2.3) using the default values contained in
              DEFT and described below.

       The software automatically starts with a simple random sampling design so that the user only
enters the minimum amount necessary to generate a sample size.  On each entry screen there is a
NEXT button which must be clicked in order to accept the values shown on that screen and advance to
the next screen.  (Note:  The ENTER key will NOT advance to the next screen; you must click
NEXT.) On all but the first entry screen there also  appears a BACK button which will allow you to
back up to the previous screen. When each screen initially appears, the fields on it  are filled in with
default values which you can either accept or change. To move between fields for the purpose of
entering or changing values, you may either click in  the desired field with the mouse or use the TAB key
to move to the next field.
DEFT                                                                     Windows Version 1.0
EPAQA/G-4D                                  9                                 September 2001

-------
                                                      Table 1.  DQOs to Enter Into DEFT
                 DQOs
           Valid
          Entries
DQO
 Step
Entry Screen
 Parameter of Interest
Parameter
Mean or Proportion

Number of Populations
One or Two
                                                                                               DEFT - Parameter Selection
                                                                                                    Select the Parameter of Interest
                                                                                                             (*~ iPppulatipn Meari

                                                                                                             (~ Population Proportion

                                                                                                    Select Number of Populations

                                                                                                             <•" One Population

                                                                                                             <~ Two Populations

                                                                                                  The parameter of interest should have been identified in
                                                                                                  Step 5, Develop a Decision Rule, of the DQO Process.

                                                                                                                         NEXT »
 Minimum Value of the Parameter
 of Interest (MIN)
For Means
MIN < MAX

For Proportions
0 if 1  population
-1 if 2 populations
 Maximum Value of the Parameter
 of Interest (MAX)
For Means
MAX > MIN

For Proportions
1 for both 1 and 2
populations
            Help

             Estimate of Minimum Value:

             Estimate of Maximum Value:

             Action Level (AL):

             Select Baseline Condition
             f~" Ho: mean >= AL vs. Ha: mean < AL
             f»" Ho: mean<-AL vs. Ha: mean> AL
      Other Bound of the Gray Region
       The Gray Region is bounded on one side by the
       action level (the lower bound). Because of the
       selected baseline condition, the other bound (the
       upper bound] must be greater than (he action level.
       Enter the upper bound.
       Upper Bound:    |75
      Estimate of Standard Deviation
      <• Use This Value:     |16.67
      <~ Coefficient of Variation: [20    % of AL (. 10 )
                                                                                          NOTE: These values should have been identified in Steps 5 and 6 of the DQO Process.

                                                                                                              « BACK    NEXT »  I            SUMMARY
EPA QA/G-4D
                              10
                                                       Windows Version 1.0
                                                             September 2001

-------
                                                      Table 1. DQOs to Enter Into DEFT
                 DQOs
           Valid
          Entries
DQO
 Step
                    Entry Screen
 Action Level (AL)
   MIN < AL < MAX

  For two populations,
         AL = 0.
 Baseline and Alternative
 Conditions
1. H0: parameters AL
vs. Ha: parameterAL
                                                                                          DEFT - One-Sample Mean Inputs
Help

 Estimate of Minimum Value:

 Estimate of Maximum Value:

 Action Level (AL):

 Select Baseline Condition
 f" Ho: mean >= AL vs. Ha: mean < AL
 f* Ho: mean <= AL vs. Ha: mean > AL
                                                                                                                     Other Bound of She Gray Region
                                                                                                                      The Gray Region is bounded on one side by the
                                                                                                                      action level (the lower bound). Because of the
                                                                                                                      selected baseline condition, the other bound (the
                                                                                                                      upper bound) must be greater than the action level.
                                                                                                                      Enter the upper bound.
                                          Upper Bound:
                                         Estimate of Standard Deviation
                                         ^ Use This Value:    [16.67
                                         <"" Coefficient of Variation: [29
 Bounds of the Gray Region (GR)
   MIN < GR < AL or
    AL < GR < MAX
 Estimate of Standard Deviation
 (SD)
 0 0
                                                                                                       Help

                                                                                                       These may have been determined in Step 3, Identify
                                                                                                       Inputs to the Decision, of the DQO Process.
                                                                                                       Laboratory Cost Input
                                                                                                         Laboratory Costs Pel Sample:  llOOO.OO

                                                                                                       Field Cos! Input
                                                                                                         Field Costs [Pel Sample):
                                                                                                            <° Per Sample
                                                                                                            (~ Total
EPA QA/G-4D
                               11
                                                        Windows Version 1.0
                                                             September 2001

-------
                                                   Table 1. DQOs to Enter Into DEFT
                DQOs
    Valid
   Entries
DQO
 Step
Entry Screen
 False Rejection (FR) and False
 Acceptance (FA) error limits at the
 bounds of the gray region
0
-------
                                                     Table 1. DQOs to Enter Into DEFT
                 DQOs
           Valid
          Entries
DQO
 Step
Entry Screen
 Additional Error Limits Above and
 Below the Gray Region

 (x = Concentration/Proportion,
 p = Probability associated with x)
 Below the Gray Region
     MIN < x < GR
               or
     MIN < x < AL

 Above the Gray Region
     GR < x < MAX
               or
     AL < x < MAX

(Limit of two additional
entries above and
below.)

        0


-------
              Parameter of Interest: The parameter of interest is a descriptive measure of some
              characteristic or attribute of the statistical population. Defining the parameter of interest
              consists of two parts - selecting the parameter (valid entries are mean and proportion2)
              and identifying the number of populations (either a single population for comparing the
              parameter to a fixed standard, or two populations for determining the difference
              between the parameters from each population).  Note:   Once the parameter of interest
              has been selected, it may not be changed. (DQO Process Step 5)

              Minimum and Maximum Values (Range) of the Parameter of Interest: If the
              parameter of interest is a population mean or the difference between two population
              means, estimates of the minimum and maximum possible values are necessary for
              scaling, graphing, and computing default values.  The range of the population mean must
              fall within the range of possible concentrations. If the parameter of interest is a
              proportion, the minimum value is automatically set to 0 and the maximum value is
              automatically  set to 1.  If the parameter of interest is the difference between two
              proportions, the minimum value is automatically set to -1 and the maximum value is
              automatically  set to 1.  These values are referred to throughout the rest of DEFT as  the
              "minimum" and "maximum" concentrations. (DQO Process Step 6)

              Action Level: The action level is a value that provides the criterion for selecting
              among alternative actions. For the one sample case, this software assumes that the
              action level  is fixed, such as a regulatory standard. For the two sample case, the default
              action level  is zero to indicate "no difference between the two population parameters."
              (DQO Process Step 5)

              Baseline and Alternative Hypotheses:  The baseline (FLJ and alternative (FQ
              conditions are used to identify which error is a false rejection error and which is a false
              acceptance error. There are two choices for the  baseline and  alternative conditions:

                      1) FL; Parameter > Action Level vs. H,: Parameter < Action Level and
                      2) FL; Parameter < Action Level vs. FL,: Parameter > Action Level.

              Once selected, these may not be changed.  Because the alternative condition is the
              opposite of the baseline condition, DEFT will only state the baseline condition after this
              selection is  made.  (DQO Process Step 6)
  2Note that determining sample sizes for testing hypotheses concerning percentiles is equivalent to determining
sample sizes for hypotheses concerning proportions. Therefore, only proportions are displayed in the software.
Chapter 3 describes the process of translating hypotheses concerning percentiles into hypotheses concerning
proportions.

                                                                             Windows Version 1.0
EPA QA/G-4D                                  14                                  September 2001

-------
               Gray Region:  The gray region is a range of possible parameter values where the
               consequences of a false acceptance decision error are relatively minor. The gray region
               is bounded on one side by the action level and on the other side by that parameter value
               where the consequences of making a false acceptance decision error begin to become
               significant. The program will automatically determine whether this bound should be less
               than or greater than the action level, based on your choice of baseline condition. (DQO
               Process Step 6)

               Estimate of Standard Deviation: When the parameter of interest is a population
               mean or the difference between two population means,  an estimate of the standard
               deviation of the population of interest is necessary for computing sample sizes.  (If the
               parameter of interest is a proportion or the difference between two proportions, an
               estimate of the standard deviation is not required.) The standard deviation is the square
               root of the variance.  An estimate of this value may be available from a pilot study or
               the user can use the DEFT default value.3 If the difference between two means is the
               parameter of interest, DEFT assumes that the standard deviations of both populations
               are equal. The standard  deviation must be greater than zero and less than or equal to
               two times the range of the population parameter (i.e., the standard deviation must be
               less than or equal to two times the maximum concentration minus the minimum
               concentration). Alternatively, the standard deviation may be specified as a fixed
               percentage of the action level. This percentage is sometimes referred to as the
               coefficient of variation, and this option may be chosen from the standard deviation entry
               screen.  (DQO Process  Step 3)

               Sampling And Analysis Costs:  The average unit cost of analyzing a sample and the
               average unit cost of collecting a sample in the field are used to compute the total cost of
               a sampling design. The average cost of analyzing a sample is referred to as the
               "laboratory cost" and the average unit cost of collecting a sample is referred to as the
               "field cost" in DEFT. Both the laboratory and field costs must be greater than or equal
               to zero.  For the field sampling cost, an alternative to specifying  a per-sample cost is to
               specify the total cost for  all samples regardless of their number.  For the case where
               sample collection and measurement analysis are one process, you should enter the cost
  3If there is no estimate available, then the (Maximum Concentration - Minimum Concentration) / 6 may be used as
a rough approximation of the standard deviation.  The default value assumes the population is normally distributed,
i.e., that 99% of the values are represented by the mean ± 3o, and, therefore, the max-min is equivalent to 60. Note
that this approximation is based on the range of the population, not the range of the population parameter and it
should only be used if there is absolutely no other information available. The approximation is only valid for the
purposes of DEFT, i.e., determining the feasibility of the DQOs. You should consult a statistician before developing
an estimate for use in the actual sampling design.

                                                                              Windows Version 1.0
EPA QA/G-4D                                   15                                   September 2001

-------
              of this process as the either the laboratory cost or the field cost and set the other cost
              equal to zero.  (DQO Process Step 3)

       •      Probability Limits on Decision Errors for the Bounds of the Gray Region:
              Limits on the probability of false rejection error and a false acceptance error must be
              specified in order to compute sample sizes. DEFT will prompt you to enter these
              probabilities after it automatically determines which error is a false acceptance error
              and which is a false rejection error. Both probabilities must be greater than 0 and less
              than or equal to 0.5. (DQO Process Step 6)

       •      Additional Limits on Decision Errors:  The DQO Process allows the planning
              team to set additional limits on decision errors besides those on the bounds of the gray
              region, although this is not required. In general, tolerable limits for making a decision
              error should decrease as the consequences of a decision error become more severe
              farther away from the Action Level. For example, the economics of making a false
              acceptance decision error may become more important as the true concentration is
              farther from the Action Level and the limits on decision error may be reduced at this
              point. DEFT will allow you to enter up to two additional limits below the lower bound
              of the gray region and up to two additional limits above the upper bound of the gray
              region. All probabilities must be greater than 0 and less  than or equal to 0.5. (DQO
              Process Step 6)

2.2    THE INPUT VERIFICATION SCREEN

       Once the DQOs are entered, DEFT displays the Input Verification Screen (Figure 2).  This
screen is used to verify the inputs from the entry screens. Any incorrect values can be corrected at this
time by pressing the Change button underneath that value.  For example, press the Change Input
Values button to change the minimum possible value for the parameter of interest.  Once the
information has been verified and corrected if necessary, press NEXT to advance to the Design/DQO
Summary Screen.

              Note: This is the last chance to adjust the minimum, maximum, and
              baseline condition as these can not be changed on the Design/DQO
              Summary Screen.

       The information on the Input Verification Screen is saved as the "Original DQOs," as this
information represents the DQOs of the planning team. This gives you the opportunity to select a
sampling design, evaluate the performance of the design based on these original DQOs, then modify the
DQOs to improve the performance of the sampling design.  You may then select a different sampling
                                                                           Windows Version 1.0
EPA QA/G-4D                                  16                                 September 2001

-------













JDEFT - Input Verification
Inputs
Parameter of Interest: Mean
M aximum M ean Concentration: 1 00
Action Level: 50
Baseline condition: mean <- 50.00
Gray Region: 50 - 60
Standard Deviation: 20
Change Input Value(s)
Cost of Sample
Analysis in the Laboratory: 1 000,00
Collection in the Field: 50.00 ge
NEXT



DE3
Decision Error Limits
concentration prab(error) type
FR
40 .01 FR
50* 0.05 FR
60" o.OI FA
80 0.05 FA
FA
* Use the Change Input Value(s)
button to change these values.
»














Figure 2. Input Verification Screen

design and restore the original DQOs in order to evaluate the new sampling design's performance
against the original DQOs (i.e., the DQOs of the planning team).

2.3    THE DESIGN/DQO SUMMARY SCREEN

       After you verify the DQOs on the Input Verification Screen, DEFT estimates sample size,
computes the total cost, and verifies that the decision error limits are satisfied using a Simple Random
Sampling Design.  This information is then displayed on the Design/DQO Summary Screen (Figure 3)
along with the DQOs and information on the current sampling design. You can investigate the feasibility
of the DQOs and save your analysis by:

              Modifying the DQOs (Section 2.3.1)
       •      Selecting a New Sampling Design (Section 2.3.2)
       •      Modifying Design-Specific Information (Section 2.3.3)
              Specifying a Sample Size or Budget (Section 2.3.4)
              Displaying the Decision Performance Goal Diagram (Section 2.3.5)
       •      Saving the Current Information (Section 2.3.6)
              Restoring the Original DQOs (Section 2.3.7)

       DEFT has a sample size limitation of 30,000 total samples. If the sample size required to meet
the DQOs exceeds this number, DEFT informs you of this in a pop-up error message. You will then
need change the DQOs (such as reduce the false rejection error rate and/or the false acceptance error
EPA QA/G-4D
17
Windows Version 1.0
     September 2001

-------
                    DEFT [Beta] - Deagn/DQO Summary
                     Sampling Parameters
                       S ampling D esign:  S impie R andom S ampling
                       Design-Specific Inputs:
                        No design-specific inputs for Simple Random Sampling.
                                  Number of Samples [per population):

                                  Total Cost:
          Change Sampling Design
                                                          36
                                                          37800.00
                   Update
                     DQO input Summary
                      Costs
                        Laboratory Costs Per Sample:  1000.00
                        Field Costs Per Sample:     50.00


                                Change Costs
  Decision Error Limits
   concentration  probferror]   type
   I         I        FR
                      Input Values
                      Parameter of Interest:  Mean
                      Action Level:       50
                      Gray Region:       50 - SO
                      Baseline condition:    mean <= 50.00
                      Standard Deviation:   20

                              Change Input Valuefs]
   |40
    so-
    so-
            am
ai
6~05
                    FA
  * Use Change Input Value(s) button
                   | Update

   Graph  Save  Original DQOs  Exit
                 Figure 3. Example Design/DQO Summary Screen

rate or increase the width of the gray region) before continuing with the DQO constraint
feasibility analysis.

        The sample size formulas used in DEFT guarantee that the decision error limits set on the
bounds of the gray region are satisfied. However, the sample size formulas do not account for
any additional decision error limits. Therefore, DEFT verifies that these additional limits are
satisfied. If a limit is not satisfied, the limit is marked "NS" in the Decision Error Limits Table.

2.3.1   Modifying the DQOs

        •       The minimum,  maximum, and baseline condition can not be changed at this
               point (see Section 2.2).

        •       The Action Level can be modified by selecting the Change Input Value(s) button.
               This will display a screen where this item may be changed.  Once the change is
               made, press the  NEXT button to return to the Design/DQO Summary  Screen.
               Sample sizes and costs are automatically updated.

        •       The Other Bound of the Gray Region can be modified by selecting  the Change
               Input Value(s) button.  This will display a screen where this item may be changed.
EPA QA/G-4D
18
                    Windows Version 1.0
                         September 2001

-------
              Once the change is made, press the NEXT button to return to the Design/DQO
              Summary Screen.  Sample sizes and costs are automatically updated.

       •      The Estimate of the Standard Deviation can be modified by selecting the Change
              Input Value(s) button.  This will display a screen where this item may be changed.
              Once the change is made, press the NEXT button to return to the Design/DQO
              Summary Screen.  Sample sizes and costs are automatically updated.

              Additional Decision Error Limits can be adjusted by changing the values in the
              space provided and new limits may be added by entering them in the space available (in
              this case, both a concentration and probability must be entered).  After changing or
              entering new limits, you must press the Update button to determine the new sample  size
              and if the additional Decision Error Limits are satisfied. If a limit is not satisfied, the limit
              is marked "A/5" in the Decision Error Limits Table.

       •      Laboratory cost and field cost estimates can be changed by selecting the Change
              Costs button to reflect the potential costs of a different sampling and/or analysis
              method.  This will display a screen where  these items may be changed. Once your
              changes are made, press the NEXT or BACK button to return to the Design/DQO
              Summary Screen.  Sample sizes and costs are automatically updated.

2.3.2  Selecting a New Sampling Design

       DEFT always starts with a simple random  sampling design but allows you to consider other
sampling designs which may perform more efficiently. To investigate other sampling designs, press the
Change Sampling Design button. You will then be prompted to select from the relevant sampling
designs shown in Table 2. For hypotheses about a single population mean, you may select either
composite sampling or stratified sampling. For hypotheses about a single population proportion, you
may select stratified sampling. These sampling designs, along with design-specific information, are
discussed in Chapter 4.
                       Table 2. Sampling Designs Available in DEFT

Mean
Proportion
One Population
- Simple Random Sampling
- Composite Sampling
- Stratified Sampling
- Simple Random Sampling
- Stratified Sampling
Two Populations
- Simple Random Sampling
- Simple Random Sampling
EPA QA/G-4D
19
Windows Version 1.0
     September 2001

-------
       The first time a sampling design is selected, you are prompted to enter the design-specific
information.  For example, an estimate of the proportion of measurement variability to the total
variability is required for the composite random sampling design. The design-specific information is
described below by sampling design and summarized in Table 3. The information first specified is
saved as part of the "Original DQOs" (see Section 2.2).  You may then modify the design-specific
information to evaluate the DQOs in relation to this sampling design.

       Composite Sampling: DEFT uses composite samples with a simple random sampling design,
       which is referred to as "composite sampling." The software computes the number of composite
       samples required to meet the DQOs based on a given number of individual samples per
       composite.  To determine the number of composite samples, DEFT requires the following
       design-specific information:

              An estimate of the ratio of the relative standard deviation of measurement error to total
              standard deviation.  This ratio must be less than one and greater than zero.

       •      The number of individual samples to be mixed to form each composite sample. This
              number should be greater than one.

       •      The cost of combining the individual samples to form a composite.

       Stratified Sampling: DEFT uses stratification with a simple random sampling design within
       each strata, which is referred to as "stratified sampling." The software computes the number of
       samples required per strata to meet the DQOs.  To estimate the sample size required for a
       stratified design, DEFT requires the following design-specific information:

       •      The number of strata. This number must be greater than one and less than six. (There
              is a limit of six strata in DEFT because the software only demonstrates feasibility of the
              DQOs and five strata should be sufficient for this purpose.)

       •      A weight factor (weight) for each stratum,  The stratum weight is the proportion of the
              volume or area of the environmental medium contained in the stratum in relation to the
              total volume or area of the study site. The sum of the strata weights must be 1, so the
              program automatically computes the weight of the final stratum.  The default weight
              corresponds to equal weighing among the  strata.

              If the population parameter is  a single mean, an estimate of the standard deviation is
              needed for each stratum. The  estimated  standard deviation for each stratum must be
              greater than zero  and less than two times the range of the population parameter,  and the
              default value is the estimated total standard deviation.
                                                                           Windows Version 1.0
EPAQA/G-4D                                  20                                 September 2001

-------
              If the population parameter is a single proportion, an estimate of the stratum proportion
              is needed for each stratum. Each estimate must be greater than zero and less than one.


                          Table 3. Summary of Design Information
Sampling
Design
Design Information
Limits
Default
Tests for a Single Mean
Composite
Sampling
Stratified
Sampling
Ratio (r) of measurement SD to
total SD
Number of individual samples (m)
per composite
Cost (c) of composting
Number of strata (L)
Stratum weights (Wh)
Stratum standard deviation (oh)
0$0.00
2
-------
samples and then DEFT adjusts the probability of a false acceptance decision error to meet your
sample size.4 You can do this by changing either the "Number of Samples" field or the "Total Cost"
field (in which case DEFT will compute the number of samples afforded by this cost using the cost input
data). To change either of these fields, first click in its box and edit the value currently appearing. Then
press the Update button to use the new value. Note that it is not valid to change both the Total Cost
and the Number of Samples before clicking the Update button — if you do, then DEFT will ignore
your cost entry and use your number of samples in computing the result.  You may enter any  sample
size greater than  1 and less than or equal to 30,000.

2.3.5  Displaying the Decision Performance Goal Diagram

       You can display the Decision Performance Goal Diagram by pressing the Graph button in the
bottom, right-hand corner of the Design/DQO Summary Screen. This diagram is discussed in detail in
Section 2.4.

2.3.6  Saving the Current Information

       Once it has been determined that the DQOs are feasible for a sampling design, you may save
the DQOs and design information to a plain text file by pressing the Save button in the lower right-hand
corner of the Design/DQO Summary Screen. This text file can then be imported into any standard
word processor.

       The first time the Save option is clicked the user is prompted for a file name into which the text
summary is saved.  If the file name chosen is the same as an existing file, you will be asked if you want
to overwrite the existing file. If you indicate you do not want to overwrite, you will be asked to  select a
new name. Once a new filename has been selected, all  subsequent uses of the SAVE option  (until the
program is exited) cause a new summary to be appended to the same file. A new file is not created for
each summary, and previous results of the current session are not overwritten.

2.3.7  Restoring the Original DQOs

       Selecting the Original DQOs button on the bottom right-hand corner of the Design/DQO
Summary Screen will restore the original DQOs (Section 2.2).  This is useful for comparing variations
of several sampling designs. For instance, if a sampling design is too expensive to satisfy the DQOs, so
you may relax some constraints to obtain a feasible sample size. After this is complete, you may want
to examine the performance of another sampling design using the Original DQOs.  This option saves
you from re-entering the original information manually.
  4When specifying a sample size, DEFT may adjust the sample size to be slightly larger than the value provided by
you, due to the way DEFT performs the calculations under these conditions.

                                                                            Windows Version 1.0
EPAQA/G-4D                                 22                                 September 2001

-------
2.4    THE DECISION PERFORMANCE GOAL DIAGRAM SCREEN

       To display the Decision Performance Goal Diagram and associated options, press the
Graph button in the bottom, right-hand corner of the Design/DQO Summary Screen. This will
bring up a Performance Goal Diagram Screen like the one shown in Figure 4.  This screen
contains the following:

       •      Decision Performance Goal Diagram - See Section 2.4.1 for a discussion of this
              diagram.
       •      DQO Summary button - This button will return you to the Design/DQO
              Summary Screen described in Section 2.3.
       •      Print Graph button - This button lets you print the diagram using the standard
              Windows print dialog.
              Copy Graph button - This button allows you to copy the graph to other Windows
              applications (Section 2.4.2).
       •      Save Graph button -  This button will let you save the  diagram to a picture file
              (Section 2.4.2)
       •      Help - This option displays help and version information.
       •      Exit - This option will exit DEFT.
                               Estimated Performance  Curve
                Simple Random Sampling
                Action Level = 50.00
                Cost = 537800.00
                Sample Size = 36
  30.0   40.0   50.0   60.0   ?0.0   80.0
True  Mean  Concentration

          Decision Error Limits
          concentration prob(E)  type
          40.00       0.010   FR
          50.00       0.050   FR
          60.00       0.100   FA
          80.00       0.050   FA
             Figure 4.  Example Design Performance Goal Diagram Screen
EPA QA/G-4D
           23
Windows Version 1.0
    September 2001

-------
2.4.1  The Decision Performance Goal Diagram

       DEFT has an option available to view the DQOs and design performance graphically on a
separate screen. This is done using a decision performance goal diagram with the performance curve
overlaid.  The performance goal diagram summarizes the gray region, the limits on decision errors, and
the action level. Information on the sample size and cost of the design are also summarized on this
screen. The performance curve can be used to determine how well a design performs in relation to the
limits on decision errors.

       The sample size reported by DEFT is always greater than or equal to 2 so that an estimate of
the standard deviation can be calculated from the data collected.  In this case, the performance curve
may satisfy a more stringent false acceptance decision error rate at the bound of the gray region than
that displayed by the software.  If so, use the option to specify the sample size (Section 2.3.4) to select
a sample size of 2 to determine the exact decision error rate satisfied by the two samples.

       Note that the performance curve displayed by DEFT is an estimate of the performance curve of
the design5.  Therefore, the curve may appear to show that a decision error limit is satisfied when it is
not.  The calculations performed in the software to determine if a particular error limit is satisfied are
more accurate than those used to draw the performance curve. Therefore, you should use the text
indication ("A/5"') in the Decision Error Limits Table to determine whether or not a limit is satisfied.

       The performance curve is always the probability of deciding that the true parameter value (such
as a mean or proportion) is greater than the action level, irrespective of the directions of the baseline
and alternative hypotheses. Thus the curve always starts at the lower left hand corner and rises to the
upper right hand corner.  This is in contrast to a statistical power curve.  For more information
regarding the performance curve, see the Guidance on the Data Quality Objectives Process (EPA
QA/G-4) (EPA, 2000c).

2.4.2  Copying and Saving the Diagram

       To  save the Performance Goal Diagram, click on Copy Graph on the menu bar of the
Performance Goal Diagram Screen. This will copy the current diagram to Windows Clipboard which
allows it to be pasted into any Windows application that supports the pasting of bitmap pictures. To
paste the diagram, open the Windows application and use the "Paste" command. This diagram can
then be saved in any format allowed by the Windows application you are using. The diagram can also
be printed or saved to a file as a Window bitmap (.bmp), using the appropriate options available on the
menu bar.
  5DEFT uses a normal distribution to approximate the power curve which is actually based on a non-central
t-distribution.

                                                                            Windows Version 1.0
EPAQA/G-4D                                  24                                 September 2001

-------
                                       CHAPTER 3
                         EXAMPLES OF DEFT APPLICATIONS

       This chapter contains four examples where DEFT is used to determine the feasibility of the
DQOs. Each example explains the planning team's choice of DQOs and shows, screen-by-screen,
what inputs were entered into DEFT. Actual text from DEFT is shown in italics and quotations; actual
buttons from DEFT are shown in italics. Note: The purpose of the examples is to show how DEFT
may be used to generate data based on various scenarios and assumptions. Although the examples
refer to various EPA requirements and standards, these are used for illustrative purposes only; they are
not examples of EPA-approved decision error limits or other data quality objectives.

3.1    TESTING A MEAN AGAINST A FIXED STANDARD - CADMIUM IN FLY ASH

       A waste incineration facility located in the Midwest routinely removes fly ash from its flue gas
scrubber system and disposes of it in a municipal landfill.  Previously, the waste fly ash was not
hazardous according to Federal environmental regulations. Due to treatment of a new waste stream,
representatives of the incineration company are concerned that the waste fly ash now contains
hazardous levels of cadmium.  If the fly ash meets the Federal standard and thus is considered non-
hazardous, it can be disposed of in a municipal landfill. If not, then the ash would have to be sent to a
higher-cost special hazardous waste disposal landfill.

Entry Screens

       Parameter of Interest: The planning team considered the population mean to be the
       appropriate parameter of interest because there is a large mixing effect when collecting the ash.
       The planning team is interested in looking at potential scenarios in preparation for making a
       decision for each load of fly ash so that only hazardous loads are disposed of in a special
       landfill.  Hence,  each load of fly ash is a separate population for which a decision is needed.
 Entry Screen 1: DEFT - Parameter Selection. Select 'Population Mean' under "Select the
 Parameter of Interest" and select 'One Population' under "Select Number of Populations" Press
 the NEXT button.
       Minimum and Maximum Values (Range) of the Parameter of Interest: The possible
       minimum value of cadmium is 0.0 mg/L and the team agreed to use a possible maximum value
       of 2.0 mg/L for planning purposes.

       Action Level: The regulatory standard for cadmium concentration in the leachate resulting
       from Toxicity Characteristic Leaching Procedure (TCLP) extraction is 1.0 mg/L.
                                                                          Windows Version 1.0
EPAQA/G-4D                                 25                                 September 2001

-------
       Baseline and Alternative Conditions:  The baseline condition is specified under the
       regulations as the case where the fly ash is considered hazardous (Baseline: Mean > 1.0 mg/L)
       and the alternative condition as the case where the waste is not considered hazardous
       (Alternative: Mean < 1.0 mg/L).

       Gray Region: The gray region is the area adjacent to the Action Level of 1.0 mg/L where the
       planning team considers the consequences of a false acceptance decision error to be minimal.
       A false acceptance error would result in unnecessary and costly disposal in a special landfill.
       The planning team specified a width of .25 mg/L for the gray region based on their preferences
       to guard against false acceptance decision errors at .75 mg/L.

       Estimate of Standard Deviation:  The planning team conducted a pilot study of the fly ash to
       determine the variability in the concentration of cadmium within loads of fly ash. This study
       showed that each load of fly ash is fairly homogenous and the standard deviation in the
       concentration of cadmium among samples within loads of ash is approximately 0.6 mg/L.
 Entry Screen 2:  DEFT - One-Sample Mean Inputs. Enter 0.0 for "Estimate of Minimum
 Value," 2.0 for "Estimate of Maximum Value," and 1.0 for "Action Level"  Under "Select
 Hypotheses" select 'H,,: mean > AL vs. }\: mean < AL.' Enter 0.75 for the 'Lower Bound' and
 0.6 for "Estimate of Standard Deviation" by "Use this Value."  Press the 7VEAT button.
       Sampling And Analysis Costs:  The cost of selecting a sample is $10.  The cost of TCLP
       analysis is $150 a sample.
 Entry Screen 3:  DEFT - Laboratory and Field Costs. Enter 150.00 for "Laboratory Costs
 per Sample," 10.00 for "Field Costs per Sample," and check the "Per Sample" box. Press the
 NEXT button.
       False Rejection Error Limit: Regulations specify a 5% false rejection decision error.
       Consequences of a false rejection error (deciding that the waste is not hazardous when it is truly
       hazardous) are that the incineration company disposes of the hazardous waste in a sanitary
       landfill, possibly endangering human health and the environment.

       False Acceptance Error Limit: The planning team set the maximum tolerable probability of
       making a false acceptance error at 20% at the bound of the gray region (0.75 mg/L). Since the
       baseline condition and the false rejection error limit are fixed by regulation, this is the only error
       limit the planning team can adjust and its primary consequence is economic. The consequence
       of a false acceptance error is an increase in unnecessary  expenses from using a special disposal
       facility when it is not needed.

                                                                          Windows Version 1.0
EPAQA/G-4D                                 26                                 September 2001

-------
 Entry Screen 4: DEFT - Decision Error Limits. Enter 0.20 for the "False Acceptance
 Error Limit" (under "Lower Bound") and 0.05 for "False Rejection Error Limit" (under
 " Upper Bound") Press the NEXT button.
      Additional Limits on Decision Errors: The planning team wanted to use additional
      decision error limits and set the maximum tolerable probability of making a false
      acceptance error at 10% when the true mean is below 0.25 mg/L.
 Entry Screen 5: DEFT - Additional Decision Error Limits. In the "Below Gray Region "
 section, enter 0.25 under "Concentration" and 0.10 under "Decision Error Limit." Press the
 NEXT button.
Input Verification Screen

       The input verification
screen (Figure 5) is used to verify
the inputs from the previous entry
screens.
 Input Verification Screen. Use
 the appropriate Change button to
 make any changes.  Press the
 NEXT button.
Design/DQO Summary Screen

       The DEFT - Design/DQO
Summary Screen (Figure 6) shows
that under simple random sampling,
the minimum number of
observations needed to satisfy the
decision error limits is 37 and the
total cost is $5,920. The incineration
company would like to hold the
study costs to around $2,500 per
load of fly ash, so the planning team
decided to investigate composite
sampling to see if it meets their
DQOs.


Inputs Decision Error Limits
Parameter of Interest: Mean concentration prob[error] type
Minimum Mean Concentration 0 I 	 I 	 pA
Action Level: 1 [025 [0~i FA
Null Hypothesis (Ho] mean>-1 ^ 	 FA
Gray Region: 0.75-1 1
Standard Deviation: 0.4 1" [o~05 ™
!ChangelnputValue(s)i | FR
Cost of Sample 1
Analysis in the Laboratory: 150.00 j ' Use the Change Input Value(s)
Collection in the Field: 10.00 Change Lost; j button to change these values.
NEXT » |


Figure 5. Input Verification Screen for Example 1


Help
Sampling Parameters
Sampling Design: Simple Random Sampling Change Sampling Design
Design-bpecinc Inputs:
No design-specific inputs for Simple Random Sampling.
	 Update Screen Data
Number of Samples: [37
. 	 I Update 1
Total Cost: $ 1 5920.00 ' 	 '
DQO Input Summary
Costs Decision Error Limits
Laboratory Costs Per Sample: 150.00 concentration ptob(error) type
Field Costs (Per Sample): 10.00 | | FA
[025 [a! FA
Change Costs . 	 ...
I nput Values , 	 „
Parameter of Interest: Mean 1* I0'05
Action Level: 1 | | FR
Gray Region: 0.75 - 1 i 	 i 	 pp
Baseline Condition: mean >= 1 .000 ' '
Standad Deviation: 0.6 'Use Change Input VaMs) button
Change I nput Vaiue(s)




                                 Figure 6. Design/DQO Summary Screen for Example 1
EPA QA/G-4D
        27
Windows Version 1.0
    September 2001

-------
       For composite sampling, the planning team needed to consider some additional
parameters. Using the results of the pilot study, the variability among subsamples within a
composite sample is expected to be negligible. Thus, the measurement standard deviation was
estimated to be a very small proportion of the total standard deviation (.0001).  Also, the
planning team decided that the load of fly ash could be easily divided into eight strata of equal
size. To form each composite sample, the containers will be divided into eight strata of equal
size, a sample taken randomly from within each stratum, and then the eight samples would be
composited.  The planning team assumed the cost for the compositing would be minimal so $0
was used as the compositing cost.
 Design/DQO Summary Screen. Press the Change Sampling Design button. On the box that
 appears, select "Composite Sampling" and press the OK button.  The "Composite Design
 Inputs" box will appear.  Enter 0.0001 for "Measurement SD/Total SD," 8 for "Aliquotsper
 Composite Sample," and 0.0 for "Cost for Compositing the Aliquots." Press the OK button.

 Press the Graph button to see the Decision Performance Goal Diagram shown in Figure 7.  To
 return to the Design/DQO Summary Screen, press the DQO Summary button.
       For this composite sampling design, the
number of samples is 6 and the cost is $1,380.
Therefore, these data quality objectives are
feasible and the planning team can continue with
Step 7 of the DQO Process, Optimize the Design.

       During Step 7 of the DQO Process, the
planning team decided to take eight composite
samples to improve the likelihood that their error
limits would be satisfied for every load of fly ash.
This design came to a total cost of $1,840 and the
false acceptance error rate has decreased from 0.2
(20%) to 0.082 (8.2%). Note: this is one potential
scenario, the planning team could have specified a
different width for the gray region or a different
false acceptance error rate depending on their

    Figure 7. Decision Performance Goal
    Diagram for Example 1
concern about costs. However, the risk to human health (controlled by the selection of the
baseline condition and the false rejection error rate) can not change as this is specified by EPA
through the regulations.
 Design/DQO Summary Screen. To changing the composite sample size from 6 to 8, enter 8
 for the "Number of Samples."  Then press the Update button to see the effect on cost and the
 false acceptance error rate.
EPA QA/G-4D
28
Windows Version 1.0
    September 2001

-------
3.2    TESTING A PERCENTILE AGAINST A FIXED STANDARD - URBAN AIR
       QUALITY COMPLIANCE

       Representatives of a metropolitan area in the Northeast want to determine if their area will meet
the PM2.5 (PM2.5, particulate matter of aerodynamic diameter less than or equal to 2.5 micrometers)
standard over the next year.  Federal regulations specify the 24-hour standard PM2 5 as a concentration
of no more than 65 //g/m3, based on the 3-year average of the annual 98th percentiles. Their sampling
network consists of three fixed-site multiple-filter gravimetric devices for measuring daily concentrations
(24-hr averages) of PM2.5.  Each of the three monitors measures concentrations once every 3 days, for
a total of 365 measurements per year.

Entry Screens

       Parameter of Interest: The population parameter of interest to the planning team was the 98th
       percentile of PM2 5 concentrations, as specified in the regulations. However, the sample size
       required to estimate a population percentile is usually determined by calculating the sample size
       needed to estimate the corresponding population proportion (see Section 4.4). Thus, the
       planning team formulated their study design requirements in terms of estimating a population
       proportion.
 Entry Screen 1:  DEFT - Parameter Selection. Select 'Population Proportion' under "Select
 the Parameter of Interest" and select 'One Population' under "Select Number of Populations." Press
 the NEXT button.
       Minimum and Maximum Values (Range) of the Parameter of Interest: For tests of a
       single proportion, the minimum value is 0 and the maximum is 1.

       Action Level: Because the 24-hour standard for PM2 5 is a concentration of no more than 65
       / 0.98, where P represents
       the proportion of daily concentrations less than or equal to 65 //g/m3.
                                                                          Windows Version 1.0
EPAQA/G-4D                                 29                                 September 2001

-------
       Gray Region: The gray region is the area adjacent to the Action Level (0.98) where the
       planning team considers the consequences of a false acceptance decision error to be minimal.
       A false acceptance error would result in the implementation of unnecessary and costly control
       strategies.  The planning team specified a width of .015 for the gray region based on their
       preferences to guard against false acceptance decision errors, thereby establishing a gray region
       of 0.98 to 0.995.
 Entry Screen 2: DEFT - One-Sample Proportion Inputs   Enter 0.98 for the "Action Lever,
 select 'H,,: proportion < AL vs. H,: proportion > AL' under "Select Hypotheses," and enter 0.995
 for the "Upper Bound " Press the NEXT button.
       Sampling And Analysis Costs:  There are no costs for the air sampling and analysis because
       the air monitoring system is already operational.
 Entry Screen 3:  DEFT - Laboratory and Field Costs. Enter 0.0 for both the "Laboratory
 Costs per Sample" and the "Field Costs per Sample."  Check the "Per Sample" box. Press the
 NEXT button.
       False Rejection Error Limit: The planning team agreed that the tolerable false rejection
       error rate should be no more than 10%.  While lowering the tolerable bound on such error was
       desirable, the planning team believed that a significantly smaller error rate was unobtainable for
       all but the most extensive and costly network designs.

       False Acceptance Error Limit: The team wanted to protect against unnecessary and costly
       control strategies (i.e., incorrectly failing to reject the baseline condition), but was willing to
       tolerate a greater probability of making this false acceptance decision error. They decided the
       limit should be no more than 30% at proportions above the upper bound of the gray region.
 Entry Screen 4:  DEFT - Decision Error Limits. Enter 0.1 for "False Rejection Error Limit"
 (under "Lower Bound") and 0.3 for the "False Acceptance Error Limit" (under "Upper Bound').
 Press the NEXT button.
       Additional Limits on Decision Errors: No additional limits on decision errors were
       specified by the planning team.
 Entry Screen 5: DEFT - Additional Decision Error Limits. Press the NEXT button.
                                                                           Windows Version 1.0
EPAQA/G-4D                                  30                                 September 2001

-------
Input Verification Screen

       The input verification screen (Figure 8) is used to verify the inputs from the previous
entry screens.
 Input Verification Screen. Use
 the appropriate Change button to
 make changes. Once the
 information is correct,  press the
 NEXT button.

MIDEFT - Input Verification UE3

Inputs Decision Error Limits
Parameter of Interest: Proporlon proportion prob(error) type
Minimum Proportion: 0.000 n 	 j 	 pp
Maximum Proportion: 1.000
Action Level: 0.9BO | | ^
Baseline condition: proporton <= 0.980 n qnnx nr: 	 : CD
Graj^ Region; 0.980-0.935 "~ ' '
0.995^ [ol FA
Change Input Valuefs] | | FA
Cost of Sample 1 '
Laboratory Cost$ Per Sample: 0.00 s Use the Change input ValuelsJ
Field Costs {Per Samdat 0.00 Change Costs butlon to change these ₯aiueSi
NEXT » |




Design/DQO Summary Screen

       The Design/DQO Summary
Screen shows that the minimum      Fi§ure 8- InPut Verification Screen for Example 2
number of observations needed to
satisfy the decision error limits with a simple random sampling design is 209. If each of the three
monitors in the network continues to sample once every 3 days, the planning team will have a
total of 365 samples for the year which will be more than sufficient.

       The planning team then continues with Step 7 of the DQO Process, "Optimize the
Design." Once the sampling design is optimized, the planning team documents the design and
quality objectives and submit this information to the appropriate regulatory body for approval.
 Design/DQO Summary Screen.  Press the Save button to save the Design/DQO Summary
 Screen to a file. Press the Exit button to exit DEFT.
3.3    TESTING THE DIFFERENCE BETWEEN TWO MEANS - CYANIDE
       CONTAMINATION IN GROUND WATER

       EPA is concerned that storage of waste materials at an abandoned factory had resulted in
environmental contamination.  Test wells at the site showed high concentrations of cyanide were
found in these wells, ranging from 1.5 parts per million (ppm) to over 300 ppm (at the plant site).
Prior activities at the site included moving the waste to a containment area and removal of the
surface and subsurface soils where the waste had been stored. EPA will now determine if
cyanide in the ground water has decreased to levels comparable to an area that was never
contaminated (a reference site). For purposes of this example, sampling costs will be less than
$12,500.  Note:  sampling costs may vary depending upon the particular scenario and applicable
requirements.
EPA QA/G-4D
31
Windows Version 1.0
    September 2001

-------
Entry Screens
       Parameter of Interest: The planning team considered the difference in population means to
       be the appropriate parameter of interest. The team is comparing the remediated site to a
       reference site to determine if the remediated site's levels of cyanide are the same as the
       reference levels; so, there are two populations of interest.
 Entry Screen 1:  DEFT - Parameter Selection. Select 'Population Mean' under "Select the
 Parameter of Interest" and select 'Two Population' under "Select Number of Populations." Press
 the NEXT button.
       Minimum and Maximum Values (Range) of the Parameter of Interest:  Because the soil
       was remediated, the most that the reference levels could exceed the site levels is by 20 ppm
       and the most that site levels could exceed reference levels is by 100 ppm. Therefore, the range
       of the difference is -20 to 100 ppm.

       Action Level: If the remediation methods worked, then the two sites should be approximately
       equal in average contamination levels, so the action level is "no difference" or 0.

       Baseline and Alternative Conditions:  The planning team designated the baseline condition
       as the case where the remediated site remains contaminated H,,: meanj - mean2 >0 where
             is the mean of the remediated site and mear^ is the mean of the reference area).
       Gray Region: The gray region is the area adjacent to the Action Level where the planning
       team considers the consequences of a false acceptance decision error to be minimal. A false
       acceptance error would result in the implementation of unnecessary and costly higher-stage
       remediation efforts. The planning team specified a width of 5 ppm for the gray region based on
       their preferences to guard against false acceptance decision errors, thereby establishing a gray
       region of -5.0 to 0.0.

       Estimate of Standard Deviation:  The planning team conducted a pilot study of the cyanide
       in the remediated wells and determined the standard deviation be 3.5 ppm. They also assumed
       the standard deviation in the reference wells would be the same.
 Entry Screen 2: DEFT - Two-Sample Mean Inputs. Enter -20.0 for "Estimate of Minimum
 Value," 100.0 for "Estimate of Maximum Value" and 0.0 for "Action Level" Under "Select
 Hypotheses" select 'H,,: mea^ - mean2 > AL vs. H,: mea^ - mean2 < AL.'  Enter -5.00 for the
 "Lower Bound' and 3.5 for "Estimate of Standard Deviation" by "Use this Value." Press the
 NEXT button.
                                                                          Windows Version 1.0
EPAQA/G-4D                                 32                                 September 2001

-------
       Sampling And Analysis Costs: The cost of selecting a sample is $150 and the cost of
       analyzing a sample is $500.
 Entry Screen 3: DEFT - Laboratory and Field Costs. Enter 150.00 for "Laboratory Costs
 per Sample," 500.00 for "Field Costs per Sample," and check the "Per Sample" box. Press the
 NEXT button.
       False Rejection Error Limit: The planning team determined that the tolerable false rejection
       decision error rate should be no more than 1% when the baseline condition is true. The
       planning team firmly believed that to protect human health and the environment it was necessary
       to have such a small false rejection error rate.

       False Acceptance Error Limit: The team wants to protect against unnecessary and costly
       higher-stage remediation efforts (i.e. incorrectly failing to reject the baseline condition), and
       decided the error limit should be no more than 5%.
 Entry Screen 4: DEFT - Decision Error Limits. Enter 0.05 for the "False Acceptance Error
 Limit" (under "Lower Bound') and 0.01 for "False Rejection Error Limit" (under "Upper
 Bound'). Press the NEXT button.
       Additional Limits on Decision Errors: No additional limits on decision errors were
       specified by the planning team.
 Entry Screen 5: DEFT - Additional Decision Error Limits. Press the NEXT button.
Input Verification Screen

       The input verification screen is used to verify the inputs from the previous entry screens.
 Input Verification Screen.  Use the appropriate Change button to make any changes. Once the
 information is correct, press the NEXT button.
Design/DQO Summary Screen

       The DEFT - Design/DQO Summary Screen (Figure 9) shows that the minimum number of
observations needed to satisfy the decision error limits with a simple random sampling design is 40 (20
per site) and the total cost is $26,000.  Since EPA would like to hold the study costs under $12,500,
                                                                         Windows Version 1.0
EPAQA/G-4D                                 33                                September 2001

-------
the DQOs are not feasible.  To reduce the
cost, the planning team decided to change
the false acceptance error limit to 15% and
the false rejection error limit to 5%.
 Design/DQO Summary Screen.  In
 the Decision Error Limits section
 underprob(error), change 0.05 to
 0.15 (next to -5.0) and change 0.01 to
 0.05 (next to 0.0). Press the Update
 button.
       The number of samples (per
population) is now 36 (18 per site) and the
total cost has decreased to $11,700.  The
planning team now has DQOs that are
feasible.  These DQOs will be used in Step
7 of the DQO Process. Then the final
sampling design and DQOs will be        Figure 9  Design/DQO Summary Screen for
documented in a Quality Assurance Project Example 3
Plan.
3.4    TESTING THE DIFFERENCE BETWEEN TWO PROPORTIONS - DIOXIN
       CONTAMINATION
       At a hazardous waste site, EPA investigators must determine whether an area suspected
to be contaminated with dioxin needs to be remediated. The potentially contaminated area (area
1) will be compared to a reference area (area 2) to see if dioxin levels in area 1 are greater than
those in area 2.  An inexpensive surrogate probe will be used to test each individual sample.

Entry Screens

       Parameter of Interest: Because the presence of dioxin in the area will be represented as
       a proportion of all samples that are contaminated, the population proportion is the
       appropriate parameter of interest.  The planning team is comparing the suspect site to a
       reference site to determine if the suspect site is contaminated with dioxin so there are two
       populations of interest.
 Entry Screen 1: DEFT - Parameter Selection. Select 'Population Proportion' under
 "Select the Parameter of Interest" and select 'Two Population' under "Select Number of
 Populations." Press the NEXT button.
EPA QA/G-4D
34
Windows Version 1.0
    September 2001

-------
       Minimum and Maximum Values (Range) of the Parameter of Interest: When
       comparing two population proportions, the minimum difference is -1 and the maximum
       difference is 1.

       Action Level: The health standard for dioxin is 1 ppb.

       Baseline and Alternative Conditions: The planning team designated the baseline condition
       as the case where the suspect site is clean and the  alternative condition as the case where the
       suspect site is contaminated (BL; meanj - mean2 < 0 where meanj is the mean of the remediated
       site and meai^ is the mean of the background area).

       Gray Region: The gray region is the area adjacent to the Action Level of 0 where the
       planning team considers the consequences of a false acceptance decision error to be minimal.
       A false acceptance error would result in the failure to clean up the contaminated site, thereby
       posing risks to human health and the environment. The planning team specified a width of 0.10
       for the gray region based on their preferences to guard against false acceptance decision errors,
       thereby establishing a gray region of 0.00 to 0.10.
 Entry Screen 2: DEFT - Two-Sample Proportion Inputs.  Under "Select Hypotheses" select
 'H: propl - prop2 < AL vs. H,: propl - prop2 > AL.'  Enter 0.10 for "Upper Bound."  Press the
 NEXT button.
       Sampling And Analysis Costs: The cost of selecting a sample is $0 and the cost of analyzing
       a sample is $17.
 Entry Screen 3: DEFT - Laboratory and Field Costs. Enter 17.00 for "Laboratory Costs per
 Sample," 0.00 for "Field Costs per Sample," and check the "Per Sample" box. Press the NEXT
 button.
       False Rejection Error Limit: The team wants to protect against unnecessary and costly
       remediation efforts so it has specified that the false rejection decision error rate should be no
       more than 10%.

       False Acceptance Error Limit:  The planning team firmly believed that to protect human
       health and the environment it was necessary to have a small false acceptance error rate.  The
       team decided that the probability of making this false acceptance decision error should be no
       more than 5%.
                                                                          Windows Version 1.0
EPAQA/G-4D                                 35                                September 2001

-------
 Entry Screen 4: DEFT - Decision Error Limits. Enter 0.05 for "False Rejection Error
 Limit" (under "Lower Bound') and 0.10 for the "False Acceptance Error Limit" (under
 " Upper Bound'). Press the NEXT button.
       Additional Limits on Decision Errors:  No additional limits on decision errors were
       specified by the planning team.
 Entry Screen 5: DEFT - Additional Decision Error Limits. Press the NEXT button.
Input Verification Screen

       The input verification screen is used to verify the inputs from the previous entry screens.
 Input Verification Screen. Use the appropriate Change button to make any changes.  Once
 the information is correct, press the NEXT button.
Design/DQO Summary Screen

       The "DEFT - Design/DQO
Summary" Screen shows that the
minimum number of observations needed
to satisfy the decision error limits with a
simple random sampling design is 644
(322 per site) and the total cost is $10,948.
Since this is close to the actual budget
allocated of $10,000, the planning team
will proceed to Step 7 of the DQO process.
(Note: there are other decision error limits
that would meet project constraints such as
cost.  This is just one example of feasible
DQOs for this problem.) During Step 7,
the planning team expects that the actual
budget will be met using these DQOs
when the sampling design is optimized.
        Cops' Graph Pfinr G>aph Save Graph Help Exi!

              Estimated Performance Curve
               True Diffe
                          in Proporti
Figure 10.  Decision Performance Goal Diagram
for Example 4
 Design/DQO Summary Screen. Press the Graph button to see the Decision Performance
 Goal Diagram shown in Figure 10. To return to the Design/DQO Summary Screen, press the
 DQO Summary button.
EPA QA/G-4D
  36
Windows Version 1.0
    September 2001

-------
                                       CHAPTER 4
                         EXTENDED APPLICATIONS OF DEFT

4.1    USING DEFT TO DETERMINE SAMPLE SIZES FOR ESTIMATION

       Although DEFT has been designed to determine the minimum sample size needed for
hypothesis testing problems, it also can be used to determine the minimum sample size needed to obtain
a sufficiently precise estimate of a population mean or a population proportion. In either case, the
estimation problem is to determine the minimum sample size needed to produce a 100 (1 - a)%
confidence interval estimate (e.g., a 95% confidence interval where a = 0.05) of a population mean or
proportion such that the maximum width of the confidence interval is less than or equal to a pre-
specified value.  That is, the confidence interval estimate should be no wider than "the point estimate  ±
1/2 of pre-specified width," where the point estimate is the sample mean or proportion.

       To use DEFT to estimate the sample size needed to obtain a confidence interval of a specified
width, make the following adjustments to the DQOs entered into DEFT.  First select an action level —
for a mean, this can be any value you like; for a proportion, this should be a preliminary estimate of the
proportion. Then set the maximum width for the confidence interval, determine what accuracy you
would like the confidence interval (for example, a 90% confidence interval or a 95% confidence
interval), and develop estimates of the standard deviation and sampling and analysis costs.  The
remaining DQOs are specified in  Table 4.
                            Table 4. Using DEFT for Estimation
DQO
Minimum Value
Maximum Value
Baseline Condition
Gray Region
False Acceptance Error Rate
False Rejection Error Rate
Additional Limits on Decision Errors
Use
Action Level minus twice the maximum
width (W), e.g., AL - 2W
Action Level plus twice the maximum
width (W), e.g., AL + 2W
mean < AL
Action Level + 1/2 the maximum width
0.50
a /2, so if you want a 90% confidence
interval, a = . 1, and the false rejection
error rate is 0.05
None
EPA QA/G-4D
37
Windows Version 1.0
     September 2001

-------
4.2    USING DEFT TO RECONCILE SAMPLE DATA WITH PROJECT DQOs

       During data quality assessment, it is necessary to determine if the project DQOs have been
achieved in order to properly interpret the results of the study. Guidance regarding this process is
provided in EPA Guidance on Data Quality Assessment (EPA/QA G-9) (EPA, 2000b).  The
process of data quality assessment has two distinct phases:

       1.     Determining if the assumptions underlying the estimation and/or hypothesis testing
              procedures are satisfied; and

       2.     Determining if the sample size (number of observations) is sufficient to make a decision
              based on the data obtained.

DEFT can be used for the second phase, i.e., determining if the sample size is sufficiently large,
assuming that the assumptions underlying the estimation or hypothesis testing procedure are satisfied
(see Chapter 4 of EPA, 2000b).  Use of DEFT for this determination is discussed in the below first for
estimation and then for hypothesis testing.

4.2.1   Estimation Problems

       If the study objective was to estimate a population parameter, the required sample size should
have been determined by specifying the maximum allowable width for a confidence interval estimate of
the parameter.  In this case, one can determine if the sample size was sufficiently large by simply
observing if the width of the actual confidence interval is less than or equal to the pre-specified
maximum width.  If so, the DQOs have been satisfied; if not, the sample size should be increased.

4.2.2   Hypothesis Testing

       If the study objective was to test a hypothesis, the required sample size should have been
determined by specifying the maximum probabilities for making decision errors — false rejection and
false acceptance of the baseline condition. If the baseline condition is rejected, the probability of false
rejection has been controlled by the critical value of the test statistic used to specify the threshold at
which the decision was made to reject the baseline condition, and there is no need to determine if the
sample size was adequate.  However, if the  baseline condition is not rejected, one needs to determine if
the sample  size is  sufficiently large to provide adequate protection against false acceptance error.
Guidance for determining if the sample size is sufficient is provided in Chapter 5 of the EPA  Guidance
on Data Quality Assessment (EPA/QA G-9) (EPA,  2000b) and DEFT can be used to implement the
calculations that are required.  The following discusses how to use DEFT to perform these
computations for each type of hypothesis test that the software supports.
                                                                            Windows Version 1.0
EPAQA/G-4D                                   38                                 September 2001

-------
       Testing a mean against a fixed standard with simple random sampling. Use the actual
       standard deviation from the sample in place of the pre-specified population standard deviation
       (Section 2.1) to calculate the required sample size. The DQO requirements have been satisfied
       if the actual sample size is greater than or equal to the required sample size.

       Testing a mean against a fixed standard with stratified simple random sampling. Use
       the actual standard deviation of the sample for each stratum instead of the pre-specified
       population standard deviation for each stratum to calculate the sample size required for each
       stratum (see Section 2.3.3). The DQOs have been satisfied if the actual sample size is greater
       than or equal to the required sample size for each stratum.

       Testing a proportion (or percentile) against a fixed standard with simple random
       sampling.  In this case, the DQOs depend on the population proportions that define the action
       level (specified in the baseline condition) and the boundary of the gray region for which the
       maximum false acceptance error rate was specified. Therefore, the power of the test does not
       depend on the actual sample proportion (or percentile), and the adequacy of the DQOs does
       not need to be verified.

       Testing a proportion (or percentile) against a fixed standard with stratified simple
       random sampling. For each stratum, use the actual proportion from the sample in place of
       the pre-specified population proportion to calculate the sample size required for each stratum
       (see Section 2.3.3).  The DQOs have been satisfied if the actual sample size is greater than or
       equal to the required sample size for each stratum.

       Testing the difference between two means with simple random sampling. Use the
       pooled standard deviation [Box 3.3-1 of iheEPA Guidance on Data Quality Assessment
       (EPA/QA G-9) (EPA, 2000b)] of the sample in place of the pre-specified population standard
       deviation to calculate the required sample size (see Section 2.3.3).  The DQOs have been
       satisfied if the actual sample size for each population is greater than or equal to the required
       sample size.

       Testing the difference between two proportions with simple random sampling.  Use the
       actual proportions from the sample in place of the pre-specified  population proportions to
       calculate the required sample size (see Section 2.3.3). The DQOs have been satisfied if the
       actual sample size for each population is greater than or equal to the required sample size.

4.3    USING DEFT FOR GRID SAMPLING DESIGNS

       The simple random sampling option may be used to estimate the sample size for a randomized
systematic sampling design (grid sampling with a random starting point). To do so, use DEFT to
                                                                          Windows Version 1.0
EPAQA/G-4D                                 39                                September 2001

-------
develop a sample size and cost estimate for a Simple Random Sample (Chapter 2) and adjust the
sampling protocols accordingly.

4.4    TESTING A PERCENTILE AGAINST A FIXED STANDARD

       A population parameter commonly of interest in environmental studies is an upper percentile
(upper proportion) because this parameter conservatively protects against extreme health affects. The
median, a measure of central tendency, is the 50th percentile. A percentile provides information
regarding extreme values and is useful when the population contains a large number of values less than
the analytical method detection limit.

       A population percentile represents the percentage of elements of a population having values less
than or equal to some threshold C. Thus, if C is the 95th percentile of a population, the values of 95%
of the elements of the population are less than or equal to C and 5% of the population have values
greater than C.  For example, if the 95th percentile of a chemical distribution is 40 ppm, then 95% of
the concentration levels are less than or equal to 40  ppm.

       Determining sample sizes for hypotheses concerning population percentiles is equivalent to
determining sample sizes for hypotheses concerning the corresponding population proportions. As  a
result, DEFT only considers proportions.  Therefore, to use DEFT for percentiles, the DQO inputs to
DEFT must be transformed. Consider the decision  to determine whether the 95th percentile of the
cadmium concentration in a load of fly ash waste is less than 1 mg/L.  The baseline condition in this case
is that the 95th percentile of cadmium is less than or equal to 1 mg/L.  Now, instead of considering the
population (the load of fly ash) to consist of differing levels of cadmium, consider the population to
consist of a binary variable that is T if the cadmium level at a particular point in the load of fly ash is 1
mg/L or less and is '0' if the level is above 1 mg/L.  In this case, the hypothesis may be changed to  a
hypothesis for a proportion so that the baseline condition becomes "the proportion of cadmium levels 1
mg/L or less in the load of fly ash is greater than 0.95."

       Once the hypotheses have been transformed, the other DQOs must also be transformed. This
includes the other bound of the gray region and additional limits on decision errors. For example, the
other bound of the gray region should have been specified as another percentile. This percentile will
also need to be converted into a proportion. Table 5 describes the conversion  necessary for all the
DQO Process information required for DEFT; column 3 contains an example of the conversion from
percentiles to proportions.
                                                                            Windows Version 1.0
EPAQA/G-4D                                  40                                 September 2001

-------
            Table 5.  Translating DQOs for Percentiles into DQOs for Proportions
DEFT Input
Parameter of
Interest
Minimum
Maximum
Action Level
Baseline and
Alternative
Conditions
Gray Region (GR)
Sampling Cost
Analysis Cost
False Rejection
Error Limit (FR)
False Acceptance
Error Limit (FA)
Translation from Percentiles
to Proportions
Use Population Proportion
0
1
Convert the action level to a
proportion by dividing by 100.
Baseline condition,
HQ! 0th Percentile > x, becomes
FL; P < Q/100
Baseline condition,
HQ! Qth Percentile < x, becomes
FL; P > Q/100
where P is the proportion with
observations coded as 1 if they are
less than x and 0 otherwise.
Convert the percentile that
describes the other bound of the
gray region to a proportion by
dividing by 100.
No change necessary.
No change necessary.
No change necessary.
No change necessary.
Example Translation



The 95th percentile becomes a
proportion of 0.95.
The baseline condition
HQ! 95th percentile > 5 ppm
translates into H,: P < 0.95 where P
is the population proportion with
observations coded as being 1 if
they are less than or equal to 5 ppm
and 0 otherwise.
The 97.5th percentile translates into
a proportion of 0.975.
No change necessary.
No change necessary.
No change necessary. The false
rejection error rate is still 5%.
No change necessary. The false
acceptance error rate is still 20%.
EPA QA/G-4D
41
Windows Version 1.0
     September 2001

-------
            Table 5.  Translating DQOs for Percentiles into DQOs for Proportions
     DEFT Input
  Translation from Percentiles
         to Proportions
      Example Translation
  Additional Error
  Limits Above or
  Below the Gray
  Region.
Convert the percentile to a
proportion by dividing by 100.  The
probabilities remain the same.
An error limit of 10% at the 99th
percentile translates into a false
acceptance error rate of 10% at a
proportion of 0.99.
EPA QA/G-4D
                      42
                 Windows Version 1.0
                      September 2001

-------
                                     CHAPTER 5
                            ALGORITHMS USED IN DEFT

       This chapter briefly describes the algorithms used in DEFT to calculate the minimum
required sample sizes under the various sampling design options. For more information
regarding these algorithms, see Gilbert (1989), Thompson (1992), or EPA (2000b).

5.1    TESTING A MEAN AGAINST A FIXED STANDARD

       A population mean represents the center of a population. This parameter is useful when
the action level is based on long-term average health effects (e.g., chronic conditions and
carcinogenicity). The mean is most useful when the population is homogeneous and has a
relatively small variance.  Estimating the mean generally requires a smaller number of samples
than estimating other population parameters.  However, the mean is not a very representative
measure of the center of the population if the underlying distribution of the population is highly
skewed, or  if the population contains a large proportion of values that are less than the analytical
method detection limit.

5.1.1   Simple Random Sampling

       The simplest probability sample is a simple random sample where every possible
sampling point has an equal probability of being selected and each sample point is selected
independently from all other sample points.  Simple random sampling is appropriate when little
or no information about a population is available. If some information is available, simple
random sampling may not be the most cost-effective sampling design.

       DEFT assumes that a t-test will be used to analyze the data.  Therefore, the corresponding
sample size formula is used in the computations:
                           =
where:   a2   =  estimated variance;
         a   =  false rejection error rate;
         P   =  false acceptance error rate;
         zp   =  the pth percentile of the standard normal distribution (from standard statistical
                tables);
         A   =  the difference between the action level and the other bound of the gray region;
         n   =  the number of samples.
                                                                      Windows Version 1.0
EPAQA/G-4D                               43                               September 2001

-------
A derivation of this formula is contained in Appendix C of the Guidance for the Data Quality
Objectives Process (EPA 2000c).  The sample size reported by DEFT is always greater than or
equal to 2 so that an estimate of the standard deviation can be calculated from the data collected.
Therefore, if the formula above yields a value less than 2, DEFT will automatically report a
sample size of 2.  If the sample size calculated is greater than 30,000 DEFT will warn the user
and make adjustments to the false rejection and false acceptance error rates (as discussed in
Section 2.3.4).

       The formula for computing the total cost of the simple random sampling design is:

             Total Cost = n ($ per field sample + $per laboratory analysis)

       The performance curve calculations are also based on the t-test. The software only
approximates this performance curve instead of computing the exact curve. As a result of this
approximation, the performance curve may appear to show that a decision error limit is satisfied
when it is not, especially on the false rejection side of the gray region. Therefore, DEFT labels
any decision error limit that is  not satisfied as "NS."  This label should be used to determine
whether or not a limit is satisfied,  rather than the graph of the performance curve.

5.1.2  Composite Sampling

       If analysis costs are high compared to sampling costs and the parameter of interest is a
mean, then it may be appropriate to use composite samples to reduce analysis costs. A composite
sample is a sample obtained by physically mixing (physically averaging) two or more samples
before analysis.  The use of composite samples in association with a sampling design can be a
cost-effective way to select a large number of sampling units and provide  better coverage of a
population without analyzing each individual sample.

       DEFT uses composite samples with a simple random sampling design, which is referred
to as "composite  sampling." The  software computes the number of composite samples, k,
required to meet the DQOs based  on a given number of individual samples, m, per composite
sample. To determine the number of composite samples, an estimate of the ratio, r, which is the
relative standard  deviation of measurement error to total standard deviation, is required, along
with the number of individual  samples, m, to be mixed to form each composite  sample. Note
that TO >1, andO
-------
where (?T is the total variance, o2x is the true variance between composite samples (i.e., the
"natural" variability with no measurement error), and (f 'e is the measurement error variance.

       If one forms composite samples of size m, then the variance between the composite
samples can be approximated as:
                    v(m)  =tfT[(l-^)/m + ^ ]                                         (3)

where r = oe / OT. DEFT uses this estimate of v(m) in place of o2 in Equation 1 for simple
random sampling. The resulting sample size is then the number of composite samples, k, of size
m that should be selected in order to satisfy the DQOs.

       The sample size reported by DEFT is always greater than or equal to 2 so that an estimate
of the standard deviation may be calculated from the data collected. Therefore, if the formula
above yields a value less than 2, DEFT will automatically report a sample size of 2. In addition,
if the sample size calculated is greater than 30,000, DEFT will adjust the false acceptance error
rate (see Section 2.3.4).

       The formula for computing the total cost of the composite sampling design is:

       Total cost = k [m($ per field sample) + ($per lab analysis) + ($per composite)}

       The performance curve calculations are also based on the t-test. The software only
approximates this performance curve instead of computing  the exact curve. As a result of this
approximation, the performance curve may appear to show that a decision error limit is satisfied
when it is not, especially on the false rejection side of the gray region. Therefore, DEFT labels
any decision error limit that is not satisfied as "NS." This label should be used to determine
whether or not a limit is satisfied, rather than the graph of the performance curve.

5.1.3  Stratified Sampling

       Stratified random sampling can be used to  improve  the precision of a sampling  design.
To create a stratified sample, the study population  is divided into two or more non-overlapping
subsets, called strata, that cover the entire population. Strata should be defined so that  physical
samples within a stratum are more similar to each  other than to  samples from other strata.
Previous data, information about concentration levels, previous studies, and knowledge about
contamination sources can be used as the basis for creating  strata. Once the strata are defined,
DEFT assumes each stratum will be sampled separately using a simple random sampling design.

       There is a limit of six strata in DEFT.  (This limit was set because the software  was
designed only to demonstrate feasibility of the DQOs and six strata should be sufficient for this
purpose.)  To determine sample sizes for  each stratum, a weighing factor (weight) and an
estimate of the standard deviation is needed. The  stratum weight is the proportion of the  volume

                                                                       Windows Version 1.0
EPAQA/G-4D                                45                                September 2001

-------
or area of the environmental medium contained in the stratum in relation to the total volume or
area of the study site.  The sum of the strata weights must be 1, so the program automatically
computes the weight of the final stratum.  The default weight corresponds to equal weighing
among the strata.  The estimated standard deviation for each stratum must be less than two times
the range of the population parameter, and the default value is the estimated total standard
deviation.

       DEFT assumes that a t-test will be used to analyze the data. Therefore, the corresponding
sample size formula6 (repeated for each stratum) is used in the computations:
                                Whah] [^]  Whoh                            (4)
                            h=\              A^

where  nh   =   the number of samples for stratum h;
        L    =   total number of strata;
        Wh  =   weight for stratum h;
        a    =   false rejection error rate;
        P    =   false acceptance error rate;
        oh   =   estimated standard deviation for stratum h;
        A    =   the difference between the action level and the other bound of the gray region;
                     yh.

                 tables).
zp   =   the p  percentile of the standard normal distribution (from standard statistical
The sample size reported by DEFT is always greater than or equal to 2 so that an estimate of the
standard deviation may be calculated from the data collected in each stratum.  Therefore, if the
formula above yields a value less than 2, DEFT will automatically report a sample size of 2. This
means that the minimum sample size for a stratified design is equal to two times the number of
strata.  If the total sample size calculated is greater than 30,000 times the number of strata, DEFT
will adjust the false acceptance error rate (see Section 2.3.4).  In addition, this sample size
formula assumes that the costs of sampling each stratum are the same. If not,  see Chapter 6 of
Methods for Evaluating the Attainment of Cleanup Standards (EPA, 1989) for a sample size
formula that accounts for unequal  stratum costs.

       The formula for computing the total cost of the stratified sampling design is:
                     L
        Total Cost =  ^3 nh$ Per fi^d sample  + $  per laboratory analysis)
                     h=l
   "This sample size formula assumes that the standard deviation is known.  Therefore, when the standard deviation
is estimated and the calculated sample size is small, consider increasing the sample size by 2 or 3 samples per
stratum.

                                                                         Windows Version 1.0
EPAQA/G-4D                                46                               September 2001

-------
       The performance curve calculations are based on the t-test. The software only
approximates this performance curve instead of computing the exact curve. As a result of this
approximation, the performance curve may appear to show that a decision error limit is satisfied
when it is not, especially on the false rejection side of the gray region.  Therefore, DEFT labels
any decision error limit that is not satisfied as "NS." This label should be used to determine
whether or not a limit is satisfied, rather than the graphical display of the  performance curve.

5.2    TESTING A PROPORTION AGAINST A FIXED STANDARD

       A proportion represents the number of objects in a population having (or not having)
some characteristic divided by the total number of objects in the population. This characteristic
may be qualitative, such as leaking drums versus non-leaking drums, or quantitative, such as the
drums with concentration levels of a contaminant greater than some fixed level. A proportion is
useful if the population consists of discreet objects such as drums or a population offish. The
following discussion assumes that the population is either infinite or extremely large.

5.2.1   Simple Random Sampling

       The simplest probability sample is a simple random  sample where every possible
sampling point has an equal probability of being selected and each sample point is selected
independently from all other sample points. Simple random sampling is appropriate when little
or no information about a problem is available. If some information is available, it may not be
the most cost-effective design.

       DEFT assumes that a  large-sample, normal approximation method will be used to analyze
the data.  Therefore, the corresponding sample size formula is used in the computations:
                   n =
                                           ZIJAL(\-AL]
(5)
where:   a    =  false rejection error rate;
         P    =  false acceptance error rate;
         zp    =  the pth percentile of the standard normal distribution (from standard
                  statistical tables);
         AL  =  action level;
         GR  =  other bound of the gray region; and
         n    =  the number of samples.

This formula is based on Box 7.2 on page 7-6 of Methods for Evaluating the Attainment of
Cleanup Standards: Volume I: Soils and Solid Media (EPA, 1989). The sample size reported
by DEFT is always greater than or equal to 2 so that an estimate of the standard deviation may be
calculated from the data collected.  Therefore, if the formula above yields a value less than 2,

                                                                      Windows Version 1.0
EPAQA/G-4D                               47                              September 2001

-------
DEFT will automatically report a sample size of 2. In addition, if the sample size calculated is
greater than 30,000, DEFT will make adjustments to the false rejection and false acceptance error
rates as explained in Section 2.3.4.

       The formula for computing the total cost of the simple random sampling design is:

                Total Cost = n ($ per field sample + $per laboratory analysis)

       The performance curve calculations are also based on the large sample approximation.
As a result of this approximation, the performance curve may appear to show that a decision
error limit is satisfied when it is not, especially on the false rejection side of the gray region.
Therefore, DEFT labels any decision error limit that is not satisfied as "NS." This label should
be used to determine whether or not a limit is satisfied, rather than the graphical display of the
performance curve.  Note that due to the approximation process, the performance curve may not
intersect the false acceptance error limit.  This is a result of a correction factor in the sample size
formula that does not cancel out in the performance curve calculations. However, the sample
size given by DEFT should satisfy the decision error limits regardless of the appearance of the
performance curve unless these limits are otherwise marked.

5.2.2   Stratified Sampling

       Stratified random  sampling is used to improve the precision of a sampling design. To
create a stratified sample, the study population is divided into two or more non-overlapping
subsets, called strata, that cover the entire population.  Strata should be defined so that physical
samples within a stratum are more similar to each other than to samples from other strata.
Previous data, information about concentration levels, previous studies, and knowledge about
contamination sources or  activities can be used as the basis for creating strata.  Once the strata
have been defined, DEFT assumes each stratum will be sampled separately using a simple
random sampling design.

       To estimate the sample size required for a  stratified design, DEFT requires information
regarding each individual stratum including a weighing factor (weight) and a preliminary
estimate of the stratum proportion.  The stratum weight is the proportion of the volume or area of
the environmental medium contained in the stratum in relation to the total volume or area of the
study population. The sum of the strata weights must be 1, so the program automatically
computes the weight of the final stratum. The default weight corresponds to equal weighing
among the strata.  The estimated stratum proportions may be based on historical information. If
there is no information available for estimating these proportions, use the action level or else the
average of the action level and the other bound of the gray region.  There is a limit of six strata in
DEFT. (This limit was set because the software was designed only to demonstrate feasibility of
the DQOs and six strata should be sufficient for this purpose.)
                                                                        Windows Version 1.0
EPAQA/G-4D                               48                               September 2001

-------
       DEFT assumes that a large sample approximation will be used to analyze the data.
Therefore, the corresponding sample size formula (repeated for each stratum) is used in the
computations:
where
        a
        P
        A
        Wh
                                                  "1-0
                                                                                     (6)
                 the number of samples for stratum h;
                 false rejection error rate;
                 false acceptance error rate;
                 the difference between the action level and the other bound of the gray region;
                 weight for stratum h;
                 estimated proportion for stratum h; and
                 the pth percentile of the standard normal distribution (from standard statistical
                 tables).
This formula is based on Box 7.7 of Methods for Evaluating the Attainment of Cleanup
Standards (EPA, 1989).

       The sample size reported by DEFT is always greater than or equal to 2 so that an estimate
of the standard deviation may be calculated from the data collected in each stratum.  If the
formula above yields a value less than 2, DEFT will automatically report a sample size of 2. This
means that the minimum sample size for a stratified design is equal to two times the number of
stratum.  If the sample size calculated is greater than 30,000 times the number of strata, DEFT
will adjust the false acceptance error rate (see Section 2.3.4). In addition, this sample size
formula assumes that the costs of sampling each stratum are the same. If not, see Chapter 6 of
Methods for Evaluating the Attainment of Cleanup Standards (EPA 1989) for a sample size
formula that accounts for unequal stratum costs.

       The formula for computing the total cost of the stratified sampling design is:
            Total Cost =    n
                                     field sample  +  $ per laboratory analysis)
where L represents the total number of strata.

       The performance curve calculations are also based on the large sample approximation.
As a result of this approximation, the performance curve may appear to show that a decision
error limit is satisfied when it is not, especially on the false rejection side of the gray region.
Therefore, DEFT labels any decision error limit that is not satisfied as "NS." This label should
be used to determine if a limit is satisfied, rather than the graph of the performance curve.
EPA QA/G-4D
                                           49
Windows Version 1.0
    September 2001

-------
5.3    TESTING THE DIFFERENCE BETWEEN TWO MEANS

       The simplest probability sample is a simple random sample where every possible
sampling point has an equal probability of being selected and each sample point is selected
independently from all other sample points. Simple random sampling is appropriate when little
or no information about a population is available.  If some information is available, simple
random sampling may not be the most cost-effective sampling design.

       DEFT assumes that a t-test will be used to analyze the data. Therefore, the corresponding
sample size formula is used in the computations:
                                                                                     (7)
where:   a2  =  estimated variance for both populations;
         a   =  false rejection error rate;
         P   =  false acceptance error rate;
         zp  =  the pth percentile of the standard normal distribution (from standard statistical
                tables);
         A   =  the difference between the action level and the other bound of the gray region;
         n   =  the number of samples for population 1; and
         m  =  the number of samples for population 2.

This formula is based on Section 3.3.1.1 of the EPA Guidance for Data Quality Assessment:
Practical Methods for Data Analysis (QA/G-9) (EPA, 2000b).

       The sample size reported by DEFT is always greater than or equal to 2 so that an estimate
of the standard deviation may be calculated from the data collected.  Therefore, if the formula
above yields a value less than 2, DEFT will automatically report a sample size of 2. In addition,
if the sample size calculated is greater than 30,000, DEFT will make adjustments to the false
rejection and false acceptance error rates as described in Section 2.3.4.

       The formula for computing the total cost of the simple random sampling design is:

             Total Cost = (m + n) ($ per field sample + $per laboratory analysis)

       The performance curve calculations are also based on the t-test.  The software only
approximates this performance curve instead of computing the exact curve.  As a result of this
approximation, the performance curve may appear to show that a decision error limit is satisfied
when it is not, especially on the false rejection side of the gray region. Therefore, DEFT labels
any decision error limit that is not satisfied as "NS." This label should be used to determine
whether or not a limit is satisfied, rather than the graphical display of the performance curve.

                                                                       Windows Version 1.0
EPAQA/G-4D                               50                               September 2001

-------
5.4    TESTING THE DIFFERENCE BETWEEN TWO PROPORTIONS

       The simplest probability sample is a simple random sample where every possible
sampling point has an equal probability of being selected and each sample point is selected
independently from all other sample points. Simple random  sampling is appropriate when little
or no information about a population is available. If some information is available, simple
random sampling may not be the most cost-effective sampling design.

       DEFT assumes that a large sample normal approximation method will be used to analyze
the data. Therefore, the corresponding sample size formula is used in the computations:
n =
                                                                                    (8)
where:   p   =   (P1 + P2)/2;
         a    =   false rejection error rate;
         P    =   false acceptance error rate;
         zp   =   the pth percentile of the standard normal distribution (from standard
                  statistical tables);
         P!   =   the action level;
         P2   =   the other bound of the gray region;
         n    =   the number of samples for population 1; and
         m   =   the number of samples for population 2.

This formula is based on Box 3.3-5 of the EPA Guidance for Data Quality Assessment:
Practical Methods for Data Analysis (QA/G-9) (EPA, 2000b).

       The sample size reported by DEFT is always greater than or equal to 2 so that an estimate
of the standard deviation may be calculated from the data collected. Therefore, if the formula
above yields a value less than 2, DEFT will automatically report a sample  size of 2.  In addition,
if the sample size calculated is greater than 30,000, DEFT will make adjustments to the false
rejection and false acceptance error rates as discussed in Section 2.3.4.

       The formula for computing the total cost of the simple random sampling design is:

               Total Cost = 2n($ per field sample + $per laboratory analysis)

       The performance curve calculations are also based on the large sample approximation.
As a result of this approximation, the performance curve may appear to show that a decision
error limit is satisfied when it is not, especially on the false rejection side of the gray region.
Therefore, DEFT labels any decision error limit that is not satisfied as "NS."  This label should
be used to determine whether or not a limit is satisfied, rather than the graphical display of the

                                                                      Windows Version 1.0
EPA QA/G-4D                               51                               September 2001

-------
performance curve. Note that due to the approximation process, the performance curve may not
intersect the false acceptance error limit.  This is a result of a correction factor in the sample size
formula that does not cancel out in the performance curve calculations. However, the sample
size given by DEFT should satisfy the decision error limits regardless of the appearance of the
performance curve unless these limits are otherwise marked.

5.5    ESTIMATING A POPULATION MEAN

       The formula used by DEFT to calculate the sample size required for testing a mean
against a fixed standard (Equation 1) is used to calculate a minimum sample size needed to
generate a 100 (l-a)% confidence interval estimate (e.g.,  a 95% confidence interval where
a = 0.05) of a population mean with a specified maximum width. In Equation 1, A = Action
Level + 1/2 the maximum  width.

5.6    ESTIMATING A POPULATION PROPORTION

       The formula used by DEFT to calculate the sample size required for testing a proportion
against a fixed standard (Equation 5) is used to calculate the minimum sample size needed to
generate a 100 (1 - a)% confidence interval estimate (e.g., a 95% confidence interval where
a = 0.05) of a population proportion with a specified maximum width. In Equation 1,
A = Action Level + 1/2 the maximum width.
                                                                     Windows Version 1.0
EPAQA/G-4D                               52                              September 2001

-------
                                   REFERENCES

Flanagan, James B., and James V. Aanstoos, 2001. Test Plan for the Data Quality Objectives
      Decision Error Feasibility Trials (DQO/DEFT) Software, Research Triangle Institute
      Report RTI/07660/009/2.2F prepared under U.S. EPA Contract 68-C-99-246, Research
      Triangle Park, NC.

Gilbert, R. O., 1987. Statistical Methods for Environmental Pollution Monitoring. John Wiley,
      New York, NY.

Thompson, S. K.,  1992.  Sampling. John Wiley, New York, NY.

U.S. Environmental Protection Agency, 2000a. The Data Quality Objectives Process for
      Hazardous Waste Sites (QA/G-4HW), EPA/600/R-00/055, Office of Environmental
      Information.

U.S. Environmental Protection Agency, 2000b. Guidance for Data Quality Assessment:
      Practical Methods for Data Analysis - QAOO Update (QA/G-9), EPA/600/R-96/084,
      Office of Research and Development.

U.S. Environmental Protection Agency, 2000c. Guidance for the Data Quality Objectives
      Process (QA/G-4), EPA/600/R-96/055, Office of Research and Development.

U.S. Environmental Protection Agency. 1989. Methods for Evaluating the Attainment of
      Cleanup Standards: Volume I:  Soils and Solid Media.  EPA/230/02-89-042, Office of
      Policy Planning and Evaluation.
                                                                     Windows Version 1.0
EPAQA/G-4D                              53                              September 2001

-------