EPA-450/4-89-015
GUIDANCE ON APPLYING THE
DATA QUALITY OBJECTIVES PROCESS
FOR AMBIENT AIR MONITORING
AROUND SUPERFUND SITES
(STAGES I & H)
By
Research Triangle Institute
Research Triangle Park, NC 27709
EPA Contract No. 68-02-4550
EPA Project Officer Darryl von Lehmden
Atmospheric Research And Exposure Assessment Laboratory
U.S. Environaental Protection Agency
7\.-'.;! 5S L'brr"^' (V:'-1")
?:;3 S. I)fc;>..r"born i"t/-->et, _.:~.a 1670
Chicago, IL 60804
Office Of Air Quality Planning And Standards
Office Of Air And Radiation
U.S. Environmental Protection Agency
Research Triangle Park, NC 27711
August 1989
-------
ACKNOWLEDGEMENTS
We would like to thank Stanley Sleva of the Office of Air Quality
Planning and Standards, as well as Work Assignment Officer Jane Leonard, for
their efforts and guidance on this project. In addition, the following
individuals provided helpful comments and suggestions:
Thomas Pritchett, OSWER
Peter Kahn, Region I
Jody Hudson, Region VII
Thomas Curran, OAQPS
Joseph Pagdett, OAQPS
m
-------
PREFACE
This document, Guidance for Applying the Data Quality Objectives Process
for Ambient Air Monitoring Around Superfund Sites, provides direction and
assistance on how the data quality objectives (DQO) process 1s used to design
an ambient air monitoring system around a Superfund site that will be adequate
for the intended use of the data. It is intended to serve as a bridge between
the Quality Assurance Management Staff (QAMS) DQO guidance and actual applica-
tion of the DQO process at Superfund sites. Specifically, this document was
written to aid the Remedial Project Managers, Enforcement Project Managers,
and the EPA Regional and Superfund contractor personnel responsible for
ambient air sampling and analysis at Superfund sites to carry out their jobs
in an efficient and effective manner.
DQOs are statements of the level of uncertainty that a decision maker is
willing to accept in results derived from environmental data. The DQO process
consists of three stages. This document gives an example of Stage I (prelimi-
nary definition of the decision) and Stage II (refinement of the decision and
requirements) for monitoring ambient air quality during remedial action at a
hypothetical Superfund site. Stage III of the DQO process, to be completed at
a later date for this hypothetical site, Involves the design of an ambient air
monitoring system capable of achieving the DQOs.
-------
TABLE OF CONTENTS
Section Page
ACKNOWLEDGEMENTS 111
PREFACE 1 v
1.0 INTRODUCTION 1
1.1 PURPOSE 1
1.2 OVERVIEW OF THE DATA QUALITY OBJECTIVES PROCESS 1
1.3 CONSIDERATIONS FOR AMBIENT AIR MONITORING AROUND
SUPERFUND SITES 3
1.4 USE AND ORGANIZATION OF THIS DOCUMENT 6
2.0 STAGE I: PRELIMINARY DEFINITION OF THE DECISION 8
2.1 PROBLEM DESCRIPTION 8
2.2 DECISION MAKERS AND KEY DATA USERS 11
2.3 DECISION DEFINITION 12
2.4 AVAILABLE RESOURCES 13
2.5 INFORMATION REQUIREMENTS 14
2.6 DATA REQUIREMENTS AND USES 15
2.7 CONSEQUENCES OF MAKING INCORRECT DECISIONS 16
3.0 STAGE II: REFINEMENT OF THE DECISION AND REQUIREMENTS 18
3.1 QUESTIONS TO BE ANSWERED 18
3.2 VARIABLES AND DOMAIN 19
3.3 SUMMARY RESULTS 22
3.4 NEED FOR NEW DATA 22
3.5 ACCEPTABLE ERROR RATES 23
3.6 SUMMARY OF THE DATA QUALITY OBJECTIVES 26
REFERENCES 32
APPENDICES
A A DISCUSSION OF FALSE POSITIVE AND FALSE NEGATIVE ERRORS A-l
B GLOSSARY B-l
-------
LIST OF FIGURES
Figure Page
1 SUPERDUMP SITE MAP 9
2 PERFORMANCE CRITERIA FOR AMBIENT AIR MONITORING AT
SUPERDUMP 25
A-l ILLUSTRATION OF THE RELATIONSHIP BETWEEN TYPE I AND
TYPE II ERRORS A-4
LIST OF TABLES
Table Page
1 DQO PROCESS SUMMARY 3
2 SUPERDUMP SITE INFORMATION 10
3 DECISION ERRORS 1 17
4 INFORMATION ON CHEMICALS AT THE SITE 20
5 EXPOSURE LIMITS FOR SELECTED INDICATOR CHEMICALS 21
6 COMPOUND-SPECIFIC DATA QUALITY OBJECTIVES 24
v1
-------
SECTION 1.0
INTRODUCTION
1.1 PURPOSE
The purpose of this document 1s to provide direction for and assistance
1n using the data quality objectives (DQO) process to design ambient air
monitoring systems around Superfund sites that will generate data of adequate
quality for use 1n decision making at the specified levels of certainty. This
guidance is also meant to assist the reader in adapting the DQO process, as
presented by the Quality Assurance Management Staff (QAMS) in Development of
Data Quality Objectives. Description of Stages I and II, to site-specific
problems: it gives an example of Stages I and II of the DQO process for
monitoring ambient air quality during remedial action at a hypothetical site.
1.2 OVERVIEW OF THE DATA QUALITY OBJECTIVES PROCESS
DQOs are statements.of the level of uncertainty that a decision maker is
willing to accept in decisions based upon environmental data. Developing DQOs
should be one of the first steps in Initiating an environmental data collec-
tion program to be conducted by or for the U.S. Environmental Protection
Agency (EPA). The DQO process helps decision makers, data users, and data
generators communicate clearly with each other about the purposes for which
environmental data will be used, the resources which can be made available for
the effort, and the level of quality required of the results to be derived
from the environmental data. For planning an appropriate ambient air monitor-
ing program at Superfund sites, communication between decision makers and data
users, such as Remedial Project Managers (RPMs) and Enforcement Project
Managers (EPMs), and data generators, such as personnel responsible for
ambient air sampling and analysis or State personnel, is Important. Equally
important is the interactive role of the decision maker and the technical
staff in defining the overall goal(s) (or decisions) of a given monitoring
program. Not only can a project manager initially select a goal that may not
be technically achieved (e.g., 10 risk for bis-2-chloromethyl ether with 95%
confidence), he may need significant Input from his technical staff just to
properly define his initial decision. For example, most project managers
would not be aware of any differences in the quality of risk determinations
-------
based upon single event, "worst case" concentrations versus risk
determinations based upon modeled, total yearly exposures using multiple
monitoring events, and historical meteorological data.
The DQO process consists of three stages with several steps. Table 1
summarizes the DQO process in terms of purpose and the Individual(s) serving
in the lead role for each stage. The decision maker takes the lead role in
defining the decision in Stage I, and the program and technical staff lead in
establishing qualitative and quantitative constraints in Stage II. The
process described in the first two stages results 1n proposed DQOs with
accompanying specifications (constraints). The specifics of monitoring or
modeling system design are not explicitly considered until Stage III. In this
stage, alternative designs for collecting the required data are developed and
quantitatively evaluated, leading to the selection of the monitoring design
which ensures that the DQOs will be met in an economical manner. The DQO
process is meant to be iterative among all stages. If the DQOs cannot be
satisfied with the given resources and other constraints, the decision maker
must decide whether to allocate more resources, relax the DQOs, or change some
other aspect of the monitoring design.
DQOs for a monitoring program are achieved by executing an experimental
design which 1s appropriate for the expected measurement data quality. DQOs
differ from measurement quality objectives (sometimes referred to in quality
assurance project plans as data quality indicator goals) in that they are
limits for the OVERALL uncertainty of a PROJECT'S results. Measurement
quality objectives, such as precision and accuracy, are limits for the uncer-
tainty of specific MEASUREMENTS. A DQO is expressed as the probability of
making a wrong decision; a measurement quality objective is expressed as a
desired value of precision, accuracy, completeness, or representativeness,,
Although they describe quality at different levels, DQOs and measurement
quality objectives are directly and quantitatively related. During Stage III
of the DQO process, monitoring program designs are evaluated for their ability
to satisfy DQOs, given the expected precision, accuracy, and completeness of
the measurement data. For example, if the precision of one sampling and
analysis procedure is not sufficient to meet a DQO, then the procedure must be
changed to require more measurements to improve its precision or a more
precise procedure must be identified and used. During or following execution
-------
TABLE 1. DQO PROCESS SUMMARY
STAGE
PURPOSE
LEAD ROLE
I
Define
Decision
and Time
Frame of
Decision
Decision
Maker
Establish
Feasibility
of the
Decision
Technical
and
Program
Staff
II
Establish
Qualitative
and
Quantitative
Constraints
Technical
and
Program
Staff
III
Design Data
Collection
Program To
Meet
Constraints
Technical
Staff
of the experimental design, the monitoring program 1s again evaluated for Its
satisfaction of DQOs by using actual precision, accuracy, and completeness
data obtained from the quality assurance/quality control (QA/QC) program.
Thus, DQOs and measurement quality objectives are recoridled 1n both the plan-
ning of the experimental design to ensure that DQOs will be met and by
analysis of QA/QC data from the monitoring program to assure the decision
maker that the DQOs were met.
1.3 CONSIDERATIONS FOR AMBIENT AIR MONITORING AROUND SUPERFUND SITES
The physical characteristic of the site Itself 1s one consideration in
designing an efficient and effective air quality monitoring program. For
example, whether or not the site acts as a point source or area source, and
whether or not the air emissions are at ground level or from some elevated
point influences the resultant monitoring strategy. Another consideration is
seasonal change. If volatile organic compounds are of concern, hot summer
months may increase the potential for emissions and thus require a more
intense and quantitative monitoring effort. On the other hand, cold winter
months, especially with snow and ice covering the ground, may minimize or
preclude any emissions and thus reduce the monitoring requirements. Another
important consideration is the time of day during which air samples are taken.
For example, diurnal variations in downwind concentrations due to changes in
the meteorological conditions such as temperature and solar radiation changes,
-------
are rarely considered even though they may be the overriding consideration.
Too many times EPA investigates odor complaints by conducting air monitoring
studies during daylight hours despite the vast majority of complaints being
logged in the early evenings or early mornings.
A general air quality monitoring strategy for a Superfund site is to
first characterize the site qualitatively and semi-quantitatively using a
quick screening procedure. Then, 1f results from the screening procedure
indicate a potential problem either in terms of the types of pollutants
present or their concentration levels, a more rigorous procedure Is used to
quantitatively characterize and monitor the site.
The routine monitoring program during the remedial response activity may
have several levels of Intensity. For example, 1f the initial screening
procedure does not indicate a problem, then scheduled periodic screenings may
be all that is needed. If, at the other extreme, the Initial or any sub-
sequent screening indicates that pollutant concentrations are near the level
of concern, then a more rigorous monitoring effort 1s required to increase the
probability of detecting an exceedance and of being able to respond to thai:
exceedance in a timely manner. The more rigorous procedure could Involve the
on-site use of compound-specific measurements in near real-time using portable
gas chromatographs, for example. More specific considerations regarding
instrumentation and exposure limits are discussed 1n the following paragraphs.
Paramount to the selection of instrumentation for ambient air monitoring
is a knowledge of the following:
pollutants of interest and their expected concentration ranges at
points where they will be measured;
required data quality (screening, qualitative, or quantitative)
to meet the DQOs;
available sampling and analysis procedures and their turnaround
times (real-time results may be necessary in some situations,
while a delay of 48 hours or longer to receive laboratory results
may be acceptable in other cases); and
available resources including funds, personnel (training,
availability, and level of experience), and equipment.
All of the above items are addressed through application of the DQO process.
-------
The first consideration for the selection of specific instrumentation for
the monitoring system is the pollutants that may be emitted from the site.
The list of compounds of interest may include all that could create a public
health problem at the site or only indicator chemicals selected on the basis
of quantities present at the site, level of hazard, volatility, persistence,
and severe odor. The concentration levels of concern may be based on regula-
tions such as applicable or relevant and appropriate requirements (ARARs), or
health risk-based calculations, as given 1n the Superfund Public Health
Evaluation Manual. ARARs may be Federal, State, or local regulations.
Volatile organic compounds (VOCs) are one class of compounds of special
Interest at Superfund sites and are used in the example in this document.
There are no Federal ambient air regulations for VOCs. If a site with a
potential ambient air VOC problem is not covered by State or local ARARs,
health risk-based calculations can be used 1f appropriate toxlcity constants
are available. During periods of remedial activity, when short-term exposures
are the concern, the health risk-based calculations are not directly appli-
cable because only long-term toxiclty constants are available. Thus, in these
instances, concentration levels of concern may need to be based on exposure
standards such as those promulgated under'the Occupational Safety and Health
Administration (OSHA), I.e., TLV/10, American Conference of Governmental
Industrial Hygienist (ACGIH), or an extrapolation to short-term exposures from
the health risk-based calculation.
The available sampling and analysis schemes that may be applicable for
ambient air monitoring for VOCs around Superfund sites Include the following:
Colorimetric detection tubes provide quick turnaround time (i.e.,
the time between sample collection and receipt of analytical
results) and are Inexpensive. Using these tubes for screening
may be appropriate if the concentrations are far above the levels
of concern to provide for a quick decision to halt remedial
activities and/or evacuate the affected population. If this
method indicates a concentration near the level of concern, a
more rigorous sampling and analysis scheme may be needed since
this method is not especially accurate. Many tubes are cross
sensitive to other compounds and may give false positives. These
measurements made on-site can be used with appropriate meteorolo-
gical data in models to predict receptor concentrations.
Portable instruments (those that can be carried by an individual)
such as flame ionization detectors and photoionization detectors
(e.g., HNU meters) may be used if quick turnaround time is needed
-------
and If the levels of concern are 1n terms of total organic con-
centrations instead of compound-specific. If these instruments
indicate a potential problem, more compound-specific analyses
will usually be needed. The accuracy of these instruments for
total organics depends somewhat on the frequency of calibration.
These instruments can be used for measurements at the source, the
fenceline, and/or the receptors.
More sophisticated transportable on-site instruments (those that
can be moved in a van, but not carried by an individual) such as
gas chromatographs (GCs) are compound-specific and provide almost
real-time results. These systems should be used when the concen-
tration levels are near the levels of concern, and good precision
and accuracy are needed to minimize the risk of making a wrong
decision. If properly used and calibrated, they provide data
quality adequate for most decision making. These systems can be
moved to follow a plume and evaluate hot spots at the fenceline
or near the receptors. They require highly trained staff to
operate and maintain.
Off-site laboratory Instrumentation, including GC, gas chromato-
graphy/mass spectrometry (GC/MS), and high performance liquid
chromatography (HPLC), can be used to analyze samples collected
on-site and transported to the laboratory. This instrumentation
usually will provide the most reliable compound-specific data,
but does not provide real-time or almost real-time data.
Decisions to Implement management practices to control ambient
air emissions, to halt remedial activities, to evacuate the
receptor population, or to continue with the remedial activities
may be made based on this type of data, depending on the concen-
tration level and the data turnaround time. These types of
laboratory analyses may be used to confirm results from other
methods discussed above.
Modeling may also be used to make decisions at Superfund sites, as when
source measurements and meteorological data are used 1n a model to predict
receptor concentration. Final selection of which sampling, analysis, and
modeling methods to use should only be made after considering all of the
issues raised in Stages I and II of the DQO process.
1.4 USE AND ORGANIZATION OF THIS DOCUMENT
This document should aid the Remedial Project Managers, Enforcement
Project Managers, and the EPA Regional and Superfund contractor personnel
responsible for ambient air sampling and analysis at Superfund sites.
Specifically, the intent of this document is to serve as a bridge between DQO
guidance documents and the site-specific DQOs that are required for ambient
air monitoring at Superfund sites.
-------
The reader should be familiar with the following:
the QAMS guidance: Development of Data Quality Objectives,
Description of Stages I and II
the two volumes by the Office of Solid Waste and Emergency Response:
Data Quality Objectives for Remedial Response Activities
the two DQO papers prepared by OAQPS: Data Quality Objectives for the
Toxic Air Monitoring Systems and Data Quality Objectives for the Urban
Air Toxic Monitoring Program
the four volumes of Procedures for Conducting Air Pathway Analyses for
Superfund Applications
The remainder of the document is organized as follows. Sections 2.0 and
3.0 present Stages I and II, respectively, of the DQO process for the hypo-
thetical site. The last subsection in Section 3.0 1s a summary of the DQOs
for this site. Appendix A is a statistical discussion oh false positive and
false negative errors 1n terms of the DQO process. Appendix B is a glossary
of Superfund program acronyms, relevant statistical terms, and terms from the
DQO process. '
-------
-------
SECTION 2.0
STAGE I: PRELIMINARY DEFINITION OF THE DECISION
During Stage I of the DQO process, the decision maker provides a prelimi-
nary definition of the decision. The definition includes a description of the
problem which in this case is a Superfund site, identification of the decision
makers and key data users and their roles 1n this program, the decision(s) and
associated actions, information and measurement data requirements and uses,
consequences of making an incorrect decision, a ranking of the seriousness of
the errors, and the resources available for the program.
2.1 PROBLEM DESCRIPTION
This hypothetical site, Superdump, 1s approximately 180 acres and is
located on the outskirts of an urban area. Figure 1 is a map of the site.
Approximately 20,000 drums containing wastes (mainly solvents) from local
industries were buried on the site from 1959 until 1980. Some of the drums
have leaked, and contamination of soil and groundwater is possible. Leakage
or accidental spills may also create an ambient air problem from vaporization,
especially during the summer months when daytime temperatures are frequently
above 90*F. The drums contain mainly spent solvents and have RCRA waste codes
of F001, F002, F003, F004, F005, and F006, according to site records. The
possibility that additional, undocumented compounds may be present at the site
must also be considered. Most of the drums were buried at a depth of about 2
meters and in a single layer. The 3-acre area marked "A" 1n Figure 1 desig-
nates where the drums were buried. The area marked "B" has a State-operated
air monitoring station which has been operating for a number of years. Its
location adjacent to Superdump is coincidental, but historical meteorological
data such as wind direction, wind speed, rainfall, and temperature will be
useful 1n designing the monitoring network. Any chemical measurements made at
the station will be of limited usefulness in detecting emissions from the site
because the station is not upwind or downwind of the site. Table 2 provides
relevant site Information.
A Remedial Investigation/Feasibility Study (RI/FS) for Superdump and
plans for remediation have been completed. The remedial action selected
following the RI/FS is to dig up the buried drums and, after stabilization
8
-------
SITE BOUNDARY
N
NOT TO SCALE
Figure 1. Superdump site map.
-------
TABLE 2. SUPERDUMP SITE INFORMATION
Site size
Contaminated area
VOC concentration 1n soil
VOC concentration 1n groundwater
Site terrain
Nearest downwind receptor
Average wind speed
Prevailing wind direction
180 acres
3 acres
0 to 500 parts per million
0 to 30 parts per million
Level.
100 meters
2 meters/second
From the northeast quadrant in
winter and from the southwest
quadrant in summer.
through use of overpacking, transport them to a nearby hazardous waste
facility. This is expected to take 12 months of on-site activity, based on an
8-hour workday, five days a week.
Even though the main medium of concern at this site is groundwater, there
is also concern for the potential release of VOCs to ambient air. During the
RI/FS, drums were disturbed and an unusual odor was detected at one of the
neighboring houses. An FID measurement indicated that VOCs had been released
into the air. The monitoring will be necessary to detect any subsequent
releases to the ambient air which would pose a threat to public health during
remediation. The guidance given in Procedures for Conducting Air Pathway
Analyses for Superfund Applications will be followed in designing the ambient
air quality monitoring system for this site. If the ambient air concentration
level of concern of one or more of the VOCs is exceeded, the decision maker
for the remediation activity must implement the appropriate action for re-
ducing the concentration or evacuating the subject population.
10
-------
There are six houses between 100 to 200 meters of the site. The safety
and health of the residents of these houses and the on-site workers during
,remediation must be assured. For example, the use of protective clothing for
the on-site workers insures their safety, and the implementation of an air
quality monitoring system that will detect with high probability elevated
levels of air pollutants in a timely manner at the receptor sites insures the
safety of the subject population.
2.2 DECISION MAKERS AND KEY DATA USERS
The organizational chart that follows presents the relationships between
the organizations Involved in Superdump during remedial action. Names of
individuals and their functions for each organization are given. The Regional
Management has final authority on the site cleanup and, as such, is the ulti-
mate decision maker in terms of setting the acceptable levels of uncertainty
in answering the question, "Has the remedial response activity resulted in
pollutant concentrations above the level of concern at receptor sites?" The
RPM is the on-site decision maker 1n terms of reviewing1 the monitoring data
and making the decision to either change the monitoring strategy, Implement
emission control procedures, stop the remedial response activity, or evacuate
the population. State on-s1te personnel are to observe the activities for
compliance with agreed-upon procedures and to further insure that the subject
population is properly protected. The Remedial Contractor Site Manager will
coordinate daily activity at the site. Environmental Services Division
personnel will monitor during remedial response activities. The RPM is the
key data user and decision maker for this site.
11
-------
Regional Management
(J. Smith, G. Davis)
Review major decisions by RPM.
State On-Site Personnel
(T. Allison, J. Boyd)
Keep RPM abreast of State
concerns.
Remedial Project Manager (RPM)
(C. Grazwell)
Decision maker.
Remedial Contractor Site
Manager
(I. Bell)
Makes day-to-day decisions
during remedial action.
Environmental Services
Division Personnel
(C. Worth, J. Burton)
Monitors during remedial
action.
2.3 DECISION DEFINITION
The main decision to be made (and the actions to be taken or not taken)
depends on the answer to the question, "Has a remedial response activity at
this site caused ambient air concentrations of VOCs to exceed the acceptable
risk-based concentrations or applicable or relevant and appropriate require-
ments (ARARs)?" If ARARs or the other concentration levels of concern are
exceeded, the decision maker must decide what action to take to protect
the public's health. The action may be:
to institute controls to lower air emissions,
to halt remedial activities, and/or
to evacuate the receptor population.
If the concentration levels are below the levels of concern, the decision maker
will elect to continue with the remedial response activity. Also, in instances
where the measured pollutant concentration increases or decreases, but remains
below the level of concern, the decision maker's action may be to change the
monitoring strategy to one with a higher probability of detecting an exceedance
in the first case or to a more relaxed strategy in the second case.
12
-------
2.4 AVAILABLE RESOURCES
The available budget for air monitoring during remedial action at
Superdump has been tentatively limited to $50,000. If adequate monitoring
cannot be performed for this cost, negotiation for more funds will be conduct-
ed. If additional funds are not available through negotiation, it may be
necessary to proceed with a monitoring system that could lead to a greater risk
of making an Incorrect decision.
Various applicable sampling, analysis, and modeling methods are available.
A portable flame lonization detector (FID) is available for instances where
measurement results are needed Immediately and when the level of concern that
will prompt action 1s in terms of a total hydrocarbon concentration instead of
a specific compound. If the decision requires compound-specific concentra-
tions, gas chromatography (GC), gas chromatography/mass spectrometry (GC/MS),
or high performance liquid chromatography (HPLC) may be chosen during the
design of the monitoring system. If quick turnaround 1s important, portable
and transportable GCs are available. In an emergency situation where pollutant
levels are suspected of being greatly above the levels of concern, a quick and
inexpensive method like a color!metric detection tube may be adequate. For
example, GC/MS air analysis with 48-hour turnaround could cost five times what
a 2-week turnaround would cost.
For Superdump, Environmental Services Division (ESD) personnel have access
to a mobile laboratory equipped with a generator and containing GCs with a-
variety of columns and detectors and portable FIDs. If appropriate, prior to
the start of remedial activities, colorimetric detector tubes can be purchased.
Also, arrangements with an off-site laboratory for sample analysis by GC/MS can
be made if deemed necessary. GC/MS analysis at the nearest off-site laboratory
has a 48-hour turnaround time. Real-time meteorological data are available
from the State air monitoring station.
Modeling may be used to logically determine the placement of monitoring
stations around Superdump and to calculate worst-case exposures at the receptor
sites based on analytical measurements taken either where the digging occurs or
at the fenceline of the site. Different levels of sophistication exist for
modeling as well as for sampling and analysis. These levels range from screen-
ing models, which assume that the contamination plume will travel directly to
13
-------
the receptor, to refined models that consider other factors, such as meteorolo-
gical conditions and variations in source strength. Modeling combined with
results from sampling and analysis may be used to characterize the site and
surrounding area.
Two technicians are available for operation of the ambient air monitoring
equipment. Additional equipment and manpower can be made available if needed,
but only at increased cost. Depending on the magnitude of additional equipment
and manpower needed, the budget may need to be increased from two to five
times. All of these details will be resolved in Stage III of the DQO process
when the monitoring system is designed. For now, it is necessary only to
gather information on available methodologies and instrumentation for analysis
of the'pollutants of interest. If an acceptable analytical method is not
available for a pollutant of interest, method development may be necessary or a
surrogate measurement, such as total VOCs, may be identified for monitoring.
2.5 INFORMATION REQUIREMENTS
At this pointr the definition of required information need not be stated
in the form of the variables to be measured (e.g., particulate matter or VOC
concentrations), but in more general terms, such as information on potential
receptors near the site, concentrations of air toxics, ARARs, and health effect
information on specific pollutants at the site. It is appropriate to begin to
consider what specific aspect of ambient air quality at the Superfund site will
bring about any particular action and what areas of ground surface and inter-
vals of time are of interest. Types of information needed to support the
design of the monitoring system are:
Information on specific pollutants present at the site (their identity,
concentration range, health effects, and concentration levels of
concern). As mentioned in section 2.1, the drums at this site contain
mainly solvents. The RCRA waste codes are F001, F002, F003, F004,
F005, and F006. The chemicals in these wastes and their air toxicity
constants are listed in Table 4 in section 3.2.
Data from monitoring of the undisturbed site (meteorological and
ambient air quality data). During the RI/FS, an FID measurement
indicated that VOCs were released when the drums were disturbed. No
compound-specific measurements were made at that time. The average
wind speed for this site is 2 meters/second and the prevailing direc-
tion is from the northeast quadrant in the winter and from the south-
west quadrant in summer.
14
-------
n°««"« !°c1i'. «
-------
scheduling of the remedial operation for a day when the wind is
forecast to be blowing away from the houses of concern,
the remedial activity to be halted and the affected population
evacuated, or
the population to be evacuated until the remedial activity is
completed.
Management practices could consist of such extreme actions as building a shed
over the site with a vacuum system that collects the air in the shed and pumps
it through an activated charcoal system before releasing it to the ambient air.
Where evaporation from soil contaminated by leakage from the drum is the
primary problem, a less costly action would be to spray the ground with foam
for short term relief. If form dissipates during strong winds, for example,
reapplication could be costly. If preliminary monitoring indicates that there
may be a problem (I.e., concentrations near the ARARs or other levels of con-
cern), more frequent or more intense monitoring (more samplers or different
types of monitors) may be deployed for the duration of the remedial action to
increase the probability of detecting an exceedance.
If the monitoring data indicate that the remedial response activity has
not created an ambient air problem, the activity will proceed, provided there
are no problems 1n other media.
2.7 CONSEQUENCES OF MAKING INCORRECT DECISIONS
Incorrect decisions include unnecessarily taking action to reduce pollu-
tion concentrations (a false positive error) or not taking action when an
action should be taken (a false negative error). The types of decision errors
are illustrated in Table 3. For example, if measurement data indicate that one
or more pollutants exceed the concentration levels of concern when in fact they
do not, unnecessary costs associated with taking action to control air emis-
sions may be incurred. There may also be unnecessary costs and inconveniences
associated with evacuating the affected population, and increasing the level of
monitoring (either by using more monitors or a more sophisticated monitoring
technique) for the duration of the remedial action. This type of error, decid-
ing to take unnecessary action, is a false positive error. A more detailed
discussion of false positive and false negative errors is given in Appendix A.
16
-------
TABLE 3. DECISION ERRORS
Take
No Action
Take
Action
Actual Concentration
Less than Level of Concern
No Error
False Positive Error
Actual Concentration
Equal to or Greater
Than Level of Concern
False Negative Error
No Error
If measurement data Indicate that pollutants do not exceed the levels of
concern when 1n fact they do, and remedial activity continues with no action
taken to reduce emissions, there may be detrimental health Impacts. This type
of error, not taking action when action 1s needed, is a false negative error.
The consequences of a false negative error depends on the seriousness of th«
undetected problem. If, for example, the concentration 1s 1n the range associ-
ated with possible health risk and 1t 1s not detected, this 1s serious. If the
concentration 1s at the level which poses an Immediate acute health risk and is
not detected, this 1s very serious. This situation should be detected by the
monitoring system with almost certainty.
The ranking of errors by their seriousness for Superdump is as follows:
ERROR
False Positive:
Concluding there is a
problem when the concen-
tration is well below a
level which poses a
health risk.
False Negative:
Concluding there 1s no
problem when the concen-
tration is in the range
associated with health risk.
False Negative:
Concluding there is no
problem when the concentra-
tion is at or near a level
which poses an immediate
acute risk to health.
LEVEL OF SERIOUSNESS
Least Serious
(Resources used
unnecessarily)
Serious
(Possible public
health problem)
Very Serious
(Likely public
health problem)
ACTION
Halting remedial
response activity or
instituting manage-
ment practices to
control air
emissions.
Taking no action to
reduce/control
emissions or to
protect the
population.
Taking no action to
reduce/control
emissions or to
protect the
population.
17
-------
SECTION 3.0
STAGE II: REFINEMENT OF THE DECISION AND REQUIREMENTS
In Stage I, a 11st of actions for protecting the population through con-
trol of emissions or evacuation has been established, together with an initial
description of the types of environmental information needed to prompt the
decision maker to take these actions. The goal of Stage II is to clarify
these initial statements and define what types of measurement data need to be
collected over what areas of space and intervals of time.
3..1 QUESTIONS TO BE ANSWERED
The decision of whether or not to take an action will be based on the
answer to the following question:
To what extent are remedial activities at Superdump creating an
ambient air pollution problem?
i
This question can be broken down into simpler, manageable questions which can
be answered using monitoring data:
1. Has the concentration level of concern, from an ARAR, health
risk-based calculation, or other criteria, been exceeded in
areas of concern by one or more VOCs (nearby off-site areas
where receptors are located)?
2. If the concentration level of concern has been exceeded in an
area of concern, was the exceedance due entirely to activities
at the site? What portion of the exceedance was due to back-
ground or other pollution sources?
The ambient air monitoring data needed to answer the first question are
the air quality measurement data at the receptor sites or air quality measure-
ments at the fenceline combined with meteorological measurements and/or model
predictions made for the receptor locations. The second set of questions need
to be answered only 1f the concentration level of concern is exceeded at any
of the off-site receptor locations. Environmental data needed to answer this
second set of questions are:
Meteorological data on wind speed and direction during the period
in which the concentration level of concern was exceeded at a
specific receptor site
18
-------
Results of downwind fence!ine and/or off-site monitoring
Results of monitoring upwind of or adjacent to the site and at
the active area of the site
Any anecdotal information on remedial response activity at
Superdump which would explain an increase In downwind concentra-
tion levels
If the level of concern is exceeded at a receptor site, the decision
maker needs to answer the second set of questions before he or she can deter-
mine if on-site emission control actions will appreciably reduce the concen-
tration at the receptor location or if the only recourse is to halt the
remedial response activity to assure the population that Superdump 1s not the
cause of the problem.
3.2 VARIABLES AND DOMAIN
For this Superfund site, the receptor population that might be exposed to
high levels of volatile organlcs in ambient air are the residents of the
nearby houses. These six houses are within about 100 meters of Superdump and
constitute the spatial domain of concern for decision making. For this
example, the workers on-s1te wear respirators and protective clothing adequate
to protect them from excessive volatile organlcs 1n ambient air. The temporal
domain is the 8-hour workday, five days a week, for the 12 months of the
remedial response activity.
Since site records Indicate that RCRA waste codes F001, F002, F003, F004,
F005, and F006 are contained 1n the drums, information on the chemicals in
these wastes (from the RCRA waste code description) and their properties (from
Appendix A of the Superfund Public Health Evaluation Manual) is needed. This
chemical-specific information is summarized 1n Table 4. The information was
reviewed, and the top six chemicals based on vapor pressure, Henry's Law con-
stant, and air toxicity constant were selected as Indicators. These top six
chemicals are a subset of the indicator chemicals selected during the baseline
risk assessment during the remedial investigation phase. These indicator
chemicals were judged to be the most toxic and most volatile ones at the site
and are listed in Table 5.
19
-------
Ul
M
CO
Ul
x
t-
co
>
u
t-4
2
UJ
x
o
u.
o
CO
l-l
K
co
1-4
ee
Ul
o
HI
cc
^
X
^
Ul
_J
CD
t
O
e
>
1- CD
l-l OU
O r- C x-x
M Z C»
X < u E
o»- «-^
(-CO 4fP>
Z u E
os o i ><
M U C
< 0
IT
t- cm
Mf- .-»
U Z CB CD
l-l< O E
x t- e"*.
OCO <*>
HZ W E
o c^
cc o
M U
H£ "^
«£
^
*
<(- 0
_IZ E
(/) 1 CO
v> E
ceo E
!Z ^fJ "^
^J A
X ^
UK
cc D'W
O CO X
eno
H
a a
10 CD
r- o>
CMCM
CM CM
aa
i i
CO O>
CO CM
CO T
CM W
a a
i i
o> a
10
t. OX
r-l-a
fH CM
a a
a a
U.U.
c4> a fH
aaa
i + i
« f- CO
co fH r-»
r~ « CM
fH
a
i i i
1 CO 1
1 CO 1
fH
CM CM tn
aaa
i i i
MT fH CM
» * K
fHCM «4>
CM fH «H
aaa
mar*
CM a in
fHCb fH
CD 10 1^
1 1 1
i0«q a
10 CM eo
i i i
fH CD CO
r-i0 a
fH
9
C
^
^C
49 k
O
0
k JC
41 e
JC 1.
U 49 N
41 C
U4>
iW -O
l^ «*i
i e o
.1-1 O U
-.0 o
fHU O
,,4
a
iiii
1 fH 1 1
1 CD 1 1
01
fH
a
iiii
i i i <>
iiia
fH
m «>
a a
iiii
1 cq 1 r»
i a i fH
fH fH
Ma CM fH
aa a a
aar~a
CM * CO eo
*H «H ^ 10
IIII
co a o» a
*H to co a
iiii
CD 10 10 O
r*. CB r-. r-
e
c
c
JE e
49 JC 41
f9 4^ J
0 0 4»
i (. c E
O 0 0 0
i. 3 N U t.
* 3
X JS JC
U I. Off O
» f»v ^ £ »
C 1 O U L.
kf» CM «^f> O iW»
^K T£1f »^f» 9 ^
1 «JC 1
CMCM U JC CM
« 1 U «
fH OH fH
vH
a
i i i i i i i i i
i i i a i i i i i
1 1 1 fH 1 1 1 1 1
,-i
i i i i i i i i i
i i i i i i i i i
i t i i i i i i i
10 (f)
a a
i i t i i i i i i
1 CD 1 CI 1 1 1 1 1
i a i «r i i i i i
CM «~
fH CM a
aa a
a a i a i i i i i
a r> i a i i i i i
tHCM r-
fH CD Hf fH m fH
iii ii i
V CO fH S CO CO
i CD !> «r i fH n i to
i i i i i i i i i
i r^fH a i CD fH i i>.
CD T a a r- co
fH fH fH
^
C
o
49
0
JC
XO
9 9 49 J
4» C 3 O C
I.J3 U 0
4» N « O C
C J= 41
U 49 X
C XJC C
CO X-t» 0 41
*> x x xjc a j:
X
a
a
u.
«H
a
+ i
r- i
CO 1
to
l i
i i
i i
Co
a
i i
a i
fH 1
.4
fH fH
aa
1 1
aa
TU>
CM fH
<4> If)
1 1
C-UJ
r-co
i i
Coco
fHCO
w
fH
^j
^
U
41
U
^
X
CI
t-
u c
H N
c
J»
o o
» t.
fc.
oz
<»
a
a
u.
CM CM a CM
aaa a
ii + ii*
a 10 v i i co
CM r* CM 1 1 fH
u> r- T in
CM
a
i i i i i i
1 1 1 1 1 fH
t I \ t t t^
^
n» 10 CM m
aaa a
i i i i i i
h> «r ra i i co
a i a CM
cor-co i a 10
CM r«- n> CMO»
w «o a fHfH CM
i i i i i i
eo » 10 w CD «
CO CO fH CO CO »
i i i i i i
co co 10 CD a 1-4
fH H
O
e
4> O
9 ^
je
_i
X 3
JE
4* O
O-D C
C C 49 C
XO 3-0
3 JC J3 J3 M
O X
1-2 WMCCffi
10
a
a
u.
i i
i i
i i
i i
i i
i i
i i
i i
i i
i i
i i
i i
i i
i i
^
o
c
41 C
49 O.
O
u
XO.
0 <-
III ^p
UJ Z
CMCM
CM fH CM
a a a
+ + t + 1
o»a i r- i
10 10 i 10 i
<4> CM fH
THCM a
aa a
+ + i + i
10 fH | |0 |
CDfH 1 CO 1 .
*
«H,H CM
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
.*
L.
aa aa>
o> «i» « a u»
i i i i i
«o r» t^ CM CM
f«-«c
^- ^r MT f 10
b- »> »» ^r
t» r» i^ i^
«M
0
<*
O
4> -U
e 9 4>
3 ««4- II
Ct<4- C
O Ul O
> K U
Ul - U
U X
X U C 49
L» « 0
o
ex
X 41 0
O 49
U *» L
e x
9 U »
3 O 0 *»
0. 1.Z
T3 t. (. «
COOL
3 »-«t. 41
*- a.
L 41 E
4> 49 0
a. o
300
C X XJ3
49 < O 0 0
a. c
4> Ul X X
o o o -o
e «->-i
O 3
V O
tH eo 10 x
*- III*
49 II
C 4> 4» 49 C
) 4. ,> » A
^ ^ ^ vQ Ot
u _ ^ .. o
-C-C-C C
XXX »^
C UIUI Ul U
u
-.«-«
u
XXXI.
«^ o
II TJ-OTJ ««-
c c c
a. ex a. c
> «<
E E E «i
o o o c
Ul 1- L l_ O
(- U. U. U. U
o
Z
(0
o
a>
c
M
^
E
M
«
u
^
c
1
t
1
1
20
-------
TABLE 5. EXPOSURE LIMITS FOR SELECTED INDICATOR CHEMICALS
Chemical
Permissible Exposure
Limit (PEL)
Short-Term
Exposure Limit (STEL)
Benzene
1 ppm for 8-hour time weighted 5 ppm for 15 minutes
average
Trichloroethylene 50 ppm for 8-hour time weighted 200 ppm for 15 minutes
average
B
Carbon
tetrachloride
2 ppm for 8-hour time weighted
average
1,1,2-
Trichloroethane 10 ppm for 8-hour time weighted
average
Carbon disulfide 4 ppm for 8-hour time weighted 12 ppm for 15 minutes
average
B
Methyl ethyl ketone 200 ppm for 8-hour time
weighted average
300 ppm for 15 minutes
B
ppm * parts per million
A
29 Code of Federal Regulations 1910.1028, Subpart Z.
D
Federal Register, 54, Thursday, January 19, 1989, pages 2332-2983.
indicates no short-term exposure limit.
21
-------
3.3 SUMMARY RESULTS
The summary results to be used 1n decision making depend on how the level
of concern is expressed. For instance, if the level of concern is expressed
as an 8-hour average, 8-hour averages will be used for decision making.
Information on permissible exposure limits (PELs) and short-term exposure
limits (STELs) for the six indicator chemicals are also presented in Table 5.
There are no ARARs for these chemicals at this site. Toxicity information
from the Superfund Public Health Evaluation Manual is not applicable to this
scenario because it is for chronic exposure* whereas during remediation, the
concern is for short-term exposure. If the excavation site is left open after
the 8:00 a.m. shift, vapors could still be released. In that event, TLVs and
STELs would not be appropriate. The exposure limits in Table 5 are from the
Occupational Safety and Health Administration (OSHA) regulations. Since these
regulations are designed for protecting workers' safety and health on an 8-
hour basis, they do not represent the decision criterion for this site, except
for the on-site workers. Workers are healthy adults, in general, but the
population hear Superdump may include children (with lower body weights),
elderly, and sick individuals. To take into account the more sensitive popu-
lation and to err on the side of safety as well, it was decided to use 1/10 of
the 8-hour PELs for making daily decisions. In addition, if an STEL is
reached, immediate action will be necessary since these are 15-minute exposure
limits for worker safety and health. Thus, there are two allowable exposure
limit summary statistics for each of the selected indicator chemicals: 8-hour
time weighted averages and 15-minute averages.
3.4 NEED FOR NEW DATA
All of the environmental data identified as needed to answer the ques-
tions in section 3.1 have to be new data to represent the temporal domain of
interest. That is, since it is expected that the emission rate from Superdump
will vary with time, and thus, ambient air pollutant concentrations will vary
in time and space, real-time (or near real-time) monitoring data will be
needed by the decision maker to effectively protect the subject population
during the remedial response activity.
22
-------
3.5 ACCEPTABLE ERROR RATES
For each of the six compounds, there are two levels of concern at the
receptors: the short-term exposure limit and 1/10 of the permissible exposure
limit. Based on consideration of the consequences of making decision errors,
the RPM established the following as acceptable error rates for this site
(Note: general statements of acceptable error rates are followed by specific
examples using benzene):
At a true average concentration of 1/2 of either concentration
level of concern, the probability of a positive finding should be
limited to less than 10%; that is, at least 90% of the time,' the
data would correctly indicate that there is no problem.
For example, if the true benzene concentration is 2.5 ppm for a
15-minute average, there should be no more than a 10% chance of
the monitoring/modeling information Indicating a concentration of
5 ppm or more (that is, the probability of a false positive error
is 10% or less).
When the true average concentration is 1 1/2 times either level
of concern, the probability of negative findings should be
limited to less than 5%; that is, at least 95% of the time, the
monitoring data would correctly indicate that there is a problem.
For example, 1f the true benzene concentration is 150 ppb for an
8-hour average, there should be no more than a 5% chance of
obtaining a determination of 100 ppb or less (that 1s, the proba-
bility of a false negative error is 5% or less).
When the true average concentration is 2 times either level of
concern, the probability of a negative finding should be limited
to less than 1%; that is, at least 99% of the time, the monitor-
ing data would correctly indicate that there is a problem.
For instance, if the true benzene concentration is 200 ppb for an
8-hour average, there should be no more than a 1% chance of
obtaining a determination of 100 ppb or less (that is, the
probability of a false negative error is 1% or less).
Based on the above error rates as specified by the decision maker and the
exposure limits given 1n Table 5, compound-specific DQOs are given in Table 5.
Figure 2 is a graphical representation of the DQOs from Table 6 and, as
described above, for short-term exposure to benzene which has a STEL of 5 ppm.
The straight lines forming rectangles show the DQOs while the S-shaped curve
shows how one sampling and analysis strategy is expected to perform. The
graph shows that for actual concentrations between 0 and 2.5 ppm (1/2 of the
STEL), the probability of taking action (the probability of a false positive)
23
-------
V)
UJ
>
H
H
Ul
-5
GO
0
>
K
M
_l
<
3
or
<
t-
<
O
u
tH
u.
M
k>
UJ
Q.
^ 4> MB
JO >
4» 4> ja
Q.
o»-o
uu. o
0 2k
< CL
c«*
o o
4>
fl 1
k > **N
4> C
e k
U X U
e e
OC« O
U ^"* U
X
4>
<» i)
JB >
0 »
J^ 4* ja
O.
9 9 CttjQ
UU, O
W 2k
< 0.
^^
c
k
e u
o c
o
4» U
X
k ^
4» O
C CM
0 vl
C >
O vl
*-
X
4>
^ 4)
ja >
4» «4>J3
WJS
uu. o o
u Q. k
< 0.
c ^
e o
43 ^
0 0
k >x-t
4» e
e k
4) 9
O M U
C *v^ C
0 VI 0
^J ^^ U
4>
k
a 3
H^O
X
^
e
3
o
CL
E
o
o
XX
vl vl
EJ3
a a.
a. a.
09 Q
vl 09
M
KM
IO IO
E-a
a. CL
a o.
IO
^ (&
10
vl
XX
s s
E rt
a. a.
O. Q.
10
CMQ
IO
E
.!?"
o"i
w
3
0
JJ
|
CD
ip
e
4)
N
C
9
GO
XX
E E
a. a.
a a
s s
^
XX
to to
E E
a. a.
a a
to
S F*
Q)
m
XX
s s
E E
C. CL
o, a.
to
B d
B
VI
E
JO
ol
W 9
3
O
^C
1
C 00
^
X
^
4^
9
o
k
o
^
JC
u
k
H-
-
1 X
1 VI
1
ja
GL
i a
i
1 B
^
^
1 K
1 to
1
j3
GL
a
i
i
i
a
Q
m
1 K
n
CL
CL
1
1
1
Q
.14
E
"
o"i
W 9
9 3
o o
* ..C
k i
O 00
^
u
4
k
^
0
4>
c
o
J3
k
^
W
1 K
1 VI
1
.E
CL
1 CL
1
1 CM
1 M
1 10
1
E
CL
CL
1
1 IO
I
1 K
1 S
.0
Cu.
CL
1
1
Q^
Q
IO
E
!?
o*E
V) 9
C 3
O
J= .C
CD
O
k
0
^
JE
u
k
^»
1
CM
94
«
vl
XX
vl vl
E.O
CLCL
CL CL
^ o
CMS
00
XX
IO IO
£.0
CL CL
CLCL
CO S
vl SI
' «P
XX
s a
E ja
CL CL
CL CL
4D 09
Q^
CM
E
JQ
o'i
3
O
^g
1
00
^
<
%.
3
d
*o
c
o
J3
k
0
U
XX
vl vl
E E
CL a
CL CL
S «
0
MM
10 IO
E E
'0. CL
CL CL
09
10 «a
^ (Q
K K
s e»
E E
CL CL
CL CL
09 09
10 vl
VI
E
J3
k
0 E
Vi 9
3
0
C -C
0 1
4> 00
0
Jg
^
X
JC
4>
9
_
X
J=
4>
0
a
^
«4>
Q **
9
e u
0
. 0
4> U
U
JO
4»
o
e
""^
M3
_l 3
UJ O
Ck W
M
jc o
4> O^
0
"o e
OD k
«09
a«s. i
> vl 4>
k k
9-00
« > C J=
4»
C k _l «.
3U(
E Ov- 3
1 JCIO JE
IO 1 4>
vl OD
k k4> 4T
0 O
k
0 9 ^
k k
m m c 9
k k
x-*^« 3
_l _l e o
UJ Ul O CL
1- Ou 0 X
O E
4> k
"iii?
~ ^ > 4*
k
o
k JC
k 3
3 «JC
O 4* O
o a c
x 9 9 m
9 4*
4>
E
k J3 U
*~ d *
4J tl TJ
i »-c e
4» 4»
k E
e k k i
JC 0 1
(O Q. U. 1
UJ
H
o
-------
9
C
a
i-
100X
90Z-
80S-
2468
15 Minute DWUAM Concvntnitiofi« ppcn
12
Figure 2. Short-term exposure DQOs for benzene at Superdump.
25
-------
should be 10% or less. Between 7.5 (1 1/2 times the STEL) and 10 ppm (2 times
the STEL), the probability of taking action (the probability of a true
positive) should be 95% or more (thus, the probability of a false negative
error is 5% or less). Finally, above 10 ppm, the probability of taking action
should be 99% or more (thus, the probability of a false negative is 1% or
less). The performance curve characterizes a sampling and analysis strategy
which achieves or surpasses these DQOs across the entire range of
concentrations.
In Stage III of the DQO process, a number of alternative monitoring/
modeling strategies will be considered, producing graphs similar to Figure 2.
Acceptable strategies are only those which are expected to achieve the DQOs
and at the same time do not exceed resource limitations.
3.6 SUMMARY OF THE DATA QUALITY OBJECTIVES
DEFINE THE DECISION
i
description of decision
The decision 1s whether action must be taken to protect the public
from VOCs in ambient air during remedial response activities.
background for the problem
The site contains 2,000 buried drums with RCRA waste codes of F001,
F002, F003, F004, F005, and F006. The remedial response activity will
be to dig up the drums, overpack them, and transport them to a
hazardous waste facility.
actions under consideration 1f a problem is found
Any of the following actions may be considered singularly or in combi-
nation with other actions:
- institute controls to lower air emissions
- halt remedial activity
- evacuate the receptor population which is at risk
- employ a more rigorous monitoring strategy
26
-------
domain of the decision
The spatial domain for this decision 1s the ambient air at the
receptor locations (six houses) and the ambient air within the site
boundaries (for on-slte workers). The temporal domain 1s the 8-hour
workday, five days a week, for the year of the remedial response
activity. Unless vapors from excavation are controlled when work 1s
not 1n progress, the above scenario 1s Inappropriate. To make sure
vapors are not being emitted during non-working hours, monitoring may
be required.
Information needs
The decision will be based on the determination of short-term (15-
mlnute) and 8-hour average concentrations of contaminants In ambient
air. The contaminants of concern and their exposure limits will be
selected based on Information on toxldty, the waste types expected,
and potentially sensitive receptors.
DEFINE THE USE OF DATA
elements of decision which require data
The elements for Superdump are the compounds of Interest. They are
benzene; trichloroethylene; carbon tetrachlorlde; 1,1,2-trichloro-
ethane; carbon dlsulflde; and methyl ethyl ketone concentrations in
ambient air. These elements are quantifiable and data-dependent. The
temporal domain is the entire time the site is active (normally 8
hours per day, 5 days per week) broken Into 15-minute segments for
short-term exposure measurements.
use of data-dependent elements
If measurement data show that the concentration of any one contaminant
of concern exceeds the STEL, then action must be taken to reduce the
VOC emissions. In addition, more rigorous monitoring will be required
to determine the extent of any off-site exceedance of the STEL. Such
an exceedance may be cause for evacuation.
If the highest average 8-hour exposure at any receptor is above the
level of concern (1/10 the PEL), then action must be taken to reduce
the VOC emissions.
27
-------
Since there are usually so many compounds, the levels of all compounds
added together may cause adverse health effects. ACGIH uses the
following formula to determine action levels for on-site personnel.
If q/TLVj + C2/TLV2 + ...Cn/TLVn ^ 1
(where Cn is the concentration of the nth component and TLVn is the
threshold limit value of the nth component), action is required.
If level < 1, then no action is taken.
RANKING OF ERRORS ACCORDING TO SERIOUSNESS OF THEIR CONSEQUENCES
The ranking of decision errors by their seriousness is as follows:
Concluding there is a problem when the concentration is well below
the level which poses a health risk is the least serious error.
Concluding there is no problem when the concentration is in the
range associated with a health risk is serious.
Concluding there is no problem when the concentration is in the
range associated with immediate acute health problems is a very
serious error.
RESTATE THE DECISION
the decision
The decision is whether actions must be taken to protect the public
from VOCs in ambient air during remedial response activities.
decision elements
Data needs: 15-minute and 8-hour average measurements of selected
target contaminants in ambient air.
Spatial domain: need to measure directly, or predict from measure-
ments using models, the concentrations of contaminants at off-site
areas where receptors are located. Data on background concentrations
will be needed if concentration thresholds are found to be exceeded.
Temporal domain: measurements will span the active period of each
working day (typically 8 hours). When the site is inactive, concen-
trations are assumed to be less because the site will be left in a
28
-------
secure state. (No disturbed drums will be left exposed, and any spil-
lage will be recovered prior to shutting down for the day, however, to
substantiate hypothesis, monitoring may still be needed.)
FORMALIZE THE DECISION PROCESS
decision rule
If either exposure limit (STEL or 1/10 of PEL) 1s exceeded for any of
the contaminants of concern, Immediate action will be taken to control
the release of volatlles.
role of data
The decision on whether or not to take action will be based on the
ambient air measurement data.
need for new data
Since the site has not yet been remediated, data for the site during
remediation are not available. All of the environmental data will
need to be new data to represent the temporal domain of Interest.
DESIRED PERFORMANCE - ACCEPTABLE PROBABILITY OF FALSE POSITIVE AND NEGATIVE
ERRORS AT SELECTED CONCENTRATION LEVELS
false positive
At a true average concentration of 1/2 of either level of concern
(STEL or 1/10 PEL), the probability of a positive finding should be
limited to less than 10%.
For example, 1f the true benzene concentration 1s 2.5 ppm for a 15-
minute average, there should be no more than a 10% chance of obtaining
a determination of 5 ppm or morel
false negatives
When the true average concentration is 1.5 times either level of
concern, the probability of a negative finding should be limited to
less than 5%.
For example, 1f the true benzene concentration 1s 150 ppb for an 8-
hour average, there should be no more than a 5% chance of obtaining
a determination of 100 ppb or less.
29
-------
When the true average concentration is 2 times either level of
concern, the probability of a negative finding should be limited to
less than 1%.
For example, 1f the real benzene concentration 1s 200 ppb for an 8-
hour average, there should be no more than a 1% chance of obtaining a
determination of 100 ppb or less.
INITIAL MONITORING SYSTEM DESIGN CONSIDERATIONS
turnaround time
Short-term measurements need to be available within 15 minutes. The
8-hour averages need to be available within 48 hours following the end
of the sampling period.
preventive maintenance / spare equipment
Spare equipment should be available so that no period of time passes
without monitoring activity. If equipment 1s not available and
operational, remedial activity will not be allowed to proceed.
limit of detection
Measurement methods should have detection limits below 1/10 the con-
centration of concern for each compound. This applies to concentra-
tions of concern for both short-term and 8-hour monitoring.
monitoring flexibility
Based on the actual compounds present and their concentration levels,
there should be several monitoring strategies ranging from periodic
screenings to on-site mobile or fixed monitoring stations with docu-
mented criteria for moving from one strategy to another.
use of models
The most cost-effective strategy may be a combination of monitoring
and modeling.
sampling error
The spatial and temporal domains for this site must both be assumed to
be heterogeneous. The VOC concentrations will vary with time and with
location around the site. This heterogeneity may introduce the greatest
amount of uncertainty to the decision making.
30
-------
measurement errors
For the measurement methods discussed 1n section 2.4, the range of
measurement error is from 1.1 to 2.0, expressed as uncertainty factors,
according to Tables 3-4 and 3-5 in Volume IV of Procedures for
Conducting Air Pathway Analyses for Superfund Applications.
31
-------
REFERENCES
U.S. Environmental Protection Agency. Superfund Public Health Evaluation
Manual. EPA/540/1-86/060, OSWER Directive 9285.4-1.
U.S. Environmental Protection Agency. Development of Data Quality Objectives,
Description of Stages I and II. Quality Assurance Management Staff, July 16,
1986.
U.S. Environmental Protection Agency. Data Quality Objectives for Remedial
Response Activities. Volume I, Development Process". EPA 540/6-87/003, March
1987.
U.S. Environmental Protection Agency. Data Quality Objectives for Remedial
Response Activities, Volume II, RI/FS Activities at a Site with Contaminate!
Soils and Ground Water. EPA 540/6-87/004, March 1987.
U.S. Environmental Protection Agency. Data Quality Objectives for the Toxic
A1r Monitoring Program (Stages I and Il7"i Office of Air Quality Planning and
Standards, December 1987.
U.S. Environmental Protection Agency. Data Quality Objectives for the Urban
Air Toxic Monitoring Program (Stages I and II).Office of Air Quality
Planning and Standards, June 6, 1988.
U.S. Environmental Protection Agency. Procedures for Conducting A1r Pathway
Analyses for Superfund Applications, Volume I. Application of Air Pathway
Analyses for Superfund. December 1988 draft.
U.S. Environmental Protection Agency. Procedures for Conducting Air Pathway
Analyses for Superfund Applications, Volume II. Procedures for Developing
Baseline Emissions from Landfills aind Lagoons. EPA-450/1-89-002, January
1989.
U.S. Environmental Protection Agency. Procedures for Conducting Air Pathway
Analyses for Superfund Applications, Volume III, Procedures for Estimating Air
Emissions Impacts from Remedial Activities at NPL Sites.EPA-450/1-89-003,
January 1989.
U.S. Environmental Protection Agency. Procedures for Conducting Air Pathway
Analyses for Superfund Applications. Volume IV, Procedures for Dispersion
Modeling and Air Monitoring for Superfund Air Pathway Analyses, December 1988
draft.
Data Quality Objectives Training Software Version 6.5, December 1988. Quality
Assurance Management Staff, U.S. Environmental Protection Agency.
32
-------
APPENDIX A
A DISCUSSION OF FALSE POSITIVE AND FALSE NEGATIVE ERRORS
-------
Environmental decisions are often framed in terms of taking or not taking
some action which is expected to help protect public health or the environ-
ment. An incorrect decision will usually result in either taking an action
which is not warranted or not taking an action which is warranted. If taking
an action is regarded as positive and not taking an action 1s negative, then
the incorrect decisions are called false positives and false negatives,
respectively. One consequence of a false positive, taking unnecessary action,
could be that the pollution control process is made too strict and/or too
costly. A consequence of a false negative, not taking the needed action,
could be that the environment does not receive the protection 1t needs and, as
a result, 1s either placed at risk or damaged.
For monitoring around Superfund sites, the greatest concern is almost
always for the false negative because the consequences (people are placed at
risk or the environment 1s damaged) are much more serious than those of the
false positive (halting the remediation or increased cost of controls). The
only case when there may be greater concern for the false positive would be
where an emergency situation exists and unnecessarily halting remediation
would have serious environmental and public health consequences.
The first step 1n evaluating potential decision errors is to list them in
order according to the level of concern for their consequences. This ranking
provides the initial basis for designing a monitoring program. The successful
monitoring program design will provide adequate protection against each type
of decision error.
Next, the technical staff and the decision maker must consider how erron-
eous experimental results could prompt the wrong decision to be made. For
each type of error, an error rate will be assigned based on the following:
Relative Importance of the result (answer to the question) in
making the decision, and
Degree of concern for the type of decision error based on
analysis of the consequences.
If a measurement result by itself prompts a decision, the probability of
drawing the wrong conclusion must relate directly to the degree of concern for
decision error.- If the measurement answers but one of many questions which
will together prompt the decision, then somewhat higher probabilities might be
acceptable.
A-l
-------
There are two ways In which the decision can be in error:
1. The null hypothesis 1s rejected, when In fact the null
hypothesis is true (false positive or Type I error), or
2. The null hypothesis is accepted, when In fact the null
hypothesis is untrue (false negative or Type II error).
Whenever a statistical test is done, the probability of the Type I error must
be selected. As part of the testing process, a researcher chooses a signifi-
cance level at which the hypothesis will be tested. If the test statistic,
whatever it might be,- falls outside this level, the null hypothesis will be
rejected. If this occurs when the null hypothesis Is true, it 1s known as a
Type I error or as a "false positive." Commonly, the significance level or
"alpha value" is chosen as 5% (or 1%). This means that if the test statistic
falls outside the range of 95% (or 99%) of Its assumed distribution, th« null
hypothesis is rejected.
Clearly, the possibility of a mistake 1s built Into this system. It is
given by definition that 5% (or 1%) of the time, the test statistic will fall
outside this range when the null hypothesis is true. This probability must be
accepted to test the hypothesis. The probability cannot be reduced to zero.
Therefore, the probability of a Type I error (false positive) 1s set at a
level which the Investigator deems acceptable. The consequences of the Type I
error are often economic ones involving the cost of unnecessarily halting the
remediation or the cost of more stringent pollution control monitoring
practices.
Concern must also focus on the other type of mistake, the "false nega-
tive" or Type II error. For monitoring at Superfund sites, these types of
error are often more serious than the Type I error. For example, if monitor-
ing data fail to detect a problem when a pollutant is actually present at
levels which cause a serious health risk, then the decision maker would not
know that some kind of protective action 1s needed. The consequence is that
the population remains at risk.
In the case of false positives, the probability is set before any stati-
stical test 1s chosen, or even before data are gathered. Unfortunately, the
probability of occurrence of the Type II error is not so easily manipulated.
This is because while there is only one null hypothesis, there are an infinite
A-2
-------
number of possible alternative hypotheses.' In the false negative case, what
Is really sought 1s the probability that some or all of the alternative situ-
ations will be missed by the test.
Knowing the probability of missing all the possible alternative hypo-
theses is Impossible. Therefore, the technique used 1s to pick one specific
alternative hypothesis, or a set of them, which the researcher would not like
to reject if they were true. For example, if a specific level of concern 1s
exceeded by a negligible amount, say 1%, 1t may be permissible to miss it. If
the level of concern 1s exceeded by a somewhat larger amount, the correct
action should be taken with higher certainty. If the ARAR or other level of
concern 1s exceeded by a very large amount (e.g., by a factor of 2), the deci-
sion maker should be very nearly certain of taking the correct action. One
way to express these desires is in terms of power of the design. The power of
a design is defined as 1 minus the probability of a Type II error. Thus, 1f a
design has 95% power when the level of concern 1s exceeded by a factor of 2,
then we would expect to correctly conclude there is a significant pollution
problem 19 out of 20 times when the level of concern actually is exceeded by a
factor of Z.
The relationship between Type I and Type II errors is graphically illus-
trated in Figure A-l on two different scales. The graph has curves which
depict two normal (i.e., Gaussian or "bell-shaped") distributions. The left-
most curve represents a set of environmental data obtained by measuring 1000
samples taken from a parcel of ambient air with a pollutant concentration of
100. Due to random error in sampling and analysis, there is a spread in the
measured values. Most of the values fall near 100 with fewer values occurring
as the distance from 100 gets larger. The dispersion of these values is the
variability of the measurement error in our sample collection and analysis.
The leftmost curve can be mathematically described by its mean, 100, and its
standard deviation, 1.5. The rightmost curve has the same dispersion
(standard deviation = 1.5), but a mean of 107. Suppose the null hypothesis to
be tested 1s that the pollutant concentration 1s 100. We plan to take some
kind of pollutant management action If the data show that the true concentra-
tion Is greater than 100. The leftmost curve shows that measurement error can
give us results greater than 100 even when the true concentration of the
A-3
-------
Nor-mal d i «.!-1 bu«. 1 on« W 1 fcn Equal Var-lanoat
and Ot^fr«i-«nt M«an« Std Oav 1. S
X
4»
c
a
x
n
JJ
0
J)
0
Shown Expanded
belo
ea
1 12
sz i oo i oa lie
Cef-ie«n«.r*a«.tati of Compoui-id X
I
0
a
x
4>
.0
0
JJ
0
1
Probability of
I error
Probability
Type II
err
102. s
103. S
10-t. S
1O2 1O3 1O4
Cono«i-i«.r-a«. t on e-P Compound X
1OS
Figure A-l. Illustration of the relationship between
Type I and Type II errors.
A-4
-------
ambient air 1s exactly 100. But we would not want measurement error to cause
us to take action when action 1s not needed, so a higher value, called an
action threshold, 1s set so that measurement error has a small probability of
causing action when the null hypothesis is true. There will be a 5% proba-
bility of causing this Type I error (false positive) if we set the threshold
at 103. The probability of the Type I error is shown as the small shaded area
to the right of 103 above the graph's X axis. A smaller probability for the
Type I error can be achieved by moving the action threshold further to the
right or by improving the precision of the measurements.
Now, suppose the true situation 1s that the mean is equal to 107. This
produces the rightmost curve in Figure A-l. The very small shaded area to the
left of 103 shows the area of the true distribution which falls below the
critical value of X=103. These values of the test statistic will not lead to
a rejection of the null hypothesis, even though it should be rejected. This
shaded area is the probability of a Type II error. The remaining area under
the righthand curve is 1 minus the probability of a Type II error, or the
power of the test.
Usually the probability of a Type I error 1s prescribed and is Indepen-
dent of the experimental design, but the Type II error can only be determined
after the conditions of the experiment are clearly defined. Thus, a specific
statistical test must be chosen, with a certain number of samples, a given
null hypothesis, a given alternative hypothesis, and certain parameters of the
4
population under study (such as means and variances). These items will be
Important in Stage III, when the monitoring system 1s designed and assessed.
At this point, 1t 1s sufficient to remember that for any given experimental
design and alternative hypothesis, a certain power exists, and that by chang-
ing certain conditions of the experiment (e.g., sample size or significance
level of the test), power can be adjusted.to achieve the desired level,
although the cost of the experiment may be intolerable.
A-5
-------
-------
APPENDIX B
GLOSSARY
-------
-------
GLOSSARY
ARARs - Applicable or relevant and appropriate requirements.
Domain - The spatial and temporal definition of the environment which is
subject to the decision and action. The purpose of sampling and analysis is
to characterize the domain.
DQIs - Data quality indicators. These include precision, accuracy, represent-
ativeness, completeness, comparability, and method detection limits.
DQOs - Data quality objectives. DQOs are statements of the level of uncer-
tainty which the decision maker is willing to accept in the results derived
from environmental measurements. For projects which are planned to test
specific hypotheses (such as whether a pollutant concentration exceeds a
standard), this uncertainty is expressed in terms of the probability of making
Type I and Type II errors. (Type I error is the rejection of a null hypothe-
sis which is actually true, and Type II error is the failure to reject a null
hypothesis which is actually false.) For projects which are to produce inter-
val estimates, this uncertainty 1s expressed as a desired confidence Interval
width.
EPA - U.S. Environmental Protection Agency.
EPMs - Enforcement Project Managers.
FIDs - Flame lonization detectors.
FS - Feasibility study.
GC - Gas chromatograpny.
GC/MS - Gas chromatography/mass spectrometry.
B-l
-------
HPLC - High performance liquid chromatography,
Hypothesis - A hypothesis is a statement about a particular population para-
meter. Spatial areas and temporal intervals to be Included must be defined.
Some value for comparison must be included. A hypothesis is tested by measur-
ing the actual value of the parameter (the statistic) in the system of
interest, and computing the probability of this measured value occurring if
the hypothesis is true. If the probability of observing the measured
statistic is very low, given that the hypothesis 1s true, then the hypothesis
is rejected.
Questions are generated to refine or test a model of the system under
study. In order to be tested statistically, general questions about the model
must be reduced to hypotheses that can be stated in terms of a specific
statistic, which 1s some function of the collected data. Each statistical
test requires a null hypothesis. Given the null hypothesis, the statistic
chosen is then examined to see 1f its value is probable or Improbable,
assuming the hypothesis 1s true. If the value 1s probable, the null hypothe-
sis is accepted. If the value is Improbable, the null hypothesis is rejected,
and some other model 1s assumed to describe the population. If our model is
the simplest one (for example, that the variable of Interest 1s normally dis-
tributed), then the null hypothesis tested would concern the mean and variance
and how well the data fit a normal distribution. If the model involves rela-
tionships between variables, a testable null hypothesis is that the cor-
relation coefficient 1s equal to 0.
The goal of the planning process is to develop one or more such statisti-
cal hypotheses for which data will be collected to test. Based on the results
of the hypothesis tests or parameter estimations, decisions will be made. The
design of a data collection program that most efficiently supports the
decision requires an early statement of these hypotheses.
Model - A description of the interrelationship between variables in the parti-
cular system being studied. It describes, in greater or lesser detail, how
the dependent variable(s) react to changes 1n the independent variables. This
model, if it is to work well and lead to increased understanding, should
B-2
-------
reflect the cause-and-effect relationships among the variables. In the early
phases of research, however, models may only show correlations, not neces-
sarily causes.
Observation - Also called a data point, a measurement, or a value. The number
derived from a measurement of a particular variable at a certain point 1n
space and time. A data base 1s a set of observations of one or more variables
over space and/or time.
OSHA - Occupational Safety and Health Administration.
Parameter - A number derived from a set of observations according to some
rule. The number may correspond to an actual observation or to a mathematical
function which combines the observations in some way. The term "parameter"
can be used to refer to the function Itself, such as the mean or the maximum,
or to the unknowable, "true" value of the parameter as estimated from a parti-
cular set of observations (see statistic).
PARCC - Precision, accuracy, representativeness, completeness, and compara-
bility.
PEL - Permissible exposure limit.
PIDs - Photoionization detectors.
Power - The probability of detecting a departure from the null hypothesis.
When the null hypothesis is true, power is the probability of Incorrectly
rejecting 1t. This 1s a Type I error. When the null hypothesis 1s false, the
term "1 minus power" is the probability of a false negative. This is a Type
II error.
QA - Quality assurance. A system of activities whose purpose 1s to provide
adequate confidence that a product or service will satisfy given needs.
QAPjP - Quality assurance project plan.
B-3
-------
QC - Quality control. The operational techniques and the activities which are
aimed at maintaining a product or service at a level of quality that will
satisfy given needs.
RCRA - Resource Conservation and Recovery Act.
RI - Remedial investigation.
RPHs - Remedial Project Managers.
Significance - One minus the probability of incorrectly rejecting a null
hypothesis which is true. Many statistical tests are applied at the 95%
significance level, meaning that the probability of a false positive is only
5% (1 in 20) when the null hypothesis is true.
Statistic - The calculated value of a population parameter from a particular
set of observations. This 1s calculated according to the appropriate rule,
given the spatial and temporal limits of the observations to be Included.
Since 1t is calculated from real-world measurements, it will Incorporate some
error due to both sampling and measurement. Thus, a statistic is an estimate
of a parameter.
Two uses can be made of a given statistic: it can be computed to esti-
mate the value of a population parameter (parameter estimation), or it can be
tested to see if it meets some pre-determined criteria (hypothesis testing).
In practice, parameter estimation and hypothesis testing are slightly
different ways of drawing conclusions from the same procedures, depending on
the purpose. In this document, the main emphasis is on hypothesis tests, but
the same principles can be applied to parameter estimation problems.
STEL - Short-term exposure limit.
B-4
-------
Type I Error or False Positive - The type of error which is made whenever a
true null hypothesis is rejected. The probability of this type of error is
called the significance level of a test.
Type II Error or False Negative - The type of error which 1s made whenever a
false null hypothesis 1s not rejected. One minus the probability of a Type II
error is power.
Variable - A characteristic of an object or system under study. Examples are
length, weight, temperature, concentration of a certain chemical, etc. A
variable can be thought of as the property that 1s quantified (to a greater or
lesser extent) by a measurement system.
Variables are often classified as Independent (or experimental) and
dependent (or response) variables. A dependent variable 1s one which is of
interest as a direct or Indirect Indicator of some process or effect. An
independent variable 1s one which 1s manipulated by the experimenter in a
controlled situation or one that 1s presumed to have ari effect on the
dependent^variable. The point of a research or monitoring program is to
obtain some measure of this effect on the dependent variable.
VOC - Volatile organic compound.
B-5
-------
TECHNICAL REPORT DATA
(Please read Instructions on the reverse before completing)
1. REPORT NO.
2.
3. RfcCIPIENTS ACCESSION NO.
4. TITLE AND SUBTITLE
2uidance for Applying the Data Quality Objective Process
"or Ambient Air Monitoring Around Superfund Sites (Stages
I and II)
5. REPORT DATE
August 1989
6. PERFORMING ORGANIZATION CODE
7. AUTHOR(S)
8. PERFORMING ORGANIZATION REPORT NO.
9. PERFORMING ORGANIZATION NAME AND ADDRESS
10. PROGRAM ELEMENT NO.
Research Triangle Institute
Research Triangle Park, N.C.
27709
11. CONTRACT/GRANT NO.
68-02-4550
12. SPONSORING AGENCY NAME AND ADDRESS
Dffice of Air Quality Planning and Standards
RTP, N.C. 27711
13. TYPE OF REPORT AND PERIOD COVERED
14. SPONSORING AGENCY CODE
15. SUPPLEMENTARY NOTES
16. ABSTRACT
This document provides direction and assistance on how the data quality objectives
process is used to design an ambient air monitoirng system around a Superfund site.
Examples are given for Stage I (preliminary decision) and Stage II (refinement of the
decision and requirements) for monitoring during remedial action at a hypothetical
Superfund site.
17.
KEY WORDS AND DOCUMENT ANALYSIS
DESCRIPTORS
b.lDENTIFIERS/OPEN ENDED TERMS C. COSATI Field/Group
.Ambient Monitoring
Superfund Air Pathways
Data Quality Objectives
DQO Process
Superfund Monitoring
Superfund Air
Pathways
18. DISTRIBUTION STATEMENT
EPA Form 2220-1 (R«v. 4-77) PREVIOUS EDITION is OBSOLETE
19. SECURITY CLASS (This Report)
21. NO. OF PAGES
3 £.
20. SECURITY CLASS (Tins page}
22 PRICE
------- |