United States
Environmental Protection
Agency
Office of Solid Waste and
Emergency Response
(5102G)
EPA 542-R-01-014
October 2001
www.epa.gov
www.clu-in.org
V EPA Current Perspectives in Site Remediation and Monitoring
CLARIFYING DQO TERMINOLOGY USAGE TO SUPPORT
MODERNIZATION OF SITE CLEANUP PRACTICE
D. M. Crumbling1
Introduction
The appropriate use of field analytical tech-
nologies and dynamic work plans could
dramatically improve the cost-effectiveness of
environmental restoration activities. EPA's
Technology Innovation Office (TIO) has been
developing classroom and Internet web-based
courses to promote adoption of these tools
and strategies. TIO's experience has been that
a common language to unambiguously com-
municate technical concepts is vital if regula-
tors, stakeholders, and practitioners are to
negotiate, plan, and implement these projects
to their mutual satisfaction.
Systematic planning is critical to the success-
ful implementation of hazardous site charac-
terization and cleanup projects. EPA's "DQO
process" has been around for many years, and
the "DQO" terminology is used extensively.
Unfortunately, over the years the terminology
has been used in ambiguous or contradictory
ways, and this has resulted in confusion about
what terms mean and how they are to be used.
It is thus useful to clarify the relationship
between DQO-related terms as descriptively
and concretely as possible. The discussion
provided here has been reviewed by the
primary DQO and data quality coordinators
within the EPA Headquarters offices of the
Office of Solid Waste, the Office of Emergen-
cy and Remedial Response, the Office of
Environmental Information, and the Quality
Staff to ensure that the concepts presented are
consistent with EPA's original intent for DQO
terminology and with the direction program
needs are currently taking. Any questions or
comments about this paper should be directed
to the EPA Technology Innovation Office
through the Clu-In "Comments" form (http://
cluin.org/gbook.cfm) or to (703) 603-9910.
This paper does not intend to provide all-
inclusive definitions that can be found else-
where in EPA guidances, nor does it attempt
to provide all-inclusive coverage of each
topic. It is intended to provide, as briefly yet
unambiguously as possible, a basic conceptual
understanding of DQO-related terms in a way
that facilitates systematic project planning
in the context of site cleanups. A list of
descriptions for DQO-related terms and
concepts appears first in this paper, followed
by a more intensive discussion of the working
interrelationships between these concepts. It
is entirely possible that other parties use terms
other than these to communicate the same
concepts. The actual terms used are less
important than the ability of parties involved
in site cleanup projects to have a "meeting of
minds" and clearly communicate the con-
cepts, since the concepts are basic to the
scientific validity of environmental decisions
and to the data that support those decisions. A
common conceptual framework could help all
within the hazardous waste community better
communicate our goals and results, fostering
more cost-effective planning and implemen-
tation of projects.
EPA, Technology Innovation Office
-------
Descriptions for DQO-Related Terms
Data Quality Objectives (DQO) Process
This is a systematic, iterative, and flexible planning
process based on the scientific method. The DQO
process was developed by EPA to provide a common
structure and terminology to practitioners designing
environmental data generation operations. The DQO
process produces quantitative and/or qualitative state-
ments (called "the DQOs," see below) that express the
project-specific decision goals. The DQOs then are used
to guide the design of sampling and analysis plans that
will be able to cost-effectively produce the "right kind of
data."
An important part of the DQO process is developing an
understanding of how uncertainties can impact the
decision-making process. A systematic planning process,
such as the DQO process, identifies what the goals are
and what the consequences may be if the decisions are
made in error. It is within the realm of values (not
science) for decision-makers (as representatives of
society as a whole) to estimate how certain (i.e.,
confident) they want to be before making decisions that
will either impact, or be impacted by, environmental
conditions. When technically feasible, an expression of
statistical certainty may be desirable because it can be
more "objective" (if it is done in a technically valid
manner). But in the environmental field, mathematical
(e.g., statistical) treatment of "uncertainty" may not
always be technically feasible or even necessary.
Qualitative expressions of decision confidence through
the exercise of professional judgment (such as a "weight
of evidence" approach) may well be sufficient, and in
some cases, may be the only option available. An
important part of systematic planning is identifying the
information gaps that could cause a decision to be made
in error. If the existence of information gaps increases
the likelihood of decision error beyond what it
acceptable, then it may be desirable to fill those gaps, if
it is feasible to do so. Planning how to gathering
environmental data that can acceptably fill information
gaps is the purpose of the DQO process. Decision-
makers should also keep in mind that, on occasional,
systematic planning may indicate that it may be more
cost-effective to simply go ahead and make the decision
to take the most conservative (protective) action, rather
than spend the resources neededto scientifically "prove"
whether the protective action is absolutely necessary or
not.
Sampling and analysis plans lay out the strategy to be
used to gather needed data. Steps 1 through 6 of the
DQO process provide the structure to help a project
team articulate their project goals and decisions, the
project's constraints (time, budget, etc.), and how much
uncertainty they can tolerate in the final decision. These
things must be thoroughly understood before the task of
developing the data-gathering plans that can meet those
goals within the given constraints is begun. Developing
project-specific sampling (i.e., determining the number
of samples, their locations, their volume, etc.) and
analysis (i.e., selecting and modifying, as needed, the
analytical preparation, cleanup, and determinative
methods, the analytical QA/QC protocols, etc.) plans is
the very last step (Step 7"optimize the design") of the
DQO process.
During Step 7, pre-existing site information should be
sought and evaluated so that uncertainties that could
impact the sampling and analysis plan can be evaluated
as much as possible prior to finalizing the plan. For
example, existing information about the mechanism(s)
of contaminant(s) distribution and their likely environ-
mental fate (degradation and/or redistribution in the
environment) can be used to develop a conceptual model
for the variability of contaminant concentrations and the
media that should be sampled. Knowledge or suspicion
that other contaminants may be present in the samples
can guide consideration of alternate analytical methods
able to cope with any analytical interferences that might
arise. [More details about the development of sampling
and analytical plans can be found in the article,
"Guidelines for Preparing SAPs Using Systematic
Planning and PBMS" (Jan/Feb 2001 issue of
Environmental Testing & Analysis; also available as a
pdf file on http://cluin.org/charl_edu.cfm#syst_plan).]
It should be noted that the DQO process is a systematic
planning process focused on generating project data.
The term "systematic planning" is often used to encom-
pass the broad range of project activities that includes
more than just data generation and interpretation
activities. (See "Systematic Planning" below.) [More
thorough discussions of DQOs and details of the DQO
process can be found in various EPA Quality Assurance
documents available on EPA's Quality Staff website:
http://www.epa.gov/quality/qa_docs.html]
Data Quality Objectives
DQOs are qualitative and quantitative statements that
translate non-technical project goals into technical
project-specific decision goals. Project planners derive
these technical DQOs from the non-technical social,
-------
economic, and/or regulatory objectives of the environ-
mental program under which the project is being
implemented. DQOs are goal-oriented statements that
establish the (technical) "bar" for overall decision
quality or tolerable decision error in accordance with
the (non-technical) objectives driving the project.
The DQOs for any particular proj ect may, or may not, be
highly specific in naming target elements, target media,
and action levels along with the intended uses of the
data. Project DQOs may be articulated at different
technical levels depending on the intended audience. For
communication with public stakeholders or in non-
technical settings, DQOs will usually be summarized as
simple, less technical statements. For communication
between technical practitioners, proj ect DQOs should be
articulated in as specific and technically-detailed manner
as possible to avoid ambiguities that could cause
confusion or misunderstandings. In either case, DQOs
summarize the outputs of the DQO (planning) process.
Example statements below provide a flavor of simply
worded DQO statements that summarize the highly
technical statements that would lie behind the simply
worded summaries and give them substance. The most
important point to note is that in no case do DQO
statements directly set criteria for the quality of data
that will be gathered during implementation of the
project. The process of determining the quality of data
that will be needed to meet the project decision goals
(i.e., "meet the DQOs") must be done after the DQOs
are established. Quantitative DQOs express decision
goals using numbers, such as quantitative expressions of
decision certainty. Qualitative DQOs express decisions
goals without specifying those goals in a quantitative
manner.
Example of a less detailed, quantitative DQO:
Determine with greater than 95% confidence that
contaminated surface soil will not pose a human
exposure hazard.
Example of a more detailed, quantitative DQO:
Determine to a 90% degree of statistical certainty
whether or not the concentration of mercury in each
bin of soil is less than 96 ppm.
Example of a detailed, qualitative DQO: Determine the
proper disposition of each bin of soil in real-time using
a dynamic work plan and a field method able to turn-
around lead (Pb) results on the soil samples within 2
hours of sample collection.
Even when expressed in technical terms, DQOs should
express "what" (i.e., what decision) the data will
ultimately support, but should not specify "how" that
data will be generated (e.g., which analytical methods
are to be used). Despite the name, "Data Quality
Objectives," DQOs should be thought of as statements
that express the project objectives (or decisions) that the
data (and its associated quality) will be expected to
support. As project objectives, DQOs serve to guide the
eventual determination of the data quality that is needed
to make good decisions, but DQOs themselves should
not attempt to directly define the specifics of that
data quality. Doing so short-circuits the systematic
planning process, hindering the ability of project
planners to optimize data collection designs to make
projects more cost-effective (Step 7 of the DQO
process). Various terms have been used that more
intuitively express the originally intended concept of
"DQO," including "Decision Quality Objectives";
"Decision Confidence Objectives (DCOsused in the
context of compliance monitoringWTQA 2001 short
course, "Regulation Writing under PBMS"); and
"Project Quality Objectives (PQOsused by EPA
Region 1 in their Quality Assurance Project Plan
Guidance)."
A discussion of the analytical flexibility inherent to U.S.
EPA's waste programs and to SW-846, the methods
manual used by these programs, is found in the paper,
Current Perspectives in Site Remediation and
Monitoring: The Relationship between SW-846, PBMS
and Innovative Analytical Technologies [document
number EPA 542-R-01-015; available on http://cluin.
org/tiopersp/].
Data Quality
Data quality is a term that tends to be rather vaguely
understood in the environmental community, despite its
importance to the decision-making process. In addition,
the term "data" is used to refer to many different kinds
of information that is derived from very different kinds
of data generation procedures. In the contextofthe DQO
process, "data" generally refers to the measurement of
some physical or chemical environmental property. Of
greatest concern to the management of hazardous waste
and contaminated sites is the measurement of toxic (or
potentially toxic) chemicals in environmental media to
which receptors maybe exposed. In this context, "good"
data quality tends to be linked in many minds with using
the most sensitive or precise analysis procedures
available. However, this view of data quality produces
problems because the information value of that kind of
data is limited not so much by the analytical procedures
used (although that is certainly possible), but by the
-------
difficult task of ensuring representative sampling in
heterogeneous environmental matrices.
Fortunately, EPA has recently clarified its intended
meaning for the term "data quality" in its broadest sense
by defining it as "the totality of features and characteris-
tics of data that bear on its ability to meet the stated or
implied needs and expectations of the customer " (i.e.,
the data user). [This definition appears in the 2000
version of the Office of Environmental Information's
Quality Management Plan, entitledManagement System
for Quality} Recent EPA guidance reinforces this
understanding of data quality by stating that "...data
quality, as a concept, is meaningful only when it relates
to the intended use of the data. Data quality does not
exist in a vacuum; one must know in what context a data
set is to be used in order to establish a relevant yardstick
for judging whether or not the data set is adequate"
[from page 0-1 of Guidance for Data Quality Assess-
ment: Practical Methods for Data Analysis (QA/G-9
QAOO Update). EPA 600/R-96/084; http://www.epa.gov/
quality/qs-docs/g9-final .pdfj.
Linking data quality directly to the data's intended use
provides a firm foundation for building a vocabulary that
distinguishes the various components of data quality.
For example, since analytical data are generated from
samples, pre-analytical considerations (such as sample
representativeness and sample integrity) are crucial
when determining whether data are of sufficient quality
to meet the user's need to make correct decisions. Data
quality can be broken broadly into the components of
analytical quality (how reliable is the analytical
procedure) and representativeness (selection of the
samples and of the analytical method is appropriate to
the intended use of the data). Non-representative sample
selection produces "bad" data (misleading or meaning-
less information), even if the analytical quality on those
samples was perfect.
Data Quality Indicators
DQIs are qualitative and quantitative measures of data
quality "attributes." Quality attributes are the descriptors
(i.e., the words) used to express various properties of
analytical data. DQIs are the measures of the individual
data characteristics (the quality attributes) that
collectively tend to be grouped under the general term
"analytical data quality." For instance, the data quality
attribute of analytical sensitivity can by measured by
different DQIs, such as instrument detection limit,
sample detection limit, or quantification limit, each of
which can be defined somewhat differently depending
on the program or laboratory. See EPA QA/G-5 (1998
version) for more discussion on the topic of DQI (http://
www.epa.gov/quality/qs-docs/g5-final.pdf). Another
guidance document, EPA/G-5i, will explicitly discuss
DQIs in much greater detail. EPA/G-5i is currently
under development. Look for a peer-review draft to be
posted in the future at http://www.epa.gov/quality/qa_
docs.html]
Quality attributes (and the facets of data quality that
they describe) include (but are not limited to) the
following:
Selectivity/specificity (describes what analytes the
technique can "see" and discriminate from other
target analytes or from similar-behaving, but non-
target, substances);
Sensitivity [depending whether "detection" or
"quantification" is specified, describes the lowest
concentration, or increment of concentration, that
the technique is able to detect (although quantifica-
tion may be highly uncertain) or quantitate with
greater confidence];
Bias (describes whether the technique produces
results with a predictable deviation from the "true"
value);
Precision (describes how much random error there
is in the measurement process or how reproducible
the technique is);
Completeness (describes whether valid data is
produced for all the submitted samples, or just some
fraction thereof); and
Comparability (describes whether two data sets can
be considered to be equivalent with respect to a
common goal).
The familiar "PARCC parameters" have been
considered to consist of 5 principal DQIs that include
measures of precision, accuracy (used in this context to
denote bias), representativeness, comparability, and
completeness. Sensitivity ("S") may also be included as
a principal DQI. Precision, bias, and sensitivity describe
properties that are measured quantitatively through an
appropriate analytical quality control (QC) program.
Comparability between data sets generated by different
analytical methods can also be established through the
use of relevant QC samples, such as standardized
performance evaluation (PE) or certified reference
-------
material (CRM) samples run by both methods, in
addition to other comparisons of each method's
performance (sensitivity, selectivity, precision, bias,
etc.).
The term "representativeness" can be used to address
either the analytical aspect or the sampling aspect of
sample analysis. Analytical methods must be selected
and designed to be representative of the parameter of
interest. Positive (i.e., causing an analytical result to be
biased high) or negative (i.e., causing an analytical result
to be biased low) interferences and unrecognized non-
selectivity for a particular target analyte can result in a
non-representative interpretation of analytical results,
leading to decision errors. For example, immunoassay
tests for environmental contaminants are usually
designed to give results that are biased high, and the kits
frequently cross-react with daughter products of the
parent contaminant or other structurally similar
compounds. A potential user of an immunoassay kit who
does not recognize these characteristics will risk serious
misinterpretation of the kit's test results. On the other
hand, users who do understand this will seek to use these
characteristics to their advantage, or will manage the
inherent uncertainties through a demonstration of
method applicability (see below) and an appropriate
quality control protocol.
The representativeness of sample selection and
collection is complicated by the extreme heterogeneity
of many of the matrices encountered in the environmen-
tal field. The concentrations of contaminants in soils,
sediments, waste streams, and other matrices can vary
tremendously on even small scales in both space and
time. Samples must be representative of the "true" site
conditions in the context of the decision to be made
based on those samples. If the decision is not specified,
a representative sampling design cannot be selected.
Sample representativeness also includes sample
preservation and subsampling issues.
Comparability and representativeness are critically
important to the scientifically valid interpretation of
analytical data, but estimating both requires the exercise
of professional judgment in BOTH the science genera-
ting the data (e.g., analytical chemistry) and in the
science involved in interpreting and using the data (e.g.,
using the data to model contaminant extent or migration
or to design a treatment system).
As noted above, there may be more than one DQI for a
single data quality attribute. For example, the attribute
of precision can be measured using mathematical
formulas for relative percent difference (RPD), relative
standard deviation (RSD), standard deviation (SD),
variance (SD2), and a variety of other calculations that
can quantitatively express the degree of random fluctua-
tion in a measurement process. The selection of a
particular DQI to measure a specific data quality
attribute (for example, selecting RPD to measure
precision) is a matter of:
Convention (what are people used to seeing or
using);
The characteristics of the analytical method (for
example, does the method generate continuous or
discontinuous data?);
The data set being evaluated (for example, the
formula for RPD cannot handle more than 2 values,
whereas the formula for RSD can handle multiple
values); or
The intended use for the data (which determines
how extensively the quality of a data set must be
documented, and what form of documentation is
most useful to the data user).
The language of "data quality attributes" and "data
quality indicators" provides data generators and data
users with the ability to establish the comparability of
different data sets and whether data are of "known and
documented quality" commensurate with intended data
use.
Measurement Quality Objectives
MQOs are project-specific analytical parameters derived
from project-specific DQOs. MQOs include acceptance
criteria for the data quality indicators (DQIssee
above) that are important to the project, such as
sensitivity (e.g., what detection or quantitation limit is
desired), selectivity (i.e, what analytes are to be
targeted), analytical precision, etc. MQOs can be used
to establish the "bar" for data performance para-
meters. MQOs are derived by considering the level of
analytical performance needed to actually achieve the
project goals (as expressed in the DQOs).
However, project MQOs are not intended to be
technology- or method-specific. As with DQOs, MQOs
specify "what" the level of data performance should be,
but not "how" that level of data performance will be
achieved. In other words, although MQOs provide the
criteria for how good the data must be, MQOs do not
-------
specify exactly how the data must be produced, and so
MQOs do not specify what analytical method or
technology is to be used.
In actual practice during project planning, the planning
team's analytical chemist will naturally be considering
which specific technologies may be applicable even in
the early stages of project planning. Evaluating and
refining analytical options is a significant part of the
iterative nature of systematic planning which seeks the
most resource-effective work strategy that can achieve
the stated project goals (i.e., the project DQOs). The
project chemist should explore whether available
innovative analytical technologies might achieve the
project MQOs (i.e., the needed data quality) to the same
degree as the conventional technology, yet be able to do
so in a way that is more resource-effective for the
project because of lower per-sample costs, economies of
scale, or more rapid turnaround times that could support
real-time project decision-making.
The following are examples of what MQOs "look like":
An MQO for one project might read: "The overall
precision of lead measurements taken on the soil in
the bins must be less than 50% RPD when at least
10 samples are taken from each bin."
An MQO for a different project might read: "The
measurement method to be chosen must be able to
detect the presence of compounds X, Y, and Z in
groundwater at a quantitation limit of 10 Fg/L with
a recovery range of 80-120% and a precision of
<20%RSD."
A large part of the variability in environmental data (and
thus in overall decision uncertainty) stems from
sampling considerations. MQOs should be developed
with this fact in mind, and requirements for analytical
MQOs should be derived in conjunction with the
development of the sampling design. The team or
individual setting the MQOs should balance the relative
contributions from analytical uncertainties and from
sampling uncertainties. In many environmental media
(especially solid media), matrix heterogeneity causes
sampling variability to overwhelm analytical variability.
Insisting on perfectly precise analyses on a few tiny
samples taken across a large heterogeneous matrix is
meaningless since two adjacent samples will probably
provide very different results. Which sample is selected,
and hence the project decision influenced by that
sample's results, will be a matter of chance. This "luck
of the draw" can only be controlled by obtaining a better
understanding of the contaminant distribution, and that
is dependent on increasing the density of sample
collection.
Depending on how the term is being applied and the
sources of uncertainty that impact an environmental
decision, measurement quality objectives may be
interpreted to include assessment of the performance of
the entire measurement system, including the uncertain-
ties in the data introduced by sampling. This is
especially true if there are more sources of uncertainty
in making the actual decision than just evaluating the
immediate data package. For example, making risk
management decisions is based not only on site-specific
data sets, but also on the non-site-specific toxicological
data sets used to derive the various reference values,
etc., all of which have their own associated uncertain-
ties. However, in some usage, the term MQO is
restricted to the analytical side of the measurement
process, and the broader concept of DQO or decision
confidence objective (DCO) is used to include the
sampling considerations. This terminology may be used
in activities such as permit compliance monitoring
where there is no perceived "uncertainty" in the
regulatory limit itself (once it has been established by
the permit). In this case, the "project decision" involves
demonstrating only that the permitted material is in
compliance to some specified level of decision
confidence. If usage of terminology such as DQO,
MQO, DCO, etc. in a particular situation is
ambiguous (as many times it is), parties should strive
to clarify what meaning is intended. Parties should
also strive to clarify how sampling uncertainties are
accounted for in data generation, assessment, and
interpretation.
Whether sampling considerations are evaluated as part
of MQOs (as the entire measurement system) or as part
of DQOs (or some other term expressing the overall
decision uncertainty), the importance of including the
sampling component when assessing overall data quality
cannot be overemphasized. It is possible to isolate the
performance of various parts of the measurement
system, and to determine the relative contributions from
the various sampling components versus the various
analytical components. [Discussions about the
partitioning of decision uncertainty can be found in
various statistical or sampling documents available on
http://cluin.org/chartext_edu.htm#stats. Since soils tend
to illustrate a "worst case scenario" for non-gaseous
environmental media, the following documents present
valuable guiding principles: the 1990 A Rationale for the
Assessment of Errors in the Sampling of Soils, and the
-------
1989 Soil Sampling Quality Assurance User's Guide.
This topic is also discussed in the paper, "Applying the
Concept of Effective Data to Environmental Analyses
for Contaminated Sites," EPA 542-R-01-013; available
from http://cluin.org/tiopersp/].
Demonstration of Method Proficiency
A Demonstration of Method Proficiency shows that a
particular operator or laboratory has the appropriate
training and equipment to accurately perform a method.
The demonstration may be done by using Performance
Evaluation (PE) samples, or by using known concentra-
tions of analytes spiked into a clean matrix. The purpose
of a demonstration of proficiency is to ensure that the
performance of the operators and equipment is capable
of producing data of known quality. [Proficiency
demonstrations are discussed in Chapter 2 of the SW-
846 Manual, available at http://www.epa.gov/epaoswer/
hazwaste/test/chap2.pdfj
Demonstration of Method Applicability
A Demonstration of Method Applicability involves a
laboratory study, pilot study, field trial, or other kind of
activity that establishes the appropriateness and
performance capability of a particular method for a site-
specific matrix and application. The purpose of a
demonstration of method applicability is to ensure that
a particular method or method modification can produce
data of known quality, able to meet the project's
decision goals, on the site-or project-specific samples to
be tested.
Systematic Planning
Systematic planning for project decision-making is the
process of clearly defining and articulating:
What the goals (i.e., primary decisions) of a project
will be (including how much uncertainty will be
tolerated in those decisions);
Identifying what potential sources of error and
uncertainty could lead to an erroneous decision;
then
Developing strategies to manage each of the
identified uncertainties and avoid decision errors;
and
Planning the most resource-effective means for
implementing those strategies.
Strategies for managing uncertainties include identifying
information or knowledge gaps and deciding how to fill
those gaps. Locating and interpreting historical
information or pre-existing data is one possible way to
fill certain knowledge gaps. Collecting new data is
another way to fill information gaps. Systematic
planning then evaluates:
What types and amounts of data will be needed to
address the information gaps; and
What mix of sampling and analytical technologies
can address both sampling and analytical uncertain-
ties to optimization the data collection design to
maximize overall cost-effectiveness for the project.
Once decisions are made, follow-up actions (such as
remedial activities) may be indicated. Systematic
planning evaluates how data gathering, decision-making,
and follow-up activities may be efficiently ordered or
merged to minimize expensive and time-consuming
remobilizations of staff and equipment back to a site. A
dynamic work plan approach develops decision trees or
other articulations of decision logic that guide real-time
decision-making in the field to allow sequential activi-
ties to be performed in fewer mobilizations. More
information about dynamic work plans can be found in
A Guideline for Dynamic Workplans and Field Ana-
lytics: The Keys to Cost-Effective Site Characterization
and Cleanup, available from http://cluin.org/download/
char/dynwkpln.pdf.
The DQO process is a systematic planning approach that
EPA has articulated to aid data collection activities. The
DQO process does not address other aspects of project
planning that are included under the broader term
"systematic planning." Systematic planning also
includes developing the work plans that will coordinate
and guide site operations related to cleanup, worker
safety, waste removal and disposal, public involvement
and other activities needed to achieve projectgoals. Key
to successful systematic planning is the involvement of
sufficient technical expertise, generally provided
through a multi-disciplinary team, that represents the
scientific and engineering disciplines needed to
adequately address all project issues. For example, the
U.S. Army Corps of Engineers uses a systematic
planning process called Technical Project Planning
(TPP) that encompasses many of the project activities
that extend beyond just data collection. The TPP manual
can be accessed at http://www.usace.army.mil/inet/
usace-docs/eng-manuals/em.htm, referto Engineering]
Mfanual] 200-1-2.
-------
EPA has policy requirements that mandate the use of
systematic planning for all projects performed under
EPA direction. EPA does not mandate the type of
systematic planning to be done, since this necessarily
will vary depending on a wide variety of factors. EPA
policy statements on systematic planning can be found
in Policy and Program Requirements for the Mandatory
Agency-Wide Quality System (EPA Order 5360.1 A2),
available at http://www.epa.gov/quality/qs-docs/5360-
l.pdf
Triad Approach
A strategy for cleaning up hazardous waste sites that
relies on the integration of systematic planning, dynamic
work plans, and real-time results (usually provided
through rapid turnaround on-site measurements) to
reduce costs and move site work along faster while
maintaining or increasing the reliability and protective-
ness of site decisions. [Discussion about the triad app-
roach can be found in the paper, Current Perspectives in
Site Remediation and Monitoring: Using the Triad App-
roach to Improve the Cost-Effectiveness of Hazardous
Waste Cleanups, EPA 542-R-01-016 available from
http://cluin.org/tiopersp/].
The Relationships Among Decision Goals, DQOs,
MQOs, and QC Protocols
During project planning, there should a logical
conceptual progression in the development of decision
goals, DQOs, MQOs, and QC acceptance criteria. In
practice, however, this will be a non-linear, iterative
process where various options for implementing a
project are explored, dissected, and recombined, the
feasibility and costs for various options are estimated
and weighed, and then the most promising option is
selected and fully developed into project work plans that
will actually be implemented. As a project's planning
documents (such as work plans, sampling and analysis
plans, quality assurance project plans, health and safety
plans) are developed and finalized, there should be a
clear presentation of (and the reasoning behind):
The general project decision goals;
The more detailed, technical expression of the
project goals (the DQOs), and the decision rules that
will guide project decision-making;
An expression of how much uncertainty decision-
makers are willing to tolerate in the final project
decisions;
An evaluation of the uncertainties (information
gaps) that could potentially lead to decision errors;
and
A discussion of the strategies that will be used to
manage each of those uncertainties to the degree
needed to needed to accommodate the desired
decision certainty.
No doubt at least one of those strategies will include the
generation of analytical chemistry data from environ-
mental samples to fill information gaps. (In contrast, it
may be possible to manage uncertainty without
generating data simply by "assuming the worst" and
taking the most protective actions. In highly specialized
instances, this might be the most cost-effective strategy
when the cost of sampling and analysis to reach a
"definitive conclusion," and the likelihood that action
will be required anyway are both high.) When data
generation is planned, the planning document should
discuss:
The roles these data are expected to play within the
context of the project or how they will be used to
support project decision-making;
A description of how data will be assessed and
interpreted according to the decision rules (e.g., how
will the results be reduced, treated statistically,
mapped, etc.);
The goals for overall data quality (the overall
MQOs, where "data" are measurements generated
from samples and sampling uncertainties must be
considered);
How the representativeness of sampling will be
ensured or assessed (how the various aspects of
sampling uncertainty will be managed);
A list of the analytical technologies and methods
that were selected, and a description of the data
attributes (analytes, detection/quantitation limits,
requirements for accuracy as bias and precision) that
is expected to be generated from the listed methods;
and
The analytical QC protocols and criteria to be used
with the methods to demonstrate that analytical data
of known quality are being generated that are
suitable for the described intended uses.
At designated completion points in the project, project
-------
reports that summarize work accomplished to date
should clearly reiterate the project goals and the means
by which these goals would be achieved. Important
uncertainties (that is, those information gaps that bear
directly on decision-making confidence) in the decision-
making process should be identified. The success of the
project plan in managing those uncertainties to the
degree desired, and an estimation of the overall decision
uncertainty should be assessed in the project report.
In the beginning of a project, high-level program
managers often set the broad, non-technical goals for
projects: Forexample, "Given a budget of $X, we want
to clean up this lead contaminated soil in accordance
with all environmental regulations and to the satisfaction
of the residents in the neighborhood." The next question,
of course, is "How do we do that?" So the next step for
the project manager or the planning team is to translate
these broad, non-technical goals into more technically
oriented goals that can address specific considerations
such as:
Regulations: What are the applicable environmental
regulations? Are applicable action levels already in
place in regulations, or do site-specific action levels
need to be derived based on risk-drivers? If there is
more than one possible regulatory action level,
which one should be used?
Confidence in the outcome: How certain do we need
to be by the end of the project that we have indeed
achieved goals such as risk reduction or regulatory
compliance? How will we demonstrate to regulatory
agencies or stakeholders that this level of certainty
has in fact been achieved (i.e., what evidence will be
used to argue that goals have been achieved)?
Constraints: What are all the constraints that need to
be accommodated (like seasonal weather, budget,
property access, etc.)?
Making sure that no important details are left out of
consideration is the purpose of a systematic planning
process such as EPA's 7-step DQO process" [Detailed
explanation of the DQO process as applied to hazardous
waste sites can be found in the document, Data Quality
Objectives Process for Hazardous Waste Site Investiga-
tions (QA/G-4HW), available through http://www.epa.
gov/quality/qa_docs.html, and will not be duplicated
here.] Statements that summarize the answers to these
and other questions constitute "the project DQOs." As
noted earlier in this paper, the project DQOs consist of
the unambiguous technical expressions of the overall
project decision goals.
The next level of technical detail geared toward data
collection involves translating the project DQOs into
project MQOs [i.e., a general characterization of the
kind of information (what parameters or analytes need
to be measured, and what level of overall data quality
for those parameters is needed) that will be needed to
achieve the project DQOs]. Analytical data quality is
most often only a very small part of the uncertainty that
needs to be controlled in order to have sufficient
confidence in the actual project decisions. An honest
examination of the "weak" links contributing to overall
decision certainty may reveal that paying for expensive
"definitive" analyses contributes nothing toward
decreasing the overall uncertainty in the project
decisions when there are larger uncertainties due to the
limitations of sampling very heterogeneous media.
Sampling uncertainty is decreased when sampling
density is increased. Composite sampling may
sometimes be used to increase sampling density while
lowering analytical costs. [Refer to EPA Observational
Economy Series Volume 1: Composite Sampling,
EPA/QA G-5S, and other statistical documents, all
available from http://cluin.org/chartext_edu.htm#stats].
Although composite sampling is undesirable in some
situations and its use should be carefully considered in
the context of how the data will be used, composite
sampling can be a highly cost-effective and informative
sampling strategy.
Another way to cost-effectively increase sampling
density is by using less expensive analytical methods
(perhaps, using screening methods) in association with
a well-planned QA/QC design and limited traditional
analyses to provide data of known quality matched to the
decision needs of the project. As long as the data quality
can be demonstrated to be compatible with the project's
decision rules, the confidence in the overall decision
reliability that is gained by increasing the sampling
density will not be lost by the use of a screening method.
For more details, see "Guidelines for Preparing SAPs
Using Systematic Planning and PBMS" in the January/
February 2001 Environmental Testing & Analysis. The
article is available through http://cluin.org/chartext_edu.
htm#planning. Additional discussion can also be found
in the issue paper, Current Perspectives in Site
Remediation and Monitoring: Applying the Concept of
Effective Data to Environmental Analyses for Contam-
inated Sites, available at http://cluin.org/tiopersp/.
When project planners wish to express desired decision
confidence objectively and rigorously in terms of
statistical certainty (that may have been specified in the
project DQOs), statistical expertise is required to
translate that goal into strategies that blend the number
-------
of samples, the expected variability in the matrix (i.e.,
heterogeneity), analytical data quality (e.g., precision,
quantitation limits), the expected contaminant concen-
trations (i.e., how close are they expected to be to
regulatory limits), sampling design (e.g., grab vs.
composite), and costs into an interlocking whole. Since
sampling design and analytical strategy interact to
influence the statistical confidence in final decisions, the
interaction between an analytical chemist, a sampling
expert, and a statistician is key to selecting a final
strategy that can achieve project goals accurately, yet
cost-effectively. Software tools can assist technical
experts to develop sampling and analysis designs. [See
http://cluin.org/cnartext_tech.htm#imp.]
The statistician is concerned with managing the overall
(or summed) variability (i.e., uncertainty) in the final
data set, and with the interpretability of that final data
set with respect to the decisions to be made. The
statistician does this during project planning by
addressing issues related to "sample support" (a concept
that involves ensuring that the volume, shape, and
orientation of extracted specimens are representative of
the original matrix under investigation), by selecting a
statistically valid sampling design, and by estimating
how analytical variability could impact the overall
variability. The field sampling expert is responsible for
implementing the sampling design while managing
contributions to the sampling variability as actual
sample locations are selected and as specimens are
actually collected. The sampling expert does this by
selecting and using sampling tools in ways that ensure
that the sample support designated in the sampling plan
is met in the field. The analytical chemist is responsible
for managing components of variability that stem from
the analytical side (including aspects of sample
preservation, storage, homogenization, subsampling,
analyte extraction, concentration, and instrumental
determinative analysis). The analytical chemist should
select analytical methods that can meet the analytical
variability limits estimated by the statistician, and design
an analytical QC program that defensibly establishes
that those goals were met in the final data set.
Managing the various sources of analytical and sampling
uncertainties (assuming no clerical or data management
errors) ensures that data of known quality are generated.
Sometimes there may be only a single option available
for a certain task, so the selection process is simple.
Other times there may be more two or more options and
cost/efficiency considerations can drive selection of the
equipment and/or the design. It should be obvious that
staff expertise (training and practical experience directly
relevant to the techniques under consideration) is very
important to project success.
The data characteristics that will control analytical and
sampling uncertainty are articulated in the MQOs.
Thus the MQOs specify "how good" the data must be at
a general level. MQOs are contrasted with DQOs, which
specify "how good" the decision must be. DQOs
certainly are the ultimate drivers of how good the data
must be, but DQOs themselves do not directly express
data quality characteristics. Sometimes, as project
planning progresses or as project implementation
proceeds, it is discovered that a DQO is unattainable
given the realities of the site conditions, the availability
of suitable technology, and financial constraints. In
collaboration with regulators and stakeholders, revision
of the project DQOs may be required. For example, it
may be discovered that current technology for a certain
analyte is unable to provide the data needed to support
risk decisions at a desired 10~6 cancer risk level. When
a risk-based DQO is unachievable with current
technology, an MQO known to be achievable with
currently available technology may be substituted for the
DQO. In other words, if it is clear that the ideal decision
goal (the DQO) is unattainable, data quality goals
(MQOs) based on the best available technology may be
substituted for the ideal DQO until a time when newer
technologies become available. It is important to note
that the technology or method itself is NOT specified by
the regulatory MQO. This allows the flexibility required
for market incentives to encourage the development of
technologies that can meet or exceed that same level of
data quality more economically.
Although project MQOs are not meant to specify
particular methods or technologies, they do serve to
guide the selection of the technologies that can most
cost-effectively meet the DQOs. As instrumentation is
selected (based on factors such as the type of data
needed, the turnaround time needed to support project
activities, the expertise and infrastructure required to
operate it, and costs), and as the analytical strategy for
the project is perfected (perhaps including a
"demonstration of method applicability"), analytical
method SOPs and QC protocols are developed that are
both method- and project-specific (i.e., tailoring an
analytical method's performance to meet the specific
data needs of the project). A QC protocol identifies the
analytical parameter or DQI to be controlled, the limits
within which results for that parameter are acceptable,
and the corrective action procedures to be followed if
those acceptance limits are exceeded. QC acceptance
criteria should be very specific and should be designed
such that if the QC acceptance criteria are consistently
met, the project MQOs will be achieved, which means
that the resulting data will be sufficient to meet the
project DQOs and support the project decisions.
10
-------
For example, an overall MQO for precision [for
example, a statistically derived objective of less than
<50% RPD between side-by-side (collocated) samples]
may be partitioned into the primary components of
variability that contribute to the overall variability.
[Discussions about the partitioning of variability can be
found in the Rationale for the Assessment of Errors in
the Sampling of Soils document, available athttp://cluin.
org/chartext_edu.htm#stats .] In the QC protocol, QC
samples are used to monitor and document these
measures of variability. The QC acceptance criteria are
used to specify the maximum allowable variation in each
component, and they might be expressed something like
this:
Analytical (instrumental) precision: "XRF instru-
ment precision shall be determined using no fewer
than 7 replicate analyses of a homogenized sample
with a lead concentration near 400 ppm (the action
level). The resulting RSD should be less than
"20%."
Combined analytical and sample preparation
precision: "Laboratory duplicates (prepared from a
single sample with at least 150 ppm lead) should
have RPDs less than "35%."
Combined analytical, sample preparation, and
sample collection precision: "Field duplicates
(collocated samples collected from a single location
with at least 150 ppm lead, with each sample
collected, prepared, and analyzed separately) should
have RPDs less than "50% (unless matrix hetero-
geneity is demonstrated to exceed the anticipated
variability)."
The figure below serves to illustrate the conceptual
progression that comprises the development of a design
for generating data based on well-defined project goals.
As stated earlier, while conceptually this process is
linear, in real-life, the development of a design is highly
iterative, as portrayed by the circular arrows. The figure
shows that the conceptual progression starts with the
project-specific decision goals, and then moves "down-
hill" from broader, higher level goals to narrower, more
technically detailed articulations of the data quality
needs. Project decisions are translated into project-
specific DQOs; then into project-specific MQOs; then
into the technology/method selection and development
of a method-specific QC protocol that blends the
QA/QC needs of the technology with the project-
specific QA/QC needs of the project. Finally, data are
generated.
Then the process reverses. The actual raw data must
then be assessed against the project MQOs to document
that the quality of the data generated do indeed meet the
decision-making needs of the project. The final step in
the chain is interpreting the data into meaningful
information (such as a statistical expression of a
contaminant concentration as an average across an
exposure unit) that is fed into the decision-making
process (e.g., further action is or is not needed). If the
"downhill" process has been conscientiously followed,
there is a very strong likelihood that the "uphill" process
of data assessment and interpretation will show that the
data are of known and documented quality, and are fully
adequate to support the project decisions.
The DQO Process
tKHbOQCrrotocolOData
so
All data are the
Project Decisions
will to and directly support the
Project Decisions.
11
------- |