Demonstrations of Method Applicability under a Triad Approach for
                         Site Assessment and Cleanup — Technology Bulletin
                                                  August 2008
Since its inception in 1995, the U.S. Environmental
Protection Agency's (EPA) Brownfields Initiative and other
revitalization efforts have grown into major national
programs that have changed the way contaminated
property is perceived, addressed, and managed in the
United States.  In addition, there has been a shift within
EPA and other environmental organizations in the way
hazardous waste sites are cleaned up. Increasingly,
project managers, regulators, technology providers, and
other stakeholders are recognizing the value of
implementing a more dynamic and flexible approach to
site cleanup that focuses on real-time  decision-making in
the field to reduce costs, improve decision certainty, and
expedite site closeout. The approach, known as Triad,
uses (1) systematic project planning, (2) dynamic work
strategies, and (3) real-time measurement technologies
designed to increase confidence in the project (Figure 1).

Figure 1.  The Triad Approach
               Real Time Measurement
                    Technologies
Triad's best management and technical practices have
been successfully implemented in a variety of regulatory
frameworks, including Brownfields, Superfund, the
Resource Conservation and Recovery Act (RCRA),
management of underground storage tanks (UST), and
voluntary cleanup programs. As a result, the EPA
Brownfields and Land Revitalization Technology Support
Center (BTSC) is preparing a series of technical bulletins
to provide additional information on implementing specific
aspects of the Triad approach.  These bulletins are
intended for technical project managers and team
members. Non-technical managers or stakeholders may
also present these bulletins to consultants and service
providers to ensure Triad best management and technical
practices are implemented appropriately at their sites.
These bulletins provide sufficient information for less
technical  project managers and team members to request
that critical Triad project elements be included in scope of
work and planning documents.
Demonstrations of Method Applicability (DMA) are a key
component of using real-time measurement technologies
and are presented in this bulletin through:
1.   Answers to frequently asked questions on key
    aspects of DMAs
2.   Examples of DMAs performed at hazardous waste
    sites:
    •   Wenatchee Tree Fruit, Wenatchee, Washington
    •   Poudre River, Fort Collins, Colorado
    •   Fort  Lewis Small Arms Firing Range, Fort Lewis,
        Washington
3.   Sources of additional information for communities and
    project teams that desire to implement DMAs and the
    Triad approach.


    About the Brownfields and Land Revitalization
         Technology Support Center (BTSC)

  EPA established the BTSC (see
  www.brownfieldstsc.orcj) to ensure that brownfields and
  other land revitalization decision-makers are aware of
  the full range of technologies and technical support
  services available for site assessments and cleanups
  and to help them make informed decisions about their
  sites. The center can help federal, state, local, and tribal
  officials evaluate strategies to streamline the site
  assessment and cleanup process at specific sites;
  identify, review, and communicate information about
  complex technology options; evaluate contractor
  capabilities and recommendations; and plan technology
  demonstrations.  BTSC is coordinated through EPA's
  Office of Superfund Remediation and Technology
  Innovation (OSRTI) and works through  EPA's Office of
  Research and Development (ORD) laboratories. The
  center also works closely with EPA's Office of
  Brownfields  Cleanup and Redevelopment and in
  partnership with the U.S. Army Corps of Engineers
  (USAGE) and Argonne National Laboratory.

  Localities can submit requests for assistance through
  the EPA Regional Brownfields Coordinators, online, or
  by calling 1-877-838-7220 toll free. For more
  information about the BTSC, contact Carlos Pachon at
  (703) 603-9904 or pachon. carlos&.epa. qov.
Office of Solid Waste and
Emergency Response
                                      EPA542-F-08-006
                                          August 2008
                                  www.brownfieldstsc.org

-------
                                                               Demonstrations of Method Applicability under a Triad Approach
What is a DMA?

A DMA is an initial site-specific performance evaluation for
a wide range of sampling, testing, and data management
tools. It is a concept founded in EPA SW-846 guidance
(www. epa. Qov/epaoswer/hazwaste/test/sw846. html) and
is based on the principles of EPA's performance-based
measurement system (PBMS) initiative
(www.epa.Qov/SW-846/pbms.htm}. A DMA usually falls
into one of two categories: (1) a comparison of a field-
based method  with a more established laboratory-based
method to demonstrate the usefulness of the field-based
method, or (2)  a test to evaluate whether a particular tool
will work on a specific site. Both types of DMAs may be
needed at a single site, and the exact format of the DMA
will depend heavily on the site characteristics, the history
of investigations at the site, and the intended use of the
data.

The DMA serves several different purposes for many
applications, including showing whether a technology will
be effective at the intended site, but also to optimize how it
will be used collaboratively with other information sources
at the site.  DMA data are of particular importance relative
to understanding the potential effects of matrix
heterogeneity and sample support on data quality.  During
the DMA, the types and frequency of  quality assurance
and quality control (QA/QC) procedures are often tested
for adequacy, and preliminary field-based action  levels are
developed for comparison with site decision criteria.
Methods for data sharing and management are also
tested to assure a project can proceed in real-time.

In addition, a DMA can be used to evaluate technologies
for generating analytical data (or other information) both in
the field and in an  off-site location that will provide
information appropriate for meeting project decision
criteria.  The ability of technologies such as X-ray
fluorescence (XRF), immunoassay (IA), ultraviolet (UV)
fluorescence, and  direct sensing tools such as the
membrane interface probe (MIP) and laser induced
fluorescence (LIF) to produce decision quality data has
been well documented (www.epa.gov/ORD/SITE and
www.epa.Qov/etv} and  their suitability to a site can  be
evaluated through a DMA. Extensive literature and
performance data are available for some technologies,
and these data should  be reviewed by project teams
before a DMA is designed.

A DMA can also provide information on cost and
performance that can be  used to optimize collaborative
data collection  using technologies for generating analytical
data (or other information) both in the field and in an off-
site location. Additionally, a DMA can offer stakeholders
an understanding of the site-specific performance of a
technology while at the same time it provides the basis to
optimize standard operating procedures (SOPs) for
deployment.

DMAs are performed easily and affordably before
mobilization, or as an early component of a field program.
Advice on the specific technology and assistance to set up
a DMA sometimes are available from equipment vendors
or service providers.

The question often arises: "How do DMAs fit into the data
quality objective (DQO) process?" Various aspects of
DMA planning  and implementation fall under a number of
DQO steps, but the most important one is Step 7: Select
the most resource-effective sampling and analysis
strategy that meets the performance criteria.

When is it necessary to perform a DMA?

DMAs may be  used when the project team works with a
technique that  previously has not been used at the site.
Site-specific factors often may render an otherwise useful
technique unsuitable and can result in high and
unnecessary project costs if they are not discovered early
in the project.  A DMA can quickly ascertain whether the
new technique is suitable for use at the site, allowing the
project team to identify an alternative if the proposed
technique is not suitable. Conversely, the project team
can proceed with confidence, realizing the benefits offered
by the technique, if the DMA suggests that it will be
effective.

A DMA may  be necessary when:

•   A project will depend heavily on field-based results to
    make real-time decisions.
•   Experience indicates a technology's performance is
    variable from site to site.
•   Heterogeneity and the cost of cleanup are high.
•   The chemistry of contaminants is complex.
•   A specific  relationship is needed between
    collaborative forms of data sets to support decision-
    making.
•   Stakeholder acceptance requires that the utility of a
    technology or approach be evaluated.

What are the benefits of performing a DMA?

A well-planned DMA can simultaneously test, refine, and
coordinate many project design parameters before full-
Office of Solid Waste and
Emergency Response
                                       EPA542-F-08-006
                                           August 2008
                                   www.brownfieldstsc.org

-------
                                                               Demonstrations of Method Applicability under a Triad Approach
scale project activities are under way. Project design
parameters often evaluated during a DMA include
sampling and analytical methods, QA/QC procedures,
data management, communication and data sharing
strategies, collaborative or comparative data needs (for
example, technologies matched with other field tools or
standard fixed-laboratory analytical methods), project
staffing, and the overall flow or sequencing of field
activities. A carefully considered DMA can help an entire
project run faster and more smoothly, resulting in lower
costs, and assuring that the data collected will be
adequate for the intended end use. Both field-based and
fixed laboratory methods have limitations, and the project
team should verify their performance during project startup
to avoid generating data or using equipment that does not
meet project requirements for precision, accuracy,
representativeness, completeness, comparability,
specificity, sensitivity, ruggedness, and reliability.

Project-specific DMAs guide the project team in selecting
and optimizing collaborative methods and assuring
adequate method  performance for site conditions and
decision criteria.  DMA results can be used to develop
project-specific action levels; analytical, sampling, and
data analysis procedures; QA/QC requirements; and
additional data requirements to assure the quality of the
        Important functions of a DMA include:

  •   Providing assurance that the proposed site
      characterization methods are suitable for the
      specific project.

  •   Generating data of known quality.

  •   Developing initial relationships between field
      methods or tools and other collaborative methods
      such as fixed laboratory. These relationships are
      used to design and focus the QC program.

  •   Testing a preliminary GSM to refine sampling
      protocols if assumptions are found to be incorrect.

  •   Setting preliminary field-based action levels to be
      used for real-time decision-making.

  •   Establishing the readiness of field personnel,
      equipment, and procedures before full-scale work
      begins at a site.

  •   Assessing alternative strategies as contingencies
      should the performance of the  intended methods
      be compromised by unanticipated problems.
decision. Furthermore, A DMA can help set acceptable
levels of uncertainty relative to decision thresholds used in
the field as part of a dynamic field decision strategy. The
fast pace of real-time cleanup projects makes DMAs
essential to avoid down time related to problems that arise
from inadequate planning for sampling and  analysis.

As noted earlier, DMAs can be used to test the suitability
of technologies for generating any data, whether the
analysis occurs in the field or in a traditional laboratory.
Chemical data-oriented DMA tasks  involve collecting,
preparing, and analyzing samples from a site-specific
matrix (soil, water, air, and tissue, for example).

DMAs also serve important non-analytical functions and
can help evaluate whether a project is ready to proceed.
DMAs can be used to test the preparedness of field
personnel and service providers, as well as to evaluate
the adequacy of logistical and data  management plans.
For example, sample throughput and analysis times can
be more accurately estimated.  Likewise,  materials and
personnel needs can be balanced and documentation
procedures clarified.  In addition, instrument compatibility
and data exchange or upload protocols can be verified
and debugged as necessary. Assessing  logistical
feasibility in this manner is especially important when the
project team uses dynamic work strategies. This aspect
of a DMA can help evaluate practical constraints for work
at the site in relation to the timing of sample collection and
analytical throughput, including field analytical equipment,
labor, sample storage, and the cost and supply of
consumables.

The DMA can also be used to "test-drive" real-time
decision support tools (DSTs).  These tools include
electronic data management procedures, global
positioning system (GPS) and surveying equipment, and
modeling, mapping,  and data display software. This
aspect of a DMA will improve the ease of use during full-
scale field mobilization by ensuring  operators can do the
job and identifying those aspects of a DST that can be
improved. In some cases, the project team may decide a
different DST is more appropriate for one particular portion
of a project versus another.

Finally, the DMA can be used to assess the
appropriateness and performance of proposed generation
techniques for data other than chemical, such as
geophysical, geotechnical, or direct sensing or probing
methods. The presence of site-specific interferences that
could compromise the performance of these tools can be
tested as a result. Interferences for geophysical
techniques could include tree leaf cover, seasonal wetland
Office of Solid Waste and
Emergency Response
                                       EPA542-F-08-006
                                            August 2008
                                   www.brownfieldstsc.org

-------
                                                                 Demonstrations of Method Applicability under a Triad Approach
 areas, low power lines, fences, shallow bedrock, salts,
 and interlocking sands.  Mineralogical interferences or
 other geological conditions can also affect the
 performance of direct-sensing equipment such as LIF
 probes. The presence of some forms of calcite or specific
 clay materials also can impair the utility of LIF.  If
 interferences are identified during the DMA, alternative
 strategies for dealing with  them can be developed before
 full-scale work is undertaken at the site.

 DMAs for direct-sensing tools include many of the same
 techniques used for evaluating other field analytical tools.
 They  can  involve building relationships between sensor
 response  and  analytical data or other forms of comparable
 information such as visible staining, free product, soil
 saturation, or physical characteristics of the matrix.  Figure
 2 provides an example of developing relationships
 between LIF response and visual core observations. At
 this site, relative fluorescence for various product types
 such  as gasoline, diesel, and oil were used to estimate the
 presence  or absence of free product. When used with LIF
 logs to estimate product thickness,  these values allowed
 members of the technical project team to estimate
 contaminant mass and optimize locations and depths for a
 product removal system.

 Figure 2: Correlations of LIF response and presence of free
 product.
Free Product At >50% Relative
  Fluorescence for Gasoline
Free Product At >75% Relative
    Fluorescence for Oil
  Fluorescence (%RE)
                                 25  50   75

                                Fluorescence (%RE)
  What are key concerns to address in designing a
  DMA?

  At least four aspects of a data-focused DMA should be
  considered during its design, including the following:

  •   What are critical aspects of the preliminary
     conceptual site model (GSM) that should be tested to
     assure project success? These aspects may include
     assumptions about the locations and nature of
    suspected releases, the degree of matrix
    heterogeneity, and the impact of sample processing
    bias.  Understanding matrix heterogeneity is critical in
    evaluating the number of samples required for
    statistically based sampling designs.  Without some
    information about the specific site and performance of
    the analytical method, the appropriate number of
    samples needed to achieve a desired level of
    decision confidence cannot be correctly identified.

•   Is it important to evaluate the cost-effectiveness and
    bias of multi-increment sampling? It may be important
    to compare the cost-effectiveness and bias of
    traditional grab sample collection and mathematical
    averaging procedures against a multi-increment
    (physical averaging) sampling strategy.

•   Do changes in sample support (the size, shape, and
    orientation of a sample) dramatically affect analytical
    data results? Is the level of effort associated with
    various sample preparation and analysis techniques
    worth the benefit of higher precision, accuracy, or
    control of bias?

•   Are project decisions of a qualitative or quantitative
    nature? In some cases, a "yes or no" answer is all
    that is needed,  while for others a more quantitative
    result is needed.  For example, in some cases the
    decision is only whether free product is present or
    absent in the subsurface. In contrast, risk estimation
    often  requires data in the form of quantitative
    concentration results.

DMAs should be designed to address those  issues that
most often provide the greatest source of uncertainty:
sample heterogeneity and  short-scale spatial variability.
The resulting mismatch between the volume of the sample
analyzed and where it is collected versus the data result
that will be extrapolated to a significantly larger volume of
material can be significant. Therefore, the DMA should
appraise sampling designs (such as multi-increment
designs), sample collection techniques (such as low-flow
purging of ground water wells versus passive diffusion
samplers), and sample preparation  procedures (such as in
situ versus ex situ readings and options for sample
homogenization or fractionization based on soil properties
such as particle size).

A DMA often involves "split samples" that are carefully
prepared to  minimize matrix heterogeneity and analyzed
by two or more different techniques to establish
relationships. Parametric and non-parametric techniques
are commonly used to evaluate these relationships and
establish decision quality for collaborative data sets.
 Office of Solid Waste and
 Emergency Response
                                                                         EPA542-F-08-006
                                                                              August 2008
                                                                     www.brownfieldstsc.org

-------
                                                               Demonstrations of Method Applicability under a Triad Approach
Parametric statistical methods use assumptions about the
data's underlying statistical distribution shape (normal,
lognormal, or other). If those assumptions are invalid, the
statistical conclusions may not be reliable. Non-
parametric techniques do not require that as many
assumptions be true, so they are more broadly applicable
to the properties of environmental data.

Comparability is quantified by establishing the frequency
that results from different techniques agree with  each
other with respect to a declared reference point. Different
points of reference can be used, but the most common
strategy in Triad projects is to establish comparability in
terms of the decision being made on  the data. These data
may require quantitative comparability (such as if or when
two data sets are combined to calculate risk assessment
parameters) or qualitative comparability for agreement at
the compliant or non-compliant decision threshold.

A comparability DMA can be used to demonstrate the
suitability of field-based technologies or project-specific
modifications to improve the performance of an
established fixed-laboratory method.  The techniques to
be compared include sampling related methods as well as
analytical methods.  Understanding the effects of sample
heterogeneity is extremely important when samples will be
split for the different analytical methods to be evaluated.
A valid comparison of data sets requires thorough
homogenization of samples before they are split to ensure
both methods see the same sample characteristics that
will be used to  make a decision.  To understand method
differences, known or blind QA samples (spikes,
replicates, reference materials, or blanks) are also often
subjected to comparative analysis to  assure technical
team members that both  methods are providing
representative  results.

What data deficiencies  can be addressed using a
DMA?

Site-specific method reporting limits (MRLs), precision,
bias, false positive rates, and false negative rates can be
assessed through the DMA process.  For example, MRLs
and sample reporting limits can be tested by analyzing
samples spiked with known amounts of target
contaminants and comparing site-specific matrices to find
the lowest concentration that can be  reliably detected and
quantified.

Data from laboratory reference methods and from a field-
based method  should be compared to see whether they
produce data that lead to the same project decision based
on established field-based action  levels or decision rules.
The ability of two methods to agree for decision-making is
an important parameter to examine when comparing
analytical methods, especially when methods with lower
analytical performance such as immunoassay methods
(which measure several closely related analytes and
report a single result for the group) are being compared
with methods with higher performance, such as gas
chromatography (GC) and mass spectrometry (MS)
(which are usually able to distinguish between closely
related analytes and measure each). In this way, the
DMA is a critical component in Triad's efforts to manage
the analytical  contribution to decision uncertainty.

How can DMA use weight of evidence and
collaborative data sets?

Terms such as "weight of evidence" or "multiple lines of
evidence" and "collaborative data sets" have been
developed to describe these layered data sets. From a
Triad perspective, there is a distinction between the two.
"Weight (or lines) of evidence" refers to combining
information from various different sources into a holistic
picture (that is, a GSM).  For example, historical
information may be used in conjunction with geological,
hydrogeological, chemical, and geophysical data to predict
contaminant fate and transport.

On the other hand, "collaborative data sets" or
"collaborative methods" refer specifically to the strategy of
using two (or more) analytical methods to measure the
"same" analyte or  a surrogate of an  analyte.  For example,
total uranium can be measured by XRF, gamma
spectroscopy, and alpha spectroscopy. Collaborative
methods are paired so that the strengths of one method
can compensate for the limitations of the other.
Frequently, a  field method is selected for its ability to
provide a much higher density of data points than an
expensive laboratory method. However, the laboratory
method will generally achieve better detection limits and
accuracy than the field method.  A DMA should be
designed to guide the "marriage" of the techniques to
produce reliable information that is not biased by the
effects of heterogeneity or analytical inaccuracy.

Additionally, alternative analytical methods, particularly
any that provide results in "real time," can be used to
optimize the decision making process.  For example, the
real-time decisions and  high data density possible with
field methods can  reduce the volume of material removed
during cleanup by more precisely defining and confirming
the actual contamination footprint. Real-time data can  in
this way improve confidence in the decision and limit
"surprises" after a project is complete.
Office of Solid Waste and
Emergency Response
                                       EPA542-F-08-006
                                           August 2008
                                   www.brownfieldstsc.org

-------
What is involved in performing a DMA?

Designing an appropriate DMA is specific to the
technology or technique being employed and to the
project, the site matrix or matrices, the effects of
heterogeneity, sample support (the size, shape, and
orientation of a sample), the expected use of the data, and
a myriad of other factors.  During systematic planning, the
project team may evaluate potential candidate
technologies or strategies for use at the site.
Technologies and strategies that can improve project
efficiency and the GSM, increase data density, and reduce
uncertainties associated with decision making are most
often targeted.  Project teams are encouraged to  employ
the services of an experienced Triad practitioner when
technologies or strategies are short listed. Although there
is no generic format for designing a DMA, a number of
activities are often involved.

•    Evaluating the strengths and limitations of
     technologies or techniques  to be used on site
     samples.

•    Evaluating sample support, throughput, ease of use,
     manipulation and storage of data, and other logistics
     so that the process is optimized.

•    Collecting and analyzing QC samples to evaluate the
     uncertainties that are the largest contributors to total
     measurement error.   Project resources can then be
     allocated to control for those activities with the
     greatest effect (Figures 3 and 4).
Figure 3. Uncertainty Sources and Associated QC Samples
I ncertainty Sources
Intrinsic (Instrumental)
Measurement Effects
Spike Prep.irntiou Effects
Preparation Method
Effects
Matrix Interference Effects
Sample Collection Effects
Sample Location Effects
Sampling Site Population
Effects
Source
Svmbol
IME
SPE
PMI
MIE
SCE
SLE
SSE
.Analytical Sample
Instrument Calibration Standard
Initial Calibration Verification Standard
Laboratory Control Sample
Matrix Interference Sample
Matrix Spike Duplicate Sample
Field Replicate (Duplicate) Sample
(Collected from same location and during same
sampling event time)
Co-Located (Same Location) Sample
(Collected 0.5-3 feet aivav from field sample)
Site field sample collected from tile environmental
site for tile study
Airalvtical Sample
Svmbol
ICS
ICY
LCS
MIS
MS MSB
FSR
CLR
SFS
-  Demonstrations of Method Applicability under a Triad Approach

 Figure 4. Output from an Uncertainty Evaluation
                  Component Relative Percent Uncertainty
            IME =    SPE    Pl.lt
         What is The measurement result?

         What ore Ihe measurement units?
         If the sample measurement is     |fkg .
         then the uncertainty interval Is 41 - 71 ,,,, at the 99% Confidence Le«
         For the above result, if itie systematic measurement error (bias) is corrected.
 •   Collecting information to establish initial relationships
     with data from the fixed laboratory or other
     collaborative information.  The collaborative
     relationship (data comparability) can be evaluated
     using a variety of options.

 •   Completing parametric statistical techniques, such as
     linear regression (Figure 5). Although commonly
     applied, caution should  be used with linear
     regression.  The correlation coefficient (R2) is
     universally used as a measure of a good relationship
     between two methods, but can  be misleading.
     Examining the slope and y-intercept can be far more
     informative and  less distorted by isolated high values.

 Figure 5. Sample Linear Regression

               ICP vx XRF (lead < 400 ppm)
                                                                   600
    500 -


    400


    300


    200


    100


     0 4
                                                                          y= 1.09X + 21
                                                                             R2 = 0.85
                                                                                   S*
                                                                      0
                                                                            100
                                                                                   200     300     400

                                                                                        ICP lead ppm
                                                                                                         500
                                                                                                                 60C
                                                                •   Using non-parametric techniques (often more useful
                                                                    for establishing comparability). These techniques are
                                                                    "common sense," but still powerful aids to decision
                                                                    making.  Examples include scatter plots, calculating
                                                                    decision error rates and establishing investigation
                                                                    levels to use with the alternative technique (Figure 6).
Office of Solid Waste and
Emergency Response
                                          EPA542-F-08-006
                                              August 2008
                                      www.brownfieldstsc.org

-------
                                                                Demonstrations of Method Applicability under a Triad Approach
Figure 6. Sample Investigation Levels
    "clean"
"unclear"
"contaminated"
               Lower Investigation
                                  Upper Investigation
                  Level (LIL)              Level (UIL)
                  "Real-time" analytical result
Figure 7. Typical Outputs of Investigation Levels and
Decisions Bins Using Non-Parametric Methods

       Relationship Between Gamma Walkver Data
                   and Thorium-230
             10K-16K         16K-20K

                    Counts per Minute (x 1000)
                                           20K +
    Using "decision bins" to establish investigation levels
    can guide confident decisions made on field data.
    (Figure 7).

    Identifying potential interferences, bias, false positive
    and false negative rates, and other issues.
    Depending on project timelines, the Quality
    Assurance Project Plan (QAPP) can be formally or
    informally updated and optimized as a result of the
    DMA to manage QA issues and produce data of
    known quality. The plan should also identify steps to
    address violations of QC criteria should they occur
    during the full-scale field effort.

    Using data collected during the DMA as the input
    values to construct a statistical sampling design. One
    of the acknowledged pitfalls associated with using
    classical statistical tools in sampling design  is that
    project teams seldom have a sound estimate of total
    measurement error to use in establishing sample
    quantities, grid sizes, and other factors.  With results
    from a DMA, project teams can use classical
    statistical tools (such as the Visual  Sample Plan
    software, http://vsD.Dnl.ciov/) more effectively in
    sampling design because they have generated site-
    specific information on method error.
•   Evaluating site-specific method error helps establish
    initial collaborative relationships that can be refined
    as the program progresses.  These relationships
    provide a framework for indicating problematic
    samples or "out of control" QC issues.

•   Providing insight into how the full set of data may be
    statistically evaluated.  Statistical methods such as
    those described in the guidance on Data Quality
    Assessment (EPA 2006b) may be examined for
    effectiveness and used to test basic project data
    assumptions, contaminant distributions, and sampling
    designs planned for use at a site.

•   Testing the suitability of data visualization and
    management strategies.

How are results of an analytical DMA applied?

If the DMA is properly designed, the data can be used for
the following:

•   Support development or refinement of the GSM
•   Estimate matrix and contaminant concentration
    variability at different spatial scales
•   Identify potential interferences
•   Ascertain whether particle size is correlated with
    contaminant concentrations
•   Evaluate the value and effectiveness of different
    sample collection and processing techniques to
    optimize SOPs
•   Establish a comparability relationship between two
    measurement systems
•   Establish proper decision logic and sequencing of
    data collection

In this section, readers will be introduced to some of the
basics of environmental decision  making. Understanding
the context in which decisions are made  is  essential for
discerning how managing associated uncertainties affects
the decisions. Managing decision uncertainties is
important to developing realistic and protective field-based
action levels for a site.  Overly protective standards can
significantly increase project costs, while less stringent
standards may lead to controversy or surprises later in the
project.

Under Triad, use of collaborative  data becomes essential
to provide sufficient density, and these data sets are then
compared to improve decision certainty.  A variety of
methods can be used to assess the comparability
between measurement systems.  "Measurement system"
refers to the combination of techniques used to collect and
Office of Solid Waste and
Emergency Response
                                                                           EPA542-F-08-006
                                                                                August 2008
                                                                        www.brownfieldstsc.org

-------
                                                                 Demonstrations of Method Applicability under a Triad Approach
process a sample in addition to the actual analysis. For
example, analysis of split samples that have been well-
homogenized (which, depending on the matrix and
analyte, may require grinding to a uniform particle size),
will be only part of the measurement system compared.
The overall measurement system for both XRF and the
compared laboratory technique must be considered for
technologies such as XRF spectrometers that can be used
for in situ sampling and where the effect of sample
properties cannot be removed from the analytical process.

Although there are a variety of mechanisms to assess the
comparability of data sets from two different measurement
methods, traditional statistical techniques are widely used
to begin the process of exploring the data.  Traditional
statistical evaluations of data sets can include summary
statistics and statistical plots (box and  whisker plots,
histograms, and probability plots) to evaluate distributional
characteristics of the sample population (normal,
lognormal, or other) that decide what types of statistical
manipulations are warranted.

Field-based and fixed-laboratory results can be compared
by developing correlation scatter plots, or by calculating
best-fit lines and correlation coefficients to describe the
mathematical relationship between the data sets.  If a field
method is shown to be biased high uniformly across a site,
the bias might be used to provide a natural safety factor
when compared with regulatory limits.  Alternatively, if the
bias is highly consistent and predictable across samples
and concentrations, adjustment for the bias is a  possibility.

One common use of a DMA is to demonstrate that a less-
rigorous analytical method correlates well with an
established method.  Best-fit lines and correlation
coefficients may be used for this  purpose.  If correlation is
lower than expected, a DMA may show that decision error
is low enough that the less-rigorous method is still
acceptable for the purpose of increasing sampling density
to delineate contaminant footprints and control data
variability on large and small spatial scales. This ability is
valuable since matrix heterogeneity and small-scale
variability are often the largest contributor to total
measurement error in environmental data.

In real life, the "true value" is unknown. Any data result,
no matter how good the analytical method, is an
approximation of the true value.  If sample heterogeneity
and interferences are controlled,  the more sophisticated
analytical method will be closer to the "truth." However,
the expense of these methods can  limit the number of
samples that can be analyzed. Non-representative data
that are biased by matrix problems can lead an unwary
decision maker into costly decision errors.  Using two
analytical methods helps ensure that matrix heterogeneity
does not mislead the decision maker.  One analytical
method (usually the fixed-laboratory method) will produce
more accurate measures of concentration, and that
method is used as the surrogate for the "true value." The
other method (usually a field method) produces a more
accurate representation of sample representativeness, but
at the expense of data accuracy. Therefore, in a DMA,
traditional  laboratory results are assigned as the "true
value"  (y axis) in Figures  8 through 10.  Field results are
assigned as the "estimated value" (x axis). "Investigation
levels" are selected through the process of comparing the
two sets of data and minimizing the  likelihood of decision
errors. Field results that fall above and below the upper
and lower investigation levels can support confident
decisions. Field results that fall within the concentration
range between the investigation levels require analysis by
the more accurate laboratory method.

Figure 8 illustrates two types of decision errors possible
when two sample analysis methods are compared.
Assume all data points fall within the ellipse.  "False
positive" decision errors (also called "false dirty" decision
errors) occur when a data result falls above an action level
when the true result is below the action level, and the
decision maker undertakes unnecessary remediation.
"False negative" decision errors (also called "false clean"
decision errors), occur when a data  result is below the
action  level when the true result is actually above the
action  level. If the decision maker accepts the data at
face value, erroneous decisions are possible that
potentially increase risk to human or ecological receptors.

Figure 8.  Misclassification Ellipse
            / BB
                   '  ESTIMATED VALUE
Xc denotes the action level, areas I and II indicate the false positive and the
false negative decision error zone, respectively. Areas AA and BB indicate
zones of consistent decisions between the data results and the true values.
Office of Solid Waste and
Emergency Response
                                        EPA542-F-08-006
                                             August 2008
                                    www.brownfieldstsc.org

-------
                                                                Demonstrations of Method Applicability under a Triad Approach
Figure 9 shows how using a safety factor below the action
level can reduce the false clean (false negative) decision
error rate.

Figure 10 shows how safety factors both above and below
the action level reduces both the false clean and false
dirty decision errors.

Figure 9.  Misclassification Ellipse with Safety Factor.
                     ESTIMATED VALUE
Xc denotes the action level, area I indicates the false positive error zone, and
area II indicates the false negative error zone that has been reduced by the
safety factor. Areas AA and BB indicate zones of consistent decisions between
the data results and the true values.


Figure 10. Multiple Investigation Levels.
       Lower      Upper
       Investigation  Investigation
       Level       Level
                                     Action
                                     Level
                Estimated Value
Both types of errors are reduced. Shaded area represents data results falling
between the investigation levels that require more accurate analysis.
What else should project managers and technical staff
consider when planning a DMA?

Refining decision criteria and decision logic based on the
results of a DMA can significantly improve project
performance. Results from the DMA should be integrated
as quickly as possible into work strategies to assure
project efficiency (See EPA 542-F-05-008, "Use of
Dynamic Work Strategies Under a Triad Approach for Site
Assessment and Cleanup — Technology Bulletin,
www. brownfieldstsc. oro/pdfs/D WSBulletin. pc/fl.

Project managers must identify resource needs to support
real-time decision-making during the DMA. These
resources include DSTs and associated expertise.  For
example, a DST may be required to assist in developing
and verifying field-based action levels.

Project managers and technical staff should refine the
type and level of field documentation required based on
the DMA.  Site-specific work plans and SOPs for field
methods with sufficient flexibility can be easily revised as
more is learned about a site, even after the DMA is
complete. Team member responsibilities should be
consistent and modified as needed based on the DMA.

Creativity and flexibility in procurement and contracting is
often needed for a DMA, or in response to a DMA.
Review EPA's procurement guide at
www. brownfieldstsc. oro/pclfs/procurement. pdi as a
starting point for procurement and contracting strategies
for Triad investigations.  Unitizing or classifying costs per
analytical sample  or borehole, for example, is an
illustration of a financial strategy that allows project
planners to accurately track costs in real time as a
dynamic investigation  progresses.  Potential vendors may
provide free resources for the DMA to market and
demonstrate the applicability of the technology, reducing
the cost to the project  — most commonly with newer and
relatively unproven technologies.

Workload balancing and task sequencing are examples of
strategies used to ensure that project team members are
aware of the project's time-critical tasks.  Team members
should work together to prioritize each task so that no task
slows the entire effort while others (drillers, samplers, or
analytical chemists) are idled, but still billing time. A DMA
can provide important  information on potential bottlenecks
in data generation and flow, so that effective field
coordination and sequencing strategies can be developed
for the main field investigation.

How is a DMA documented?

A DMA is documented through a variety of formal and
informal means.  Project plans such as Sampling and
Analysis Plans (SAPs) and QAPPs are formal documents
that undergo mandatory review.  When they are written for
a Triad investigation, these documents outline the DMA to
refine the data collection schemes and strategies to
manage uncertainty. Site-specific SOPs for field methods
are useful for documenting the outcome of a DMA in the
Office of Solid Waste and
Emergency Response
                                        EPA542-F-08-006
                                            August 2008
                                    www.brownfieldstsc.org

-------
                                                              Demonstrations of Method Applicability under a Triad Approach
form of exact procedures to be used at a given site,
especially when there are deviations from vendor-
recommended procedures or published methods (such as
SW-846 methods).

More informal methods, such as memoranda of
understanding, meeting notes,  project Web sites,
E-rooms, and electronic bulletin boards, also document
the DMA process. These informal methods are
particularly useful to document stakeholder participation
and buy-in for Triad investigations.  Informal discussions
with stakeholders after the DMA can also be useful to
accelerate document comment, revision, and submittal.

Regardless of the method used to document a DMA, good
records are essential to scientifically validating and legally
defending selection and use of analytical methods and,
ultimately, the conclusions based on data generated in the
field. Project teams are strongly encouraged to plan for
and complete DMA documentation.

Are DMAs difficult, time-consuming, or costly?

Generally, DMAs can be completed with 20 or fewer
samples, but the level of effort can be scaled to the
magnitude of expected site work. However, more data
can be extremely valuable in the case of many real-time
technologies where costs per sample are relatively low. If
linear regressions will be generated, a good rule of thumb
is at  least 10 paired samples; however, 20 or more can
provide exceedingly robust statistical evaluations. A key
concept for analytical  DMAs is to focus sample pairs
around action or decision points (for example, 5 low
values, 5 higher values, and 10 in areas around action
levels). Using real-time measurements provides a level of
assurance that samples submitted for fixed-laboratory
comparative analysis  are in the  range of interest. Data
sets with  high percentages of non-detected pairs are not
beneficial for statistical evaluations.

Information collected during the DMA will provide a basis
for establishing QA/QC protocol, sample support,
preliminary relationships for collaborative data sets, load
balancing, and sequencing field activities. The DMA
results are a means for optimizing use of resources and
become part of the final data set that will be compiled to
reach project decision points.

The cost  and time required for a well-designed DMA are
usually a small fraction of the cost of a full-scale field
program. When the project team designs a DMA, the cost
and time  allotted should be proportionate to the impact of
         the DMA on reduction in uncertainty about a site condition.
         It is expected that relationships evaluated under the DMA
         will continue to be refined as more data become available.
         Another cost savings consideration is the use of archived
         material (where appropriate) for comparative analyses that
         may have already been completed. These samples
         provide the advantage that concentration ranges of
         contaminants of concern (COCs) to target samples in the
         primary areas of interest will be known.

         In any project, the methods being used will be under
         scrutiny. When a DMA is not conducted on a limited
         number of samples, the data collected during the full site
         investigation must be used to demonstrate method
         performance.  The DMA therefore provides an opportunity
         to change tactics affordably if a method does not perform
         as expected, compared with the alternative of having to
         change tactics after the full site investigation.

         Appropriate professional expertise and good
         communication among team members about their data
         needs are  critical to planning a successful DMA. The cost
         of a DMA is recouped many times over though cost
         avoidance of unusable data and recharacterization efforts.

         Are DMAs appropriate in all cases?

         Some investigation and cleanups may prescribe set
         sample numbers or recommend limited sampling through
         guidance.  In these cases, DMAs may not be appropriate.
         Similarly, resources may not be  adequate at some sites
         with limited grant funding to accommodate a DMA.
         Projects with adequate resources to employ  established
         mobile or fixed-laboratory methods at sufficient density
         may be inappropriate, while those that require method
         modifications or careful examination of sampling and
         spatial uncertainties may benefit significantly from DMAs.
         Even if only fixed-laboratory methods are used,  a DMA
         may be considered if there is any question about matrix
         interference effects. A relatively limited number of pilot
         samples can save large sums of money by detecting
         extraction issues and interference problems  at the start of
         a program.

         It should be noted, however, that a DMA is beneficial for
         most applications precisely because a particular field
         analytical technique, direct-sensing tool, or innovative
         strategy is identified as potentially applicable to  cost-
         effectively  increase data density, refine the GSM, or
         address small-scale variability and matrix heterogeneity.
         In some cases, sampling locations for an early
         assessment are obvious (for example, areas of visible
Office of Solid Waste and
Emergency Response
10
    EPA542-F-08-006
        August 2008
www.brownfieldstsc.org

-------
                                                               Demonstrations of Method Applicability under a Triad Approach
staining, product, lagoons, or discharge points), while
other cases are more complicated, and identifying
appropriate sampling locations is a problem. Depending
on the nature of the suspected contamination, some
sample material can be archived and potentially used later
as part of a DMA for an expanded assessment or
additional investigation.

Adding limited additional cost for analyzing field analytical
methods, direct-sensing tools, or other innovative
technologies does not significantly raise project expenses
at sites with elevated expenditures associated with
collecting deeper subsurface samples. In these cases,
inexpensive analytics and direct sensing tools can provide
greater vertical density to help target locations for more
expensive traditional laboratory samples. Furthermore,
the increased density can support a more efficient design
of cleanup strategies when required, leading to lower
project lifecycle costs.

Finally, the definition of a Brownfield — "a property,
redevelopment, or reuse which may be complicated by the
presence or potential presence, of a hazardous
substance, pollutant, or contaminant" — underscores the
need for higher data density and collaborative data sets
that accompany DMAs. Regardless of whether significant
contamination or just the perception of contamination is
present at a property, DMAs used to optimize sampling
schemes with innovative tools provide a higher data
density that facilitates timely revitalization.  These well-
designed data sets are  particularly helpful to address
stakeholder concerns and provide a level of comfort that
allows developers,  insurance partners, risk partners,
public stakeholders, state agencies, local agencies, and
others to be involved, invested, and reassured with a
project outcome.

 EXAMPLES OF DMA IMPLEMENTATION

Example #1: Immunoassay:  Wenatchee Tree Fruit
Test Plot Site, Wenatchee, Washington
                       0
The Wenatchee Tree Fruit Test Plot area contained soil
contaminated with pesticide compounds from agricultural
research conducted from 1966 through the mid-1980s.
The U.S. Public Heath Service (PHS) and EPA used the
         test plot area to evaluate the effectiveness of various land
         disposal methods for pesticides. In 1997, USAGE
         conducted an integrated site characterization and
         remediation program that allowed characterization,
         excavation, and segregation of contaminated material in
         real time. Work was completed under a voluntary  cleanup
         program with regulatory oversight of the Washington
         Department of Ecology.

         A DMA was conducted to provide critical input to the
         project design because the project would use IA methods
         to drive the dynamic work strategy.  The DMA was
         structured to evaluate both the utility of the IA kits and to
         develop field-based  action levels.
            Site Facts

            •/  Disposal area of an agricultural research facility.

            S  Reuse scenarios not identified. Changing land use
                nearby increased concern that the area should
                undergo investigation and remediation.

            •/  Principal threats included off-site migration,
                contamination of other media, and direct contact.

            •/  COCs included organochlorine, organophosphorous,
                and other pesticide compounds.
         Highlights of the DMA

         The DMA confirmed that the IA test kits were intentionally
         biased 100 percent high by the manufacturer to reduce
         the chance of false negative results. The DMA
         accommodated the response of the kits to structurally
         similar compounds beyond the target compounds. Taking
         into consideration the high bias and correlations with
         fixed-laboratory results, the DMA showed that the DDT
         test kit result exceeding 5 parts per million (ppm) could
         indicate elevated levels of DDT, DDE, or ODD. Likewise a
         cyclodiene kit response of 0.1  ppm indicated the
         possibility that regulatory action levels for endrin or
         dieldrin were exceeded.  Therefore, these values were
         selected as the investigation levels to make decisions
         based on the kit results.

         Several modifications to the IA kit procedures were made
         based on DMA results.  For one, pure methanol was used
         instead of a water-methanol mix, and extraction volumes
         were doubled to 20 milliliters (ml) to bracket action levels
         based on cross reactivity and sensitivity results. The
Office of Solid Waste and
Emergency Response
11
    EPA542-F-08-006
        August 2008
www.brownfieldstsc.org

-------
                                                               Demonstrations of Method Applicability under a Triad Approach
resulting 20-milliliter (ml) extracts were sufficient to run
both the DDT and cyclodienes IA analyses.

Some fixed-laboratory detection methods for collaborative
data were also modified. The project team used MS
detection instead of the method specified nitrogen and
phosphorus detector (NPD) to improve selectivity and
meet project required quantitation limits for the
organophosphorus pesticides. Similarly, the team used a
GC NPD method instead of high-pressure liquid
chromatography (HPLC) for the carbamate pesticides to
reduce interference. In  addition, the surrogate compound
was changed to a compound rarely used in agricultural
applications.  Non-target compounds and tentatively
identified compounds (TICs) from fixed-laboratory
methods also became crucial to understanding IA kit
response from a broad range of contaminants.

During the characterization phase, the project team
continued to collect a subset of samples for fixed-
laboratory analysis. These results, used in conjunction
with DMA data, indicated that the 5 ppm investigation level
for the DDT IA kit was overly conservative. With regulator
approval, the DDT IA investigation level was raised  to 10
ppm to complete excavation and waste segregation.

No false negative  decision errors for the action levels for
individual pesticides were encountered. A low percentage
of false positive errors (usually associated with of the
presence of endosulfans in the samples) was encountered
for the cyclodiene kit. Use of the DMA and Triad
principles resulted in an estimated savings of 50 percent
for total project costs.

More information on the DMA and Triad work conducted
for this site is available at the Triad Resource Center Web
page:  www.triadcentral.orQ.

Example #2: Poudre River Site, Fort Collins, Colorado
The Poudre River Site is located in Fort Collins, Colorado,
along the Cache La Poudre River.  The presence of coal
tar in the river and fuel-related ground water
contamination prompted EPA to initiate a Targeted
Brownfields Assessment in May 2003. Two DMAs were
         conducted as part of the Targeted Brownfields
         Assessment.
         '  Site Facts
            •/  Site includes a former manufactured gas plant (MGP)
                that operated from approximately 1900 to 1930.

            •S  Site includes a former municipal burn landfill that
                operated from the late 1930s to the early 1960s.

            •/  Proposed reuse was recreational, commercial, and
                industrial.

            •/  COCs included chlorinated solvents and petroleum-
                related substances.
         Highlights of the DMAs

         One DMA focused on demonstrating the capability of
         passive soil gas samplers from EM FLUX (now known as
         Beacon Environmental) to detect volatile organic
         compounds in the subsurface.

         A full-scale soil gas survey was implemented using 333
         devices after the passive soil gas DMA successfully
         demonstrated the use of the EMFLUX passive soil gas
         samplers. The data from the study were used to create
         isoconcentration maps for target  analytes, helping to
         refine the GSM and optimize the field investigation drilling
         program.

         A DMA was also performed for a modified EPA SW-846
         Method 8260 used for the analysis of ground water
         samples on site via a mobile laboratory.  The project team
         used the results from the ground  water DMA to set the
         applicable detection and reporting limits for GC/MS
         results, design appropriate initial  calibration  and QC
         protocols, and evaluate the types and concentrations of
         contaminants expected in ground water at the site. The
         DMA also provided site-specific information  about the
         accuracy and precision of the method.

         Another aspect of the Poudre River Site field program that
         showed the usefulness of a DMA study was geophysical
         survey work. A ground penetrating radar (GPR) survey
         conducted at the site suffered from poor  signal penetration
         because of soil conditions.  However, the performance-
         based contract used for the work did not allow the cost of
         this GPR work to be billed against the program. Had the
         GPR vendor conducted a DMA, this problem would likely
Office of Solid Waste and
Emergency Response
12
    EPA542-F-08-006
        August 2008
www.brownfieldstsc.org

-------
                                                                Demonstrations of Method Applicability under a Triad Approach
have been discovered earlier and at significantly lower
cost to the vendor.

More information on the DMA and Triad work conducted
for this site is available at the Triad Resource Center Web
page:  www.triadcentral.orQ.
Example #3: X-ray Fluorescence: Fort Lewis Small
Arms Firing Range, Fort Lewis, Washington
In 2003, USAGE used the Triad approach to expedite site
characterization and remediation of contaminated soil at
the former Evergreen Infiltration Training Range in Fort
Lewis Washington. A dynamic sampling and analytical
strategy based on rapid field-based analytical methods
was used to streamline site activities and save resources
while increasing confidence in remediation decisions.

Initial evaluations included a suite of metals associated
with small arms firing ranges (antimony, arsenic, copper,
iron, lead, tin, and zinc).  The DMA indicated that lead was
the primary risk driver given regulatory thresholds and site
action levels. After the DMA, the remaining
characterization and remediation work at the Fort Lewis
site focused on lead.
   Site Facts

   •/  Site is a former small arms firing range that operated
       over an unknown time period between 1917 and 1965.

   •/  Proposed reuse was for military barracks.

   •/  Principal threat is direct contact with contaminated soil.

   •/  COCs were metals including lead and arsenic.
Highlights of the DMA

At the beginning of the field investigation, a DMA was
conducted using field-portable XRF and fixed-laboratory
methodologies (EPA SW-846; sections 6010 and 6020).
Forty samples were collected and analyzed by both
         methods.  The DMA established a strong correlation
         between XRF and laboratory data for lead (see Figure 11),
         even with  minimal soil sample homogenization. The
         measured correlation coefficient (R2) between the
         methods was 0.96; however, inspection of the slope and
         y-intercept indicate some loss of linearity.  Examining
         concentrations for individual sample pairs indicates that
         XRF results tend to under-report concentrations as
         concentrations increase above percent levels (10,000
         ppm).  Under-reporting was not a concern for the project
         team, however, since various action levels for lead were
         all less than 1,000 ppm.

         Figure 11.  Correlation Curve.
            50000 -,

          3 45000 -

          |> 40000 -

          IT 35000 -
          o
          ra 30000 -
          4-1
          O 20000 -
          O
          •g 15000 -

          ^ 10000 -

          X  5000 -
                 0         20000       40000       60000
                       Fixed-lab Lead Concentration (mg/kg)
         XRF data are plotted against data from EPA methods 6010/6020 for Fort Lewis
         Small Arms Firing Range DMA.

         A regression was also generated using results for a
         subset of split samples in the 0 to 1,000 ppm range for
         lead to evaluate XRF performance in this critical area.
         The DMA confirmed that the XRF reliably quantified lead
         concentrations down to 45 milligrams per kilogram
         (mg/kg), and so was accurate in locating both "clean" and
         "dirty" areas. Through the DMA, it was assured that data
         of "known and  documented" quality could be produced.
         The level of data quality was shown to be sufficient for the
         project's decisions.  Although more  intensive sample
         preparation or  use of substitution methods for non-
         detected XRF values may have produced a better
         regression, the project's data needs did not require the
         additional precision.

         More information on the DMA and Triad work conducted
         for this site is available at the Triad Resource Center Web
         page: www.triadcentral.orQ.
Office of Solid Waste and
Emergency Response
13
    EPA542-F-08-006
        August 2008
www.brownfieldstsc.org

-------
                                                             Demonstrations of Method Applicability under a Triad Approach
SOURCES OF ADDITIONAL INFORMATION

The Triad approach is encountering ever greater
acceptance by regulatory agencies, as well as by
professional and industry organizations.  Communities
and project teams interested in implementing the Triad are
encouraged to contact the BTSC for more information on
these organizations and for successful examples of Triad
applications. More detailed information on DMA and on
the Triad approach can be found in the Brownfields
Technology Primer Series document Using the Triad
Approach to Streamline Brownfields Site Assessment and
Cleanup, which is available at www.brownfieldstsc.om.
Project profiles, case studies, and other  information on
applying the Triad approach can be found at
www. triadcentral.org.  The BTSC provides other technical
bulletins related to best practices embodied in the Triad
approach such as "Use of Dynamic Work Strategies
Under a Triad Approach for Site Assessment and Cleanup
— Technology Bulletin." Additional documents providing
critical information on related issues such as Green
Remediation and Vapor Intrusion are also available.
REFERENCES

Brownfields and Land Revitalization Technology Support
Center Web Site. 2008. Accessed August.
www. brownfieldstsc. om

Fort Lewis, Washington. 2003. "Fort Lewis Agreed Order
Rl Demonstration of Method of Applicability Sampling and
Analysis Plan Addendum Former Small Arms Ranges."
October.  Available  at:
www. triadcentral. oro/user/doc/TPP-FortLewis-
DMAMemo.pdf

Gilbert, R. 1987. "Statistical Methods for Environmental
Pollution Monitoring. Wiley.

Lesnik, B.  2000. "Method Validation for the Resource
Conservation and Recovery Act Program."  LC/GC
Magazine, Volume 18, Number 10. October.

Pacific Northwest National Laboratory.  "Visual Sampling
Plan Web page." Accessed August. Available at:
http://vsp.pnl.Qov

Triad Resource Center Web Site.  2008. Accessed July.
Available at:  www.triadcentral.pro/index.cfm
         U.S. Environmental Protection Agency (EPA).  2000.
         "Guidance for Data Quality Assessment: Practical
         Methods for Data Analysis" (QA/G-9).  EPA 600-R-96-084.
         July.  Available at www.epa.Qov/oust/cat/epaQaQ9.pdf

         EPA. 2003.  "Using the Triad Approach to Streamline
         Brownfields Site Assessment and Cleanup." Brownfields
         Technology Primer Series. EPA 542-B-03-002. June.
         Available at:
         http://www.brownfieldstsc.orQ/pdfs/Triadprimer.pdf

         EPA. 2004a. "Case Study of the Triad Approach:
         Expedited Characterization of Petroleum Constituents and
         PCBs Using Test Kits and a Mobile Chromatography
         Laboratory at the Former Cos Cob Power Plant Site."
         EPA542-R-04-008.

         EPA. 2004b. "Innovations in Site Characterization Case
         Study: Expedited Characterization and Excavation of Lead
         in Soil Using X-Ray Fluorescence at the Ross Metals Site,
         Rossville, Fayette County, Tennessee." December.

         EPA. 2005a. "Understanding Dynamic Work Strategies
         Under a Triad Approach for Site Assessment and
         Cleanup—Technology Bulletin."  September. Available at:
         www. brownfieldstsc. org/pdfs/D WSBulletin. pdf

         EPA. 2005b. "Understanding Procurement for Sampling
         and Analytical Services Under a Triad Approach." EPA
         542-R-05-022. June.  Available at:
         www.brownfieldstsc.org/pdfs/procurement.pdf

         EPA. 2005c. "Use of Field Portable X-Ray Fluorescence
         (FPXRF) and the Triad Approach to Investigate the Extent
         of Lead Contamination at a Small Arms Training Range,
         Fort Lewis, WA." August.

         EPA. 2006a. "Data Quality Assessment: A Reviewer's
         Guide." EPA 240-B-06-002. February. Available at:
         www. epa. Qov/Quality/QS-docs/Q9r-final.pdf

         EPA. 2006b. "Data Quality Assessment: Statistical
         Methods for Practitioners." EPA 240-B-06-003.  February.
         Available at: www.epa.Qov/Quality/Qs-docs/Q9s-final.pdf

         EPA. 2006c. "Guidance on Systematic Planning Using
         the Data Quality Objectives Process." EPA/240/B-06/001.
         February. Available at: www.epa.Qov/Quality/QS-docs/Q4-
         final.pdf
Office of Solid Waste and
Emergency Response
14
    EPA542-F-08-006
        August 2008
www.brownfieldstsc.org

-------
EPA.  2006d.  "SW-846 Manual."  Last updated October
18,2006. Available at:
www. epa. Qov/epaoswer/hazwaste/test/sw846.htm

EPA.  2008a.  "Environmental Technology Verification
(ETV) Web page." Accessed August. Available at:
EPA.  2008b.  "Performance-Based Measurement System
(PBMS) Web page."  Accessed August. Available at:
www. epa. Qov/SW-846/pbms.htm

EPA.   2008c.  Information Quality Guidelines Web Site.
Accessed July. Available at:
www. epa. Qov/Quality/informationciuidelines
        —  Demonstrations of Method Applicability under a Triad Approach

         NOTICE AND DISCLAIMER

         This bulletin was prepared by EPA's Office of Solid Waste
         and Emergency Response under EPA Contract Nos. 68-
         W-02-034 and EP-W-07-078. The information in this
         bulletin is not intended to revise or update EPA policy or
         guidance on how to investigate or cleanup Brownfields or
         other revitalization sites.  Mention of trade names or
         commercial products does not constitute endorsement or
         recommendation for use.

         This bulletin can be downloaded from EPA's Brownfields
         and Land Revitalization Technology Support Center at
         www.brownfieldstsc.org.  For technical inquiries regarding
         this bulletin please contact Stephen Dyment of EPA at
         (703) 603-9903, or dyment.stephen@,epa.ciov.
Office of Solid Waste and
Emergency Response
15
    EPA542-F-08-006
        August 2008
www.brownfieldstsc.org

-------