United States
Environmental Protection
Agency
Environmental Monitoring
Systems Laboratory
Las Vegas, NV 89193-3478
Research and Development
EPA/600/S4-91/008 April 1992
Project Summary
Direct Delayed
Response Project:
Soil Characterization
Comparison
L.K. Fenstermaker, G.E. Byers, T.H. Starks,
M.J. Miah, C.J. Palmer, and K.D. Lauckner
A large amount of soil characteriza-
tion data has been collected as a com-
ponent of the Direct Delayed Response
Project (DDRP) in the acid rain Aquatic
Effects Research Program. An
interlaboratory comparison study was
undertaken to identify the comparabil-
ity of this data to that obtained from
representative soil characterization
laboratories. Participating laboratories
were selected at random from four re-
gions of the U.S. and two regions of
Canada. Two original DDRP contract
laboratories also participated. Duplicate
samples of six soil audit materials and
two liquid soil extracts were sent to
each of the laboratories in two sepa-
rate batches. Laboratories used their
own protocols to perform the analyses
requested except for the contract labo-
ratories which followed the DDRP pro-
tocol. The largest number of different
methods used was for the measure-
ment of cation exchange capacity. The
results between the DDRP soil survey
data and this study's results were com-
pared using Youden-pair plots, and
standard statistical tests. Overall, the
DDRP data were comparable to the data
from this study. However, out of the
total 141 comparisons involving results
from six or more laboratories, the re-
sults from the two contract laborato-
ries did not meet the comparison crite-
ria in 19 cases. Since there was never
a case in which both contract laborato-
ries failed, it would appear that the 19
cases which were not comparable were
due to random analytical errors, incor-
rectly reported results, or misapplica-
tion of DDRP protocol.
This Project Summary was developed
by EPA's Environmental Monitoring
Systems Laboratory, Las Vegas, NV, to
announce key findings of the research
project that is fully documented in a
separate report of the same title (see
Project Report ordering information at
back).
Introduction
This study was designed and imple-
mented to answer questions regarding data
comparability and accuracy related to the
Direct Delayed Response Project (DDRP)
soil surveys. An important component of
the quality assurance program within the
DDRP is the assessment of the data qual-
ity. One of the attributes of data quality is
comparability. Comparability is of particu-
lar importance for two reasons. First, it
was recognized that it would be neces-
sary to develop a standardized analytical
methods manual for the DDRP soil survey
analyses. Information was compiled from
many soil methods manuals in the devel-
opment of the DDRP methods manual.
Although this manual was subjected to
extensive peer review, the question still
remains as to the comparability of data
resulting from these methods to those ob-
tained by the methods presently in use by
soil characterization laboratories in the U.S.
and Canada. Second, although the DDRP
soil surveys do cover broad land areas of
particular interest to the issue of acid depo-
sitions effects, there are other regions in
the U.S. and Canada of similar impor-
tance. To make predictions for these re-
gions, it would be necessary to rely on
existing soil characterization information.
Are these data comparable to the DDRP
Printed on Recycled Paper
-------
data, or will there need to be adjustments
in the data or in the DDRP models to
account for existing differences? If the data
from this study are comparable, then it is
possible that the various soil characteriza-
tion data bases in existence elsewhere
may be comparable to the DDRP data
bases. The objective of this study was to
investigate whether the analytical data ob-
tained by laboratories using DDRP soil
analysis protocols are comparable to data
for the same audit materials analyzed by
other soil characterization laboratories in
Canada and the U.S. not using DDRP
protocols.
Procedure
The purpose of this study was to deter-
mine whether data obtained by using the
DDRP protocol to measure soil param-
eters are comparable with data obtained
from other soil characterization laborato-
ries using their own individual protocols.
Laboratories were selected at random from
four regions of the U.S. and two regions
of Canada. Two original DDRP contract
laboratories also participated. Six different
soil audit samples used in the study were
chosen from bulk soil samples which had
been previously prepared for use in the
Northeastern and Southern Blue Ridge
Province DDRP soil surveys (A, Bs, Bw,
and C horizons), and the Mid-Appalachian
DDRP soil survey (O, A, Bw, and B). The
bulk soil samples originated in the north-
eastern United States and represented the
major soils of the region. The audit
samples included five mineral soils and
one organic soil. Two batches of the soil
samples were shipped to the laboratories
at separate times for analysis. Detailed
procedures for the soil analytical protocols
to be used by the noncontract laborato-
ries were not specified for this study. Only
the general method was specified for each
group of parameters, e.g., calcium ex-
changeable in ammonium acetate. The
purpose of this study was to compare the
analytical results from different laborato-
ries, not to compare their protocols.
For this study, the quantitative criterion
used to determine whether the DDRP pro-
tocol provided results comparable to those
obtained by other soil characterization
laboratories is: if the sample means from
the contract laboratories are not extremes
in a box plot of all participating laborato-
ries sample means for a particular param-
eter and soil audit material, then the re-
sults are comparable for that parameter
and soil audit material combination. There-
fore, in this study, if the results from the
contract laboratories using the DDRP pro-
tocol are near the center of the results
distribution from the noncontract laborato-
ries, then the DDRP protocol are consid-
ered to produce results that are "compa-
rable" to results being obtained by soil
laboratories outside the DDRP program.
Youden-pair plots derived from box plots
were prepared for visual determination of
data comparability for each parameter and
soil sample combination.
After reviewing the data, it was decided
that certain standard tests of equality of
means and variances would provide de-
scriptive information for the reader. How-
ever, since the number of samples and
replicates is so limited, these tests do not
have sufficient power to address data com-
parability. The standard tests addressed
the following issues: time effect, (i.e., did
the soil samples change over time); ho-
mogeneity of laboratory results; pairwise
comparisons to determine which laborato-
ries were significantly different from each
other; within and between group variabil-
ity; and the comparability of combined pa-
rameters, (i.e. groups of similar param-
eters).
Results
The contract laboratories results were
found to be comparable to results from
noncontract laboratories in 122 out of 141
combinations of parameters and audit ma-
terials. In all 19 failures to meet the com-
parability criterion, only one, but not al-
ways the same one, of the two contract
laboratories had measurements outside the
prescribed range. By itself, not too much
should be made of this since the number
of laboratories reporting results is less than
8, and it is impossible for the criterion to
indicate that both contract laboratories are
too large or both too small. However, a
check found only one of the 19 cases
where if one contract laboratory mean is
too large, the other contract laboratory
had the next smaller mean value, or where
one contract laboratory mean was too
small, the other contract laboratory had
the next larger mean. That case is "A1
extractable in pyro-PO4" in the Bs soil
sample. Thus, these 19 cases provide little
evidence that the DDRP protocol is caus-
ing extreme measurements. In 9 of these
19 cases, the failure was caused by only
one of the two measurements by a con-
tract laboratory being outside the pre-
scribed range. The only place where there
seems to be something approaching a
consistent pattern of such failures was in
the measurement of sand content. Six of
the 19 failures reported are related to the
measurement of sand content, where the
failing measurements were all too large.
The results for the supplemental statis-
tical analyses provide an indication that
the DDRP databases may be comparable
to data generated by other soil character-
ization laboratories. However, the conclu-
sions drawn from the supplemental tests
may be limited due to the very small de-
grees of freedom. These tests differed
from the Youden-pair plots in that the plots
compared the contract versus the
noncontract laboratory data, and the tests
compared the DDRP soil survey data to
the noncontract laboratory data. Overall,
the supplemental tests indicate that the
DDRP and noncontract data are compa-
rable.
Conclusions
The general conclusion is that the DDRP
protocol does provide results comparable
to those results obtained by other soil
laboratories for 30 parameters. In 122 of
141 comparisons involving results from
six or more laboratories, the results from
the two contract laboratories using the
DDRP protocol met the comparability cri-
terion. There was never a case in which
both contract laboratories failed to meet
the comparability criterion and only one
case where one failed and the other con-
tract laboratory had the next most ex-
treme sample mean. Therefore, it would
appear that the 19 failures to obtain com-
parability are either a result of random
analytical errors in individual contract labo-
ratories, incorrectly reported results, or
from the misapplication of the DDRP pro-
tocol. In the case of the sand results,
where there is a pattern of measurements
by one laboratory that is too large, it is
possible that interpretation or adherence
to the protocol may have changed.
The information in this document has
been funded wholly or in part by the United
States Environmental Protection Agency
under EPA Cooperative Agreement Num-
ber CR814701-01, to the Environmental
Research Center, University of Nevada
and EPA Contract Number 68-03-3249 to
Lockheed Engineering and Sciences Co.
It has been subject to the Agency's peer
and administrative review, and it has been
approved for publication as an EPA docu-
ment.
Mention of trade names or commercial
products does not constitute endorsement
of recommendation for use.
&U.S. GOVERNMENT PRINTING OFFICE: 1992 - 64S-080/402.10
-------
-------
L.K. Fenstermaker, T.H. Starks, C.J. Palmer, and K.D. Lauckner are with the
Environmental Research Center, University of Nevada, Las Vegas 89154;
G.E. Byers and M.J. Miah are with Lockheed Engineering and Sciences Com-
pany, Las Vegas, NV 89119.
L.J. Blume andD.T. Heggem are the EPA Project Officers (see below).
The complete report, entitled "Direct Delayed Response Project: Soil Characteriza-
tion Comparison, "(OrderNo. PB92-153428/AS; Cost:$43.00;subjecttochange)
will be available only from:
National Technical Information Service
5285 Port Royal Road
Springfield, VA 22161
Telephone: 703-487-4650
The EPA Project Officers can be contacted at:
Environmental Monitoring Systems Laboratory
U.S. Environmental Protection Agency
Las Vegas, NV 89193-3478
United States
Environmental Protection
Agency
Center for Environmental
Research Information
Cincinnati, OH 45268
BULK RATE
POSTAGE & FEES PAID
EPA
PERMIT No. G-35
Official Business
Penalty for Private Use $300
EPA/600/S4-91/008
------- |