United States
Environmental Protection
Agency
Environmental Monitoring
Systems Laboratory
Las Vegas, NV 89193-3478
Research and Development
EPA/600/S4-91/008  April 1992
 Project  Summary
Direct  Delayed
Response Project:
Soil  Characterization
Comparison
L.K. Fenstermaker, G.E. Byers, T.H. Starks,
M.J. Miah, C.J. Palmer, and K.D. Lauckner
  A large amount of soil characteriza-
tion data has been collected as a com-
ponent of the Direct Delayed Response
Project (DDRP) in the acid rain Aquatic
Effects   Research   Program.  An
interlaboratory comparison study was
undertaken to identify the comparabil-
ity of this data to that obtained from
representative  soil characterization
laboratories.  Participating laboratories
were selected at random from four re-
gions of the  U.S. and  two regions of
Canada.  Two original  DDRP contract
laboratories also participated. Duplicate
samples of six soil audit materials and
two liquid soil  extracts were sent to
each of the  laboratories in two sepa-
rate batches. Laboratories used their
own protocols to perform the analyses
requested except for the contract labo-
ratories which followed the DDRP pro-
tocol. The largest number of different
methods  used  was  for the  measure-
ment of cation exchange capacity. The
results between the  DDRP soil survey
data and this  study's results were com-
pared using  Youden-pair plots, and
standard  statistical tests. Overall,  the
DDRP data were comparable to the data
from this study. However, out  of  the
total 141 comparisons involving results
from six or more laboratories, the re-
sults from the two contract laborato-
ries did not meet the comparison crite-
ria in 19 cases. Since there was never
a case in which both contract laborato-
ries failed, it  would appear that the 19
cases which were  not comparable were
due to random analytical errors, incor-
rectly reported  results, or misapplica-
tion of DDRP protocol.
  This Project Summary was developed
by EPA's  Environmental  Monitoring
Systems Laboratory, Las Vegas, NV, to
announce key findings of the research
project that is  fully documented in  a
separate report of the same title (see
Project Report ordering information at
back).

Introduction
  This study  was  designed and imple-
mented to answer questions regarding data
comparability and accuracy related to the
Direct Delayed Response Project (DDRP)
soil surveys. An  important component of
the quality assurance program within the
DDRP is the assessment of the data qual-
ity. One of the attributes of data quality  is
comparability. Comparability is of particu-
lar importance for  two reasons. First,  it
was recognized  that it would be neces-
sary to develop a standardized analytical
methods manual for the DDRP soil survey
analyses. Information was compiled from
many soil methods manuals in the devel-
opment of  the DDRP  methods  manual.
Although this manual was  subjected to
extensive peer review, the  question still
remains as to the  comparability of data
resulting from these methods to those ob-
tained by the methods presently in use by
soil characterization laboratories in the U.S.
and Canada. Second, although the DDRP
soil surveys do cover broad land  areas of
particular interest to the issue of acid depo-
sitions effects, there are other regions  in
the U.S. and Canada of similar impor-
tance. To make  predictions for these re-
gions, it would be  necessary to rely on
existing soil characterization information.
Are these data comparable to the DDRP
                                                  Printed on Recycled Paper

-------
 data, or will there need to be adjustments
 in the  data or in the DDRP  models to
 account for existing differences? If the data
 from this study are comparable, then it is
 possible that the various soil characteriza-
 tion data bases  in existence  elsewhere
 may  be comparable  to  the DDRP data
 bases.  The  objective of this study was to
 investigate whether the analytical data ob-
 tained  by laboratories using  DDRP soil
 analysis protocols are comparable to data
 for the  same audit materials analyzed by
 other soil characterization laboratories in
 Canada and the U.S. not  using DDRP
 protocols.

 Procedure
   The purpose of this study was to deter-
 mine whether data obtained by using the
 DDRP  protocol to measure soil  param-
 eters are comparable with  data obtained
 from other soil characterization laborato-
 ries using their own  individual  protocols.
 Laboratories were selected at random from
 four regions of the U.S. and two regions
 of Canada.  Two  original DDRP contract
 laboratories  also participated. Six different
 soil audit samples used in the study were
 chosen from bulk soil  samples  which had
 been previously prepared for  use in the
 Northeastern and Southern Blue  Ridge
 Province DDRP soil surveys (A, Bs, Bw,
 and C horizons), and the Mid-Appalachian
 DDRP soil survey (O,  A, Bw, and B). The
 bulk soil samples originated in the north-
 eastern United States and represented the
 major  soils of  the  region. The audit
 samples included five mineral  soils and
 one organic soil. Two batches  of the soil
 samples were shipped to the laboratories
 at separate  times for analysis. Detailed
 procedures for the soil analytical protocols
 to be used by the noncontract laborato-
 ries were not specified for this study. Only
 the general method was specified for each
 group  of parameters,  e.g.,  calcium ex-
 changeable  in  ammonium  acetate. The
 purpose of this study was to compare the
 analytical results  from different laborato-
 ries, not to compare their protocols.
   For this study, the quantitative criterion
 used to determine whether the DDRP pro-
tocol provided results comparable to those
obtained by other soil characterization
 laboratories is: if the sample means from
the contract laboratories are not extremes
 in a box plot of all participating laborato-
 ries sample means for a particular param-
eter and soil audit material, then the re-
sults are comparable  for that  parameter
and soil audit material combination. There-
fore, in  this study, if the results from the
 contract laboratories using the DDRP pro-
 tocol are near the center  of  the results
 distribution from the noncontract laborato-
 ries, then the DDRP protocol are consid-
 ered to produce results that are "compa-
 rable"  to results being obtained  by soil
 laboratories outside  the  DDRP program.
 Youden-pair  plots  derived from box plots
 were prepared for visual  determination of
 data comparability  for each parameter and
 soil sample combination.
   After reviewing the data, it was decided
 that certain standard tests of equality of
 means and variances  would  provide de-
 scriptive information for the reader.  How-
 ever, since the number  of samples and
 replicates is so limited, these tests do not
 have sufficient power to address data com-
 parability. The standard  tests addressed
 the following issues: time effect, (i.e., did
 the soil samples change over time); ho-
 mogeneity of  laboratory  results; pairwise
 comparisons to determine which laborato-
 ries were significantly different from  each
 other;  within  and between group variabil-
 ity; and the comparability  of combined pa-
 rameters, (i.e.  groups  of similar  param-
 eters).

 Results
   The  contract laboratories  results  were
 found  to be  comparable to results  from
 noncontract laboratories in 122 out of 141
 combinations of parameters and audit ma-
 terials. In all 19 failures to meet the  com-
 parability  criterion, only one,  but  not al-
 ways the same  one, of the two contract
 laboratories had measurements outside the
 prescribed range. By itself,  not too much
 should be made of this since the number
 of laboratories reporting results is less than
 8, and  it is impossible  for the criterion to
 indicate that both contract laboratories are
 too large  or both too small. However, a
 check  found  only  one of the 19 cases
 where  if one  contract laboratory mean is
 too large,  the  other contract  laboratory
 had the next smaller mean value, or where
 one  contract  laboratory  mean was too
 small,  the other contract laboratory had
 the next larger mean.  That  case is "A1
 extractable in pyro-PO4"  in  the Bs soil
 sample. Thus, these 19  cases provide little
 evidence that the DDRP protocol is caus-
 ing extreme measurements. In 9 of these
 19 cases, the failure was caused by only
one of  the two measurements by a  con-
tract laboratory  being  outside  the  pre-
 scribed range. The  only place where there
 seems  to be  something  approaching a
consistent pattern of such failures was in
 the measurement of sand content. Six of
 the 19 failures reported are related to the
 measurement of sand content, where the
 failing measurements were all too large.
   The results for the supplemental statis-
 tical analyses provide an indication that
 the DDRP databases may be comparable
 to data generated by other soil character-
 ization laboratories. However, the conclu-
 sions drawn  from the supplemental tests
 may be limited due to the very small de-
 grees  of  freedom.  These tests  differed
 from the Youden-pair plots in that the plots
 compared  the  contract  versus  the
 noncontract laboratory data, and the tests
 compared the DDRP soil survey data to
 the noncontract  laboratory data. Overall,
 the supplemental tests  indicate that  the
 DDRP and noncontract data are compa-
 rable.

 Conclusions
   The general conclusion is that the DDRP
 protocol does provide results comparable
 to those  results  obtained  by other soil
 laboratories for 30 parameters.  In 122 of
 141  comparisons involving  results  from
 six or more laboratories, the  results from
 the two  contract laboratories using the
 DDRP protocol met the  comparability cri-
 terion. There was never a case  in which
 both contract laboratories failed to  meet
 the comparability criterion and  only  one
 case where one failed and the other con-
 tract laboratory  had the  next  most ex-
 treme sample mean. Therefore,  it would
 appear that the 19 failures to  obtain com-
 parability  are either a result of random
 analytical errors in individual contract labo-
 ratories,  incorrectly reported results, or
 from the misapplication of the DDRP  pro-
 tocol. In the case  of the  sand  results,
 where there is a pattern of measurements
 by one  laboratory that is too large,  it  is
 possible that interpretation or adherence
 to the protocol may  have changed.
  The  information in this document  has
 been funded wholly or in part by the United
 States Environmental Protection Agency
 under EPA Cooperative Agreement Num-
 ber CR814701-01, to the Environmental
 Research  Center, University  of  Nevada
 and EPA Contract Number 68-03-3249 to
 Lockheed  Engineering and Sciences Co.
 It has been subject  to the Agency's  peer
 and administrative review, and it has been
 approved for publication  as an EPA docu-
 ment.
  Mention of trade names or  commercial
products does not constitute endorsement
of recommendation for use.
                                                                        &U.S. GOVERNMENT PRINTING OFFICE: 1992 - 64S-080/402.10

-------

-------
  L.K. Fenstermaker,  T.H. Starks, C.J. Palmer, and K.D.  Lauckner are with the
    Environmental Research Center, University of Nevada,  Las Vegas 89154;
    G.E. Byers and M.J. Miah are with Lockheed Engineering and Sciences Com-
    pany, Las Vegas, NV 89119.
  L.J. Blume andD.T. Heggem are the EPA Project Officers (see below).
  The complete report, entitled "Direct Delayed Response Project: Soil Characteriza-
    tion Comparison, "(OrderNo. PB92-153428/AS; Cost:$43.00;subjecttochange)
    will be available only from:
          National Technical Information Service
          5285 Port Royal Road
          Springfield, VA 22161
          Telephone: 703-487-4650
  The EPA Project Officers can be contacted at:
          Environmental Monitoring Systems Laboratory
          U.S. Environmental Protection Agency
          Las Vegas, NV 89193-3478
 United States
 Environmental Protection
 Agency
Center for Environmental
Research Information
Cincinnati, OH 45268
      BULK RATE
POSTAGE & FEES PAID
         EPA
   PERMIT No. G-35
Official Business
Penalty for Private Use $300
EPA/600/S4-91/008

-------