United States Environmental Protection Agency Environmental Monitoring Systems Laboratory Las Vegas, NV 89193-3478 Research and Development EPA/600/S4-91/008 April 1992 Project Summary Direct Delayed Response Project: Soil Characterization Comparison L.K. Fenstermaker, G.E. Byers, T.H. Starks, M.J. Miah, C.J. Palmer, and K.D. Lauckner A large amount of soil characteriza- tion data has been collected as a com- ponent of the Direct Delayed Response Project (DDRP) in the acid rain Aquatic Effects Research Program. An interlaboratory comparison study was undertaken to identify the comparabil- ity of this data to that obtained from representative soil characterization laboratories. Participating laboratories were selected at random from four re- gions of the U.S. and two regions of Canada. Two original DDRP contract laboratories also participated. Duplicate samples of six soil audit materials and two liquid soil extracts were sent to each of the laboratories in two sepa- rate batches. Laboratories used their own protocols to perform the analyses requested except for the contract labo- ratories which followed the DDRP pro- tocol. The largest number of different methods used was for the measure- ment of cation exchange capacity. The results between the DDRP soil survey data and this study's results were com- pared using Youden-pair plots, and standard statistical tests. Overall, the DDRP data were comparable to the data from this study. However, out of the total 141 comparisons involving results from six or more laboratories, the re- sults from the two contract laborato- ries did not meet the comparison crite- ria in 19 cases. Since there was never a case in which both contract laborato- ries failed, it would appear that the 19 cases which were not comparable were due to random analytical errors, incor- rectly reported results, or misapplica- tion of DDRP protocol. This Project Summary was developed by EPA's Environmental Monitoring Systems Laboratory, Las Vegas, NV, to announce key findings of the research project that is fully documented in a separate report of the same title (see Project Report ordering information at back). Introduction This study was designed and imple- mented to answer questions regarding data comparability and accuracy related to the Direct Delayed Response Project (DDRP) soil surveys. An important component of the quality assurance program within the DDRP is the assessment of the data qual- ity. One of the attributes of data quality is comparability. Comparability is of particu- lar importance for two reasons. First, it was recognized that it would be neces- sary to develop a standardized analytical methods manual for the DDRP soil survey analyses. Information was compiled from many soil methods manuals in the devel- opment of the DDRP methods manual. Although this manual was subjected to extensive peer review, the question still remains as to the comparability of data resulting from these methods to those ob- tained by the methods presently in use by soil characterization laboratories in the U.S. and Canada. Second, although the DDRP soil surveys do cover broad land areas of particular interest to the issue of acid depo- sitions effects, there are other regions in the U.S. and Canada of similar impor- tance. To make predictions for these re- gions, it would be necessary to rely on existing soil characterization information. Are these data comparable to the DDRP Printed on Recycled Paper ------- data, or will there need to be adjustments in the data or in the DDRP models to account for existing differences? If the data from this study are comparable, then it is possible that the various soil characteriza- tion data bases in existence elsewhere may be comparable to the DDRP data bases. The objective of this study was to investigate whether the analytical data ob- tained by laboratories using DDRP soil analysis protocols are comparable to data for the same audit materials analyzed by other soil characterization laboratories in Canada and the U.S. not using DDRP protocols. Procedure The purpose of this study was to deter- mine whether data obtained by using the DDRP protocol to measure soil param- eters are comparable with data obtained from other soil characterization laborato- ries using their own individual protocols. Laboratories were selected at random from four regions of the U.S. and two regions of Canada. Two original DDRP contract laboratories also participated. Six different soil audit samples used in the study were chosen from bulk soil samples which had been previously prepared for use in the Northeastern and Southern Blue Ridge Province DDRP soil surveys (A, Bs, Bw, and C horizons), and the Mid-Appalachian DDRP soil survey (O, A, Bw, and B). The bulk soil samples originated in the north- eastern United States and represented the major soils of the region. The audit samples included five mineral soils and one organic soil. Two batches of the soil samples were shipped to the laboratories at separate times for analysis. Detailed procedures for the soil analytical protocols to be used by the noncontract laborato- ries were not specified for this study. Only the general method was specified for each group of parameters, e.g., calcium ex- changeable in ammonium acetate. The purpose of this study was to compare the analytical results from different laborato- ries, not to compare their protocols. For this study, the quantitative criterion used to determine whether the DDRP pro- tocol provided results comparable to those obtained by other soil characterization laboratories is: if the sample means from the contract laboratories are not extremes in a box plot of all participating laborato- ries sample means for a particular param- eter and soil audit material, then the re- sults are comparable for that parameter and soil audit material combination. There- fore, in this study, if the results from the contract laboratories using the DDRP pro- tocol are near the center of the results distribution from the noncontract laborato- ries, then the DDRP protocol are consid- ered to produce results that are "compa- rable" to results being obtained by soil laboratories outside the DDRP program. Youden-pair plots derived from box plots were prepared for visual determination of data comparability for each parameter and soil sample combination. After reviewing the data, it was decided that certain standard tests of equality of means and variances would provide de- scriptive information for the reader. How- ever, since the number of samples and replicates is so limited, these tests do not have sufficient power to address data com- parability. The standard tests addressed the following issues: time effect, (i.e., did the soil samples change over time); ho- mogeneity of laboratory results; pairwise comparisons to determine which laborato- ries were significantly different from each other; within and between group variabil- ity; and the comparability of combined pa- rameters, (i.e. groups of similar param- eters). Results The contract laboratories results were found to be comparable to results from noncontract laboratories in 122 out of 141 combinations of parameters and audit ma- terials. In all 19 failures to meet the com- parability criterion, only one, but not al- ways the same one, of the two contract laboratories had measurements outside the prescribed range. By itself, not too much should be made of this since the number of laboratories reporting results is less than 8, and it is impossible for the criterion to indicate that both contract laboratories are too large or both too small. However, a check found only one of the 19 cases where if one contract laboratory mean is too large, the other contract laboratory had the next smaller mean value, or where one contract laboratory mean was too small, the other contract laboratory had the next larger mean. That case is "A1 extractable in pyro-PO4" in the Bs soil sample. Thus, these 19 cases provide little evidence that the DDRP protocol is caus- ing extreme measurements. In 9 of these 19 cases, the failure was caused by only one of the two measurements by a con- tract laboratory being outside the pre- scribed range. The only place where there seems to be something approaching a consistent pattern of such failures was in the measurement of sand content. Six of the 19 failures reported are related to the measurement of sand content, where the failing measurements were all too large. The results for the supplemental statis- tical analyses provide an indication that the DDRP databases may be comparable to data generated by other soil character- ization laboratories. However, the conclu- sions drawn from the supplemental tests may be limited due to the very small de- grees of freedom. These tests differed from the Youden-pair plots in that the plots compared the contract versus the noncontract laboratory data, and the tests compared the DDRP soil survey data to the noncontract laboratory data. Overall, the supplemental tests indicate that the DDRP and noncontract data are compa- rable. Conclusions The general conclusion is that the DDRP protocol does provide results comparable to those results obtained by other soil laboratories for 30 parameters. In 122 of 141 comparisons involving results from six or more laboratories, the results from the two contract laboratories using the DDRP protocol met the comparability cri- terion. There was never a case in which both contract laboratories failed to meet the comparability criterion and only one case where one failed and the other con- tract laboratory had the next most ex- treme sample mean. Therefore, it would appear that the 19 failures to obtain com- parability are either a result of random analytical errors in individual contract labo- ratories, incorrectly reported results, or from the misapplication of the DDRP pro- tocol. In the case of the sand results, where there is a pattern of measurements by one laboratory that is too large, it is possible that interpretation or adherence to the protocol may have changed. The information in this document has been funded wholly or in part by the United States Environmental Protection Agency under EPA Cooperative Agreement Num- ber CR814701-01, to the Environmental Research Center, University of Nevada and EPA Contract Number 68-03-3249 to Lockheed Engineering and Sciences Co. It has been subject to the Agency's peer and administrative review, and it has been approved for publication as an EPA docu- ment. Mention of trade names or commercial products does not constitute endorsement of recommendation for use. &U.S. GOVERNMENT PRINTING OFFICE: 1992 - 64S-080/402.10 ------- ------- L.K. Fenstermaker, T.H. Starks, C.J. Palmer, and K.D. Lauckner are with the Environmental Research Center, University of Nevada, Las Vegas 89154; G.E. Byers and M.J. Miah are with Lockheed Engineering and Sciences Com- pany, Las Vegas, NV 89119. L.J. Blume andD.T. Heggem are the EPA Project Officers (see below). The complete report, entitled "Direct Delayed Response Project: Soil Characteriza- tion Comparison, "(OrderNo. PB92-153428/AS; Cost:$43.00;subjecttochange) will be available only from: National Technical Information Service 5285 Port Royal Road Springfield, VA 22161 Telephone: 703-487-4650 The EPA Project Officers can be contacted at: Environmental Monitoring Systems Laboratory U.S. Environmental Protection Agency Las Vegas, NV 89193-3478 United States Environmental Protection Agency Center for Environmental Research Information Cincinnati, OH 45268 BULK RATE POSTAGE & FEES PAID EPA PERMIT No. G-35 Official Business Penalty for Private Use $300 EPA/600/S4-91/008 ------- |