AIL United States Environmental Protection Agency Environmental Monitoring Systems Laboratory Las Vegas NV 89193-3478 Research and Development EPA/600/S4-86/011 Jan. 1988 SEPA Project Summary National Surface Water Survey Eastern Lake Survey (Phase I - Synoptic Chemistry) Quality Assurance Report M.D. Best, S.K.Drouse, LW. Creelman, DJ. Chaloud, and D.W. Sutton The quality assurance program for the Eastern Lake Survey - Phase I included quality assurance and quality control activities for field and laboratory operations and for verification and validation of the data. A combination of blank, duplicate, and audit samples were analyzed to provide an external check on the quality of 32 physical and chemical parameters and to allow early detection of problems in sample collection, processing, and analysis. The statistical analysis of the verified data set included estimates of instrumental and system detection limits, system decision limits, overall and analytical within-batch precision, and overall and analytical among-batch precision. Quantita- tion limits also were calculated for use in evaluating the precision of the survey data. The results of these statistical analyses were compared to the data quality objectives for detectability and precision. This report was submitted in partial fulfillment of Contract Numbers 68-03-3050 and 68-03- 3249. This report covers a planning, implementation, and data review period from March 1983 to January 1986, and work was completed as of December 1986. This Project Summary was developed by EPA's Environmental Monitoring Systems Laboratory, Las Vegas, NV, to announce key findings of the research project that is fully documented in a separate report of the same title (see Project Report ordering information at back). Introduction The National Surface Water Survey (NSWS) is a three-phase project within the National Acid Precipitation Assessment Program (NAPAP). The U S. Environmental Protection Agency (EPA) initiated the NSWS in 1983. The purpose of Phase I of the NSWS was to document the present chemical status of lakes and streams in areas of the United States that are potentially susceptible to the effects of acidic deposition. This report provides an overview of the quality assurance (QA) activities and results for one component of the NSWS, the Eastern Lake Survey - Phase I (ELS-I), which evaluated the chemical status of 1,798 lakes in the eastern United States. The lakes were selected from three regions east of the Mississippi River that are potentially susceptible to acidification. The EPA Environmental Monitoring Systems Laboratory in Las Vegas, (EMSL-LV), Nevada, had primary responsibility for the ELS-I QA program and sampling operations. Thirty-two chemical and physical parameters were selected for in situ or laboratory measurement. Data from these measurements of single lake samples provided information to evaluate the present status of lakes on a regional basis. Data quality objectives (DQOs) were defined in terms of the precision and accuracy of measured values for ------- each parameter and of the representativeness, comparability, and completeness of the resulting data base. The anticipated range of values and the required detection limits were also specified for each measurement. Equipment and protocols for sampling, chemical analysis, and data processing were based on the best available methods and were standardized in order to achieve the DQOs. A draft QA plan and a draft analytical methods manual were used during pilot studies which were conducted to test all aspects of the ELS-I research plan. Activities tested include the lake selection process, the proposed sampling and analytical protocols, the DQOs, the QA program, and the data management system. The final QA plan and the final methods manual incorporated revisions based on results obtained from the pilot studies and on changes implemented during the ELS-I. Sampling and Analytical Methodologies The primary QA goals of ELS-I base site operations were to obtain accurate physicochemical and geographical data at each lake site, to collect representative lake samples without introducing contamination, to preserve the integrity of samples until their analysis at contract laboratories, and to perform selected analyses at field laboratories. Field personnel used helicopters equipped with fixed floats to collect samples. Activities of field laboratory personnel included: 1. Receiving lake and QA samples and field data from each sampling team and assessing sample condition upon receipt. 2. Reviewing field data forms and field laboratory forms for accuracy and completeness. 3. Incorporating audit samples with lake samples to form a batch of samples for analysis. 4. Analyzing the batch of samples for pH, dissolved inorganic carbon (DIG), true color, and turbidity. 5. Performing aluminum extrac- tions. 6. Filtering, preserving, and shipping samples to contract analytical laboratories for additional analyses. 7. Coordinating sample shipment information with the EPA Sample Management Office in Alexandria, Virginia, and with the EMSL-LV in Las Vegas. 8. Distributing copies of data and data forms to the appropriate offices. Standardized forms were developed to record measurements made at each lake and at the field and contract analytical laboratories. One copy of each form was sent to Oak Ridge National Laboratory for entry into the NSWS data base, and a second copy was sent to QA personnel in Las Vegas. Figure 1 shows the flow of samples and data from the field to the field laboratory and to the analytical laboratory. Data Comparability Studies Standardized techniques for sampling and chemical analyses of water samples from the ELS-I ensured that the effect (if any) of variance from sampling and analysis procedures on the differences between lakes could be identified. Studies were also undertaken to determine whether ELS-I data could be compared to survey data from other countries and to data obtained by using different analytical methods. Subsamples from 215 ELS-I samples collected in the southern Blue Ridge Mountains were shipped via commercial courier to Norway for chemical analysis of 14 parameters. Similarly, 105 subsamples from the Adirondack Mountains were analyzed in Canada for 18 chemical parameters. A second study used 2,047 split samples from the ELS-I to compare data from chemical analyses by flame atomic absorption spectroscopy (AAS) and inductively coupled plasma emission spectroscopy (ICPES). The ICPES analyses were performed at the EPA Environmental Research Laboratory in Corvallis, Oregon, and by ELS-1 contract laboratories. The flame AAS analyses were performed by ELS-I laboratories only. Procedures Operational Quality Assurance and Quality Control Program The QA program used a combination of blank, duplicate, and audit samples to provide an external check on the quality of the data obtained from measurements of the 32 physicochemical parameters and to allow early detection of problems in sample collection, processing, and analysis. Quality control (QC) protocols for field sampling, field and analytical laboratory activities, and data base management were implemented ensure reliability of the data. The estimated number of samples be analyzed and the estimated rate sample collection required contract! with more than one analytical laborati to meet the QA and QC requiremer Laboratories that submitted low bids response to the formal bidding proce were evaluated on the basis of th analytical laboratory performance. The that passed were visited to verify th qualifications and capabilities. Data quality also depended on i ability of the field and laboratc personnel to properly collect, proce and analyze the samples. Training v\ essential to ensure consistent applicati of all operational and QA-C procedures. Coordination of the ELS operations required close communicat among all participants to ensure that 1 program objectives were met. Da monitoring of field sampling, fie laboratory, and analytical laboratc activities facilitated safety and logisti coordination. During the actual sampli phase, the most critical lines communication were between t logistics personnel in Las Vegas and I field stations and between the < personnel in Las Vegas and the contr analytical laboratories. Logisti personnel also coordinated and track the shipment of samples to the analyti laboratories and coordinated t shipment of supplies to field stations. 1 QA staff made daily calls to the analyti laboratories to ensure that the < procedures were being implement according to survey requirements a that the samples were being handled z analyzed properly. Sampling and Field Laboratc Quality Control Field sampling QC procedur included daily calibration of the Hydro units, measurements of QC che samples for pH and conductance, a sampling site verification. The < procedures for field laboratory operatic included daily instrument calibrate measurements of QC check samples < laboratory duplicates, and preservation sample aliquots for additior measurements by the analytii laboratories. The objectives of samp preservation at the field laboratories w to (1) inhibit chemical and biologi activity, (2) prevent changes due volatilization, and (3) prevent effects c ------- Routine Lake Samples Field Blank Samples 1 1 — — — - 11 Daily S< of Sam Field Duplicate Sample \ Field Laboml itch pies __ Field/Laboratory Audit Sample(s) ory \ r Relabeling Analysis (DIC. pH, Turbidity, True Color) OC Check Batch Trailer Duplicate Samples Samples Sample -*• Data "•*- Aliquot Preparation Aluminum Extraction Preservation Shipment to Analytical Laboratory Analytical Laboratory Analysis r Internal OC Samples Laboratory Blank, Matrix Spike, OC Sample Check t Batch Samples Laboratory Duplicate -»- Data -*- Figure 1. Flow of samples and data through field and analytical laboratories, Eastern Lake Survey - Phase I. ------- to precipitation or adsorption. Seven preserved aliquots were prepared from each bulk sample (routine, field duplicate, field blank, or field audit). Four parameters (pH, DIG, true color, and turbidity) required immediate analysis. Use of a mobile field laboratory for these analyses permitted these four measurements to be completed within 16 hours of sample collection. Analytical Laboratory Quality Control Protocols Analytical laboratory QC included the use of QC charts, calibration of instruments and analysis of QC check samples (QCCS), detection limit QCCS, and duplicate samples. A maximum sample holding time, from the time of sample collection to sample analysis, was established for each parameter measured in the analytical laboratories. These holding times were based upon information from the literature, the best scientific judgment related to the defined needs, and the logistical demands and limitations of the ELS-I. Data Base Quality Assurance Oak Ridge National Laboratory (ORNL) personnel managed the data base for the ELS-I. The data are stored in four major data sets: (1) a raw data set of field and analytical laboratory data, (2) a verified data set, (3) a validated data set, and (4) a final data set. Oak Ridge personnel entered the field and laboratory data into the data base and used a double entry and comparison process to minimize data entry errors. The verified data set provides a data base in which any values that are questionable on the basis of known physicochemical relationships are qualified with a flag. The validated data set provides data that have been evaluated, using all available information, for internal and regional consistency. The final data set provides a representative summary of sample values for use in generating population estimates. Data Verification The raw data were verified by: (1) communicating daily with the field and analytical laboratories; (2) assessing the completeness and consistency of each data package (one per sample batch) on receipt, and reviewing any comments or questions associated with the batch or sample under evaluation; (3) evaluating the preliminary routine and QA sample data; (4) obtaining confirmation, correction, or reanalysis data from the laboratories as needed to address atypical values; and (5) providing correcting entries to ORNL for establishing the verified data set. EMSL-LV QA personnel developed a computer software package (AQUARIUS) to automate this procedure as much as possible. These five steps produced a verified data set in which all values that did not meet a number of criteria based on physicochemical relationships were flagged, replaced with either corrected or reanalyzed data, or replaced with codes indicating a missing value. Data Validation The data validation process identified potential errors in chemical analyses that could not be revealed by the verification procedures. The validation process also evaluated the quality of nonchemical variables. Data validation employed pre-existing software to identify possible outliers and to evaluate possible systematic error in the measurement process. Both of these aspects were exploratory (as opposed to test-oriented). The objective of data validation was to identify individual values or sets of values that warrant special attention or caution when used for analysis of survey results or when used for building models based upon survey data. Final decisions regarding data quality were based on all available information. Data Base Review An independent data audit team inspected routine and duplicate data from each of the four data sets (raw, verified, validated, and final) that constitute the ELS-I data base. The data base review included an evaluation of the adequacy of documentation for value changes made between the raw and the final data sets. For some of the parameters, the audit team evaluated the documentation by a search and review of all of the data. For other parameters, they selected a random subsample of the changed values for review. The review consisted of a comparison of the old and new values against the field and laboratory records and the verification reports. The review also included a determination of the correctness of data entry. The completeness of the data base was estimated for each of the parameters by examining the justifications for missing value codes. The correctness of the calculations of population estimates was verified I implementing the estimation algorithr on a test data set which was a subset the final ELS-I data set. Results Operations Evaluation The ELS-I was completed with the time required by the research plj and the statistical requirements f adequate sample size were achieved all field stations. There were no maj interruptions in field operations due accidents, weather, or equipment failui The sampling and laboratory protocc were successful for most procedure This survey design should serve as model for future field studies of a simi nature. Lake and Sample Information Approximately 90 percent of tl lakes initially selected for sampling we visited by sampling crews. Of those lak visited, 96 percent (1,612) were actua sampled. In addition to those lakes, 1 special-interest lakes were sampled 1 a total sample of 1,798 lakes. Less than 20 percent of the lak visited were sampled at a depth otr than the one originally specified. Only percent of the lakes sampled we thermally stratified; thus, 95 percent of samples were acceptable in terms of t research plan requirement that a sin( water sample was to be collected frc each lake during a period when the lak were isothermal. In total, 2,389 routine, field duplica and field blank samples were deliver from the field laboratories to the contr< analytical laboratories. One matrix spi and one laboratory duplicate we analyzed for each batch of sampl making a total of 2,639 sets of analyses Field Problems and Resolution; The field crews collected samples planned during the ELS-I with average yield of 20 processed sampl per operating day from each fie laboratory. Problems which we identified and corrected prior to or duri field operations included an inaccur. pH meter, use of contaminated bla sample water at one field laboratory, a destruction of one set of split samples the commercial courier. At the conclusion of the survey, number of recommendations were ma for improving field operations. A syst< for efficient and complete informati transfer should be used between t management team and field personnel ------- ensure that all new developments or modifications in operational protocols are consistently disseminated and understood. Field personnel should be prepared to make special arrangements for shipment of samples to the analytical laboratories on weekends. All field samplers should receive comprehensive instruction to ensure consistency of sample collection and processing activities. Rotation or replacement of field personnel should be minimized because it may lead to inconsistencies in data reporting and it requires additional time and effort for training. Analytical Laboratory Problems and Resolutions Several problems were identified and corrected during the analytical laboratory operations. Two significant difficulties centered on the inaccurate calibration method used with silica measurement at one laboratory and on aluminum contamination problems at three laboratories. Methods Evaluation Methods studies addressed a number of analytical questions before and during the survey operations. The measurement of free dissolved fluoride was found to be impractical, and ion- selective electrodes (ISE) were favored over ion chromatography (1C) for measurement of total dissolved fluoride. The Gran analysis method was found to be useful for determinations of acid- neutralizing capacity (ANC); however, problems were encountered with the calculations for base-neutralizing capacity (BNC). The procedure for measuring total extractable aluminum and the design of the pH sample chamber were modified during the pilot study. Effects of filtration on dissolved iron and aluminum concentrations in the audit samples were detected during the survey operations and were attributed to sample instability. A nitrate contamination problem in blank samples was also encountered. To meet analytical laboratory quality control specifications, nitrate concentrations in field blank samples were required to be less than 0.01 mg I/1. During the pilot study, up to 18 mg L'1 nitrate were detected in field blanks, suggesting a serious contamination problem. This contamination was not present in the analytical laboratory blank samples. During the pilot study, two sources of contamination were found to be the deionized water and the sample containers. Contamination from the filtration apparatus was a third source. The ELS-I was about half completed before nitrate contamination from the filtration apparatus was isolated and eliminated. Nitrate data from samples that were processed on or before the change in the filtration protocol were replaced with data from reanalyzed samples. Evaluation of Split Sample Analyses from Norway and Canada In a study undertaken to determine whether data obtained using analytical methods used in this survey are comparable to the data obtained by methods used in other regions where surface water surveys take place, 110 pairs of aliquots (split samples) from 97 lakes in the southern Blue Ridge Mountains were sent to the Norwegian Institute for Water Research in Oslo and to an ELS-I analytical laboratory. Each laboratory analyzed the split samples for 11 chemical parameters. This study also used 105 split samples from 92 lakes in the Adirondack Mountains. Some of these split samples were shipped to the Water Quality National Laboratory of the Canada Centre for Inland Waters in Burlington, Ontario, and to an ELS-I analytical laboratory. Other split samples were sent to the Water Quality Section of the Ontario Ministry of the Environment in Rexdale and to an ELS-I analytical laboratory. Analytical data from the three laboratories were compared for 17 parameters by using linear regressions and sign tests. The data from the pairs of laboratories that analyzed the samples for the same parameters are comparable for most of the parameters as indicated by the high values of the regression coefficients (r2 > 0.95 for 38 of 56 comparisons and r2 > 0.90 for 48 of 56 comparisons). The linear regression analysis also indicated that there are significant differences (uniform bias or concentration-dependent bias) between the ELS-I and Norwegian measurements and between the ELS-I and Canadian measurements for nearly all of the parameters. The sign test results show that as a set, the ELS-I values are greater in magnitude than the Norwegian values for 5 of the 11 parameters and less than the Norwegian values for 3 of the parameters; there is no significant difference between the ELS-I and Norwegian values for the remaining 3 parameters. The magnitude of the set of ELS-I values is greater than the magnitude of the Canadian values for 11 of the 17 parameters and it is less than the magnitude of the Canadian values for 4 of the parameters; there is no significant difference between the ELS-I and Canadian values for the remaining 2 parameters. The components of overall variability in these split sample data include random error (measurement imprecision) and systematic error (accuracy and specificity of the method). Measurement imprecision and differences in the sample composition over time may have contributed to the overall variability for all of the parameters measured by the ELS-I laboratories and by the Norwegian and Canadian laboratories. Differences in analytical methodology contributed to the overall variability for some parameters. Evaluation of Data Base Quality A review of the data base showed that less than 3 percent of the raw data was classified as reporting errors. These errors were corrected in the verified data set. Sample reanalysis was requested for less than 4 percent of the originally reported raw data values. Less than 1 percent of the reported data required correction because of transcription or data entry errors. An independent review of the data base indicated that nearly all of the value changes had been documented; that there were few if any data entry errors; that the data base was essentially complete; and that the calculations of population estimates were correct. Evaluation of Quality Assurance Data Blank Sample Data Field blanks and analytical laboratory blanks were analyzed during the ELS-I. The 245 field blanks were analyzed for turbidity at the field laboratory and for 21 physicochemical parameters at the analytical laboratory. The analytical laboratories used calibration blanks or reagent blanks to determine background levels and to calculate instrumental detection limits. Evaluation of the blank sample data demonstrated that the goals for instrumental detection limits were generally achieved. However, to interpret the data, results from the field blanks ------- must also be taken into consideration. Extremely low detection limits achieved in the laboratory are of limited value in defining usable data when they are lower than the system detection limit (the background from sample collection and handling). The system decision limit should be considered during data interpretation when comparing population estimates. Duplicate Sample Data Analysis of field duplicate samples provided data for estimating the overall within-batch precision. The 125 field duplicate samples that were processed by the field laboratories were analyzed with the routine samples and field blanks at the analytical laboratories. The field and analytical laboratories also performed duplicate chemical analyses on one sample per batch as a QC check on analytical within-batch precision. Overall within-batch precision estimates were numerically larger, as was expected, than the analytical within-batch precision estimates for all but five parameters. The estimated overall and analytical within-batch precision was considered to be adequate to meet the DQOs. Audit Sample Data Six types of audit samples (2 field naturals, 2 field synthetics, and 2 laboratory synthetics) were analyzed for 23 parameters during the ELS-I. Field naturals and field synthetics were handled in the same manner as were routine, field blank, and field duplicate samples to estimate the overall among- batch precision (including the effects of sample processing, but not of sample collection). Laboratory synthetics were prepared as processed aliquots by a support laboratory; these were relabeled at the field laboratory and were incorporated with the sample batch to estimate the analytical among-batch precision. The among-batch precision estimates from field natural audit sample measurements were within the DQOs for all of the parameters except total extractable Al, total Al, and Cl- in the FN2 (low ANC) samples, and they were within the DQOs for all of the parameters in the FN3 (high ANC) samples. The among-batch precision estimates from field and laboratory synthetic audit sample measurements were generally within the DQOs for parameters with sample values above the quantitation limit, and they were generally not within the DQOs for parameters with values below the quantitation limit. The natural audit samples were also used to estimate the amount of relative interlaboratory bias by comparing measured values from the contract laboratories with each other and with the referee laboratory measurements. The synthetic audit samples were used to provide information on absolute interlaboratory bias, and thus had to be prepared from solutions of known composition. Examination of the theoretical values and the measured values for the synthetic audit samples indicated that the actual sample composition may have differed on different days; the amounts of measurement imprecision and interlaboratory bias were thought to be small by comparison to the differences in sample composition. Quantitation limits were a useful means of classifying the data in order to objectively evaluate the among-batch precision estimates for the 23 parameters. High relative precision (percent relative standard deviation) was not expected for measurements close to the detection limits, and it was not achieved for any parameters with means less than quantitation limits except dissolved organic carbon (DOC). Conversely, high relative precision was expected and was generally achieved for measurements with higher mean values. The overall and analytical within-batch precision estimates for DOC measurements showed patterns which were opposite to the patterns shown by all other parameters; i.e., measurements at higher concentrations exhibited greater variability than measurements at lower concentrations. An independent statistical analysis concluded that, overall, the imprecision of a perfectly calibrated instrument under constant conditions is small in comparison to the bias and trend introduced by procedural variations and by changes in the sample composition over time, that precision estimates from field natural audit samples are the best indicators of measurement uncertainty in the routine samples; that measurements of laboratory synthetic audit samples provide estimates of accuracy and relative interlaboratory bias; that daily lots of audit samples should be prepared and divided into aliquots and the aliquots should be assigned to different analytical laboratories at random; that interlaboratory comparisons should be made using data from the same audi sample lot; that audit sample compositioi should be varied among the lots in orde to estimate the precision and accuracy 2 various concentrations and to prever recognition of the audit samples by th< analytical laboratories; and that a empirical approach should be used t calculate detection limits. An EMSL-L1 review of this independent analysi questioned two of the statistic* assumptions: a zero median wa assumed for blank sampl measurements, and the measuremer error variance was assumed to b independent of the analyte concentratio for some purposes but to be proportion; to concentration for other purpose; Other issues raised by the EMSL-L reviewers concern the evidence used t support the statement describing th blank sample data as non-normal; us of the root mean square for estimatin the mean of the percent relative standar deviation; use of a single slope i correcting the bias and trend in aud sample data; use of standard deviatic instead of standard error to estimal precision in the corrected data; an omission of the DQOs as a basis for ft data quality evaluations. Variability Four types of precision estimate (overall within-batch, overall amont batch, analytical within-batch, an analytical among-batch) identify th amounts of data variability that can t attributed to sample collectioi processing, storage, and analysis. Fi the ELS-I, each type of precisic estimate was used to estimate a differe aspect of data variability. Overall within-batch precisic estimates were expected to t numerically larger than analytical withi batch precision estimates by an amou equal to the variability from samp collection, processing, and storag Similarly, overall among-batch precisii estimates were expected to t numerically larger than analytic among-batch precision estimate Analytical and overall among-bat< precision estimates were expected to I numerically larger than the correspond'!! analytical and overall within-bati precision estimates by amounts that we equal to the temporal variability. Exceptions to the expect! relationships were generally associat with the presence of one or me extreme outliers in the verified data s with values close to the detection limit, with a methodological problem. Ma ------- exceptions involved small differences in the estimated precision. In several cases it was necessary to retain confirmed but questionable values in the verified data set; these values were later deleted during data validation. The confirmed questionable values influenced the statistical evaluation of the ELS-I data. For subsequent surveys, a data qualifier was added to ensure that such values are retained in the verified data set but are not included in statistical calculations. Summary Data quality objectives were established for precision, accuracy, representativeness, comparability, and completeness. Pilot studies were conducted prior to the ELS-I to identify and correct problem areas in field and laboratory operations. Field and laboratory QA and QC protocols were implemented during the survey to detect and eliminate equipment and procedure problems. QA and QC procedures in the data entry and verification provided a means for detecting and correcting transcription, transposition, and typographical errors as well as analytical errors in the data base. The raw data were verified using physicochemical relationships. Values that did not meet criteria were flagged, corrected, or replaced with codes for missing values. The data quality objectives that were specified for detectability and precision were generally achievable. On the basis of a limited amount of information the amount of bias in the data appeared to be small in relation to the imprecision. Field operations were a successful means for obtaining samples and field data consistent with the ELS-I research plan. The statistical requirments for adequate sample size were achieved at all field stations. Comparability of data obtained by different analytical methods and different laboratories was addressed by independent studies. Less than 3 percent of the raw data was classified as reporting errors, and these were corrected in the verified data set. ------- M.D. Best, S.K. Drowse', LW. Creelman, D.J. Chaloud, and D. W. Sutton are with Lockheed Engineering and Management Services Co. ,lnc.. Las Vegas. NV 89119. Eugene P. Meier is the EPA Project Officer (see below). The complete report, entitled "National Surface Water Survey, Eastern Lake Survey (Phase I - Synoptic Chemistry) Quality Assurance Report," (Order No. PB 88-1337491 AS; Cost: $19.95, subject to change) will be available only from: National Technical Information Service 5285 Port Royal Road Springfield, VA22161 Telephone: 703-487-4650 The EPA Project Officer can be contacted at: Exposure Assessment Research Division Environmental Monitoring Systems Laboratory Las Vegas, Nevada 89193-3478 United States Environmental Protection Agency Center for Environmental Research Information Cincinnati OH 45268 BULK RATE POSTAGE & FEES PAID EPA PERMIT No. G-35 Official Business Penalty for Private Use $300 EPA/600/S4-86/011 0000329 PS ------- |