O Baflelle . . . Putting Technology To Work Environmental Technology Verification Program Advanced Monitoring Systems Pilot Test/QA Plan for Verification of Portable Analyzers etVetVety ------- TEST/QA PLAN for VERIFICATION OF PORTABLE ANALYZERS December 8, 2000 Prepared by Battelle 505 King Avenue Columbus, OH 43201-2693 ------- Table of Contents Page 1.0 Introduction 1 1.1 Test Description 1 1.2 Test Objective 1 1.3 Organization and Responsibility 2 1.3.1 Battelle 2 1.3.2 Vendors 5 1.3.3 EPA 6 2.0 Verification Approach 7 2.1 Scope of Testing 7 2.2 Experimental Design 8 2.3 Test Samples 9 2.3.1 QC Samples 10 2.3.2 PT Samples 10 2.3.3 Environmental Samples 11 2.4 Reference Method 13 3.0 Materials and Equipment 14 3.1 Laboratory Supplies 14 3.2 Field Supplies 15 3.3 Reference Instrument 15 4.0 Procedures 16 4.1 Test Sample Preparation and Storage 16 4.2 Sample Identification 16 4.3 Sample Analysis 17 4.3.1 Reference Method 17 4.3.2 Analyzers Undergoing Verification 17 4.4 Schedule 18 5.0 Data Handling and Reporting 20 5.1 Data Acquisition and Review 20 5.2 Statistical Calculations 21 5.2.1 Accuracy 21 5.2.2 Precision 21 5.2.3 Linearity 22 5.2.4 Method Detection Limit 22 5.2.5 Matrix Interferences 22 5.2.6 Operator Bias 23 ------- Table of Contents (Continued) Page 5.2.7 Inter-Unit Reproducibility 23 5.2.8 Rate of False Positives/False Negatives 23 5.3 Data Review 24 5.4 Reporting 25 6.0 Quality Assurance/Quality Control 26 6.1 QC of Reference Method 26 6.2 Audits 27 6.2.1 Performance Evaluation Audit 27 6.2.2 Technical Systems Audit 27 6.2.3 Audit of Data Quality 28 6.3 QA/QC Reporting 28 6.4 Corrective Action 28 7.0 References 30 APPENDIX A Summary of Test Samples for Selected Analytes 31 APPENDIX B Battelle Standard Operating Procedure for At-Sea Collection of Hydrographic Data Using CTD and Rosette System 35 LIST OF TABLES Table 1. Schedule of Verification Test Days 19 Table 2. Summary of Data Recording Process for the Verification Test 24 LIST OF FIGURES Figure 1. Organization Chart for the Verification Test 3 ------- DISTRIBUTION LIST Ms. Elizabeth A. Betz U.S. Environmental Protection Agency National Exposure Research Laboratory MD-44 Research Triangle Park, NC 27711 Mr. Robert Fuerst U.S. Environmental Protection Agency National Exposure Research Laboratory MD-46 Research Triangle Park, NC 27711 Ms. Elizabeth Hunike Quality Assurance Specialist U.S. Environmental Protection Agency National Exposure Research Laboratory ERC Annex, MD-46 Research Triangle Park, NC 27711 Mr. Raymond Loebker USEPA Facilities 26 West Martin Luther King Drive MS 525 Cincinnati, OH 45268 Mr. Geoff Dates River Watch Network RR 1 Box 209 Hartland, VT 05048 Ms. Marty Link Groundwater Section Nebraska Dept. of Environmental Quality The Atrium, 1200 N St. Ste. 400, P.O. Box 98922 Lincoln, NE 68509-8922 Mr. Vito Minei Acting Director, Division of Environmental Quality Suffolk County, NY, Department of Health Services 220 Rabro Drive Hauppauge, NY 11788 Dennis Goldman, Ph.D., R.G. Executive Director for IEE & Program Development Officer The Geological Society of America 3300 Penrose Place P.O. Box 9140 Boulder, CO 80301-9140 ------- Portable Water Analyzers Test/QA Plan Page 1 of 30 Version 1 Date: 12/08/00 1.0 Introduction 1.1 Test Description This test/quality assurance (QA) plan provides procedures for a verification test of portable analysis technologies that determine various contaminants in water. The verification test will be conducted under the auspices of the U.S. Environmental Protection Agency (EPA) through the Environmental Technology Verification (ETV) program. The purpose of ETV is to provide objective and quality assured performance data on environmental technologies, so that users, developers, regulators, and consultants can make informed decisions about purchasing and applying these technologies. The verification test will be performed by Battelle, of Columbus, Ohio, which is managing the ETV Advanced Monitoring Systems (AMS) Center through a cooperative agreement with EPA. The scope of the AMS Center covers verification of monitoring technologies for contaminants and natural species in air, water, and soil. In performing the verification test, Battelle will follow the procedures specified in this test/QA plan, and will comply with the data quality requirements in the "Quality Management Plan for the ETV Advanced Monitoring Systems Pilot" (QMP).1 1.2 Test Objective The verification of portable water analyzers will be conducted to quantify the analytical and operational performance characteristics of these technologies. A variety of quality control, performance evaluation, and environmental water samples will be analyzed to assess the capabilities of the analyzers relative to accepted reference methods. ------- Portable Water Analyzers Test/QA Plan Page 2 of 30 Version 1 Date: 12/08/00 1.3 Organization and Responsibility The verification test will be performed by Battelle with the participation of the interested vendors who will be having their analyzers verified. The testing will occur at Battelle's Columbus, Ohio laboratories, at local waterways within Columbus, and at Massachusetts Bay near Duxbury, Massachusetts. The organizational chart below shows the individuals from Battelle, the vendor companies and the EPA who will have responsibilities in the verification test. The specific responsibilities of these individuals are detailed in Figure 1. 1.3.1 Battelle Mr. Adam Abbgy is the AMS Center Verification Testing Coordinator. In this role, Mr. Abbgy will have overall responsibility for ensuring that the technical, schedule, and cost goals established for the verification test are met. More specifically, Mr. Abbgy will: • Assemble a team of qualified technical staff to conduct the verification test. • Direct the team in performing the verification test in accordance with the test/QA plan. • Ensure that all quality procedures specified in the test/QA plan and in the QMP are followed. • Prepare the draft test/QA plan, verification reports, and verification statements. • Revise the draft test/QA plan, verification reports, and verification statements in response to reviewers' comments. • Respond to any issues raised in assessment reports and audits, including instituting corrective action as necessary. • Serve as the primary point of contact for vendor representatives. ------- Portable Water Analyzers Test/QA Plan Page 3 of 30 Version 1 Date: 12/08/00 Figure 1. Organization Chart for the Verification Test ------- Portable Water Analyzers Test/QA Plan Page 4 of 30 Version 1 Date: 12/08/00 • Establish a budget for the verification test and monitor staff effort to ensure that budget is not exceeded. • Ensure that confidentiality of vendor information is maintained. Dr. Thomas J. Kelly is the Verification Testing Leader for the AMS Center. As such, Dr. Kelly will provide technical guidance and oversee the various stages of verification testing. He will: • Support Mr. Abbgy in preparing the test/QA plan and organizing the testing. • Review the draft test/QA plan. • Review the draft verification reports and statements. Ms. Karen Riggs is Battelle's manager for the AMS Center. As such, Ms. Riggs will: • Review the draft test/QA plan. • Review the draft verification reports, and verification statements. • Coordinate distribution of the final test/QA plan, verification reports and statements. • Ensure that necessary Battelle resources, including staff and facilities, are committed to the verification test. • Ensure that vendor confidentiality is maintained. • Support Mr. Abbgy in responding to any issues raised in assessment reports and audits. • Maintain communication with EPA's technical and quality managers. Battelle Technical Staff will conduct the testing of the analyzers during the verification test and associated experimental activities. The responsibilities of these technical staff include: ------- Portable Water Analyzers Test/QA Plan Page 5 of 30 Version 1 Date: 12/08/00 • Assist in the collection of samples. • Analyze samples for the verification test as described in this test/QA plan. Mr. Chuck Lawrie is Battelle's Quality Manager for the AMS Center. As such Mr. Lawrie will: • Review the draft test/QA plan. • Conduct a technical systems audit once during the verification test. • Audit at least 10% of the verification data. • Prepare and distribute an assessment report for each audit. • Verify implementation of any necessary corrective action. • Issue a stop work order if self audits indicate that data quality is being compromised; notify Battelle's Center Manager if stop work order is issued. • Provide a summary of the quality assurance/quality control (QA/QC) activities and results for the verification reports. • Review the draft verification reports and statements. • Have an overall responsibility for ensuring that the test/QA plan is followed. 1.3.2 Vendors Vendor representatives will: • Review the draft test/QA plan. • Approve the test/QA plan. • Provide two off-the-shelf models of the analyzers to be verified for the duration of the verification test. ------- Portable Water Analyzers Test/QA Plan Page 6 of 30 Version 1 Date: 12/08/00 • If necessary, instruct Battelle personal on how to operate and maintain the analyzers prior to testing. • Review their respective draft verification report and statement. 1.3.3 EPA EPA's responsibilities in the AMS Center are based on the requirements stated in the "Environmental Technology Verification Program Quality and Management Plan of the Pilot Period (1995-2000)" (QAMP). The roles of the specific EPA staff are as follows: Ms. Elizabeth Betz is EPA's Quality Manager. For the verification test, Ms. Betz will: • Review the draft test/QA plan. • Perform at her option one external technical system audit during the verification test. • Notify the Battelle AMS Center Manager to facilitate a stop work order if external audit indicates that data quality is being compromised. • Prepare and distribute an assessment report summarizing results of external audit. • Review draft verification reports and statements. Mr. Robert Fuerst is EPA's manager for the AMS Center. As such, Mr. Fuerst will: • Review the draft test/QA plan. • Approve the final test/QA plan. • Approve the final verification reports. • Review the draft verification statements. ------- Portable Water Analyzers Test/QA Plan Page 7 of 30 Version 1 Date: 12/08/00 2.0 Verification Approach 2.1 Scope of Testing This test/QA plan specifically addresses verification testing of 1) portable analyzers that provide quantitative measurements of metals and other inorganic contaminants in water, and 2) portable test kits that provide qualitative or semi-quantitative measurements. The quantitative analyzers consist of a portable electronic instrument that often requires a specific reagent solution. Typically the reagent and the water sample are mixed, and the mixture is inserted into the analyzer and probed either photometrically or electrochemically to provide a quantitative determination of the target contaminant. These analyzers report results via a digital display or electronic output signal. Technologies which provide only qualitative results are typically test strips or reagent solutions, which when exposed to the water sample indicate the presence of the analyte through a visible color change. These approaches are designed primarily to indicate the presence or absence of the target analyte relative to some regulatory or health-based concentration level. Semi-quantitative results can be obtained using these same technologies, by comparison of the color to that of standards run with the samples or to a color comparison chart provided by the manufacturer. These comparators typically have discrete color levels indicating different analyte concentrations, and the results are based on subjective visual comparisons made by the user. In some cases quantitative results can be obtained by submitting the samples to a laboratory and analyzing them with a colorimeter. Both quantitative and qualitative analyzers are designed to be operated by non-technical users. Each of these technologies, whether quantitative or qualitative, may be capable of detecting a variety of aqueous analytes including dissolved metals, and other inorganic cations and anions. The verification of all portable water analyzers will involve testing them with known calibration standards, and through the analysis of realistic samples by both the analyzers being verified and appropriate reference methods. Statistical comparisons of the analytical results from the reference methods and the analyzers being verified will provide a basis for quantitative ------- Portable Water Analyzers Test/QA Plan Page 8 of 30 Version 1 Date: 12/08/00 performance evaluations of the analyzers. Each of the analyzers will also be evaluated in terms of ease of use, cost, and sample throughput. The quantitative analyzers (and semi-quantitative technologies) provide at least some measure of the analyte concentration, and will be evaluated in terms of: • accuracy • precision • linearity • method detection limit • matrix interference effects • operator bias. The purely qualitative technologies indicate only the presence or absence of a color change associated with a given analyte. The color change can be semi-quantified by comparison to a color chart. As such, the performance of these technologies will be verified in terms of: • rate of false positives/false negatives • lowest calibration concentration producing a positive response • highest calibration concentration producing a negative response • matrix interference effects • operator bias. 2.2 Experimental Design Two units of each water analyzer being tested will undergo verification testing. The verification results for the two units will be reported, and intercompared to assess unit-to-unit reproducibility. The verification test will involve challenging the analyzers being verified with a variety of test samples, including a set of fresh water samples and a set of salt water samples representative of those likely to be analyzed using these devices. All samples will be analyzed by the analyzers being verified, and by an appropriate reference method. Comparison of the results from the analyzers to those from the reference method will be used to quantitatively ------- Portable Water Analyzers Test/QA Plan Page 9 of 30 Version 1 Date: 12/08/00 assess accuracy, linearity, and detection limit. Multiple aliquots of each test sample will be analyzed separately to assess precision. For each analyzer, identical sets of samples will be analyzed independently by two separate operators (a technical and a non-technical Battelle staff member) to test for the existence of operator bias on analyzer performance. The analyzers are designed for non-technical operators. The non-technical staff member will have little prior knowledge of the analyzer being verified. Interferences and matrix effects will be assessed by separately evaluating accuracy, precision, and linearity on distinctly different sample matrices, i.e., prepared samples, drinking water, fresh water, and salt water samples. Sample throughput will be estimated based on the time required to analyze a sample set. Performance parameters, such as ease of use and reliability, will be based on documented observations of the operators. Each analyzer will be used in a field environment, as well as in a laboratory setting, to assess the impact of field conditions on performance. Unit-to-unit reproducibility will be evaluated by intercomparing results from two units of each technology tested. 2.3 Test Samples Test samples to be used in this verification test will include quality control (QC) samples, performance test (PT) samples, and environmental water samples. Tables listing the number and type of different samples to be analyzed for selected analytes are provided in Appendix A. The QC and PT samples will be prepared from purchased standards. The QC sample concentrations for most analytes will be targeted to the EPA maximum contaminant level (MCL) for drinking water or other regulatory guidelines as are applicable. The PT samples will cover the range from 10 percent to 1000 percent of that guideline level. The environmental water samples indicated in Appendix A will be collected from various drinking water and surface water sources. All samples will be analyzed both by the two units of each analyzer undergoing testing, and by a laboratory reference method. Every tenth sample will be analyzed twice by the reference method, to assess the reference method's precision. ------- Portable Water Analyzers Test/QA Plan Page 10 of 30 Version 1 Date: 12/08/00 2.3.1 QC Samples Prepared QC samples will include both laboratory reagent blanks (RB), and laboratory fortified matrix (LFM) samples. The RB samples will be prepared from ASTM type II deionized water and will be exposed to identical handling and analysis procedures as other prepared samples, including the addition of all reagents. These samples will be used to help ensure that no sources of contamination are introduced in the sample handling and analysis procedures. The LFM samples will be prepared as aliquots of environmental samples, spiked in the field to increase the analyte concentration by 10 ppb. The spike solution used to prepare the LFM will be prepared in the laboratory and brought to the field site. These samples will be used to help identify if matrix effects have an influence on the analytical results. At least 10% of all the prepared samples to be analyzed will be RBs and at least one sample taken from each sampling site will be an LFM. Quality control standards (QCS) will be used as a calibration check to verify that the analyzers being verified and the reference instruments are properly calibrated and reading within defined control limits. These standards will be purchased from a commercial supplier and subject only to dilution as appropriate. The calibration of all instruments will be verified using a QCS before and after the testing period, as well as after every tenth sample. Additional standards will also be purchased from an independent supplier, for use in a performance evaluation audit, as described in Section 6.2.1. 2.3.2 PT Samples In general, two types of PT samples will be used in this verification test. All PT samples will be prepared in the laboratory using ASTM Type II water as the water source. One type of PT solution will include only the single analyte at various concentrations and will be prepared specifically to help determine the analyzer accuracy, linearity, and detection limit. To determine the detection limit of the analyzers, a solution with a concentration five times the vendor's reported detection limit will be used. Seven nonconsecutive replicate ------- Portable Water Analyzers Test/QA Plan Page 11 of 30 Version 1 Date: 12/08/00 analyses of this solution will be made to obtain precision data with which to determine the method detection limit (Appendix A). Additionally, solutions will be prepared to assess the linearity over a broad concentration range. Four aliquots of each of these solutions will be prepared and analyzed separately, to assess the precision of the analyzers (Appendix A). The second type of PT sample to be used in this test will help establish the effects of potential matrix interferences on the performance of the analyzers. These samples will be prepared from solutions with known concentrations of the analytes (see Appendix A), and will be spiked with potentially interfering species likely to be found in typical water samples. The first sample will contain low levels of interferences which will consist of 1 mg iron, 3 mg sodium chloride, and 0.1 mg of sulfate per liter at a pH of 6. The second sample will contain high levels of interferences which will consist of 10 mg iron, 30 mg sodium chloride, and 1.0 mg of sulfate per liter at a pH of 3. Four replicate samples of each of these solutions will be analyzed. 2.3.3 Environmental Samples Environmental samples, including tap water (well and community sources), fresh surface water, and salt water will be collected from a variety of sources, and will be used to evaluate technology performance. Samples will be collected from the following sources: • Drinking fountain within Battelle • Residential tap (community water) • Residential tap (well water) • Alum Creek Reservoir • Olentangy River • Scioto River Massachusetts Bay. ------- Portable Water Analyzers Test/QA Plan Page 12 of 30 Version 1 Date: 12/08/00 In all cases the technologies undergoing verification will be used to analyze the water samples as soon as possible after collection. The results of those analyses will be compared to subsequent reference method analyses of the same samples in the laboratory. The drinking water samples from the Battelle drinking fountain and the two different residential sites will be collected directly from the tap into 2-L high density polyethylene (HDPE) containers. The samples will be split into 100-mL aliquots. Four aliquots of each sample will be analyzed at the time of collection by each of the analyzers being verified. Four aliquots of each sample will be returned to Battelle for reference analysis. These aliquots will be preserved and stored as appropriate for the target analyte and analyzed within appropriate holding times. Fresh water samples from the reservoir and from each river will be collected in 500-mL HDPE containers. The samples will be collected at the surface of the water near the shoreline by submerging the containers no more than one inch below the surface of the water. The samples will be split into four 100-mL aliquots. Each body of water will be sampled at four distinct locations. Two aliquots of each sample will be analyzed in the field at the time of collection by the analyzers being verified. One aliquot of each sample will be returned to Battelle for reference analysis for each target analyte. This aliquot will be preserved and stored as necessary for the target analyte and analyzed within appropriate holding times. Salt water samples will be collected from the Massachusetts Bay using a Rosette system as described in Battelle Duxbury Operations Standard Operating Procedure No. 5 27S-01 (Appendix B). Five 100-mL aliquots of each sample will be obtained for each analyzer undergoing testing. Samples will be collected from the surface of the bay and from the sediment/water column interface at four distinct locations. One aliquot of each sample will be analyzed at the time of collection by each unit of the analyzer being verified, and by each operator. One aliquot of each sample will be returned to Battelle in Columbus, OH for analysis by the reference method. The field testing will occur on three separate days. These days do not need to be consecutive. One day will be used to collect and test tap water within Battelle, at a residential site using community water, and at a residential site using well water. A second will be spent collecting and testing fresh water samples from the Scioto and Olentangy Rivers which are near ------- Portable Water Analyzers Test/QA Plan Page 13 of 30 Version 1 Date: 12/08/00 Battelle's Columbus facilities, and from Alum Creek Reservoir north of Columbus. A third day will be spent collecting salt water samples from aboard a boat in the Massachusetts Bay. In these field testing efforts, the technologies being verified will be transported, handled, and used under normal field conditions, as a test of real-world reliability and performance. Field conditions (temperature, humidity, weather conditions) will be noted at least twice on each day of field testing. 2.4 Reference Method Technology verification will involve, in part, comparison of the results from each analyzer being verified to the results obtained from an appropriate reference method. The reference method chosen for this verification test is an EPA standard method for the analysis of water. Samples containing metal ions will be analyzed using inductively-coupled plasma mass spectrometry (ICP-MS) according to EPA Method 200.8.2 Samples containing anions will be analyzed using ion chromotography (IC) according to EPA Method 300.13 ------- Portable Water Analyzers Test/QA Plan Page 14 of 30 Version 1 Date: 12/08/00 3.0 Materials and Equipment In general, this verification test relies on the materials and equipment provided by the vendors. Battelle will provide the following equipment and materials. 3.1 Laboratory Supplies The following laboratory supplies will be needed for the preparation of the PT samples and the QC samples: • ASTM type II water • trace metal grade nitric acid • 1-L, 250-mL, 100-mL Class A volumetric flask • 10-mL Class A volumetric pipets • 0.5-mL, 1.0-mL micro pipets • Micro pipet tips • NIST traceable reference standard for target analyte • 100 ppm iron standard • sodium chloride • sodium sulfate • HDPE containers • pH meter or strips capable of reading pH levels of 6 and 3. ------- Portable Water Analyzers Test/QA Plan Page 15 of 30 Version 1 Date: 12/08/00 3.2 Field Supplies The following supplies will be needed for the collection of field samples: • ASTM type II water • 125-mL, 500-mL, 1000-mL HDPE containers • 1-mL micro pipet • 1-mL micro pipet tips • 100-mL HDPE volumetric flasks. • Coolers and blue ice packs for sample storage • Thermometer (to determine air and water temperature) 3.3 Reference Instrument The reference method for analysis of metal will be performed on a Perkin Elmer Sciex 6000 ICP-MS or equivalent. The reference method for anions will be performed on a Dionex 600 Ino Chromatograph or equivalent. ------- Portable Water Analyzers Test/QA Plan Page 16 of 30 Version 1 Date: 12/08/00 4.0 Procedures 4.1 Test Sample Preparation and Storage QC and PT samples will be prepared from commercially available NIST traceable standard solutions. Purchased solutions will be diluted to appropriate concentrations using distilled, deionized water in Class A volumetric glassware. In some cases, additional species will be added to the solutions to assess the effect of interferences on the performance of the technologies. These interferences will be added to simulate levels of contaminants which may be found in typical water sources. The QC and PT samples will be prepared within two days of analysis, and stored at approximately 4°C until use. Environmental water samples will be collected from the sources indicated in Section 2.3.3, and will be stored in HDPE containers. Sample analysis will be performed at the time of collection by the analyzers being verified. The samples to be analyzed by the reference methods will be stored at 4°C until analysis and preserved with nitric acid at a pH of less than 2 for metal analytes. The reference analyses will be performed within 14 days of collection or the field sampling will be repeated. 4.2 Sample Identification Aliquots to be analyzed will be drawn from the prepared standard solutions or from environmental samples and placed in uniquely identified sample containers for subsequent analysis. The sample containers will be identified by a unique identification (ID) number. A master log of the samples and sample ID numbers for each analyzer will be kept by Battelle. The ID number, date, person collecting, sample location, and time of collection will be recorded on a chain- of-custody form for all field samples. ------- Portable Water Analyzers Test/QA Plan Page 17 of 30 Version 1 Date: 12/08/00 4.3 Sample Analysis 4.3.1 Reference Method The reference instrument will be operated according to the recommended procedures in the instruction manual, and samples will be analyzed according to an appropriate reference method. Analysis for metals will be conducted according to EPA Method 200.8, "Determination of Trace Elements in Waters and Wastes by Inductively Coupled Plasma - Mass Spectrometry."2 Analysis for anions will be performed using EPA Method 300.1 "Determination of Inorganic Anions by Ion Chromotography". Results from the reference analyses will be recorded electronically and compiled by the laboratory performing the analyses into a report format, including the sample ID and the analyte concentration for each sample. 4.3.2 Analyzers Undergoing Verification Each vendor will be required to provide two units of his portable water analyzer. Each unit will be subjected to the test procedure independently, and separate verification results will be reported for each unit. Those results will then be intercompared to assess unit-to-unit reproducibility. Each of the analyzers being verified will be used to analyze the full set of samples for the target analyte chosen by the respective vendor. As shown in Tables 1 and 2, the sample set will include replicates of each of the PT, QC, and environmental samples. Analysis of the complete set of samples will be performed twice for each of the analyzers — once by a non- technical staff member of Battelle, and once by a technical staff member using the same sample aliquot. The analyses will be performed according to the manufacturer's recommended procedures as described in the user's instructions or manual for the respective analyzers. Calibration and maintenance of the analyzers will be performed as specified by the manufacturer. Results from the analyzers being verified will be recorded manually by the operator on appropriate data sheets. In addition to the analytical results, the data sheets will include records ------- Portable Water Analyzers Test/QA Plan Page 18 of 30 Version 1 Date: 12/08/00 of the time required for sample analysis and operator observations concerning the use of the analyzer (i.e., frequency of calibration, ease of use, maintenance, etc.). 4.4 Schedule The verification test described here will take place over a four-day period at Battelle's laboratories in Columbus, Ohio, followed by a one-day period at Battelle's Ocean Sciences Laboratory in Duxbury Massachusetts. The one-day period need not follow immediately after the four-day period. Table 1 lists the activities to be conducted on those test days. The samples referred to in Table 1 are those listed in Appendix A. All participating analyzers will undergo verification testing on the same days. The same samples analyzed by the instruments undergoing testing will be analyzed by the reference method. All analyzers being tested for a given analyte will be challenged with the same set of samples. Separate aliquots will be drawn from a single sample for each type of technology. No direct comparison will be made between the results from different analyzers; however, it is to the benefit of potential users of the analyzers that test conditions be as similar as possible. It will be necessary for participating vendors to provide their analyzers to Battelle one week before the start of testing, so that project staff may become familiar with operating the units before testing begins. This period will also be used to clarify any questions about the analyzer's operation or maintenance. Vendor staff may need to be present for this familiarization stage to provide training in operating the analyzers. During the verification test, all analyzers will be operated by Battelle staff. Vendors are encouraged to stay and observe during the duration of the tests. Analyzers and associated equipment (if not consumables) will be returned to the vendors at the completion of testing. ------- Portable Water Analyzers Test/QA Plan Page 19 of 30 Version 1 Date: 12/08/00 Table 1. Schedule of Verification Test Days Test Day Testing Location Activity Day One Battelle Columbus Laboratory Analysis of PT samples and associated QC samples with operator #1. Day Two Battelle Columbus Laboratory Analysis of PT samples and associated QC samples with operator #2. Day Three Columbus Field Location Collection and analysis of environmental samples and LFM samples at four tap water locations. Day Four Columbus Field Location Collection and analysis of environmental samples and LFM samples at four locations within three fresh water sites. Day Five Transport to Battelle Duxbury, Mass Shipping and handling of analyzers undergoing verification to field test site. Day Six Duxbury, Mass Field Location Collection and analysis of environmental samples and LFM samples at salt water locations; shipping of environmental samples to Columbus for subsequent reference analysis. ------- Portable Water Analyzers Test/QA Plan Page 20 of 30 Version 1 Date: 12/08/00 5.0 Data Handling and Reporting 5.1 Data Acquisition and Review A variety of data will be acquired and recorded electronically or manually by Battelle staff in this verification test. Operation, maintenance, and results from the analyzers being verified and sampling procedures, will generally be documented on data sheets or in laboratory record books. Results from the reference instruments will be compiled in electronic format. Records received by or generated by Battelle staff in the verification test will receive a one- over-one review within two weeks after receipt or generation, respectively, before these records are used to calculate, evaluate, or report verification results. These records may include electronic records; laboratory record books; sampling records from the field test; or equipment calibration records. This review will be performed by a Battelle technical staff member involved in the verification test, but not the staff member that originally received or generated the record. The review will be documented by the person performing the review by adding his/her initials and date to a hard copy of the record being reviewed. This hard copy will then be returned to the Battelle staff member who received or generated or who will be storing the record. In addition, data calculations performed by Battelle will be spot-checked by Battelle technical staff to ensure that calculations are performed correctly. Calculations to be checked include reference analysis results and statistical calculations described in this test/QA plan. The data obtained from this verification test will be compiled and reported independently for each analyzer being verified. No intercomparison of the results from one vendor's technology to another's will be made. However, intercomparisons will be made between the results for duplicate analyzers from each vendor. ------- Portable Water Analyzers Test/QA Plan Page 21 of 30 Version 1 Date: 12/08/00 5.2 Statistical Calculations 5.2.1 Accuracy Accuracy of the analyzers being verified will be assessed relative to the results obtained from the reference analyses. Samples will be analyzed by both the reference method and the analyzers being verified. The results for each set of analyses will be averaged, and the accuracy will be expressed in terms of a relative average bias (B) as calculated from the following equation: where d is the average difference between the readings from the analyzer being verified and those from the reference method, and CR is the average of the reference measurements. Accuracy will be assessed independently for each analyzer to determine inter-unit reproducibility. Additionally, the results will be analyzed independently for the readings obtained from the two operators to determine if significant operator bias exists. 5.2.2 Precision The standard deviation (S) of the results for the replicate samples will be calculated and used as a measure of instrumental precision at each concentration. where n is the number of replicate samples (see Table 1), Ck is the concentration measured for the k'h sample, and C is the average concentration of the replicate samples. The instrumental d B =^xl00 CR (2) (3) ------- Portable Water Analyzers Test/QA Plan Page 22 of 30 Version 1 Date: 12/08/00 precision at each concentration will be reported in terms of the relative standard deviation (RSD), e.g., S RSD C x 100 (4) 5.2.3 Linearity Linearity will be assessed by linear regression with the analyte concentration measured by the reference method as independent variable, and the reading from the analyzer being verified as dependent variable. Linearity will be expressed in terms of the slope, intercept, and the coefficient of determination (r2). 5.2.4 Method Detection Limit The method detection limit (MDL)3 for each analyzer will be assessed from the seven replicate analyses of a fortified sample with an analyte concentration of five times the vendor's estimated detection limit (see Tables 1 and 2). The MDL will be calculated from the following equation: MDL = txS (5) where t is the Student's value for a 99% confidence level, and S is the standard deviation of the replicate samples. The MDL values for the two units of each analyzer will be reported as separate results. 5.2.5 Matrix Interferences The effect of interfering matrix species on the response of an analyzer to a given analyte will be calculated as the ratio of the difference in analytical response to the concentration of interfering species. For example, if the addition of 500 ppb of an interfering species results in a ------- Portable Water Analyzers Test/QA Plan Page 23 of 30 Version 1 Date: 12/08/00 difference of 10 ppb in the analytical result, the relative sensitivity of the analyzer to that interferent is calculated as 10 ppb/500 ppb = 2%. 5.2.6 Operator Bias To assess operator bias for each analyzer, the results obtained from each operator will be compiled independently and subsequently compared. The existence of statistically significant operator bias will be assessed through a Mest of the data. 5.2.7 Inter-Unit Reproducibility The results obtained from two identical units of each analyzer will be compiled independently for each analyzer and for each operator, and compared to assess inter-unit reproducibility. The results will be interpreted using a Mest to assess if significant differences exist between the units tested. 5.2.8 Rate of False Positives/False Negatives The rate of false positives/false negatives of the qualitative analyzers for each analyte will be assessed relative to the guidance level. Analyte reported as being above that level by the analyzer being verified, but below that level by the reference method, will be considered a false positive. Analyte not reported as being above the guidance level by the analyzer being verified, but reported as above that level by the reference method, will be considered a false negative. The rate of false positives/false negatives will be expressed as a percentage of total samples analyzed for each matrix. ------- Portable Water Analyzers Test/QA Plan Page 24 of 30 Version 1 Date: 12/08/00 5.3 Data Review Records generated in the verification test will receive a one-over-one review within two weeks after generation, before these records are used to calculate, evaluate, or report verification results. Table 2 summarizes the types of data to be recorded. These records may include laboratory record books or reference method analytical results. This review will be performed by a Battelle technical staff member involved in the verification test, but not the staff member that originally generated the record. EPA/contractor and/or vendor staff will be consulted as needed to clarify any issues about the data records. The review will be documented by the person performing the review by adding his/her initials and date to a hard copy of the record being reviewed. This hard copy will then be returned to the Battelle staff member who generated or who will be storing the record. Table 2. Summary of Data Recording Process for the Verification Test Data to be Recorded Responsible Party Where Recorded How often Recorded Disposition of data(a) Dates, times of test events Battelle Laboratory record books Start/end of test, and at each change of a test parameter. Used to organize/check test results; manually incorporated in data spreadsheets as necessary. Test parameters (temperature, analyte/ interferant identities and concentrations, gas flows, etc.) Battelle Laboratory record books When set or changed, or as needed to document stability. Used to organize/check test results, manually incorporated in data spreadsheets as necessary. Reference method sampling data Battelle Laboratory record books At least at start/end of reference sample, and at each change of a test parameter. Used to organize/check test results; manually incorporated in data spreadsheets as necessary. Reference method sample analysis, chain of custody, and results Battelle Laboratory record books, data sheets, or data acquisition system, as appropriate Throughout sample handling and analysis process Transferred to spreadsheets (a) All activities subsequent to data recording are carried out by Battelle. ------- Portable Water Analyzers Test/QA Plan Page 25 of 30 Version 1 Date: 12/08/00 5.4 Reporting The data obtained in the verification test will be compiled separately for each vendor's analyzer, and the statistical evaluations described in Section 4 will be applied to each data set without reference to any other. At no time will data from different vendor's analyzers be intercompared or ranked. Following completion of the statistical evaluations, a draft verification report will be prepared for each vendor's analyzer, stating the verification test procedures and documenting the performance observed. The draft verification reports will each be submitted to the respective vendors for review and comments. The comments provided by each vendor on his draft verification report will be the basis for revision of that report. The revised reports will then be submitted to EPA and AMS Center stakeholders for peer review. The reports will then be revised again to address the peer review comments and submitted for final EPA approval. In parallel with preparation of the verification reports will be preparation of the verification statement for each analyzer. The verification statement is a two- to three-page summary of the technology, the test procedures, and the test results. Each draft verification statement will be submitted to the respective vendor for review, and then will follow the same revision and EPA review process as the reports. Upon approval by EPA, each verification statement will be signed by a senior manager of Battelle and by an EPA laboratory director. Final verification reports and statements will be posted on the ETV website (http://www.epa.gov/etv), and original signed verification statements will be provided to the respective vendors for use in marketing their products. ------- Portable Water Analyzers Test/QA Plan Page 26 of 30 Version 1 Date: 12/08/00 6.0 Quality Assurance /Quality Control The QA/QC activities associated with this verification test will focus primarily on reference analysis, sample preparation and handling, and data recording and analysis. An independent audit covering each of these areas will be performed by the Battelle Quality Manager to ensure the quality of the verification test. 6.1 QC of Reference Method Analysis of QC samples throughout the verification test will be used to document the performance of the reference methods. RB samples will be analyzed to ensure that no sources of contamination are present. If the analysis of an RB sample indicates a concentration above the MDL for the reference instrument, contamination will be suspected. Any contamination source(s) will be corrected, and proper blank readings will be achieved, before proceeding with the verification test. The accuracy of the reference methods will be verified before the beginning and after the conclusion of each testing day. The instruments to be used for reference will be initially calibrated according to the procedures specified in the reference method. The instrument calibration will be verified using an appropriate QCS. If the QCS analysis differs by more than ±10% from the true value of the standard, the instrument will be recalibrated before continuation of the test. LFM samples will be analyzed to assess if matrix effects influence the results of the reference methods. The percent recovery (R) of the spiked solution will be calculated from the following equation: C -C R=— x 100 s (6) ------- Portable Water Analyzers Test/QA Plan Page 27 of 30 Version 1 Date: 12/08/00 where Cs is the analyzed concentration of the spiked sample, C is the analyzed concentration of the unspiked sample, and s is the concentration equivalent of the analyte spike. If the percent recovery of an LRM falls outside the range from 85-115%, a matrix effect will be suspected. 6.2 Audits 6.2.1 Performance Evaluation Audit A performance evaluation (PE) audit will be conducted to assess the quality of the reference measurements made in this verification test. A performance evaluation audit involves challenging the instruments used for reference methods with standards that are independent of those used to calibrate the instruments for the test. For the PE audit, an independent standard will be obtained from a vendor that is different from the one that supplied the QC standards. This comparison of the QC and performance evaluation standards will be done once during the verification test, on the day that the portable analyzers are in transit to the Massachusetts field sampling location. Agreement of the standards within 10% is required for the measurements to be considered as acceptable. Failure to achieve this agreement will trigger recalibration of the instruments with the original QC standards, and a repeat of the performance evaluation comparison. Failure in the second comparison requires obtaining another set of standards, and repeating the performance audit. 6.2.2 Technical Systems Audit The Battelle Quality Manager will conduct a technical systems audit at least once during the course of the verification test. The purpose of this audit is to ensure that the verification test is being performed in accordance with this test/QA plan and the AMS Center QMP1, and that all procedures described in this test/QA plan are being followed. This audit will review the reference standards and methods used, compare actual test procedures to those specified in this test/QA plan, and review data acquisition and handling procedures. An independent technical ------- Portable Water Analyzers Test/QA Plan Page 28 of 30 Version 1 Date: 12/08/00 systems audit may also be performed by EPA Quality Management staff during the verification test, at EPA's discretion. 6.2.3 Audit of Data Quality At least 10% percent of the data acquired during the verification test will be audited during the verification test. Battelle's Quality Manager will trace the data from the initial acquisition, through reduction and statistical analysis, to final reporting, to ensure the integrity of the reported results. All calculations performed on the data undergoing the audit will be checked. 6.3 QA/QC Reporting Each assessment and audit will be documented in accordance with Section 2.9.7 of the QMP for the AMS Center. The results of the technical systems audit and the audit of data quality will be sent to the EPA. Assessment reports will include the following: • Identification of any adverse findings or potential problems • Response to adverse findings or potential problems • Recommendations for resolving problems • Confirmation that solutions have been implemented and are effective • Citation of any noteworthy practices that may be of use to others 6.4 Corrective Action The Battelle or EPA Quality Managers during the course of any assessment or audit will identify to the technical staff performing experimental activities any immediate corrective action that should be taken. If serious quality problems exist, the Battelle Quality Manager is authorized to stop work. Once the assessment report has been prepared, the Verification Test Coordinator ------- Portable Water Analyzers Test/QA Plan Page 29 of 30 Version 1 Date: 12/08/00 will ensure that a response is provided for each adverse finding or potential problem, and will implement any necessary follow-up corrective action. The Battelle Quality Manager will ensure that follow-up corrective action has been taken. ------- Portable Water Analyzers Test/QA Plan Page 30 of 30 Version 1 Date: 12/08/00 7.0 References 1. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Pilot, U.S. EPA Environmental Technology Verification Program, prepared by Battelle, Columbus, Ohio, September 1998. 2. U.S. EPA Method 200.8, Determination of Trace Elements in Waters and Wastes by Inductively Coupled Plasma-Mass Spectrometry, Revision 4.4, April 1991. 3. U.S. Code of Federal Regulations title 40 Part 136 Appendix B. ------- APPENDIX A SUMMARY OF TEST SAMPLES FOR SELECTED ANALYTES ------- Table A-l. Summary of Test Samples3 for Lead Type of Sample Sample Characteristics Concentration No. of Samples Quality Control RBb ~0 10% of all LFM b 30 ppb 1 per site QCS b 30 ppb 10% of all Performance Test For the determination of detection limit Five times the manufacturer's estimated detection limit 7 Lead 1.5 ppb 4 Lead 5 ppb 4 Lead 15 ppbc 4 Lead 45 ppb 4 Lead 150 ppb 4 Analyte spiked with interference 45 ppb with low interference 8 Analyte spiked with interference 45 ppb with high interference 8 Environmental Drinking fountain Unknown 4 Community water Unknown 4 Well water Unknown 4 Alum Creek Reservoir Unknown 4 Olentangy River Unknown 4 Scioto River Unknown 4 Massachusetts Bay surface water Unknown 4 Massachusetts Bay water at sediment/water column interface Unknown 4 a Listing is for clarity; samples will be analyzed in randomized order for the verification testing. b See Section 2.3.1 for descriptions of these samples. c MCL for lead. ------- Table A-2. Summary of Test Samples3 for Nitrate Type of Sample Sample Characteristics Concentration No. of Samples Quality Control RBb ~0 10% of all LFM b 2 ppb above native level 1 per site QCS b 2 ppb 10% of all Performance Test For the determination of detection limit for nitrate Five times the manufacturer's estimated detection limit 7 Nitrate 0.2 ppb 4 Nitrate 0.6 ppb 4 Nitrate 2.0 ppb 4 Nitrate 6.0 ppb 4 Nitrate 20 ppb 4 Analyte spiked with interference 3.0 ppb with low interference 8 Analyte spiked with interference 3.0 ppb with high interference 8 Environmental Drinking fountain Unknown 4 Community water Unknown 4 Well water Unknown 4 Alum Creek Reservoir Unknown 4 Olentangy River Unknown 4 Scioto River Unknown 4 Massachusetts Bay surface water Unknown 4 Massachusetts Bay water at sediment/water column interface Unknown 4 a Listing is for clarity; samples will be analyzed in randomized order for the verification testing. b See Section 2.3.1 for descriptions of these samples. ------- Table A-3. Summary of Test Samples3 for Arsenic Type of Sample Sample Characteristics Concentration No. of Samples Quality Control RBb ~0 10% of all LFM b 10 ppb above native level 1 per site QCS b 10 ppb 10% of all Performance Test For the determination of detection limit Five times the manufacturer's estimated detection limit 7 Arsenic 1 ppb 4 Arsenic 3 ppb 4 Arsenic 10 ppbc 4 Arsenic 30 ppb 4 Arsenic 100 ppb 4 Analyte spiked with interference 10 ppb with low interference 8 Analyte spiked with interference 10 ppb with high interference 8 Environmental Drinking fountain Unknown 4 Community water Unknown 4 Well water Unknown 4 Alum Creek Reservoir Unknown 4 Olentangy River Unknown 4 Scioto River Unknown 4 Massachusetts Bay surface water Unknown 4 Massachusetts Bay water at sediment/water column interface Unknown 4 a Listing is for clarity; samples will be analyzed in randomized order for the verification testing. b See Section 2.3.1 for descriptions of these samples. 0 WHO Provisional Guideline Value for arsenic in drinking water. ------- APPENDIX B BATTELLE STANDARD OPERATING PROCEDURE FOR AT-SEA COLLECTION OF HYDROGRAPHIC DATA USING CTD AND ROSETTE SYSTEM ------- |