Battelle 7/ht cj Innovation Environmental Technology Verification Program Advanced Monitoring Systems Center Test/QA Plan for Verification of Multi-Parameter Water Monitors for Distribution Systems ETv^iTviTv ------- TEST/QA PLAN for Verification of Multi-Parameter Water Monitors for Distribution Systems August 2, 2004 Prepared by Battelle 505 King Avenue Columbus, OH 43201-2693 ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 2 of 35 Version I August 4, 2004 A2 TABLE OF CONTENTS Section Page A PROJECT MANAGEMENT AI Title Page I A2 Table of Contents 2 A3 Distribution List 5 A4 Verification Test Organization 6 A5 Background 12 A6 Verification Test Description and Schedule 12 A7 Quality Objectives 14 A8 Special Training/Certification 14 A9 Documentation and Records 15 B MEASUREMENT AND DATA ACQUISITION Bl Experimental Design 16 B2 Reference Sample Collection 23 B3 Sample Handling and Custody Requirements 23 B4 Laboratory Reference Methods 25 B5 Quality Control Audits and Requirements 25 B6 Instrument/Equipment Testing. Inspection, and Maintenance 26 B7 Instrument Calibration and Frequency 26 B8 Inspection/Acceptance of Supplies and Consumables 26 B9 Non-Direct Measurements 27 B10 Data Management 28 C ASSESSMENT AND OVERSIGHT CI Assessments and Response Actions 30 C2 Reports to Management 32 D DATA VALIDATION AND USABILITY I) I Data Review. Validation, and Verification Requirements 33 D2 Validation and Verification Methods 33 D3 Reconciliation with User Requirements 33 E REFERENCES 34 ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 3 of 35 Version I August 4, 2004 List of Figures Page Figure I Organization Chart 7 Figure 2 Verification Schedule 14 List of Tables Table I Injected Contaminants for Identification 20 Table 2 Description of Reference Samples 24 Table 3 Reference Methods 27 Table 4 Summary of Data Recording Process 29 ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 4 of 35 Version I August 4, 2004 ETV Advanced Monitoring Systems Center Test/QA Plan for Verification of Multi-Parameter Water Monitors for Distribution Systems Version 1 August 4, 2004 APPROVAL: Name Company Date ------- A3 DISTRIBUTION LIST Alex Gordon Prominent Fluid Controls Inc. 136 Industry Drive Pittsburgh, PA 15275 Beth Clark Water Security and Technologies, Inc. 159 East Bri mfield Road Holland. MA 01521 Jim Standard U.S. Filter 1901 W. Garden Rd. Vineland. NJ 08360 John Becker Analytical Technologies 6 Iron Bridge Drive Collegeville, PA 19426 Richard Baril Rosemount Analytical 2400 Barranca Parkway Irvine. CA 92606 Uwe Michalak Sensicore. Inc. 745 Phoenix Drive Ann Arbor. MI 48108 Daniel Kroll Hach Company 5600 Lindbergh Drive Loveland. CO 80538-8998 Martin Harmless Clarion Sensing Systems 3901 W. 30th St. Indianapolis. IN 46222 Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 5 of 35 Version I August 4, 2004 Man-Tech Associates 600 Main Street Tonawanda. NY 14150 Elizabeth A. Betz U.S. Environmental Protection Agency- HEASI) National Exposure Research Laboratory E205-01 EPA Mailroom Research Triangle Park, NC 27711 Robert Fuerst U.S. Environmental Protection Agency- HEASI) National Exposure Research Laboratory D205-05 EPA Mailroom Research Triangle Park, NC 27711 Karen Riggs Amy Di tidal Ryan James Zachary Willenberg Battelle 505 King Ave. Columbus. OH 43201 Robert Menegotto ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 6 of 35 Version I August 4, 2004 SECTION A PROJECT MANAGEMENT A4 VERIFICATION TEST ORGANIZATION The verification test will be conducted under the auspices of the U.S. Environmental Protection Agency (EPA) through the Environmental Technology Verification (ETV) Program. It will be performed by Battelle. which is managing the ETV Advanced Monitoring Systems (A MS) Center through a cooperative agreement with EPA. The scope of the AMS Center covers verification of monitoring technologies for contaminants and natural species in air. water, and soil. The day to day operations of this verification test will be coordinated and supervised by Battelle personnel, with the participation of the vendors who will be having the performance of their multi-parameter water monitors verified. The testing will occur at the EPA's Testing and Evaluation (T&E) Facility in Cincinnati. Ohio. Staff from the T&E Facility will participate in this test by interfacing the technologies with the T&E Facility pipe loops, which use water from the City of Cincinnati distribution system, performing the reference analyses, and when necessary, altering the conditions of the water within the pipe loops. The pipe loops used for testing are designed to simulate conditions within water distribution systems. Vendor representatives will install, maintain, and operate their respective technologies throughout the test unless they give written consent for Battelle staff to carry out these activities. Quality assurance (QA) oversight will be provided by the Battelle Quality Manager and the EPA AMS Center Quality Manager at her discretion. The organization chart in Figure I identifies the responsibilities of the organizations and individuals associated with the verification test. Roles and responsibilities are defined further below. A4.1 Battelle Dr. Rvan James is the AMS Center Verification Test Coordinator. In this role. Dr. James will have overall responsibility for ensuring that the technical, schedule, and cost goals ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 7 of 35 Version I August 4, 2004 Zachary Willenberg Battelle AMS Center Quality Manager Battelle Management Karen Riggs Battelle AMS Center Manager Amy Dindal Verification Testing Leader Ryan James Verification Test Coordinator Battelle Technical Staff Robert Fuerst EPA AMS Center Manager Elizabeth Betz EPA AMS Center Quality Manager Vendor Representatives Figure 1. Organization Chart ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 8 of 35 Version I August 4, 2004 established for the verification test are met. Specifically, he will: • Assemble a team of qualified technical staff to conduct the verification test. • Direct the team < Battelle. EPA, and T&E Facility staff) performing the verification test in accordance with the test/QA plan. • Ensure that all quality procedures specified in the test/QA plan and in the A MS Center Quality Management Plan1 (QMP) are followed. • Prepare the draft and final test/QA plan, verification reports, and verification statements. • Revise the draft test/QA plan, verification reports, and verification statements in response to reviewers' comments. • Respond to any issues raised in assessment reports and audits, including instituting corrective action as necessary. • Serve as the pri mary point of contact for vendor representatives. • Coordinate distribution of the final test/QA plan, verification reports, and statements. • Establish a budget for the verification test and manage staff to ensure the budget is not exceeded. • Ensure that confidentiality of sensitive vendor information is maintained. Ms. Amy Pi tidal is a Verification Testing Leader for the AMS Center. Ms. Di tidal will provide technical guidance and oversee the various stages of verification testing. She will • Support Dr. James in preparing the test/QA plan and organizing the testing. • Review the draft and final test/QA plan. • Review the draft and final verification reports and verification statements. Ms. Karen Riggs is Battelle"s manager for the AMS Center. Ms. Riggs will • Review the draft and final test/QA plan. • Review the draft and final verification reports and verification statements. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 9 of 35 Version I August 4, 2004 • Ensure that necessary Battelle resources, including staff and facilities, are committed to the verification test. • Ensure that confidentiality of sensitive vendor information is maintained. • Support Dr. James in responding to any issues raised in assessment reports and audits. • Maintain communication with EPA's technical and quality managers. • Facilitate a stop work order if Battelle or EPA QA staff discovers adverse findings that will compromise test results. Battelle Technical Staff will conduct the testing of the multi-parameter water monitors during the verification test. Battelle staff will be on-site at the EPA T&E Facility during the entire verification test. The responsibilities of the technical staff will be to: • Maintain and operate the technologies if desired by the vendors and proper training is provided. • Collect the reference samples from the distribution system. • Prepare contaminant solutions for injection into the T&E Facility pipe loops. • Perform the verification testing as described in the test/QA plan. • Make qualitative observations about the maintenance and operation of the multi- parameter water monitors. • Collect the data from each multi-parameter water monitor and transmit it to the Verification Test Coordinator on a daily basis. • Troubleshoot any problems with the multi-parameter water monitors and communicate them to the Verification Test Coordinator immediately. Mr. Zacharv Willenberg is Battelle"s Quality Manager for the AMS Center. Mr. Willenberg will: • Review the draft and final test/QA plan. • Conduct a technical systems audit once during the verification test, or designate other QA staff to conduct the audit. • Audit at least 10% of the verification data. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 10 of 35 Version I August 4, 2004 • Prepare and distribute an assessment report for each audit. • Verify implementation of any necessary corrective action. • Issue a stop work order if self audits indicate that data quality is being compromised: notify Battelle's AMS Center Manager if a stop work order is issued. • Provide a summary of the QA/QC activities and results for the verification reports. • Review the draft and final verification reports and verification statements. • Assume overall responsibility for ensuring that the test/QA plan is followed. A4.2 Vendors The responsibilities of the vendor representatives are as follows: • Review the draft test/QA plan. • Approve the test/QA plan prior to test initiation. • Provide two off-the-shelf multi-parameter water monitors for evaluation during the verification test. • Provide all other equipment/supplies/reagents/consumables needed to operate their monitors for the duration of the verification test. • Either supply a representative to install, maintain, and operate their technology throughout the test, or provide written consent and instructions for Battelle staff to carry out these activities. • Provide written instructions for operation of multi-parameter water monitors, including a daily checklist of maintenance activities. • Review the draft verification report and statement. A4.3 EPA EPA's responsibilities in the AMS Center are based on the requirements stated in the "Environmental Technology Verification Program Quality Management Plan" (EPA QMP)2. The roles of the specific EPA staff are as follows: ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 11 of 35 Version I August 4, 2004 Ms. Elizabeth Betz is EPA's A MS Center Quality Manager. For the verification test. Ms. Betz will: • Review the draft test/QA plan. • Perform at her option one external technical system audit during the verification test. • Notify the EPA AMS Center Manager of the need for a stop work order if external audit indicates that data quality is being compromised. • Prepare and distribute an assessment report summarizing results of external audit. • Review draft verification reports and statements. Mr. Robert Fuerst is EPA's manager for the AMS Center. Mr. Fuerst will: • Review the draft test/QA plan. • Approve the final test/QA plan. • Review the draft verification reports and statements. • Oversee the EPA review process for the verification reports and statements. • Coordinate the submission of verification reports and statements for final EPA approval. Mr. Rov Haught is the Branch Chief at EPA's Water Quality Management Branch within the Office of Research and Development. National Risk Management Research Laboratory. Dr. Haught will host the verification test at EPA's T&E Facility and: • Provide Battelle and vendor personnel access to the T&E Facility during the verification test. • Provide T&E Facility technical staff (EPA or contractor) to assist in interfacing the multi-parameter water monitors with the distribution system. • Provide T&E Facility technical staff (EPA or contractor) and equipment to perform standard laboratory reference method analyses of the water quality parameters being measured by the multi-parameter water monitors. • Provide all reference measurement data to Battelle electronically, in mutually agreed upon format. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 12 of 35 Version I August 4, 2004 • Provide T&E Facility technical staff (EPA or contractor) and equipment to control the flow of pipe loop water as directed by the test/QA plan or the Verification Test Coordinator. • Provide T&E Facility technical staff (EPA or contractor) and equipment to change the conditions of the pipe loop water as directed by the test/QA plan or the Verification Test Coordinator. • Provide training records and information, upon request, regarding education and experience of each EPA or contractor staff involved in the verification test. • Assist in Battelle's reporting of the reference measurements and associated QA/QC results. • Review portions of the draft verification reports to assure accurate descriptions of the T&E Facility operations and to provide technical insights on verification results. • Provide necessary safety instructions to Battelle technical staff and vendor representatives for operations at the T&E Facility. EPA and the T&E Facility will maintain stop work authority as it applies to safety and pipe loop operational issues. A5 BACKGROUND The ETV Program's A MS Center does third-party verification testing of commercially available technologies that detect natural species and contaminants in air. water, and soil. A stakeholder committee of buyers and users of such technologies recommend the technology categories and technologies within those categories as priorities for testing. Multi-parameter water monitors for distribution systems were identified as a priority technology category through the A MS Center stakeholder process. A6 VERIFICATION TEST DESCRIPTION AND SCHEDULE A6.1 Summary of Technology Category The multi-parameter water monitors for distribution systems to be tested during this verification test consist of instrument packages that can be connected to or inserted into ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 13 of 35 Version I August 4, 2004 distribution system pipes for continuous monitoring. Also included in this technology category are technologies that can be programmed to automatically sample and analyze distribution system water at regular intervals, as well as handheld technologies requiring technicians to manually collect samples and perform the analyses. The monitors must be able to measure residual/free chlorine as well as at least one other water quality parameter. Residual/free chlorine is a particularly important water-quality parameter because changes in its concentration can indicate the presence of contamination within a distribution system and the majority of water utilities in the United States use chlorination for disinfection. Other water-quality parameters that these technologies measure may include: alkalinity. pH. dissolved oxygen, oxidation-reduction potential, temperature, turbidity, conductivity, ammonia, calcium, total organic carbon, and monochloramine. The number and selection of water quality parameters will be determined by each monitor's capabilities. In addition to measuring the above water quality parameters, some of the multi-parameter water monitors for distributions systems have the added capability of using the measured water quality parameter data to determine the identity of an injected contaminant. This verification test will assess the performance of each multi-parameter water monitor for distribution systems relative to key verification parameters including accuracy, inter- unit reproducibility, and including comparison to reference measurements where possible. In performing the verification test. Battelle will follow the technical and QA procedures specified in this test/QA plan and will comply with the data quality requirements in the AMS Center QMP1. A6.2 Verification Schedule As shown in Figure 2, the verification test of multi-parameter water monitors for distribution systems will begin in August 2004 and last through October 2004. At the close of testing, individual reports will be drafted for each technology, reviewed, and submitted to EPA for final signature. All documents will be submitted to EPA in electronic < WordPerfect and Adobe portable document format [pdf |) and hard copy formats. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 14 of 35 Version I August 4, 2004 Testing Stage August 2004 September 2004 October 2004 1 - Accuracy 2 - Contaminant Injection 3 - Extended Deployment 4 - Contaminant Identification Figure 2. Verification Schedule A7 QUALITY OBJECTIVES This verification test will evaluate the performance of multi-parameter water monitors for distribution systems. This will include a comparison of the monitor results to the results of standard laboratory reference methods. The quality of the reference measurements will be monitored by the inclusion of blank, duplicate, and performance evaluation (PE) audit samples. The PE audit samples will be analyzed by an instrument or calibration standards that are different from those used for the rest of the reference analyses. These samples are meant to independently confirm that the reference measurements are being performed correctly with accurate results. Control limits on the duplicate and PE samples are given in Section CI. A8 SPECIAL TRAINING/CERTIFICATION Documentation on training related to standard analytical chemistry methodology is maintained for Battelle and T&E Facility technical staff in training files at the respective locations. The Battelle Quality Manager will verify the presence of appropriate training records prior to the start of testing. If the vendors request that Battelle technical staff operate and maintain their monitors during the verification test, the vendors will be required to train the Battelle technical staff prior to the start of testing. Battelle will document this training with a consent form, signed by the vendor, that states which specific Battelle technical staff have been trained to use their monitor. Battelle technical staff will have a minimum of a bachelor" s degree in science/engineering or have equivalent work experience. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 15 of 35 Version I August 4, 2004 A9 DOCUMENTATION AND RECORDS The records for this verification test will be contained in the test/QA plan, the protocols, chain of custody forms, laboratory record books (LRB), data collection forms, electronic files < both raw data and spreadsheets), and the final verification report. All of these records will be maintained in the Verification Test Coordinator's office during the test and will be transferred to permanent storage at Battelle's Records Management Office at the conclusion of the verification test. All Battelle LRBs are stored indefinitely, either by the Verification Test Coordinator or Battelle's Records Management Office. EPA will be notified before disposal of any files. The results from the reference measurements made by the T&E Facility technical staff will be submitted to Battelle immediately upon making the measurement and obtaining the results of the analyses. Section B10 further details the data recording practices and responsibilities. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 16 of 35 Version I August 4, 2004 SECTION B MEASUREMENT AND DATA ACQUISITION B1 EXPERIMENTAL DESIGN This verification test will specifically address verification of multi-parameter water monitors for distribution systems by evaluating the accuracy of the water quality measurements made by each technology in finished drinking water, the response to changes in the water system due to intentional contamination of the water in the pipe loop, and the durability and ruggedness of each monitor. Also, an optional portion of the test will involve the evaluation of technologies that can detect a change in the water quality parameters of the water in a distribution system, and can use that information to qualitatively determine the identity of specific contaminants injected into a distribution system. Because assessing the precision of continuous measurements of flowing distribution system water is difficult to accomplish, i.e. the system is never truly stable, precision of these monitors will not be assessed during this verification test. Two identical monitors from each vendor will be connected to a pipe loop and tested simultaneously to compare their results. The drinking water analyzed by the multi-parameter water monitors during this verification test will be from the Cincinnati. Ohio distribution system. Because only one source of water with a limited number of controlled parameters < residual/free chlorine, temperature, and pH) will be used for this verification test, this is not intended to be an exhaustive study or to represent all possible water types that could be tested. The multi-parameter water monitors will be evaluated for the following parameters: • Accuracy - Comparison to results from standard laboratory reference analyses • Response to injected contaminants - Detection of changes in pipe loop water chemistry • Inter-unit reproducibility - Comparison of results from two simultaneously operating monitors • Ease of use - general operation, data acquisition, set-up. demobilization, required maintenance • Presence and identification of injected contaminants (as applicable) ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 17 of 35 Version I August 4, 2004 BI. I Stage I - Accuracy The first stage of this verification test will evaluate the accuracy of the measurements made by the multi-parameter water monitors plumbed to a recirculating pipe loop. The pipe loop, that is I (X) feet long and consists of ductile iron pipe that is six inches in diameter, will contain approximately 240 gallons of Cincinnati water with a flow rate of approximately 1 ft/sec. and have a residual/free chlorine concentration of approximately 1 mg/L, The accuracy of the monitors will be evaluated by comparing each of their measurements to the result for the same target analytes/water quality parameters generated by a standard laboratory reference method. A reference sample will be collected every hour during a 4 hour testing period and analyzed within the holding times required by each standard reference method. Each multi-parameter water monitor will be connected to the pipe loop simultaneously via a multi-spigot tap that ensures that each multi-parameter water monitor will be making measurements on very similar water. Each technology will be plumbed separately to the multi-spigot tap to ensure that there is no possibility of one technology affecting the performance of another. One spigot will be dedicated to collecting the reference samples, and the rest will be connected to the multi-parameter water monitors. The water quality parameters measured made by each monitor will be determined by each vendor and agreed to by the Verification Test Coordinator. It would be difficult to simulate the characteristics of every municipal distribution system in order to verify the performance of these monitors under multiple water quality conditions. However. Stage I testing will attempt to address this issue by changing two key variables. pH and temperature. The testing described above, consisting of four hours of continuous analysis with reference sampling performed every hour, will be done using five different sets of pH and temperature conditions. Three sets of conditions will involve varying only the pH. Those sets will consist of a pH of approximately seven, eight (T&E Facility ambient), and nine and a temperature between 16 to 18°C (T&E Facility ambient). Two other sets will include changing the water temperature to between 10 and 12°C and perform testing at pHs of approximately eight and nine. Prior to each testing period with unique conditions, the water in the pipe loop will be equilibrated until the pH and temperature is at the desired level (approximately 12 hours), as ------- determined by the standard reference methods. Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 18 of 35 Version I August 4, 2004 Bl.2 Stage 2 - Contaminant Injection The second stage of the verification test will test the responsiveness of the monitors to changes in water quality parameters by the injection of contaminants into the pipe loop. Three contaminants will be injected (in duplicate) into the recirculating pipe loop containing ambient Cincinnati water (pH ~8. 16 to 18°C). The first injection will consist of a solution of aldicarb and the second injection will be a solution of arsenic trioxide. Each injection solution will be prepared in two gallons of pipe loop water at a concentration adequate to make the pipe loop water approximately 10 mg/L in each contaminant. The content of the third injection will consist of a either a solution of non-pathogenic E. Coli bacteria or a solution of nicotine. The bacteria injection would be preferred if the T&E Facility determines that its injection is allowable under the waste water regulations by which they must abide. A decision in that regard will be made just prior to the time of injection. If the bacteria injection is not approved by the T&E Facility, a two gallon solution of nicotine at a concentration adequate to make the pipe loop water 10 mg/L will be prepared for injection. For all three sets of injections, a reference sample will be collected prior to the injections and again three. 15, and 60 minutes after the injection. The difference between the two results will indicate the approximate change in water quality caused by the injected contaminant which should be measured by each of the multi-parameter water quality monitors being tested. For the monitors that are not designed to operate continuously, the verification staff will make triplicate measurements using those monitors on an aliquot of the same samples collected for the reference analyses. Each contaminant solution injected will be prepared in a volume of two gallons. One concentration of each contaminant will be injected. After each injection, the pipe loop will be allowed to re-equilibrate so that each multi-parameter water monitor is measuring a steady baseline (approximately three days). Each water quality parameter may not be exactly the same as pre-injection conditions, but the return to a steady baseline will allow for any change in parameters due to the injection of a contaminant to be detected. National Institute of Standards and Technology (MIST) traceable standards (when available) will be used for preparation of the ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 19 of 35 Version I August 4, 2004 injected contaminant solutions. These solutions will be prepared by dissolving the standards into pipe loop water and diluting the solutions to the appropriate concentrations. Battelle QA staff will audit the gravimetric preparation of these solutions as well as any necessary dilutions. B1.3 Stage 3-Extended Deployment The third stage of this verification test will evaluate the performance of the multi- parameter water monitors during a deployment 45-60 days in length. During this time, the multi- parameter water monitors will operate continuously while connected to the recirculating pipe loop containing ambient Cincinnati water. The monitors will receive the minimum amount of maintenance required (if the vendor does not operate and maintain the monitor, a daily maintenance checklist provided by the vendor will be carried out by Battelle). No matter if the vendors or Battelle are performing the maintenance and operation of the monitors, all maintenance activities will be documented by those performing the work in the project laboratory record book. For the monitors that are not designed to operate continuously, the vendor or trained Battelle technical staff will make one measurement per day (in triplicate). These monitors will also receive the minimum maintenance suggested by the vendor. In order to track the performance of the monitors with respect to the reference results, reference samples will be collected and analyzed for the selected parameters at least once per day for the duration of Stage 3. If Battelle staff are operating and maintaining the monitor and there is a problem during the extended portion of the verification test, as with any other portion of the verification test, the Battelle technical staff will troubleshoot the problem to the best of their ability and then contact the vendor for assistance. The final phase of Stage 3 will consist of a thorough evaluation of performance after the extended placement. This will be done in two steps. First, a reference sample will be collected every hour during a four-hour analysis period and analyzed within the holding times required by the standard reference methods. Second, the responsiveness of the monitors will be evaluated by repeating injection number one (aldicarb) from Stage 2 in duplicate. During this final component of contaminant injections, reference samples will be collected as they were during Stage 2. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 20 of 35 Version I August 4, 2004 B1.4 Stage 4 - Contaminant Identification (as applicable) The Stage 4 testing is a component of the verification test that will evaluate the performance of the monitors that have been designed to. not just identify the occurrence of the injection of a contaminant into the distribution based on the observed change in the water quality parameters, but to identify what contaminant was injected. No reference measurements will be made during this optional stage of the test because the two parameters being evaluated are the ability of the monitors to report whether or not an injection event was detected and if so, was the injected contaminant identified correctly. This portion of the test will be done by interfacing the monitors with this capability to a straight pipe made from ductile iron that is three inches in diameter and 1,500 feet in length. The contaminants listed in Table I will be individually injected into the pipe so the concentration of each contaminant in the pipe will become approximately 10 mg/L. Each injection will be done in duplicate and all injections will be done blindly so that the monitor operator does not know what contaminant is being injected. Table 1. Injected Contaminants for Identification Contaminants to be Injected Approximate Concentration in Pipe Water Aldicarb. arsenic trioxide, colchicine, carbarvl. dichlorvos. dicamba. E. Coli bacteria1, glyphosate, lead nitrate, malathion. mercuric chloride, methanol, nicotine, phorate. potassium ferricvanide. and sodium fluoroacetate. 10 mg/L 'If T&E Facility determines E. Coli may be injected into the pipe loop. B1.5 Statistical Analysis B 1.5.1 Percent Difference Results from the multi-parameter water monitors being verified in stages one through three will be compared to the results obtained from analysis of a grab sample by the reference method. Because no reference samples will be analyzed during Stage 4, a percent difference calculation ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 21 of 35 Version I August 4, 2004 would not be appropriate. The results for each sample will be recorded, and the accuracy will be expressed in terms of the percent difference (%D), as calculated from the following equation: %D= C" ~d x 100% £r where CR is the concentration determined by the reference method and d is the average measurement from the monitor across the entire time the reference sample was collected. This calculation will be performed for each reference sample analysis for each of the parameters by each monitor. Ideally, if the monitor and reference method measurements are the same, there would be a percent difference of zero percent. Readings of pH will be converted to the hydronium ion concentration, and temperature readings will be converted to absolute units (Kelvin) prior to making this calculation. Percent difference will be assessed independently for each monitor provided by a vendor to determine inter-unit reproducibility. A plot of the continuous measurements will be reported with the reference results added as individual data points. To evaluate the responsiveness of the multi-parameter water monitor readings to contaminant injections, the pre- and post-injection reference samples will be compared to the monitor readings using the percent difference calculation described earlier in this section. Again, small percent differences will indicated good agreement between the monitors and the reference methods. The data collected during the extended deployment data will also be compared with the reference samples in a similar way. In addition, a plot of the continuous measurements during the injection of the contaminants as well as during the extended deployment will be reported with the reference results added as individual data points to clearly show the effect each injection had on the water chemistry of the water in the pipe loop and the stability of the monitors during non- injection periods of operation. B 1.5.2 False Positive/False Negative Identification Rate During Stage 4. Contaminant Identification, two assessments will be reported upon the injection of each contaminant: 1) whether or not an injection event was detected, and 2) if the ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 22 of 35 Version I August 4, 2004 injection was detected, was the injected contaminant identified correctly. For each of these results categories, the percentage of false positive and negative results out of the total number of injections will be reported. HI.5.3 Inter-Unit Reproducibility The results obtained from identical units of each monitor will be compiled independently for each analyzer and compared to assess inter-unit reproducibility. The results will be interpreted using a /-test, or other appropriate comparison, to assess whether significant differences exist between the units tested. One evaluation will be a regression analysis and plot of unit one versus unit two from each vendor. A slope of unity would indicate ideal inter-unit comparability. B1.6 Reporting The statistical comparisons described above will be conducted separately for each of the monitors being tested, and information on the additional performance parameters such as ease of use, level of maintenance, calibration, and set-up/demobilization will be compiled and reported. Separate verification reports will be prepared for each monitor that was tested. Each report will show separate verification results from the duplicate monitors undergoing testing, along with calculations of the inter-unit reproducibility of the technology. For each test, the verification report will present the test procedures and test data, as well as the results of the statistical evaluation of those data. All interaction with the monitors (such as during maintenance, cleaning, and calibration) will be documented at the time of the test and reported. In addition, descriptions of the data- recording procedures, use of vendor-supplied proprietary software, consumables used, and required reagents will be presented in the report. The verification report will briefly describe the ETV program, the A MS Center, and the procedures used in verification testing. These sections will be common to each verification report. The results of the verification test will then be stated quantitatively, without comparison to any other technology tested or comment on the acceptability of the technology's performance. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 23 of 35 Version I August 4, 2004 B2 REFERENCE SAMPLE COLLECTION As described above. Stage I testing of the multi-parameter water monitors will consist of five separate four-hour evaluation periods. The water in the recirculating pipe loop will have different combinations of pH and temperature during each of these evaluation periods. Battelle technical staff will collect the reference samples during all stages of the verification test. The samples will be collected following guidelines set in each standard reference method listed in Section B4. The methods describe the appropriate sampling containers, preservation techniques, and maximum holding times. With the first sample being collected at the start of the evaluation period, one reference sample of adequate volume to perform each reference analysis will be collected from the reference sample spigot every hour during each of these evaluation periods for a total of five reference samples will be collected during each period. One of these five reference samples will be split into two samples and analyzed as a duplicate and one American Standard Testing and Materials Type II deionized (ASTM DI) water blank will be analyzed as a laboratory blank. During Stage 2, reference samples will be collected before each injection of contaminants and then at three. 15, and 60 minutes post-injection. Therefore, four grab samples will be collected for each contaminant injection. For each injection, one of the four reference samples will be split and analyzed as a duplicate and one DI water blank will be analyzed as a laboratory blank. Stage 3 reference samples will be collected daily for the duration of the extended deployment. Each week, one sample will be split and analyzed as a duplicate and one DI water blank will be analyzed as a laboratory blank. The samples will be treated as specified for Stage 1 testing. The optional Stage 4 testing will not involve reference sample analysis. Table 2 summarizes the reference samples to be collected during each stage of the verification test. B3 SAMPLE HANDLING AND CUSTODY REQUIREMENTS Sample custody will be documented throughout collection, shipping (if necessary), and ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 24 of 35 Version I August 4, 2004 analysis of the reference samples. Sample chain-of-custody procedures will be in accordance with the Battelle SOP ASAT.I-009-00 Footnote 3 Sample Chain of Custody (Draft)3. The chain-of- custodv form summarizes the samples collected and analyses requested. The chain-of-custody form will track sample release from the sampling location to the analysis laboratory. Chain-of- custody forms will be used regardless of whether the samples are being transferred within the Table 2 Description of Reference Analyses Stage Number of Sampling Periods Sampling Period Reference Sample Frequency Number of Reference Samples Per Sampling Period QA Analyses per Sampling Period Total Reference Samples per Sampling Period Including QA analyses I 5 4 hours One at start and one every hour therefafter 5 I duplicate I DI water blank 7 2 6 I injection I pre- injection I @ 3. 15, and 60 minutes post- injection 4 I duplicate I DI water blank 6 3 I Several Weeks once daily -30 I duplicate and 1 DI water blank each week -42 4 No reference analyses to be performed. T&E Facility or to an external location. Each chain-of-custody form will be signed by the person relinquishing samples once that person has verified that the chain-of-custody form is accurate. The original sample chain-of-custody forms accompany the samples; the shipper will keep a copy. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 25 of 35 Version I August 4, 2004 Upon receipt at the laboratory, chain-of-custody forms will be signed by the person receiving the samples once that person has verified that all samples identified on the chain-of-custody forms are present in the shipping container. Any discrepancies will be noted on the form and the sample receiver will immediately contact the Verification Test Coordinator to report missing, broken, or compromised samples. Copies of all chain-of-custody forms will be delivered to the Verification Test Coordinator and maintained with the test records. B4 LABORATORY REFERENCE METHODS Table 3 provides the standard laboratory methods that will be used for the sample collection and reference analyses during this verification test. Also included in the table is each method's detection limit, method of preservation, and maximum holding time. The collection of the reference samples will be the responsibility of Battelle technical staff while the analyses will be performed by T&E Facility technical staff. The T&E Facility is responsible for providing calibrated instrumentation, performing all method QA/QC, and providing calibration records for any instrumentation used. The T&E Facility will be required to provide Battelle all reference sample results immediately upon the analysis of the reference samples. The laboratory reference method for oxidation-reduction potential is known to be biased when compared with the same measurement done in a flowing pipe. The reference method will still be used, but the results will be interpreted in that context. B5 QUALITY CONTROL AUDITS AND REQUIREMENTS As described in Section B2. blank and duplicate samples will be required to be analyzed by each standard reference method with each set of reference samples collected. The blank reference method results are required to be less than the method detection limit (except for temperature and pH) for each reference method and the duplicate measurements will be required to be within the acceptable tolerances provided in Table 3. Samples producing results not meeting these requirements shall be reanalyzed by the reference method. If the results are still outside the ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 26 of 35 Version I August 4, 2004 required tolerance, the reference instrument will be recalibrated and the reference samples reanalyzed. If the outlying results persist, the repeat of the appropriate parts of the verification test may be considered. B6 INSTRUMENT/EQUIPMENT TESTING, INSPECTION, AND MAINTENANCE The instruments used for the reference analyses will be tested and inspected as per the standard operating procedures of the T&E Facility or the standard methods being used to make each measurement. If Battelle operates and maintains the monitors, it will be done as directed by the vendor. Otherwise, operation and maintenance will be the responsibility of the vendor. B7 INSTRUMENT CALIBRATION AND FREQUENCY The instruments used for the reference analyses will be calibrated per the standard reference methods being used to make each measurement or the standard operating procedures of the analysis laboratory. If the monitor is being operated by Battelle. the vendor will provide Battelle technical staff with a checklist of tasks to be completed daily to properly maintain each monitor. All calibrations performed will be documented by Battelle in the project laboratory record book. Calibration of the multi-parameter water monitors will be done for each measured parameter as often as suggested by the vendor. This calibration will use NIST-traceable standards when applicable. Vendors may choose to supply the necessary calibration solutions and devices specific to the monitors being verified. Balances and pipettes used during injection solution preparation will be maintained and calibrated as required by the T&E Facility standard operating procedures which will be reviewed by the Battelle Quality Manager prior to the verification test. B8 INSPECTION/ACCEPTANCE OF SUPPLIES AND CONSUMABLES All materials, supplies, and consumables will be ordered by the Verification Test Coordinator or designee. Where possible. Battelle will rely on sources of materials and consumables that have been used previously as part of ETV verification testing without problems. Battelle will also rely on previous experience or recommendations from T&E Facility technical ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 27 of 35 Version I August 4, 2004 Table 3. Reference Methods Parameter Method Method Detection Limits Preservation Holding Time Acceptable Tolerance Ammonia EPA 350.14 0.01 mg/L as CaC03 H2S04, pH<2 28 d @ 4°C 25% Calcium EPA 215.1s 0.01 mg/L HN03, pH<2 6 mo @ 4°C 25% Conductivity SM 25106 2 fimiio none Immediate analysis 25% Dissolved Oxygen SM 4500-07 0.2 mg/L none Immediate analysis 25% Free Chlorine SM 4500-C1-G8 0.01 mg/L as ci2 none Immediate analysis 25% Monochloramine SM 4500-C1-G8 0.01 mg/L as Nl I.CI none Immediate analysis 25% Oxidation Potential1 SM 2580-B9 NA none Immediate analysis 25% pH EPA 150.110 NA none Immediate analysis +0.3 pH units Temperature EPA 170.1" NA none Immediate analysis +1°C Total Alkalinity EPA 310.112 20 mg/L none 14 d @ 4°C 25% Total Organic Carbon EPA 415.113 0.01 mg/L H2S04, pH<2 28 d @ 4°C 25% Turbidity EPA 180.114 0.067 ntu none 48 h @ 4°C 25% 'The reference method for the measurement of oxidation-reduction potential is known to be biased due to the difference in potential in a flowing pipe to that measured from the grab sample. staff to guide selection of manufacturers and materials. The manufacturer's criteria for acceptance/purity will be required to be met. B9 NON-DIRECT MEASUREMENTS Data published previously in the scientific literature will not be used during this verification test. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 28 of 35 Version I August 4, 2004 BIO DATA MANAGEMENT Various types of data will be acquired and recorded electronically or manually by Battelle technical staff during this verification test. Table 4 summarizes the type of data to be recorded. All data and observations for the operation of the monitors will be documented by vendors or Battelle technical staff on data sheets or in laboratory record books. Results from the laboratory reference instruments will be compiled by T&E Facility staff in electronic format and submitted to Battelle immediately upon obtaining results. Records received by or generated by any of the Battelle or T&E Facility technical staff during the verification test will be reviewed by a Battelle staff member within two weeks of receipt or generation, respectively, before the records are used to calculate, evaluate, or report verification results. If a Battelle staff member generated the record, this review will be performed by a Battelle technical staff member involved in the verification test, but not the staff member that originally received or generated the record. The review will be documented by the person performing the review by adding his/her initials and date to the hard copy of the record being reviewed. In addition, data calculations performed by Battelle or T&E Facility technical staff will be spot-checked by Battelle technical staff to ensure that calculations are performed correctly. Calculations to be checked include any statistical calculations described in this test/QA plan. The data obtained from this verification test will be compiled and reported independently for each multi-parameter water monitor. Results for multi-parameter water monitors from different vendors will not be compared with each other. During the course of any assessment or audit, the Battelle Quality Manager will inform the technical staff of any immediate corrective action that should be taken. If serious quality problems exist, the Battelle Quality Manager is authorized to stop work. Once the assessment report has been prepared, the Verification Test Coordinator will ensure that a response is provided for each adverse finding or potential problem, and will implement any necessary follow-up corrective action. The Battelle Quality Manager will ensure that follow-up corrective action has been taken. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 29 of 35 Version I August 4, 2004 Table 4. Summary of Data Recording Process Data to Be Recorded Where Recorded How Often Recorded By Whom Disposition of Data Dates, times, and details of test events H I V data sheets and testing notebook Start/end of test, and at each change of a test parameter Bat telle and T&E Facility Used to organize/check test results; manually incorporated in data spreadsheets as necessary Calibration information H I V data sheets and testing notebook Prior to sample preparation Bat telle Manually incorporated in data spreadsheets as necessary Multi-parameter water monitor results Recorded electronically by each monitor and then downloaded to computer at the close of each day. Recorded continuously. Bat telle Comma delimited text files. Reference method procedures H I V laboratory record books, or data recording forms Throughout sample analysis process T&E Facility Transferred to spreadsheets or laboratory record book ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 30 of 35 Version I August 4, 2004 SECTION C ASSESSMENT AND OVERSIGHT CI ASSESSMENTS AND RESPONSE ACTIONS Every effort will be made in this verification test to anticipate and resolve potential problems before the quality of performance is compromised. One of the major objectives of the test/QA plan is to establish mechanisms necessary to ensure this. Internal quality control measures described in this test/QA plan, which is peer reviewed by a panel of outside experts, implemented by the technical staff and monitored by the Verification Test Coordinator, will give information on data quality on a day-to-day basis. The responsibility for interpreting the results of these checks and resolving any potential problems resides with the Verification Test Coordinator. Technical staff have the responsibility to identify problems that could affect data quality or the ability to use the data. Any problems that are identified will be reported to the Verification Test Coordinator, who will work to resolve any issues. Action will be taken to control the problem, identify a solution to the problem, and minimize losses and correct data, where possible. Independent of T&E Facility and EPA QA activities. Battelle will be responsible for ensuring that the following audits are conducted as part of this verification test. CI. I Performance Evaluation Audits A PE audit will be conducted to assess the quality of the reference measurements made in this verification test. Each type of reference measurement will be compared with an independent instrument or a NIST-traceable standard that is independent of those used during the testing as part of the PE audit. The results of the PE audit must be within the acceptable tolerances provided in Table 3. If the results do not meet this requirement, they will be repeated. If the outlying results persist, a change in reference instrument, and a repeat of the PE audit may have to be considered. This audit will be performed once at the start of the verification test, and will be the responsibility of the Verification Test Coordinator or designee. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 31 of 35 Version I August 4, 2004 C1.2 Technical Systems Audits The Battelle Quality Manager will perform a technical systems audit (TSA) at least once during this verification test. The purpose of this audit is to ensure that the verification test is being performed in accordance with the A MS Center QMP1, this test/QA plan, published reference methods, and any SOPs used by the T&E Facility. In this audit, the Battelle Quality Manager, or designee, may review the reference methods used, compare actual test procedures to those specified or referenced in this plan, and review data acquisition and handling procedures. A TSA report will be prepared, including a statement of findings and the actions taken to address any adverse findings. The EPA AMS Center Quality Manager will receive a copy of Battelle"s TSA report. At EPA's discretion. EPA QA staff may also conduct an independent on-site I SA during the verification test. The TSA findings will be communicated to technical staff at the time of the audit and documented in a TSA report. C1.3 Data Quality Audits The Battelle Quality Manager will audit at least 10% of the verification data acquired in the verification test. The Battelle Quality Manager will trace the data from initial acquisition, through reduction and statistical comparisons, to final reporting. All calculations performed on the data undergoing the audit will be checked. C1.4 QA/QC Reporting Each assessment and audit will be documented in accordance with Section 3.3.4 of the AMS Center QMP1. The results of the technical systems audit will be submitted to EPA. Assessment reports will include the following: • Identification of any adverse findings or potential problems • Response to adverse findings or potential problems • Recommendations for resolving problems • Confirmation that solutions have been implemented and are effective • Citation of any noteworthy practices that may be of use to others. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 32 of 35 Version I August 4, 2004 C2 REPORTS TO MANAGEMENT The Battelle Quality Manager, during the course of any assessment or audit, will identify to the technical staff performing experimental activities any immediate corrective action that should be taken. If serious quality problems exist, the Battelle Quality Manager is authorized to stop work. Once the assessment report has been prepared, the Verification Test Coordinator will ensure that a response is provided for each adverse finding or potential problem and will implement any necessary follow-up corrective action. The Battelle Quality Manager will ensure that follow-up corrective action has been taken. The test/QA plan and final report are reviewed by EPA AMS Center quality assurance staff and the EPA A MS Center program management staff. Upon final review and approval, both documents will then be posted on the ETV website (www.epa.gov/etv). ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 33 of 35 Version I August 4, 2004 SECTION I) DATA VALIDATION AND USABILITY D1 DATA REVIEW, VALIDATION, AND VERIFICATION REQUIREMENTS The key data review requirements for the verification test are the collection of QC samples as outlined in the test/QA plan, a comparison of field data sheet comments against final data to flag any suspect data, and a review of final data to resolve any questions about apparent outliers. The QA audits, as described within this document are designed to assure the quality of this data. 1)2 VALIDATION AND VERIFICATION METHODS Section CI, the Assessment and Response section, provides a thorough description of the validation safeguards employed for this verification test. 1)3 RECONCILIATION WITH USER REQUIREMENTS The data will be compiled into an ETV verification report. The report will be submitted to EPA in Word Perfect and Adobe pdf format and subsequently posted on the ETV website. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 34 of 35 Version I August 4, 2004 SECTION E REFERENCES 1. Quality Management Plan for the ETV Advanced Monitoring Systems Center. Version 5.0, U.S. EPA Environmental Technology Verification Program, Battelle. Columbus. Ohio. March 2004. 2. "Environmental Technology Verification Program Quality Management Plan". December 2002. EPA/600/R-03/021. 3. Battelle SOP A SAT 1-009. Footnote 3, Sample Chain of Custody, Draft June 2004. 4. U.S. EPA. EPA Method 350.1, Nitrogen, Ammonia in Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79/020, March 1983. 5. U.S. EPA. EPA Method 215.1, Calcium in Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79/020, March 1983. 6. American Public Health Association, et al.. SM 2510 Conductivity in Standard Methods for the Examination of Water and Wastewater. 19th Edition. Washington, D.C., 1997. 7. American Public Health Association, et al.. SM 4500-0 Dissolved Oxygen in Standard Methods for the Examination of Water and Wastewater. 19th Edition. Washington, D.C., 1997. 8. American Public Health Association, et al.. SM 4500-G Residual Chlorine in Standard Methods for the Examination of Water and Wastewater. 19th Edition. Washington, D.C., 1997. 9. American Public Health Association, et al.. SM 2580-B Electrochemical Potential in Standard Methods for the Examination of Water and Wastewater. 19th Edition. Washington, D.C., 1997. 10. U.S. EPA. EPA Method 150.1, pH in Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79/020, March 1983. 11. U.S. EPA. EPA Method 170.1, Temperature in Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79/020, March 1983. 12. U.S. EPA. EPA Method 310., Alkalinity in Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79/020, March 1983. 13. U.S. EPA. EPA Method 415.1, Total Organic Carbon in Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79/020, March 1983. ------- Multi-Parameter Water Monitors for Distribution Systems Test/QA Plan Page 35 of 35 Version I August 4, 2004 14. U.S. EPA, EPA Method 180.1, Turbidity in Methods for Chemical Analysis of Water and Wastes, EPA/600/4-79/020, March 1983. ------- |