April 2011 Environmental Technology Verification Report INSTRUMENTATION NORTHWEST INC. AQUISTAR® TEMPHION™ SMART SENSOR AND DATALOGGER NITRATE-SPECIFIC ION- SELECTIVE ELECTRODE FOR GROUNDWATER REMEDIATION MONITORING Prepared by Balteile //?L? Husincss oj Innovation Under a cooperative agreement with f/EPA ET1/ET1/ET1/ ------- April 2011 Environmental Technology Verification Report ETV Advanced Monitoring Systems Center INSTRUMENTATION NORTHWEST INC. AQUISTAR® TEMPHION™ SMART SENSOR AND DATALOGGER NITRATE-SPECIFIC ION- SELECTIVE ELECTRODE FOR GROUNDWATER REMEDIATION MONITORING by Andrew Barton, Chris Gardner, Dale Rhoda, and Amy Dindal, Battelle John McKernan, U.S. EPA ------- Notice The U.S. Environmental Protection Agency, through its Office of Research and Development, funded and managed, or partially funded and collaborated in, the research described herein. It has been subjected to the Agency's peer and administrative review. Any opinions expressed in this report are those of the author(s) and do not necessarily reflect the views of the Agency, therefore, no official endorsement should be inferred. Any mention of trade names or commercial products does not constitute endorsement or recommendation for use. 11 ------- Foreword The EPA is charged by Congress with protecting the nation's air, water, and land resources. Under a mandate of national environmental laws, the Agency strives to formulate and implement actions leading to a compatible balance between human activities and the ability of natural systems to support and nurture life. To meet this mandate, the EPA's Office of Research and Development provides data and science support that can be used to solve environmental problems and to build the scientific knowledge base needed to manage our ecological resources wisely, to understand how pollutants affect our health, and to prevent or reduce environmental risks. The Environmental Technology Verification (ETV) Program has been established by the EPA to verify the performance characteristics of innovative environmental technology across all media and to report this objective information to permitters, buyers, and users of the technology, thus substantially accelerating the entrance of new environmental technologies into the marketplace. Verification organizations oversee and report verification activities based on testing and quality assurance protocols developed with input from major stakeholders and customer groups associated with the technology area. ETV consists of six environmental technology centers. Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/. Effective verifications of monitoring technologies are needed to assess environmental quality and to supply cost and performance data to select the most appropriate technology for that assessment. Under a cooperative agreement, Battelle has received EPA funding to plan, coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air, Water, and Soil" and report the results to the community at large. Information concerning this specific environmental technology area can be found on the Internet at http ://www. epa.gov/etv/centers/centerl .html. in ------- Acknowledgments The authors wish to acknowledge the support of all those who helped plan and conduct the verification test, analyze the data, and prepare this report. The U.S. Department of Agriculture/Agricultural Research Service (USDA/ARS) National Laboratory for Agriculture and the Environment, and specifically Tom Moorman, Beth Douglas, and Jerry Hatfield, provided a tremendous amount of support for this verification test, including reference analyses, field site access, and laboratory support. Dr. Stuart Nagourney (New Jersey Department of Environmental Protection [NJDEP]) and Michael Brody (EPA) provided extensive input to the planning of the verification test and participated in document review. We also acknowledge the contributions to this report from Kati Lentz and Rose Filoramo, students at the College of New Jersey, who performed data and statistical analyses under the direction of Dr. Nagourney. Data storage, graphing, and mapping capabilities were performed by Mark Kram and Cliff Frescura of Groundswell Technologies, Inc. Quality assurance (QA) oversight was provided by Michelle Henderson, EPA, Amy Bowman, NJDEP, and Zachary Willenberg, Rosanna Buhl, and Kristen Nichols, Battelle. The authors also thank Charles Spooner, EPA, Ken Wood, DuPont, and Dr. Jacob Gibs, U.S. Geological Survey for their review of the test/QA plan and/or this verification report. IV ------- Contents Pat Foreword iii Acknowledgments iv Figures vi Tables vi List of Abbreviations viii Chapter 1 Background 1 Chapter 2 Technology Description 2 Chapters Test Design and Procedures 4 3.1 Introduction 4 3.2 Nitrate Analysis Reference Method 4 3.3 Test Design 5 3.3.1 Laboratory Testing Stage 5 3.3.2 Field Testing Stage 6 3.4 Test Procedures 9 3.4.1 Accuracy 9 3.4.2 Variability 10 3.4.3 Duplication 10 3.4.4 Effect of Changes in Water Quality 10 3.4.5 Operational and Sustainability Factors 10 3.5 Analysis of Baseline Concentrations 11 3.6 Verification Schedule 11 Chapter 4 Quality Assurance/Quality Control 12 4.1 Laboratory Sample Analysis QA/QC 12 4.1.1 Instrument Calibration Checks 12 4.1.2 Initial Calibration Checks 12 4.1.3 Laboratory Reagent Blanks 13 4.1.4 Laboratory Fortified Blanks 13 4.1.5 Laboratory Fortified Sample Matrix 13 4.1.6 Laboratory Duplicate Samples 13 4.2 Field QA/QC 15 4.2.1 Field Duplicate Samples 15 4.2.2 Equipment Blanks 15 4.3 Audits 15 4.3.1 Performance Evaluation Audits 15 4.3.2 Technical Systems Audit 16 4.3.3 Data Quality Audits 17 ------- Chapters Statistical Methods 18 5.1 Accuracy 18 5.2 Variability 19 5.3 Duplication 20 5.4 Effect of Changes in Water Quality 20 Chapter 6 Test Results 21 6.1 Accuracy 21 6.2 Variability 25 6.3 Duplication 25 6.4 Effect of Changes in Water Quality 27 6.5 Operational and Sustainability Factors 29 Chapter 7 Performance Summary 31 Chapters References 33 Figures Figure 2-1. Aquistar® TempHion™ Smart Sensor 3 Figure 2-2. Sensor Collection and Data Transmission Schematic 3 Figure 3-1. Laboratory Test Cell Configuration 6 Figure 3-2. Schematic Showing Well and Sensor Layout in Bioreactor 7 Figure 3-3. Figure Showing Nitrate Sensors in Bioreactor 8 Figure 3-4. Nitrate Sensor with attached Conventional Sample Tubing 8 Figure 6-1. Time Series Graph of Concentrations in Well 1 (Inlet) 30 Figure 6-2. Plume Map Showing Dissolved Nitrate Concentrations in Test Cell 30 Tables Table 3-1. Summary of Nitrate Sensor Laboratory Testing 6 Table 3-2. Summary of Nitrate Sensor Verification Samples 9 Table 3-3. Summary of Background Water Quality 11 Table 3-4. Verification Schedule 11 Table 4-1. Summary of Laboratory and Field QA/QC Samples for Nitrate Results 14 Table 4-2. Summary of PE Audit Results 16 Table 6-1. Summary of Phase 1 t-Test and MAE Results 22 Table 6-2. Summary of Phase 2 t-Test and MAE Results for all Sensors Combined 23 Table 6-3. Summary of Field Testing t-Test and MAE Results for all Wells 23 Table 6-4. Well-Specific MAE Estimates 23 Table 6-5. Summary of RPD for Laboratory Test for All Nitrate Concentrations 26 Table 6-6. Summary of RPD for Field Test 26 Table 6-7. Summary of Variability in Phase 1 Laboratory Test 27 Table 6-8. Summary of Duplication Results in Laboratory Test 28 Table 6-9. Summary of the Effects of Changes in Water Quality 28 VI ------- Appendices Appendix 1: Well-Specific Time Series Nitrate Concentration Plots 34 Appendix 2: Accuracy Data Plots 38 vn ------- List of Abbreviations AMS Advanced Monitoring Systems ANOVA analysis of variance ARS Agricultural Research Service bgs below ground surface CCC continuing calibration check DI deionized ECC end calibration check EPA U.S. Environmental Protection Agency ETV Environmental Technology Verification 1C ion chromatography ICAL initial calibration ICC initial calibration check INW Instrumentation Northwest, Inc. ISE ion-selective electrode LFB laboratory fortified blank LFSM laboratory fortified sample matrix LRB laboratory reagent blank MAE mean absolute error MDL method detection limit MS/MSD matrix spike/matrix spike duplicate MSE mean square error NELAC National Environmental Laboratory Accreditation Conference NJDEP New Jersey Department of Environmental Protection NRMRL National Risk Management Research Laboratory NTU nephelometric turbidity unit PVC polyvinyl chloride QA quality assurance QC quality control QCS quality control sample QMP Quality Management Plan RFIC reagent-free ion chromatography RPD relative percent difference Vlll ------- TQAP Test/QA Plan TSA technical systems audit UCL upper confidence limit USD A U. S. Department of Agriculture VTC Verification Test Coordinator IX ------- Chapter 1 Background The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology Verification (ETV) Program to facilitate the deployment of innovative environmental technologies through performance verification and dissemination of information. The goal of the ETV Program is to further environmental protection by accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal by providing high- quality, peer-reviewed data on technology performance to those involved in the design, distribution, financing, permitting, purchase, and use of environmental technologies. ETV works in partnership with recognized testing organizations; with stakeholder groups consisting of buyers, vendor organizations, and permitters; and with the full participation of individual technology developers. The program evaluates the performance of innovative technologies by developing test plans that are responsive to the needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer-reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance (QA) protocols to ensure that data of known and adequate quality are generated and that the results are defensible. EPA's National Risk Management Research Laboratory (NRMRL) and its verification organization partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center recently evaluated the performance of Instrumentation Northwest, Inc.'s (INW's) nitrate-specific ion-selective electrode (ISE) for measuring nitrate concentrations in groundwater. This evaluation was carried out in collaboration with the U.S. Department of Agriculture / Agricultural Research Service (USDA/ARS) National Laboratory for Agriculture and the Environment. ------- Chapter 2 Technology Description The objective of the ETV AMS Center is to verify the performance characteristics of environmental monitoring technologies for air, water, and soil. This verification report provides results for the verification testing of INW's Aquistar® TempHion™ Smart Sensor and Datalogger nitrate-specific ISE. The following description of the sensor is based on information provided by the vendor. This section describes INW's Aquistar® TempHion™ Smart Sensor and Datalogger, which can be outfitted with sensors for temperature, pH, specific ions (chloride, bromide, or nitrate), or redox elements. The products used in this verification test were outfitted only with nitrate ISEs and temperature electrodes, and the test focused only on the nitrate ISE. The following technology description was provided by the vendor and was not verified in this test. The Aquistar® TempHion™ Smart Sensor is a submersible water quality sensor and datalogger capable of measuring and recording pH, specific ions, redox, level and temperature. Each unit comes with a thermistor-based temperature element and a pressure/level element, with the option of adding up to three pH, ISE, or redox elements. The TempHion™ Smart Sensor logs data, operates on low power, and comes with its own software. The sensor has two digital output protocols, Modbus or Sdil2; both options are license-free digital communication languages. Several TempHion™ sensors, or a combination of TempHion™ sensors and other INW Smart Sensors, can be networked together and controlled from one location, either directly from a computer or through a WaveData® Wireless Data Collection System. The sensor used in this verification test is shown in Figure 2-1. Data were collected automatically using a cellular modem link, INW's auto-collection program (Aqua4Push), and Groundswell Technology's visualization software. A schematic showing the data collection and transmission process is shown in Figure 2-2. The TempHion™ Smart Sensor can be powered internally with two AA alkaline batteries, or with an auxiliary power supply for data intensive applications. The unit is programmed using a laptop or desktop Windows®-based computer via its RS485/RS232 or USB port and INW's Aqua4Plus utility software. Once programmed, the unit will measure and collect data internally on a variety of time intervals. The internal processor in the TempHion™ Smart Sensor allows for calibration using the calibration utilities in INW's Aqua4Plus software. Once calibrated, the calibration data are stored in non-volatile memory within the Smart Sensor and are applied to the collected data. ------- .Top-cap/cable harness 2 AA batteries Smart Sensor recording circuit board 5-channels with digital output Nitrate module Embedded Thermistor Patented reference electrode Figure 2-1. Aquistar® TempHion™ Smart Sensor Data Flow Overview Located at EPA ETV Site Located Off Site Copyright 2010 Groundswell Technologies Figure 2-2. Sensor Collection and Data Transmission Schematic ------- Chapter 3 Test Design and Procedures 3.1 Introduction This verification test was conducted over a nine-week period beginning in April 2010 and ending in July 2010, according to procedures specified in the Test/QA Plan (TQAP) for Verification of Nitrate Sensors for Groundwater Remediation Monitoring (1). As indicated in the test/QA plan, the testing conducted satisfied EPA QA Category III requirements. The test/QA plan and/or this verification report were reviewed by: • Stu Nagourney, NJDEP • Kenneth Wood, DuPont • Michael Brody, U. S. EPA • Charles Spooner, U.S. EPA (test/QA plan only) • Jacob Gibs, U.S. Geological Survey (report only). The verification was based on comparing the nitrate concentration results from the Aquistar® TempHion™ nitrate ISE to those from a laboratory-based reference method. The reference method for nitrate analysis was ion chromatography (1C), performed by USDA/ARS according to EPA Method 300.1 "Determination of Inorganic Anions by Ion Chromatography" (2) (see Section 3.2). The nitrate sensors were calibrated using a one- or two-point calibration method through INW's proprietary Aqua4Plus software for Microsoft Windows. The nitrate sensors were verified in the laboratory by challenging the sensors with solutions of known nitrate concentrations with and without the addition of selected interference parameters. The sensors were then deployed in the field for nine weeks. Sensor output was verified against groundwater samples collected and analyzed using the EPA laboratory method. 3.2 Nitrate Analysis Reference Method All conventional groundwater samples collected were analyzed following the EPA laboratory method for determination of nitrate. Samples were collected in 125 mL plastic containers that were rinsed with deionized (DI) water, preserved by refrigeration to ± 2°C, and analyzed within 48 hours of collection. The collection of the samples was the responsibility of USD A and Battelle staff. For the reference analysis, a Dionex ICS-2000 Reagent-Free Ion Chromatography (RFIC) System was operated by USDA staff according to instrument procedures (see Appendix E of the TQAP) and the manufacturers' instructions, including those for warm-up and stabilization time before testing. The USDA laboratory was responsible for coordinating the analysis of the samples with associated QA/quality control (QC). Calibration and maintenance ------- documentation for the Dionex ICS-2000 and all results of the reference analyses were provided as part of the data dissemination process. A laboratory audit addressing 1C data collection was performed by the NJDEP (see Section 4.2.2) according to guidelines provided by the 2003 National Environmental Laboratory Accreditation Conference (NELAC) Standard. 3.3 Test Design INW's nitrate sensor was verified based on the following performance parameters: • Accuracy • Variability of readings • Duplication of readings • Effect of nitrite, turbidity, and chloride on nitrate sensor readings • Operational and sustainability factors, including ease of use, downloading of data, timely dissemination of data, and environmental impacts of using nitrate sensors for real-time remote data collection. The verification test involved two separate stages: a laboratory testing stage in which sensors were challenged with known nitrate concentrations and interference parameter concentrations, and a field application stage in which several sensors were placed in monitoring wells and streamed data to a remote server. Nitrate sensor concentration data were compared to laboratory 1C data to determine a number of verification parameters including accuracy and variability. 3.3.1 Laboratory Testing Stage The laboratory stage of the verification test was performed in the USDA/ARS laboratory, and involved challenging the nitrate sensors by measuring solutions of known nitrate concentrations in two clear polyvinyl chloride (PVC) test cells measuring 4-ft high with a 2-inch diameter (Figure 3-1). One test cell contained a single nitrate sensor suspended approximately 6 inches below the base of the test cell, whereas the other test cell contained two duplicate sensors suspended at the same depth as in the first test cell. During Phase 1 of the laboratory stage, nitrate solutions of known concentration were added to the test cells and sensors were programmed to begin collecting data in one-minute intervals for a predetermined period of time (20 minutes) for each nitrate concentration. After sensor data collection, a water sample for 1C analysis was collected from each cell through the attached stopcock located at the base of the test cell. Phase 2 of the experiment followed the same methods, but interference parameters (chloride, nitrite, and turbidity) at varying concentrations were also added to the nitrate solutions. The sensors were programmed to collect readings at one minute intervals for a period of 10 minutes for each nitrate/interference parameter concentration combination performed during Phase 2. Further details of the experimental design for the laboratory stage are located in the Test/QA Plan for Verification of Nitrate Sensors for Groundwater Remediation Monitor ing (1). Table 3-1 provides a summary of the nitrate and interference parameter concentrations used in the laboratory testing. ------- Figure 3-1. Laboratory Test Cell Configuration Table 3-1. Summary of Nitrate Sensor Laboratory Testing Phase 1 (Nitrate Only) 2 (Nitrate Plus Interference) Interference Parameter None Chloride Nitrite-N Turbidity Interference Parameter Concentration Chloride = ND Nitrate-N = ND Turbidity = ND lOOmg/L 500mg/L 2,500 mg/L Img/L 2 mg/L 4 mg/L 1NTU 5NTU Nitrate-N Concentration" (mg/L) 1.0 (4.4) 1.0 (4.4) 1.0 (4.4) 1.0 (4.4) 3.0(13) 3.0(13) 3.0(13) 3.0(13) 6.0 (26) 12 (53) 12 (53) 12 (53) 12 (53) - - - a: Equivalent nitrate concentrations in parentheses NTU = nephelometric turbidity unit ND = non detect 3.3.2 Field Testing Stage Field testing consisted of nitrate sensor deployment in an end-of-tile bioreactor located at the Kelly Farm research site in Ames, IA. The bioreactor is an excavated below-grade cavity filled ------- with wood chips at the downstream end of a series of subsurface tiles that are used to promote drainage in the surrounding agricultural area. The tile drainage water is routed through the bioreactor, where the wood chips naturally support populations of microorganisms that remove the nitrate through denitrification. The bioreactor contains inlet and outlet piping, and seven monitoring wells that are screened from 2 to 6 feet below ground surface (bgs) and used to monitor water quality (see Figure 3-2). I Inlet Welll (Inlet) Nitrate Sensor ID: W1_ISE(3) Sensor Depth: 4 ft bgs Well 2 Nitrate Sensor ID: W2_ISE(5) Sensor Depth: 4 ft bgs WellS Nitrate Sensor IDs: W3_ISE(7), W3_ISE2(14) Sensor Depths: 3 ft bgs, 5 ft bgs Well 4 Nitrate Sensor ID: W4_ISE(9) Sensor Depth: 4 ft bgs WellS Nitrate Sensor ID: W5_ISE(11) Sensor Depth: 4 ft bgs Well6 Nitrate Sensor ID: W6_ISE(13) Sensor Depth: 4 ft bgs Welll Wei I 2 Well 3 (Shallowand Deep Sensor) Wei 14, WellS Wei I 6 Outlet Wells with Sensors Well without Sensors Not to scale North Figure 3-2. Schematic Showing Well and Sensor Layout in Bioreactor Sensors measured continuous nitrate concentrations in 15-minute intervals from seven locations within the bioreactor (Figures 3-2 and 3-3) for a period of nine weeks. Sensors were deployed in the inlet to the bioreactor, and in four 2-inch PVC monitoring wells (two sensors were deployed in one of the wells at varying depths to evaluate vertical gradients within the test cell) (see Figure 3-2 for sensor deployment depths). Data were transmitted wirelessly to the vendor's server, and then forwarded to a Web site for download. Conventional groundwater samples were collected weekly for the nine-week deployment period using a low-flow purging technique, whereby dedicated tubing was installed in each well and attached to the sensor with a zip tie so that the tubing inlet was located at the same depth where the sensor reading was collected (see Figure 3- 4). In addition to the weekly monitoring, two days of intensive conventional sampling events 7 ------- were performed at the beginning and end of the test, during which samples were collected once per hour for eight hours. It should be noted that although nitrate sensors were deployed in seven locations, only four of the locations (Wells 1, 2, 3 [shallow and deep], and 6) were used for data evaluation, as outlined in the TQAP. In addition, water-level sensors were placed in each of the monitored wells. Data from the additional sensors and from the additional wells were used for a broader evaluation performed by NJDEP to evaluate the spatial distribution of water quality within the test cell, and was conducted separately from this ETV test; accordingly, the data generated from these sensors were not evaluated in this verification report. Figure 3-3. Figure Showing Nitrate Sensors in Bioreactor Figure 3-4. Nitrate Sensor with attached Conventional Sample Tubing ------- 3.4 Test Procedures Comparisons were made between nitrate concentrations measured using the nitrate sensor and those measured in the laboratory using the EPA laboratory method from analysis of samples collected using the conventional groundwater sampling technique. It is assumed that the nitrate concentration value in the sample collected using the conventional sampling technique represented the actual or target nitrate concentration present in the well against which the nitrate sensor concentration was being evaluated. Table 3-2 summarizes the types and numbers of samples that were used to verify the performance of INW's nitrate sensors. The test procedures used to evaluate the performance of the nitrate sensors are presented in the following subsections. Table 3-2. Summary of Nitrate Sensor Verification Samples Sample Type Phase 1 laboratory water samples Phase 2 laboratory water samples Field groundwater samples User observations Approximate Number of Samples or Readings 80 240 200 All Associated QC Samples Equipment rinsates "Field" duplicates Laboratory QA/QC Equipment rinsates "Field" duplicates Laboratory QA/QC Equipment rinsates Field duplicates Laboratory QA/QC Not Applicable Uses Accuracy, variability, duplication, user agreement, operational factors Accuracy, duplication, effect of changes in water quality, user agreement, operational factors Accuracy, effect of changes in water quality, user agreement, operational factors Operational factors 3.4.1 Accuracy Prior to deployment and testing, each nitrate sensor was calibrated by the vendor. Immediately after calibration, the sensor was programmed to take a few readings while the sensor was still in the reference standard to verify the accuracy of the initial calibration. The accuracy of the nitrate sensor in the field and in the laboratory was determined by comparing nitrate sensor readings to simultaneous measurements made using conventional (low flow) groundwater sampling techniques. The comparison of accuracy was made statistically and graphically by plotting nitrate sensor readings (concentrations) against the nitrate concentrations measured in groundwater samples collected using conventional techniques. In the laboratory testing (see Tables 3-1 and 3-2), nitrate concentrations were generated from a concentrated stock solution spanning the range of anticipated field concentrations, and evaluated with the nitrate sensors and conventional EPA method sample analysis. Additionally, in Phase 2 of the laboratory testing, concentrations of interference parameters (chloride, nitrite, and turbidity) were introduced into the test cells to evaluate the ability of the nitrate sensor to accurately measure nitrate concentrations under simulated field conditions. ------- 3.4.2 Variability Variability of nitrate sensor concentration readings refers to the consistency in reported nitrate concentrations. Variability was assessed in Phase 1 of the laboratory evaluation using 20 readings made by each of three sensors deployed in separate test cells at four reference nitrate concentrations (see Table 3-1). Variability was equated to drift and expressed as percentage change in concentration as a function of time compared to the reference concentrations. 3.4.3 Duplication The degree of agreement of nitrate concentrations reported simultaneously using duplicate nitrate sensors was assessed in the laboratory in the two test cells. As discussed in Section 3.2.1, one test cell housed two nitrate sensors to evaluate intra-well duplication within the test cell, and a second test cell housed one nitrate sensor to evaluate inter-well duplication between the two test cells. The three nitrate sensors were synchronized and programmed to record nitrate concentrations at one-minute intervals throughout Phase 1 and Phase 2 of the laboratory test to directly compare nitrate concentration data. 3.4.4 Effect of Changes in Water Quality The effect of water quality (i.e., interference parameters) on nitrate sensor response to nitrate concentrations was evaluated in Phase 2 of the laboratory testing by exposing nitrate sensors to constant nitrate concentrations under different water quality conditions. The laboratory testing schedule is described in Section 3.2.1 and summarized in Table 3-1. The ability of the nitrate sensors to accurately measure nitrate concentrations was evaluated under each of the 24 different scenarios. In addition to the laboratory testing, conventional groundwater field and associated QC samples were analyzed for the presence and level of nitrite as nitrogen (nitrite-N) until negligible concentrations (<1 mg/L) were verified in successive monitoring events. An initial sampling event also was conducted at each well in the test cell prior to sensor deployment to evaluate background concentrations of nitrate, nitrite, and chloride in groundwater (see Section 3.5). 3.4.5 Operational and Sustainability Factors Operational factors associated with use of the nitrate sensors were evaluated based on the comments and observations of verification test staff (i.e., Battelle and USD A) in laboratory and field testing. Such observations addressed the convenience and environmental impact of using the nitrate sensors, the completeness of nitrate sensor readings (percent data collected), their reliability under differing conditions, the apparent consistency of nitrate sensor readings, and acceptability as a groundwater monitoring tool. Observations also included any noted biofouling at the end of the field testing period. In addition, data dissemination was evaluated, including ease of data transmission, timeliness of data dissemination, ease of data downloading, and usability of downloaded data. Cost for the nitrate sensor and associated data transmission equipment also was reported. 10 ------- 3.5 Analysis of Baseline Concentrations Prior to the field verification testing, conventional groundwater samples were collected by field personnel from Battelle and USDA from each location within the test cell to evaluate background nitrate concentrations. In addition, the samples were analyzed for nitrite and chloride to better understand the background water quality. The background laboratory analyses were performed by USDA. The results from these analyses are summarized in Table 3-3. As discussed in Section 3.4.4, groundwater samples were analyzed for nitrate during the initial stages of the field test to understand baseline levels before starting the field testing. Because nitrite concentrations were consistently well below the 1 mg/L threshold throughout the first day of intensive sampling, continued monitoring for nitrite was unnecessary. Table 3-3. Summary of Background Water Quality Sampling Location Well 1 (Inlet) Well 2 Well 3 S Well 3D Well 6 Nitrate (mg/L) 8.59 8.54 8.51 7.82 8.61 Nitrite (mg/L) 0.01 0.02 0.02 0.02 0.03 Chloride (mg/L) 14.27 14.47 14.20 14.02 14.17 3.6 Verification Schedule Table 3-4 summarizes the schedule for verification testing, data analysis, and reporting. Table 3-4. Verification Schedule Date Verification Activity April 19-22, 2010 Completed Performance Evaluation Audit April 26-29, 2010 Completed Phase 1 and Phase 2 of the laboratory evaluation Completed Technical Systems Audit Installed nitrate sensors in test cell Collected initial groundwater samples from test cells for analysis of nitrate and potential interference parameters Began field test Completed first day of initial intensive sampling event April 29, 2010 through July 12, 2010 May 3, 2010 July 13-14, 2010 September 2, 2010 September 8-22, 2010 Performed field test Completed two Audits of Data Quality Completed second day of initial intensive sampling event Completed final intensive sampling event Completed final Audit of Data Quality Peer review of draft report 11 ------- Chapter 4 Quality Assurance/Quality Control QA/QC procedures were performed in accordance with the Quality Management Plan (QMP)for the ETV Advanced Monitoring Systems Center (3) and the TQAP for this verification test (1). QA/QC procedures and results are described below. 4.1 Laboratory Sample Analysis QA/QC Quality of the laboratory reference nitrate measurements were ensured by a calibration of the Dionex ICS-2000 RFIC before testing began. A pre-testing calibration curve was prepared for each analytical run; the curve was required to be linear with the coefficient of determination (R2) greater than or equal to 0.995 before proceeding with analysis. Calibration was verified throughout the analytical run by inserting calibration check standards and reagent blanks with every set of 10 samples. The calibration and all verifications are incorporated into the run alongside the samples and visually evaluated by the instrument operator to meet the reference laboratory's QC criteria. A complete description of the USDA's current Dionex ICS-2000 RFIC analytical procedures including equipment, standards, reagents, and calibration is included in Appendix E of the TQAP. The following subsections summarize the results of the laboratory sample analysis QA/QC procedures. 4.1.1 Instrument Calibration Checks Instrument calibration checks were performed on each batch of samples submitted for laboratory analysis. A nitrate-N standard concentration of 10 or 20 mg/L was used for instrument calibration. Three separate calibration checks were performed, one during the initial portion of the laboratory run (initial calibration check [ICC]), one during the laboratory run (continuing calibration check [CCC]), and one near the end of the laboratory run (end calibration check [ECC]). If the determined concentrations were not within 90% to 110% of the stated values, performance of the determinative step of the method was unacceptable and would be repeated. The results of the instrument calibration checks are summarized in Table 4-1, and indicate that the instrument calibration check QC criteria were met for all samples. 4.1.2 Initial Calibration Checks To establish the ability to generate acceptable precision results, the laboratory analyzed 10 replicates of a mid-range standard within the range of anticipated field concentrations as an initial calibration (ICAL) check. The results of the replicates were used to compute the average percent recovery and the standard deviation for the analyte. A linear calibration curve with the 12 ------- R2 greater than or equal to 0.995 is required for acceptance. The results of the ICAL checks are summarized in Table 4-1, and indicate that the ICAL QC criteria were met. 4.1.3 Laboratory Reagent Blanks A laboratory reagent blank (LRB), consisting of filtered DI water, was included in each laboratory batch run. Although the acceptance criteria for the LRB were not defined in the TQAP or in the laboratory protocol, discussions with USDA ARS personnel indicated that QC criteria were met if LRB concentrations were below the nitrate-N method detection limit (MDL) (<0.3 mg/L). The LRB results are summarized in Table 4-1, and indicate that the LRB QC criteria were met with the exception of the absence of LRB samples in the third and eighth field sampling events. 4.1.4 Laboratory Fortified Blanks A laboratory fortified blank (LFB), consisting of filtered DI water spiked to a nitrate-N concentration of 20 mg/L, was included in each laboratory batch run. QC criteria were met if the determined concentrations were within 85% to 115% of the stated value. The LFB results are summarized in Table 4-1, and indicate that the LFB QC criteria were met. 4.1.5 Laboratory Fortified Sample Matrix Laboratory fortified sample matrix (LFSM), or matrix spike/matrix spike duplicate (MS/MSD) samples, were prepared and analyzed at a rate of 5% of the total number of samples. For this analysis, the selected field water sample was divided after filtering and aliquotted into Dionex PolyVials and stored in the refrigerator. Following analysis to determine the background concentration, these reserved samples were spiked with a concentrated solution of nitrate-N to achieve concentrations 2 to 3 times above background with a minimal (<2%) change in sample volume. QC criteria were met if the determined recovery was within 75% to 125% of the stated value. The LFSM (MS/MSD) results are summarized in Table 4-1, and indicate that the LFSM QC criteria were met. 4.1.6 Laboratory Duplicate Samples Duplicate analyses were performed in 10% of the field samples, with QC criteria being met if the relative percent difference (RPD) was ±10%. The laboratory duplicate sample results are summarized in Table 4-1, and indicate that the laboratory duplicate sample QC criteria were met for all samples. It should be noted that the concentrations of many of the laboratory duplicate samples collected during the final intensive sampling events were well below the MDL, so the RPD was not calculated. 13 ------- Table 4-1. Summary of Laboratory and Field QA/QC Samples for Nitrate Results Sampling Event Phase 1 Laboratory Test Phase 2 Laboratory Test Initial Intensive Sampling Event Day 1 Initial Intensive Sampling Event Day 2 Field Sampling Event 1 Field Sampling Event 2 Field Sampling Event 3 Field Sampling Event 4 Field Sampling Event 5 Field Sampling Event 6 Field Sampling Event 7 Field Sampling Event 8 Field Sampling Event 9 Final Intensive Sampling Event Day 1 Final Intensive Sampling Event Day 2 Date 04/27/10 04/28/10 04/29/10 05/03/10 05/10/10 05/17/10 05/24/10 06/01/10 06/07/10 06/17/10 06/22/10 06/28/10 07/06/10 07/13/10 07/14/10 Laboratory QA/QC Instrument Calibration Check Result (%) ICC 99.6 100 98.8 101 99.5 99.6 100 99.8 100 100 101 100 100 100 100 100 100 ccc 99.7 101 99.6 100 101 99.3 101 101 100 101 101 101 102 99.6 99.7 100 100 101 100 100 101 101 101 101 100 99.6 100 101 101 100 ECC 101 101 100 101 99.0 101 99.7 100 101 101 99.9 100 101 101 101 99.9 102 ICAL (R2 Value) 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 LRB (mg/L) 0.005 0.007 0.006 0.005 0.006 0.005 0.008 NA 0.004 0.001 0.009 0.005 NA 0.033 0.001 0.001 0.001 LFB (%) 99.8 99.8 100 100 100 101 99.8 99.7 101 100 99.7 101 101 101 100 99.8 99.5 99.4 99.7 99.9 99.6 99.5 99.9 99.8 99.8 99.5 99.9 100 99.9 100 101 101 101 LFSM (%) 102 93.8 102 101 102 101 98.5 101 103 103 102 101 101 102 113 102 103 99.2 101 101 101 101 101 101 100 100 99.8 100 Laboratory Duplicate (RPD) 0.05 2.8 0.42 0.80 7.1 2.0 0.77 3.8 0.43 3.7 0.03 2.1 0.04 0.58 2.4 0.59 0.21 2.4 4.4 8.2 0.03 1.9 0.02 0.09 5.0 NC NC NC NC NC NC 0.19 Field QA/QC Field Duplicate (RPD) NS 0.03 0.17 2.6 0.49 0.19 4.6 3.3 0.47 0.58 4.6 0.24 1.3 3.8 4.2 0.17 1.1 NS NS NS 3.1 NS 0.04 0.62 NS NS 14 NC NC NC NC NC Rinsate Blank (mg/L) 0.02 0.01 0.03 0.02 4.1 2.4 0.03 0.01 0.01 0.01 0.02 0.02 0.02 0.04 0.01 <0.01 NA - not analyzed NS - no sample collected NC - not calculated; laboratory duplicate concentrations were at or below the method detection limit. 14 ------- 4.2 Field QA/QC During field and laboratory groundwater sampling activities, QC samples, including field duplicates and equipment blanks, were collected to ensure the reliability of field data. The field QC samples are discussed in the following sections. 4.2.1 Field Duplicate Samples Duplicate groundwater samples were collected at a frequency of one for every 10 samples (i.e., 10%) to evaluate the reproducibility of analytical results. If 10 samples were not collected during a sampling event, one duplicate sample per sampling event was collected. Duplicate samples were collected simultaneously with the original sample into identical sample containers. The QC criteria were met if the RPD was ±10%. The field duplicate sample results are summarized in Table 4-1, and indicate that the QC criteria for sample collection frequency was not met on several occasions, and the RPD was slightly exceeded on one occasion. It should be noted that the concentrations of many of the field duplicate samples collected during the final intensive sampling events were well below the MDL, so the RPD was not calculated. 4.2.2 Equipment Blanks Equipment blanks, also referred to as rinsate blanks, were collected to evaluate the potential for sample cross-contamination from the sampling equipment used. Equipment rinsate blanks were collected daily during sampling to ensure that nondedicated groundwater sampling equipment had been decontaminated effectively. Daily equipment blanks were collected after collection of at least one field sample and after the equipment was decontaminated. The equipment blank for groundwater sampling equipment was laboratory-provided DI water that was passed through or over the sampling equipment used to collect samples (i.e., Teflon® polyethylene tubing). The QC criteria were met if the analytical results from the equipment blank sample were <2 mg/L nitrate-N. The equipment rinsate sample results are summarized in Table 4-1, and indicate that the QC criteria were not met on two occasions near the beginning of the field study, and based on the elevated concentration, may be indicative of improper rinsate blank sample collection. 4.3 Audits Three types of audits were performed during the verification test: a performance evaluation (PE) audit of the laboratory analysis method, a technical systems audit (TSA) of the verification test performance, and three data quality audits. Audit procedures are described further in the following subsections. 4.3.1 Performance Evaluation Audits A PE audit was performed to confirm the accuracy of the laboratory nitrate analysis reference method. Prior to the laboratory and field investigations, five blind samples of varying nitrate concentrations within the range of anticipated field concentrations were generated from a stock solution and shipped to the USDA ARS laboratory on 19 April 2010 for analysis on 21 April 2010. Table 4-2 summarizes the results of the PE audit of laboratory nitrate analysis reference method, showing the stock solution generated nitrate concentration, the laboratory 1C nitrate concentration, and the RPD between the two concentrations. Table 4-2 shows that all of the RPD values for generated nitrate concentrations were below 4%, and a graphical plot of the data 15 ------- indicated an R2 value of 1.00 The PE audit results were within the target RPD tolerances of 10% set forth in Appendix E of the TQAP. Table 4-2. Summary of PE Audit Results Generated Nitrate-N Concentration (mg/L as N) 6.0 3.0 12 0 1.0 Laboratory 1C Result (mg/L Nitrate-N) 6.09 3.03 12.5 0.026 1.03 RPD 1.5 1.0 3.8 Not applicable o o 3.3 4.3.2 Technical Systems Audit A Quality Auditor from the NJDEP (Amy Bowman) conducted a ISA at the USDA ARS laboratory and field test site on 27-29 April 2010 to ensure that the verification test was being conducted in accordance with the TQAP (1) and the AMS Center QMP (3). This audit was designed to achieve the following objectives: • Evaluate all activities related to the installation and verification testing of the nitrate sensors • Review laboratory elements of the TQAP for Verification of Nitrate Sensors for Groundwater Remediation Monitoring, prepared by the ETV Program • Assess data from Performance Evaluation samples analyzed by 1C • Audit laboratory operations and 1C instrument operations at the USDA ARS research facility for method and QA/QC compliance. During this TSA, the NJDEP Quality Auditor performed a review of 1C data and operations. Issues related to run logs and inclusion of the required QC samples were reviewed with the laboratory analyst, and the run logs were revised to include the required QC samples. In addition, a review of the laboratory sample receipt procedures was performed, and sensor testing and data recording were observed in the laboratory. The initial groundwater sampling event (for collection of background nitrate and interference parameter concentrations) was observed, as was a partial installation of a nitrate sensor array into one of the field monitoring points. During sensor installation, the vendor was interviewed about sensor installation and calibration in the field. The TSA of both the laboratory and field testing portions resulted in eight findings and nine observations. The corrective actions taken in response to significant findings of the TSA were as follows; • Inclusion of a temperature blank to measure sample temperature upon sample receipt in the laboratory 16 ------- • Revision of the initial intensive sampling schedule from a Thursday/Friday schedule to a Thursday/Tuesday schedule to ensure the sample holding time was not exceeded • Revision of the laboratory run logs to include QC samples required by the analytical method. The remaining findings and the observations noted documentation errors, need for improvements to the manner in which samples were processed, and QC sample frequency deficiencies. The findings and observations were discussed onsite with the field team and subsequently with the verification test coordinator (VTC) and project team via a conference call; immediate changes based on the discussed improvements were implemented. A TSA report was prepared, and a copy was distributed to the EPA. 4.3.3 Data Quality Audits Records generated in the verification test received a review from a technical person independent of the person generating the data before these records were used to calculate, evaluate, or report verification results. Data were reviewed by a Battelle technical staff member involved in the verification test. The person performing the review added his/her initials and the date to a hard copy of the record being reviewed. All of the verification test data were reviewed for quality by the VTC, and at least 10% of the data acquired during the verification test were audited. The data were traced from the initial acquisition, through reduction and statistical analysis, to final reporting to ensure the integrity of the reported results. All calculations performed on the data undergoing the audit were checked. Three data quality audits were performed. The first data quality audit, which covered the laboratory test investigation, resulted in 11 findings, five observations, and two recommendations. The first audit results were related to laboratory and field QC procedures, laboratory reporting and documentation issues, laboratory instrument calibration procedures, and data transcription errors. The second data quality audit, which covered the first intensive sampling event, resulted in three findings and three observations. The second audit results were related to laboratory and field QC procedures and documentation errors. The third data quality audit, which covered the weekly field sampling events and the second intensive sampling event, resulted in four findings and two observations. The third audit results related to laboratory and field QC procedures and documentation errors. The data quality audit issues were addressed with procedural changes and references to the TSA audit recommendations that were implemented to refine the field and laboratory QC procedures. Three data quality audit reports were prepared, and copies were distributed to the EPA. 17 ------- Chapter 5 Statistical Methods The statistical methods used to evaluate the quantitative performance factors listed in Section 3.1 are presented in this chapter. The methods described below are consistent with those outlined in the approved TQAP; the additional methods not outlined in the TQAP were selected by a Battelle statistician to provide an additional data evaluation approach, and were based on several iterations of representative statistical methods. Qualitative observations also were used to evaluate verification test data. 5.1 Accuracy Accuracy was determined by comparing nitrate sensor readings to water samples collected during laboratory and field testing. The water samples were analyzed in the laboratory using EPA-approved analysis methods. The accuracy of the nitrate sensor concentrations with respect to the laboratory measured concentrations was assessed graphically and by evaluating the differences between paired concentrations (concentration residuals) from measurements collected simultaneously at the same location. The nitrate sensor concentration reading (reported every 15 minutes) collected closest in time to the collection of the reference monitoring sample was initially used for paired comparison. Two statistical measurements were used to assess the accuracy: (1) inference about the mean difference, and (2) estimation of the mean absolute error (MAE). The inference about the observed difference included estimation of the mean difference and a statistical hypothesis about whether the mean difference was equal to or different from zero (using a paired-sample t-test). The hypothesis tests were conducted on the natural log of the concentration data in order to more closely approximate the normal or Gaussian distribution and thereby satisfy the assumptions of the t-test. The log of the laboratory concentration was subtracted from the log of the sensor concentration and the null hypothesis was that the resulting difference had a mean of zero. If the p-value of the hypothesis test was below 0.05, the null hypothesis was rejected and there was strong evidence to suggest that the sensor and laboratory concentrations were not equal. If the p- value was larger than 0.05, there was not strong evidence to reject the null hypothesis. (It should be noted that even when there are no differences in two underlying population means, differences in random samples drawn from those populations [due only to sampling error] should be expected.) When considering the p-value for a specific hypothesis test, the p-value is the proportion of times that a difference of the magnitude observed in these data, or larger, would be expected by chance due only to sampling error if the null hypothesis were true (the null hypothesis often states that there is no underlying difference between the two population means). If the p-value is smaller 18 ------- than 0.05, the difference is large enough to be expected fewer than five out of every 100 experiments (5/100 = 0.05), even if there are no underlying differences between the two groups. In these cases, the null hypothesis was rejected and it was concluded that there was noteworthy evidence of an underlying difference. The MAE was calculated for the concentration differences as follows: n 1V MAE = - > (laboratory concentrationj — nitrate sensor concentrationj | where n is the number of paired nitrate concentration measurements. The MAE was used to represent the average absolute difference in the two measurement methods. The statistical analyses for accuracy, as outlined in the TQAP (1), called for hypothesis tests where the null hypothesis was strict equality between the mean sensor and lab measurements. Based on discussions with personnel involved with sensor technology and nitrate field monitoring, and the desire to further evaluate the overall objective of this nitrate sensor evaluation, the null hypothesis was modified to evaluate whether the sensor measurements were, on average, within a select percentage (i.e., 25%) of the lab results. To perform this additional evaluation, the RPD for each sensor reading was computed, and an estimate of the upper confidence limit for that quantity was made. For example, if the upper confidence limit was below 25%, a null hypothesis that says that the average sensor error was smaller than 25% would not be rejected. It should be noted that these calculations are not intended to be used for strict acceptance criteria or for apportioning the variance in observations between different components of variation. For the purpose of evaluation, an RPD value of <20% was considered to represent general agreement between sampling methods. The RPD was calculated as follows, and assumes that the laboratory concentration is the accepted (benchmark) concentration value for comparison purposes: RPD = 100 x (nitrate sensor concentration — laboratory concentration) laboratory concentration The accuracy estimates were calculated separately for Phase 1 and Phase 2 of the laboratory evaluation, for the two intensive hourly sampling events at the beginning and end of the field evaluation, and for the weekly sampling conducted during the field evaluation. In addition, comprehensive accuracy estimates were calculated using all of the paired data sets from the field and laboratory evaluation. Well-specific MAE values also were calculated. Time series plots showing sensor and conventional monitoring data collected during the field investigation also were used to evaluate the accuracy. 5.2 Variability Variability was assessed by observing the spread of nitrate sensor readings made at constant nitrate concentrations (equated to drift) using stock solutions in the laboratory portion of the investigation. Variability of the nitrate sensor concentration readings was evaluated using data from Phase 1 of the laboratory evaluation using the multiple readings (20) made by each of three sensors deployed in separate test cells at four reference nitrate concentrations (1, 3, 6, and 12 19 ------- mg/L nitrate-N). Variability was expressed as the standard deviation of the sensor concentration readings calculated two ways: (1) using the reference concentrations as the average (mean) values for comparison, and (2) using the measured mean concentrations (geometric mean from log-transformed results) from the samples. Standard deviation values were calculated for each of the four reference concentrations. 5.3 Duplication Nitrate sensor duplication of readings was assessed by comparing nitrate sensor readings made by placing duplicate nitrate sensors in a single test cell in the laboratory so they were exposed to identical concentrations simultaneously during Phase 1 and Phase 2 of the laboratory tests. The degree of agreement of nitrate concentrations reported simultaneously using duplicate nitrate sensors was assessed in this laboratory evaluation. The first test cell housed two nitrate sensors to evaluate intra-well duplication within the test cell, whereas the second test cell housed a single sensor to evaluate inter-well duplication between the two test cells. The degree of agreement between each pair of reported nitrate concentrations (inter-well and intra-well) was assessed by calculating the intra-well mean square error (MSB) and inter-well MSB using a random-effects analysis-of-variance (ANOVA) model. 5.4 Effect of Changes in Water Quality The effect of nitrite, turbidity, and chloride on nitrate sensor readings was assessed in the laboratory by comparing nitrate sensor readings exposed at constant nitrate concentrations with varying interference parameter (nitrite, turbidity, and chloride) concentrations. Consistency of nitrate sensor readings over time was assessed in the accuracy of nitrate sensor readings made in the field and in the laboratory over time, verified against laboratory analyses. The field testing evaluation (particularly the readings collected near the end of the field period) served to evaluate the effects of sensor fouling on the accuracy and duplication performance parameters. The effect of changes in water quality on nitrate sensor performance was assessed using the data from Phase 2 of the laboratory evaluation by calculating the accuracy, variability, and duplication of nitrate sensor readings (Sections 5.1 through 5.3) of the test data at each of the water quality conditions outlined in Table 3-1. The results were compared to indicate whether changes in nitrate, turbidity, and chloride concentrations have any apparent effect on the nitrate sensor performance at constant nitrate concentrations. Accuracy or variability results that differed by more than 5% or nitrate sensor duplication results that differed by more than 20% were taken as evidence of a significant water quality effect. 20 ------- Chapter 6 Test Results The statistical methods used to evaluate the quantitative performance factors listed in Section 3.2 are presented in this chapter. Qualitative observations also were used to evaluate verification test data. 6.1 Accuracy As discussed in Section 5.1, several statistical measurements were used to assess the accuracy, including the paired t-test, and calculation of the MAE. Table 6-1 presents the MAE and t-test results for Phase 1 of the laboratory investigation and a graphical representation of the data are presented in Appendix 2. The MAE increases with increasing nitrate concentrations, ranging from 0.13 to 1.7 mg/L for summing the data from the sensors for each target concentration. The overall MAE for Phase 1 was below 1 mg/L (0.79 mg/L). As noted in Section 5.1, the hypothesis tests for equality between the sensors and laboratory values was strict, with no margin for disagreement. For every sensor at every concentration during Phase 1, the hypothesis of equality was rejected, although the hypothesis was accepted when summing the sensor data for a nitrate-N concentration of 1 mg/L. Table 6-2 presents the MAE and t-test results for Phase 2 of the laboratory investigation and a graphical representation of the data are presented in Appendix 2. Similar to that observed in Phase 1, the MAE increased with increasing nitrate concentrations. Excluding the chloride interference at a concentration of 2,500 mg/L, the MAE ranged from 0.33 to 0.96 mg/L, 0.60 to 2.0 mg/L, and 2.3 to 5.3 mg/L for target nitrate levels of 1, 3, and 12 mg/L, respectively. The chloride interference at a concentration of 2,500 mg/L posed a problem for two of the three sensors regardless of the underlying nitrate concentration. The MAE for the chloride concentration of 2,500 mg/L was 6, 7, and 9 mg/L for target nitrate levels of 1, 3, and 12 mg/L, respectively. The Phase 2 data show that varying the nitrite and turbidity levels have little effect on the sensor performance at the respective nitrate concentrations, whereas increasing chloride concentrations have a significant effect on sensor performance. Table 6-2 shows that the hypothesis of strict equality was rejected for the majority of nitrate and interference parameter concentration combinations. Table 6-3 presents the MAE and t-test results for the field investigation and a graphical representation of the MAE data are presented in Appendix 2. For the field evaluation, all wells were summarized into a single set of descriptive statistics for the respective test period (initial and final intensive sampling and weekly sampling) and the overall field test. The MAE increases with time, from 3.3 mg/L during the initial intensive sampling to 7.3 mg/L for the final intensive 21 ------- sampling. The overall field test MAE is 5.9 mg/L. The hypothesis of strict equality was rejected for the intensive sampling events and the overall field test, but was not rejected for the weekly sampling portion of the field test. Table 6-1. Summary of Phase 1 t-Test and MAE Results Target Nitrate-N Concentration (mg/L) 1 3 6 12 1 o 5 6 12 All Sensor W3-1 W3-2 W4 W3-1 W3-2 W4 W3-1 W3-2 W4 W3-1 W3-2 W4 All All All All All N 20 20 20 20 20 20 20 20 20 20 20 20 60 60 60 60 240 t Statistic -13.7 9.53 32.3 23.3 23.4 No variation1 No variation1 -62.5 -17.1 -390 -41.3 55.2 1.83 36.5 -12.0 -6.83 -4.66 p-Value 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001 O.0001 O.0001 0.0001 O.0001 O.0001 0.0001 0.073 O.0001 0.0001 0.0001 O.0001 Decision Reject Reject Reject Reject Reject Reject Reject Reject Reject Reject Reject Reject Fail to reject Reject Reject Reject Reject MAE (mg/L) 0.13 0.11 0.15 0.25 0.26 0.35 1.8 1.2 0.24 3.2 1.4 0.54 0.13 0.29 1.1 1.7 0.79 1 - Upon peer review of the information presented in this table, it was noted that the issue of 'no variation' could be addressed by doing a multi-way analysis of variance rather than a sequence of t-tests. The multi-way analysis would compute a single pooled estimate of variation using all of the data and use the pooled estimate to calculate the effects associated with of sensor and nitrate concentration. Although it is agreed that this would be a reasonable approach, it would mask the informative result that under two of the conditions presented in the table, a sensor gave 20 consecutive identical readings. The first figure in Appendix 2 illustrates graphically that the sensors were very consistent in Phase 1, if not always accurate. 22 ------- Table 6-2. Summary of Phase 2 t-Test and MAE Results for all Sensors Combined Target Nitrate-N Cone. (mg/L) 1 3 12 Interference Parameter Chloride Nitrite Turbidity Chloride Nitrite Turbidity Chloride Nitrite Turbidity Interference Parameter Cone. lOOmg/L 500 mg/L 2,500 mg/L Img/L 2 mg/L 4 mg/L 1NTU 5NTU 100 mg/L 500 mg/L 2,500 mg/L 1 mg/L 2 mg/L 4 mg/L 1NTU 5NTU 100 mg/L 500 mg/L 2,500 mg/L Img/L 2 mg/L 4 mg/L 1NTU 5NTU N 33 47 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 t Statistic 0.412 17.9 21.6 -3.40 -5.13 -4.32 -1.20 1.17 -6.50 1.85 14.0 -8.25 -8.47 -8.74 -4.21 -2.71 -4.19 -1.11 5.17 -6.47 -5.74 -9.41 -4.06 -2.53 p-Value 0.683 O.0001 0.0001 0.002 O.0001 0.0001 0.240 0.249 O.0001 0.074 0.0001 O.0001 0.0001 0.0001 O.0001 0.011 0.0001 0.276 0.0001 0.0001 O.0001 O.0001 O.0001 0.017 Decision Fail to reject Reject Reject Reject Reject Reject Fail to reject Fail to reject Reject Fail to reject Reject Reject Reject Reject Reject Reject Reject Fail to reject Reject Reject Reject Reject Reject Reject MAE (mg/L) 0.33 0.95 6.4 0.17 0.32 0.35 0.46 0.96 0.60 1.1 6.8 0.90 0.80 1.2 1.1 2.0 2.3 2.8 9.2 4.3 2.7 5.2 4.0 5.3 Table 6-3. Summary of Field Testing t-Test and MAE Results for all Wells Field Data Period Initial Intensive Sampling Weekly Sampling Final Intensive Sampling All N 63 55 80 187 t Statistic -6.10 -0.312 3.80 2.52 p-Value O.0001 0.7583 0.0003 0.0127 Decision Reject Fail to reject Reject Reject MAE (mg/L) 3.3 7.0 7.3 5.9 23 ------- Table 6-4 presents the well-specific MAE estimates for the field evaluation. The data in Table 6- 4 show that the comparison of nitrate concentrations in Well 2 exhibited the lowest MAE values for the initial intensive sampling event and the weekly sampling event, data that are supported by the time series data plots presented in Appendix 1. Well 3D exhibited the lowest MAE values for the final intensive sampling event. Several extreme sensor concentration values that differed significantly from the paired laboratory concentration value resulted in high MAE values for Well 1 (weekly sampling) and Well 6 (initial intensive sampling). Table 6-4. Well-Specific MAE Estimates Well 1 2 3S 3D 6 MAE (mg/L) Initial Intensive Sampling 6.7 1.1 7.2 8.5 570 Weekly Sampling 87 2.8 7.8 4.4 13 Final Intensive Sampling (Day 1, Day 2) 9.4, 9.5 4.6, 11 1.7, 10 0.74, 2.0 11, 14 Table 6-5 presents a summary of the RPD evaluation for the Phase 1 and 2 of the laboratory test, and lists the mean RPD, a 95% upper confidence limit (UCL) for the RPD, and the minimum, maximum, and 25th, 50th, and 75* RPD percentiles for each different phases of the laboratory test. In Phase 1, when there are no interferences present, the 95% UCL for RPD was 15%; this indicates that the sensor measurements were within 15% of the laboratory 1C concentrations on average and they were always within 31% (maximum) of the laboratory-derived concentrations. The average RPD for Phase 1 met the <20% criteria discussed in Section 5.1 that indicates reasonable agreement between the INW sensor and the 1C measurements. In Phase 2, the 95% UCLs ranged from 27% to 77%. The 95%UCL for the highest chloride parameter level was substantially greater, indicating a large interference. The data from the field portion of the experiment are characterized by a number of extreme sensor concentration values that differ significantly from the paired laboratory concentration value. The final intensive sampling event of the field portion of the experiment was especially problematic because many of the laboratory concentrations were near the limit of detection, so the RPD was significantly high. Table 6-6 presents a summary of the RPD evaluation for the field test, including parameters similar to those presented in Table 6-5. The data presented in Table 6-6 indicates that the 95% UCL is above 100% for each of the three evaluation periods when all data are included in the analyses. Due to the presence of significant number of extreme sensor concentration values in the data, additional analyses were performed to selectively remove the most extreme sensor and laboratory concentration pairings for the first intensive sampling and the weekly sampling periods. In the first intensive sampling event, the 95% UCL was 47% when the nine most extreme values were removed. For the weekly data collection period, the 95% UCL was 100% when the two most extreme values were removed. For the second intensive sampling event, the number and magnitude of extreme sensor concentrations 24 ------- was such that the selective removal of a reasonable percentage did not result in a significant reduction in the 95% UCL. It should be noted that the 95% UCL calculations use the simplifying assumption that the observations in each portion of the experiment are statistically independent. That is to say that they ignore the clustering by factors like sensor, well, laboratory measurement, and target nitrate concentration. The goal of these calculations is to investigate broadly the rough order of magnitude of a "buffer zone" that could be built into a null hypothesis in this experiment to have it not be rejected, and these RPD calculations are useful for that purpose. However, it should be noted that these calculations are not intended to be used for strict acceptance criteria or for apportioning the variance in observations between different components of variation. Time series plots showing nitrate concentrations measured weekly using laboratory 1C methods and at 15-minute increments using the INW nitrate sensors are included in Appendix 1. With the exception of Well 3S, nitrate sensor data were capable of reporting relative changes in nitrate concentration over nine continuous weeks. 6.2 Variability Variability of the nitrate sensor concentration readings was evaluated using data from Phase 1 of the laboratory evaluation using the multiple readings (20) made by each of three sensors deployed in separate test cells at four reference nitrate concentrations (1, 3, 6, and 12 mg/L nitrate-N). Variability was expressed as the standard deviation of the sensor concentration readings calculated two ways: (1) using the reference (laboratory) concentrations as the average (mean) values for comparison, and (2) using the measured mean concentrations from the samples. The variability results are summarized in Table 6-7, and show that the variability in readings increases with increasing nitrate concentrations, ranging from 0.13 to 1.5 mg/L using the mean values, and from 0.13 to 2.0 mg/L using the target values. 6.3 Duplication Nitrate sensor duplication of readings was assessed by comparing nitrate sensor readings made by placing duplicate nitrate sensors in a single test cell in the laboratory so they were exposed to identical concentrations simultaneously during Phase 1 and Phase 2 of the laboratory tests. The degree of agreement between each pair of reported nitrate concentrations (inter-well and intra- well) was assessed by calculating the intra-well MSB and inter-well MSB using a random-effects ANOVA model. Table 6-8 summarizes the results of the duplication analyses, and indicates that there is strong evidence for different mean levels (MAE ^ 0) from sensors both within (intra) and between (inter) wells. The inter- and intra-well MAE values for Phase 1 all were below 1 mg/L, but increased above 1 mg/L in a significant majority for the Phase 2 testing. All of the p-values except one intra-well difference are significantly lower than 0.05, indicating the inter- and intra- well results are not equal. 25 ------- Table 6-5. Summary of RPD for Laboratory Test for All Nitrate Concentrations Phase 1 2 Interference Parameter NA Chloride Nitrite Turbidity Interference Parameter Cone. NA lOOmg/L 500 mg/L 2,500 mg/L Img/L 2 mg/L 4 mg/L 1NTU 5NTU N 240 99 113 99 99 99 99 99 99 Relative Percent Difference (RPD) Mean 14% 24% 60% 336% 28% 27% 40% 39% 68% Standard Error Mean 1% 2% 4% 34% 2% 2% 2% 3% 5% 95%UCL 15% 27% 67% 392% 32% 30% 44% 43% 77% Minimum 2% 1% 16% 15% <1% 8% 0% 2% 2% 25th Percentile 8% 7% 29% 86% 12% 11% 11% 15% 51% Median 13% 25% 38% 241% 18% 19% 57% 18% 53% 75th Percentile 18% 41% 113% 351% 52% 48% 60% 67% 76% Maximum 31% 46% 147% 966% 72% 60% 63% 80% 205% Table 6-6. Summary of RPD for Field Test Period Initial Intensive Sampling Weekly Sampling Final Intensive Sampling Extreme Data Points Removed None 2 9 None 1 2 None N 72 70 63 45 44 43 80 Relative Percent Difference (RPD) Mean 618% 151% 40% 437% 237% 85% > 1,000% Standard Error Mean 370% 40% 4% 249% 153% 9% >1,000% 95%UCL >1,000% 217% 47% 847% 489% 100% > 1,000% Minimum <1% 1% <1% 2% 2% 2% 98% 25th Percentile 9% 9% 8% 47% 44% 41% 100% Median 52% 46% 18% 78% 77% 76% > 1,000% 75th Percentile 82% 81% 77% 139% 127% 109% > 1,000% Maximum > 1,000% >1,000% 86% >1,000% >1,000% 253% > 1,000% 26 ------- Table 6-7. Summary of Variability in Phase 1 Laboratory Test Target Nitrate Concentration (mg/L) 1 3 6 12 N 20 20 20 20 StdDev (mg/L) (Ref=Mean) 0.13 0.05 0.69 1.5 StdDev (mg/L) (Ref=Target) 0.13 0.06 1.4 2.0 6.4 Effect of Changes in Water Quality The results of the Phase 2 laboratory data evaluation of accuracy, variability, and duplication (see Sections 6.1 through 6.3, respectively) were compared to indicate whether changes in nitrate, turbidity, and chloride concentrations have any apparent effect on the nitrate sensor performance at constant nitrate concentrations. Accuracy or variability results that differed by more than 5% or nitrate sensor duplication results that differed by more than 20% were taken as evidence of a significant water quality effect. Table 6-9 summarizes the changes in MAE, standard deviation computed both ways (reference equal to the mean, and reference equal to the laboratory target concentration), and intra- and inter-well MSB resulting from changes in water quality (i.e., different interference parameter concentrations). The information presented in Table 6-9 indicates that the changes in MAE, standard deviation (computed both ways), and MSB for both intra-well and inter-well measurements exceed the threshold differences of 20% and 5%. 27 ------- Table 6-8. Summary of Duplication Results in Laboratory Test Phase 1 2 Interference Parameter NA Chloride Nitrite Turbidity Interference Parameter Concentration NA lOOmg/L 500 mg/L 2,500 mg/L Img/L 2 mg/L 4 mg/L INTO 5NTU Target Nitrate Concentration (mg/L) 1 3 6 12 1 3 12 1 3 12 1 3 12 1 3 12 1 3 12 1 3 12 1 3 12 1 3 12 Intra-well MSB (mg/L) 0.64 0 0.22 0.35 2.4 0.74 1.6 1.4 2.0 2.4 6.8 4.3 3.8 2.2 2.0 0.32 5.6 1.3 2.2 1.8 4.0 4.0 1.2 0.81 0.45 7.8 12 1.3 Inter-well MSB (mg/L) 0.40 0.01 0.98 0.85 2.0 0.67 1.2 1.7 1.5 0.85 2.0 2.9 2.6 0.70 0.66 5.8 0.02 0.21 0.75 4.9 0.72 1.2 11 16 11 8.8 3.0 18 Intra-well P O.001 0.704 0.001 0.001 O.001 0.001 O.001 O.001 0.001 O.001 0.001 O.001 O.001 0.001 O.001 0.019 0.001 0.001 O.001 O.001 0.001 O.001 0.001 O.001 O.001 O.001 O.001 0.001 Inter-well P O.001 0.001 0.001 0.001 O.001 0.001 O.001 O.001 0.001 O.001 0.001 O.001 O.001 0.001 O.001 0.001 0.001 0.001 O.001 O.001 0.001 O.001 0.001 O.001 O.001 O.001 O.001 0.001 28 ------- Table 6-9. Summary of the Effect of Changes in Water Quality Interference Parameter Chloride Nitrite Turbidity Target Nitrate Concentration (mg/L) 1 3 12 1 3 12 1 3 12 Change in MAE >1,000% > 1,000% 294% 106% 51% 94% 109% 72% 33% Change in StdDev (Ref=Mean) 775% 687% 178% 60% 61% 33% 117% 65% 47% Change in StdDev (Ref = Target) 777% 418% 133% 57% 67% 61% 144% 39% 24% Change in Intra-well MSB 396% 479% 147% 215% 201% > 1,000% 564% > 1,000% 188% Change in Inter-well MSB 18% 333% 209% >1,000% 247% 664% 23% 453% 60% 6.5 Operational and Sustainability Factors The TempHion™ Smart Sensor was calibrated and installed in each well during the laboratory and field test by a representative from INW without significant problems. The calibration procedure was simple to perform, and was taught to field personnel from Battelle and the USDA ARS laboratory prior to deployment for the field test. During the test, operator observations on sensor performance were recorded on field activity and sampling logs. Nitrate sensor data from each well were transmitted wirelessly to the vendor's server on 15 minute increments, and subsequently forwarded on to a web site for download for near real-time viewing and analysis (see Figure 2-2 for data transmission schematic). During the field investigation, the nitrate sensors each achieved a 100% data collection standard, indicating completeness in data collection. A power outage at the test site did result in a stoppage in data transmittal, but the nitrate sensor data were stored internally within the sensors, and subsequently recovered. The Web site used for data download was easily accessible and data were provided in usable format (i.e., comma-delimited worksheets and Microsoft Excel spreadsheets). The web site used for data download and storage also provided real-time graphics nitrate sensor concentrations, including well-specific time series plots (see Figure 6-1) and plume maps (see Figure 6-2). Upon removal from the test wells, no biofouling was noted on any of the sensors. Review of the laboratory data collected during the field test showed that the nitrate concentrations were at the very low end of this range (typically below 15 mg/L). According to the vendor, the nitrate sensors used in the verification test were capable of detecting and reporting nitrate-N concentrations ranging from approximately 3 to 10,000 mg/L. The vendor indicated that tailoring the operational range of the sensors to anticipated field conditions (i.e., smaller range) and performing a three-point calibration procedure could improve the accuracy of the sensors. The cost of a single TempHion™ nitrate sensor at the time of this verification test was $1,495. TA/f Remote groundwater well monitoring using TempHion nitrate sensors have potential cost and long-term sustainability advantages with fewer field visits and reduced sampling and laboratory 29 ------- analysis costs. The nitrate sensors used in this investigation were programmed to collect nitrate measurements at 15 minute intervals, which equates to 129,600 readings during a traditional quarterly sampling schedule. Yearly monitoring costs for a single well monitored conventionally on a quarterly schedule can exceed $2,000, including labor, equipment, disposal of purge water, and frequent transportation to and from the field site. Figure 6-1. Time Series Graph of Sensor Nitrate Concentrations in Well 1 (Inlet) Figure 6-2. Plume Map Showing Dissolved Sensor Nitrate Concentrations in Test Cell 30 ------- Chapter 7 Performance Summary The evaluation of the accuracy of the nitrate sensors indicated that the MAE typically increased with increasing nitrate concentrations. In Phase 1 of the laboratory investigation, the overall MAE was below 1 mg/L, and in Phase 2, the MAE ranged from 0.33 to 0.96 mg/L, 0.60 to 2.0 mg/L, and 2.3 to 5.3 mg/L for target nitrate levels of 1, 3, and 12 mg/L, respectively (excluding the chloride interference at a concentration of 2,500 mg/L, which had MAE values above 6 mg/L). For the field evaluation, the MAE increased with time, from 3.3 mg/L during the initial intensive sampling to 7.3 mg/L for the final intensive sampling, with an overall MAE of 5.9 mg/L. The hypothesis of strict equality was rejected for every sensor at every concentration during Phase 1, for the vast majority of nitrate and interference parameter concentration combinations in Phase 2, and for the intensive sampling events and the overall field test, but was not rejected for the weekly sampling portion of the field test. The RPD evaluation indicated that the sensor measurements were within 15% of the laboratory 1C concentrations. The average RPD for Phase 1 met the <20% criteria discussed in Section 5.1 that indicates general agreement between sampling methods. In Phase 2 RPD evaluation, the 95% UCLs ranged from 27 to 77%, with one exception. The data from the field portion of the verification indicated extreme variations between paired sensor concentrations and laboratory analytical results. The accuracy readings indicate strong correlation between methods in Phase 1, but suggest that accuracy is reduced when elevated levels of interference parameters are introduced, and decreases with time during field deployment. With the exception of Well 3S, nitrate sensor data were capable of demonstrating relative changes (trending) in nitrate concentrations over nine continuous weeks based on time series plots. ------- Variability in nitrate sensor readings increases with increasing nitrate concentrations, ranging from 0.13 to 1.5 mg/L using Phase 1 mean values, and from 0.13 to 2.0 mg/L using Phase 1 target values. Nitrate sensor duplication of readings indicates that there is strong evidence for different mean levels (MAE ^ 0) from sensors both within (intra) and between (inter) wells. During Phase 1, the inter- and intra-well MAE values all were below 1 mg/L, but increase above 1 mg/L in a significant majority of the Phase 2 test. All of the p-values except one for one intra- well difference are significantly lower than 0.05, indicating the inter- and intra-well results are not equal. When evaluating the effect of changes in water quality, changes in MAE, standard deviation (computed both ways), and MSE for both intra-well and inter-well measurements exceed the threshold differences of 20% and 5%. The TempHion™ Smart Sensor was calibrated and installed in each well during the laboratory and field test by a representative from INW without problems. The calibration procedure was simple to perform, and was taught to field personnel from Battelle and the USDA ARS laboratory prior to deployment for the field test. Nitrate sensor data from each well were transmitted wirelessly to the vendor's server on 15-minute increments, and subsequently forwarded on to a web site for download for near real-time viewing and analysis with 100% data collection reported. The web site used for data download was easily accessible and data were provided in usable format with real-time graphics capabilities. Upon removal from the test wells, no biofouling was noted on any of the sensors. The vendor indicated that tailoring operational range of the sensors to anticipated field conditions could improve the accuracy of the sensors. The cost of a single TempHion™ nitrate sensor at the time of this verification test was $1,495. Remote groundwater well monitoring using TempHion™ nitrate sensors have potential cost and long-term sustainability advantages with fewer field visits and reduced sampling and laboratory analysis costs. When reviewing the results of the nitrate sensor performance analysis, consideration should be given to the objectives of the long-term monitoring effort and the threshold concentration for nitrate, particularly considering the percentage of error associated with sample collection and laboratory analysis. These errors, in addition to the magnitude of the threshold of the site- specific nitrate concentration and the cost and sustainability of the technique, should be taken into account when selecting a sampling approach for long-term monitoring. 32 ------- Chapter 8 References (1) Battelle. 2010. Test/QA Plan for Verification of Nitrate Sensors for Groundwater Remediation Monitoring for the ETV Advanced Monitoring Systems Center. Environmental Technology Verification Program, April. (2) EPA. 1997. Method 300.1: Determination of Inorganic Anions in Drinking Water by Ion Chromatography. (3) Battelle. 2008. Environmental Technology Verification Program Advanced Monitoring Systems Center Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Center, Version 7.0. December. 33 ------- Appendix 1 Well-Specific Time Series Nitrate Concentration Plots 34 ------- o 0 41 U O U 5/2/2010 Field Analysis: Well 1 2/22/2010 6/11/2010 7/1/2010 Date of Simultaneous Sampling -1C -Sensor 7/21/2010 0 5/2/2010 Field Analysis: Well 2 -1C •Sens 2/22/2010 6/11/2010 7/1/2010 Date of Simultaneous Sampling 7/21/2010 35 ------- 25 20 E a 315 n O Field Analysis: Well 3S -1C 'Sensoi 5/2/2010 2/22/2010 6/11/010 7/1/2010 Date of Simultaneous Sampling 7/21/2010 Field Analysis: Well 3D 0 5/2/2010 2/22/2010 6/11/010 7/1/2010 Date of Simultaneous Sampling 7/21/2010 36 ------- Field Analysis: Well 6 35 - 30 - a -S 25 - \l 0 «J 1 15 . c o § 10 - u 5 . 0 - ^""^* r^ I • /v •v^^^^y : *— (-, _ , *^ * * * * * -1C fr-Sensor 5/2/2010 2/22/2010 6/11/010 7/1/2010 7/21/2010 Date of Simultaneous Sampling 37 ------- Appendix 2 Accuracy Data Plots 38 ------- Phase 1 Accuracy Evaluation Plot Individual Value Plot of Sensor Nitrate (ppm) 14- 12- Q. 10- 0. ^ Q _ E £ 6- o 8 4- & 2- 0- XMK IMK w Sensor Target Nitrate (ppm) 39 ------- Phase 2 Accuracy Evaluation Plot - Target Nitrate Level of 1 mg/L E a. a. o M I Individual Value Plot of Sensor Nitrate (ppm) Target Nitrate Level = 1 ppm 10H 6- 4- 2- m Interference Level fV \ \ 40 ------- Phase 2 Accuracy Evaluation Plot - Target Nitrate Level of 3 mg/L E o. a. re Individual Value Plot of Sensor Nitrate (ppm) Target Nitrate Level = 3 ppm 14- 12- 10- 8- 6- 4- 2- 0- Sensor Interference Level ^> <\ 41 ------- Phase 2 Accuracy Evaluation Plot - Target Nitrate Level of 12 mg/L E a a ^> a> *> re o 10 I Individual Value Plot of Sensor Nitrate (ppm) Target Nitrate Level = 12 ppm JU- 25- 20- 15- 10- 5- 0- w M *^ f 3MMt ^^tf^^v^ «^fe ^99C uftift )Mi{^ «R MR J* * M ^^^ j^K ^^^ 12 Sensor Interference Level 42 ------- MAE During Field Sampling Sensor Minus Lab 10 15 5 10 15 5 Consecutive Sample Number 10 15 43 ------- |