August 2003 Environmental Technology Verification Report Monitoring Technologies International, Pty. Ltd. PDV6000 Portable analyzer Prepared by %s? Baffefie Putting Technology To Work Battel le Under a cooperative agreement with vvEPA U.S. Environmental Protection Agency ETV ElV ElV ------- August 2003 Environmental Technology Verification Report ETV Advanced Monitoring Systems Center Monitoring Technologies International, Pty. Ltd. PDV 6000 Portable Analyzer by Tim Kaufman Patty White Amy Dindal Zachary Willenberg Karen Riggs Battelle Columbus, Ohio 43201 ------- Notice The U.S. Environmental Protection Agency (EPA), through its Office of Research and Development, has financially supported and collaborated in the extramural program described here. This document has been peer reviewed by the Agency and recommended for public release. Mention of trade names or commercial products does not constitute endorsement or recommendation by the EPA for use. 11 ------- Foreword The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the nation's air, water, and land resources. Under a mandate of national environmental laws, the Agency strives to formulate and implement actions leading to a compatible balance between human activities and the ability of natural systems to support and nurture life. To meet this mandate, the EPA's Office of Research and Development provides data and science support that can be used to solve environmental problems and to build the scientific knowledge base needed to manage our ecological resources wisely, to understand how pollutants affect our health, and to prevent or reduce environmental risks. The Environmental Technology Verification (ETV) Program has been established by the EPA to verify the performance characteristics of innovative environmental technology across all media and to report this objective information to permitters, buyers, and users of the technology, thus substantially accelerating the entrance of new environmental technologies into the marketplace. Verification organizations oversee and report verification activities based on testing and quality assurance protocols developed with input from major stakeholders and customer groups associated with the technology area. ETV consists of seven environmental technology centers. Information about each of these centers can be found on the Internet at http://www.epa.gov/etv/. Effective verifications of monitoring technologies are needed to assess environmental quality and to supply cost and performance data to select the most appropriate technology for that assessment. In 1997, through a competitive cooperative agreement, Battelle was awarded EPA funding and support to plan, coordinate, and conduct such verification tests for "Advanced Monitoring Systems for Air, Water, and Soil" and report the results to the community at large. Information concerning this specific environmental technology area can be found on the Internet at http://www.epa.gov/etv/centers/centerl .html. 111 ------- Acknowledgments The authors wish to acknowledge the support of all those who helped plan and conduct the verification test, analyze the data, and prepare this report. In particular we would like to thank Rosanna Buhl, Adam Abbgy, and Bea Weaver of Battelle, and Mike Madigan and Rick Linde of the Ayer, Massachusetts, Department of Public Works Water Division. We also acknowledge the assistance of Jeff Adams of EPA and AMS Center stakeholders Vito Minei and Marty Link, who reviewed the verification reports. iv ------- Contents Page Notice ii Foreword iii Acknowledgments iv List of Abbreviations viii 1 Background 1 2 Technology Description 2 3 Test Design and Procedures 4 3.1 Introduction 4 3.2 Test Design 4 3.3 Test Samples 5 3.3.1 QC Samples 5 3.3.2 PT Samples 7 3.3.3 Environmental Samples 7 3.4 Reference Analysis 8 3.5 Verification Schedule 8 4 Quality Assurance/Quality Control 9 4.1 Laboratory QC for Reference Method 9 4.2 Audits 12 4.2.1 Performance Evaluation Audit 12 4.2.2 Technical Systems Audit 12 4.2.3 Data Quality Audit 13 4.3 QA/QC Reporting 13 4.4 Data Review 13 5 Statistical Methods 15 5.1 Accuracy 15 5.2 Precision 15 5.3 Linearity 16 5.4 Method Detection Limit 16 5.5 Matrix Interference Effects 16 5.6 Inter-Unit Reproducibility 16 5.7 Rate of False Positives/False Negatives 16 6 Test Results 18 6.1 QC Samples 18 6.2 PT and Environmental Samples 19 6.2.1 Accuracy 22 6.2.2 Precision 23 v ------- 6.2.3 Linearity 23 6.2.4 Method Detection Limit 23 6.2.5 Matrix Interference Effects 24 6.2.6 Inter-Unit Reproducibility 25 6.2.7 Rate of False Positives/False Negatives 25 6.3 Other Factors 28 6.3.1 Ease of Use 28 6.3.2 Analysis Time 28 6.3.3 Reliability 28 6.3.4 Waste Material 29 6.3.5 Cost 29 7 Performance Summary 30 8 References 32 Figures Figure 2-1. MTI Pty. Ltd., PDV 6000 Portable Analyzer 2 Figure 2-2. Example Voltammograms for 10 ppb and 50 ppb Arsenic Standards 3 Figure 6-1. Linearity of PDV 6000 Results 24 Figure 6-2. Comparison of PDV 6000 Test Results for Units #1 and #2 25 Tables Table 3-1. Test Samples for Verification of the PDV 6000 6 Table 3-2. Schedule of Verification Test Days 8 Table 4-1. Reference Method QCS Analysis Results 10 Table 4-2. Reference Method LFM Results 11 Table 4-3. Reference Method Duplicate Analysis Results 12 Table 4-4. Reference Method PE Audit Results 13 Table 4-5. Summary of Data Recording Process 14 Table 6-1. RB Sample Results for the PDV 6000 19 Table 6-2. LFM Sample Results for the PDV 6000 19 Table 6-3. PDV 6000 and Reference Sample Results 20 vi ------- Table 6-4. Accuracy Results for the PDV 6000 22 Table 6-5. Precision Results for the PDV 6000 23 Table 6-6. Detection Limit Results for the PDV 6000 24 Table 6-7. Rate of False Positives for PDV 6000 26 Table 6-8. Rate of False Negatives for PDV 6000 27 vii ------- List of Abbreviations AMS Advanced Monitoring Systems EPA U.S. Environmental Protection Agency ETV Environmental Technology Verification HDPE high-density polyethylene ICPMS inductively coupled plasma mass spectrometry LFM laboratory-fortified matrix MDL method detection limit MTI Monitoring Technologies International NIST National Institute of Standards and Technology ppb parts per billion ppm parts per million PE performance evaluation PT performance test QA quality assurance QA/QC quality assurance/quality control QC quality control QCS quality control standard QMP Quality Management Plan R correlation coefficient RB reagent blank RPD relative percent difference RSD relative standard deviation TSA technical systems audit viii ------- Chapter 1 Background The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology Verification (ETV) Program to facilitate the deployment of innovative environmental technologies through performance verification and dissemination of information. The goal of the ETV Program is to further environmental protection by accelerating the acceptance and use of improved and cost-effective technologies. ETV seeks to achieve this goal by providing high- quality, peer-reviewed data on technology performance to those involved in the design, distribution, financing, permitting, purchase, and use of environmental technologies. ETV works in partnership with recognized testing organizations; with stakeholder groups consisting of buyers, vendor organizations, and permitters; and with the full participation of individual technology developers. The program evaluates the performance of innovative tech- nologies by developing test plans that are responsive to the needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing peer- reviewed reports. All evaluations are conducted in accordance with rigorous quality assurance (QA) protocols to ensure that data of known and adequate quality are generated and that the results are defensible. The EPA's National Exposure Research Laboratory and its verification organization partner, Battelle, operate the Advanced Monitoring Systems (AMS) Center under ETV. The AMS Center recently evaluated the performance of the Monitoring Technologies International (MTI) Pty. Ltd., PDV 6000 portable analyzer for the measurement of heavy metal ions. The use of the PDV 6000 for the measurement of arsenic in water was evaluated in this test. 1 ------- Chapter 2 Technology Description The objective of the ETV AMS Center is to verify the performance characteristics of environmental monitoring technologies for air, water, and soil. This verification report provides results for the verification testing of the PDV 6000 portable analyzer for the measurement of heavy metal ions (Figure 2-1). The detection of arsenic in water was verified in this test. The following is a description of the analyzer, based on information provided by the vendor. The information provided below was not verified in this test. The PDV 6000 comprises a small analytical cell assembly and handheld controller used together as a portable tool for field screening for particular heavy metals. The PDV 6000 can be powered from a main power supply, a portable battery pack, or internal 9-volt batteries. When used in conjunction with VAS Version 2.1 software, a Windows application provided with the PDV 6000 that runs on a personal computer or laptop, the PDV 6000 is capable of metal ion analysis in the field as well as the laboratory. The performance of the PDV 6000 in conjunction with the VAS software was verified in this test. The principal of analysis used by the PDV 6000 is anodic stripping voltammetry (ASV). A reducing potential is applied to the working electrode. When the electrode potential exceeds the ionization potential of the analyte metal ion in solution, it is reduced to the metal which plates onto the working electrode surface. The longer the potential is applied, the more metal is reduced and plated onto the electrode surface (also known as the "deposi- tion" or "accumulation" step). When sufficient metal has been plated onto the working electrode, the metal is stripped (oxidized) off the electrode by increasing, at a constant rate, the potential applied to the working electrode. Figure 2-1. MTI Pty. Ltd., PDV 6000 For a given electrolyte solution and electrode, Portable Analyzer eac'1 metal has a specific potential at which the oxidation reaction will occur. The electrons released by this process form a current, which is measured and may be plotted as a function of applied potential to give a "voltammogram" (Figure 2-2). The current at the oxidation or stripping potential for the analyte metal is seen as a peak. To calculate the sample concentration, 2 ------- •50 mV OmV 50 rnV 100 rnV 150 mV 200 mV 250 mV 300 mV 350 mV 400 mV 450 mV Figure 2-2. Example Voltammograms for 10 ppb and 50 ppb Arsenic Standards the peak height or area is measured and compared to that of a known standard solution analyzed under the same conditions. The sample result is provided as a digital readout on the handheld controller, or if VAS software is being used, on the computer monitor screen. Sample results can be stored electronically using the VAS software. The vendor provides instructions for the analysis of water samples with concentrations ranging from five parts per billion (ppb) to 1,000 ppb. 3 ------- Chapter 3 Test Design and Procedures 3.1 Introduction This verification test was conducted according to procedures specified in the Test/QA Plan for Verification of Portable Analyzers,(1) The verification was based on comparing the arsenic results from the PDV6000 to those from a laboratory-based reference method. The reference method for arsenic analysis was inductively coupled plasma mass spectrometry (ICPMS) performed according to EPA Method 200.8.(2) The PDV 6000 performance was verified by analyzing laboratory-prepared performance test samples, treated and untreated drinking water samples, and fresh surface water samples. All samples were tested using both the PDV 6000 and the reference method. The test design and procedures are described further below. 3.2 Test Design The PDV 6000 was verified by evaluating the following parameters: ¦ Accuracy ¦ Precision ¦ Linearity ¦ Method detection limit (MDL) ¦ Matrix interference effects ¦ Inter-unit reproducibility ¦ Rate of false positives/false negatives. All sample preparation and analyses were performed according to the vendor's recommended procedures. Results for each sample were hand-recorded and most were also stored electronically on a laptop computer. The test/QA plan specified that all analyses would be performed by a technical operator and a non-technical operator to evaluate operator bias. However, the technical and non-technical operators were not able to successfully set up and operate the analyzer using the materials and instructions provided by the vendor. Consequently, all samples were analyzed by an MTI representative and operator bias was not evaluated. The results from the PDV 6000 were compared to those from the reference method to assess accuracy and linearity. Multiple aliquots of performance test samples, drinking water samples, and surface water samples were analyzed to assess precision. Multiple aliquots of a low-level performance test sample were analyzed to assess the detection limit of the PDV 6000. Potential matrix interference effects were assessed by challenging the PDV 6000 with performance test 4 ------- samples of known arsenic concentrations that contained both low levels and high levels of inter- fering substances. All samples were analyzed using two identical PDV 6000 units (designated unit #1 and unit #2). Results of analyses from the two units were statistically compared to evaluate inter-unit reproducibility. The rates of false positive and false negative results were evaluated relative to the 10-ppb maximum contaminant level for arsenic in drinking water.(3) Other factors that were quali- tatively assessed during the test included ease of use, time required for sample analysis, and reliability. 3.3 Test Samples Three types of samples were analyzed in the verification test, as shown in Table 3-1: quality control (QC) samples, performance test (PT) samples, and environmental water samples. The QC and PT samples were prepared from National Institute of Standards and Technology (NIST) traceable standards purchased from a commercial supplier and subject only to dilution as appro- priate. Under the Safe Drinking Water Act, the EPA lowered the maximum contaminant level for arsenic from 50 ppb to 10 ppb in January 2001; public water supply systems must comply with this standard by January 2006.(3) Therefore, the QC sample concentrations targeted the 10 ppb arsenic level. The PT samples ranged from 10% to 1,000% of the 10 ppb level (i.e., from 1 ppb to 100 ppb). The environmental water samples were collected from various drinking water and surface freshwater sources. Each sample was assigned a unique sample identification number when prepared in the laboratory or collected in the field. The PT and environmental samples were submitted blind to the operator and were analyzed randomly to the degree possible. 3.3.1 QC Samples QC samples included laboratory reagent blanks (RB), quality control samples (QCS), and laboratory-fortified matrix (LFM) samples (Table 3-1). The RB samples consisted of the same ASTM Type I water used to prepare all other samples and were subjected to the same handling and analysis procedures as the other samples. The RB samples were used to verify that no arsenic contamination was introduced during sample handling and analysis. RB samples were analyzed at a frequency of 10%. The QCS consisted of standards analyzed initially to calibrate the PDV 6000, then after every fifth sample and at the end of the analysis run to verify the calibration. The QCS, which were referred to as standards in the vendor's operation manual, were prepared and analyzed according to the vendor's instructions and consisted of PDV 6000 electrolyte solution spiked to concentrations of 10 ppb and 50 ppb arsenic with a NIST-traceable standard. 5 ------- Table 3-1. Test Samples for Verification of the PDV 6000 Type of Sample Sample Characteristics Arsenic Concentration (a) No. of Replicates Quality Control Reagent Blank (RB) ~ 0 ppb 10% of all Quality Control Sample (QCS) 10 ppb Beginning, end, and every 5th sample Laboratory Fortified Mixture (LFM) 10 ppb above native level 1 per site Performance Prepared arsenic solution 1 ppb 4 Test Prepared arsenic solution 3 ppb 4 Prepared arsenic solution 10 ppb 4 Prepared arsenic solution 30 ppb 4 Prepared arsenic solution 100 ppb 4 Prepared arsenic solution for detection limit determination 25 ppb 7 Prepared arsenic solution spiked with low levels of interfering substances 10 ppb 4 Prepared arsenic solution spiked spiked with high levels of interfering substances 10 ppb 4 Environmental Battelle drinking water <0.5 ppb 4 Ayer untreated water 8.08 ppb 4 Ayer treated water 0.98 ppb 4 Falmouth Pond water <0.5 ppb 4 Taunton River water 1.31 ppb 4 (a) Target concentration for Quality Control and Performance Test samples; measured concentration for environmental samples (average of four replicate measurements). The LFM samples consisted of aliquots of environmental samples that were spiked in the field to increase the arsenic concentration by 10 ppb. The spike solution used for the LFM samples was prepared in the laboratory and brought to the field site. One LFM sample was prepared from each environmental sample. 6 ------- 3.3.2 PT Samples Three types of PT samples used in this verification test (Table 3-1): spiked samples ranging from 1 ppb to 100 ppb arsenic, a low-level spiked sample for evaluation of the PDV 6000's detection limit, and matrix interference samples that were spiked with potential interfering substances. All PT samples were prepared in the laboratory using ASTM Type I water and NIST-traceable standards. Five PT samples containing arsenic at concentrations from 1 ppb to 100 ppb were prepared to evaluate PDV 6000 accuracy and linearity. Four aliquots of each of these samples were analyzed to assess precision. To determine the detection limit of the PDV 6000, a PT sample was prepared with an arsenic concentration approximately five times the vendor-stated detection limit (i.e., 5 ppb x 5 = 25 ppb). Seven non-consecutive replicates of this 25 ppb arsenic sample were analyzed to provide precision data with which to estimate the method detection limit (MDL). The matrix interference samples were spiked with 10 ppb arsenic as well as potentially inter- fering species commonly found in natural water samples. One sample contained low levels of interfering substances that consisted of 1 part per million (ppm) iron and 0.1 ppm sulfide. The second sample contained high levels of interfering compounds at concentrations of 10 ppm iron and 1.0 ppm sulfide. Four replicates of each of these samples were analyzed. Although the test/QA plan specified the addition of sodium chloride to these samples, this compound was not added to the samples because the PDV 6000 electrolyte solution was more saline than the target sodium chloride sample concentration given in the test/QA plan. 3.3.3 Environmental Samples The environmental samples listed in Table 3-1 included three drinking water samples and two surface water samples. All environmental samples were collected in 20-L high density poly- ethylene (HDPE) carboys. The Battelle drinking water sample was collected directly from a tap without purging. Untreated and treated groundwater samples from the Ayer, Massachusetts Department of Public Works Water Treatment Plant were collected directly from spigots, also without purging. Four aliquots of each sample were analyzed using the PDV 6000 in the Battelle laboratory as soon as possible after collection. One aliquot of each sample was preserved with nitric acid and submitted to the reference laboratory for reference analysis. One surface water sample was collected from a pond in Falmouth, Massachusetts and another was collected from the Taunton River near Bridgewater, Massachusetts. These samples were collected near the shoreline by submerging a 2-L HDPE sample container no more than one inch below the surface of the water, and decanting the water into a 20-L HDPE carboy until full. Each water body was sampled at one accessible location. These samples could not be analyzed at the field location as planned because of persistent, severe winter weather conditions. Therefore, the samples were returned to a storage shed at the Battelle laboratory, which was heated but not serviced by running water. The storage shed was intended to simulate realistic field conditions under which the PDV 6000 might be used. Four aliquots of each surface water sample were 7 ------- analyzed in the storage shed as soon as possible after collection. One aliquot of each sample was preserved with nitric acid and submitted to the reference laboratory for reference analysis. 3.4 Reference Analysis The reference arsenic analyses were performed in a Battelle laboratory using a Perkin Elmer Sciex Elan 6000 ICPMS according to EPA Method 200.8, Revision 5.5.(2) The sample was introduced through a peristaltic pump by pneumatic nebulization into a radiofrequency plasma where energy transfer processes caused desolvation, atomization, and ionization. The ions were extracted from the plasma through a pumped vacuum interface and separated on the basis of their mass-to-charge ratio by a quadrupole mass spectrometer. The ions transmitted through the quadrupole were registered by a continuous dynode electron multiplier, and the ion information was processed by a data handling system. The ICPMS was tuned, optimized, and calibrated daily. The calibration was performed using a minimum of five calibration standards at concentrations ranging between 0.5 and 250 ppb, and a required correlation coefficient of a minimum of 0.999. Internal standards were used to correct for instrument drift and physical interferences. These standards were introduced in line through the peristaltic pump and analyzed with all blanks, standards, and samples. 3.5 Verification Schedule The verification test took place from February 20 through February 25, 2003. Table 3-2 shows the daily activities that were conducted during this period. The reference analyses were per- formed on March 7 and March 13-14, 2003, approximately one to two weeks after sample collection. Table 3-2. Schedule of Verification Test Days Sample Collection Date Sample Analysis Date Testing Location Activity 2/21/03- 2/25/03 2/21/03- 2/25/03 Battelle Laboratory and Storage Shed Preparation and analysis of PT and associated QC samples 2/12/03 2/20/033 Battelle Laboratory Collection and analysis of Ayer untreated and treated water and associated QC samples 2/20/03 2/20/03 Battelle Laboratory Collection and analysis of Battelle drinking water and associated QC samples 2/21/03 2/21/03 Battelle Storage Shed Collection and analysis of Falmouth Pond water and associated QC samples 2/23/03 2/24/03 Battelle Storage Shed Collection and analysis of Taunton River water and associated QC samples Subsamples for reference method analysis were collected on 2/20/03. 8 ------- Chapter 4 Quality Assurance/Quality Control Quality assurance/quality control (QA/QC) procedures were performed in accordance with the quality management plan (QMP) for the AMS Center(3) and the test/QA plan for this verification test.(1) QA/QC procedures and results are described below. 4.1 Laboratory QC for Reference Method Reference analyses were conducted on March 7 and March 13-14, 2003. Laboratory QC for the reference method included the analysis of RB, QCS, LFM, and analytical duplicate samples. Laboratory RB samples were analyzed to ensure that no contamination was introduced by the sample preparation and analysis process. The test/QA plan stated that if arsenic was detected in a RB sample above the MDL for the reference instrument, then the contamination source would be identified and removed and proper blank readings achieved before proceeding with the reference analyses. All of the laboratory RB samples analyzed were below the reporting limit for arsenic (i.e., below the concentration of the lowest calibration standard) except for several blanks that were analyzed at the end of the day on March 7. The two test samples that were associated with these RB samples were re-analyzed on March 14, with acceptable blank results. On March 7 and 13, the instrument used for the reference method was calibrated using nine calibration standards, with concentrations ranging from 0.5 to 250 ppb arsenic. On March 14, it was calibrated using eight standards ranging in concentration from 0.1 to 25 ppb arsenic for more accurate analysis of low level samples. The accuracy of the calibration was verified after the analysis of every 10 samples by analyzing a QCS of a known concentration. The percent recovery of the QCS was calculated from the following equation: R = — xlOO ^ s where Cs is the measured concentration of the QCS and 5 is the spike concentration. If the QCS analysis differed by more than 10% from the true value of the standard, the instrument was recalibrated before continuing the test. As shown in Table 4-1, all QCS analyses were within the required range. 9 ------- Table 4-1. Reference Method QCS Analysis Results Sample ID Analysis Date Measured (ppb) Actual (ppb) Percent Recovery CCV25 3/7/2003 24.96 25.00 100% QCS 25 3/7/2003 26.81 25.00 107% CCV25 3/7/2003 24.50 25.00 98% CCV25 3/7/2003 25.39 25.00 102% CCV25 3/7/2003 25.73 25.00 103% CCV25 3/7/2003 25.81 25.00 103% CCV25 3/7/2003 25.64 25.00 103% CCV25 3/7/2003 25.30 25.00 101% CCV25 3/7/2003 24.90 25.00 100% CCV25 3/7/2003 22.67 25.00 91% QCS 25 3/13/2003 27.06 25.00 108% CCV25 3/13/2003 25.07 25.00 100% CCV25 3/13/2003 24.15 25.00 97% CCV25 3/13/2003 25.79 25.00 103% CCV25 3/13/2003 24.89 25.00 100% CCV25 3/13/2003 24.34 25.00 97% QCS 25 3/14/2003 24.90 25.00 100% CCV2.5 3/14/2003 2.74 2.50 110% QCS 2.5 3/14/2003 2.70 2.50 108% CCV2.5 3/14/2003 2.58 2.50 103% CCV2.5 3/14/2003 2.65 2.50 106% CCV2.5 3/14/2003 2.66 2.50 106% CCV2.5 3/14/2003 2.61 2.50 104% CCV2.5 3/14/2003 2.60 2.50 104% LFM samples were analyzed to assess whether matrix effects influenced the reference method results. The LFM percent recovery (R) was calculated from the following equation: R = Cs ~C xlOO (2) where Cs is the measured concentration of the spiked sample, C is the measured concentration of the unspiked sample, and 5 is the spike concentration. If the percent recovery of an LFM fell outside the range from 85% to 115%, a matrix effect was suspected. As shown in Table 4-2, all of the LFM sample results were within this range. 10 ------- Table 4-2. Reference Method LFM Results Amount Unspiked Spiked Spiked Percent Sample ID Matrix Analysis Date (ppb) (ppb) (ppb) Recovery CAA-22 ASTM Type I water 3/7/2003 11.02 37.20 25.00 105% CAA-25 R4 ASTM Type I water 3/7/2003 0.95 22.76 25.00 87% CAA-28 R2 ASTM Type I water 3/7/2003 3.45 30.64 25.00 109% CAA-29 R4 ASTM Type I water 3/7/2003 34.98 60.37 25.00 102% CAA-37 R4 CAA-41 R4 CAA-48 CAA-47 R4 Drinking water Drinking water Surface water Surface water 3/7/2003 3/7/2003 3/7/2003 3/7/2003 0.52 1.24 12.26 1.07 28.20 28.88 39.40 28.41 25.00 25.00 25.00 25.00 111% 111% 109% 109% CAA-95 R1 ASTM Type I water 3/13/2003 11.34 38.46 25.00 108% CAA-32 R3 ASTM Type I water 3/13/2003 103.70 128.05 25.00 97% CAA-90 R2 CAA-96 CAA-27 R1 Drinking water Surface water ASTM Type I water 3/13/2003 3/13/2003 3/14/2003 8.06 18.86 2.56 32.88 43.21 4.73 25.00 25.00 2.50 99% 97% 87% CAA-37 R3 CAA-47 R1 CAA-88 R3 CAA-88 R4 Drinking water Surface water Drinking water Drinking water 3/14/2003 3/14/2003 3/14/2003 3/14/2003 0.45 1.36 0.43 0.42 3.11 4.16 3.16 3.18 2.50 2.50 2.50 2.50 107% 112% 109% 111% Duplicate samples were analyzed to assess the precision of the reference analysis. The relative percent difference (RPD) of the duplicate sample analysis was calculated from the following equation: RPD= ^C~Cd) xl0o (3) (C + CD)/2 where C is the concentration of the sample analysis, and Co is the concentration of the duplicate sample analysis. If the RPD was greater than 10%, the instrument was recalibrated before continuing the test. As shown in Table 4-3, the RPDs for the duplicate analyses were all less than 10%. The RPD for one duplicate pair was 9.5%; however, the reported concentrations were below the reporting limit for the reference method (i.e., below the concentration of the lowest calibration standard). 11 ------- 4.2 Audits Three types of audits were performed during the verification test: a performance evaluation (PE) audit of the reference method, a technical systems audit of the verification test performance, and a data quality audit. Audit procedures are described further below. Table 4-3. Reference Method Duplicate Analysis Results Sample ID Analysis Date Sample Concentration (PPb) Duplicate Concentration (ppb) Relative Percent Difference CAA-4 3/7/2003 9.33 9.20 1.4% CAA-70 3/7/2003 10.93 10.82 1.0% CAA-26 R1 3/7/2003 1.14 1.13 1.4% CAA-28 R3 3/7/2003 3.49 3.45 1.1% CAA-31 R1 3/7/2003 111.89 112.20 0.3% CAA-38 3/7/2003 11.96 11.90 0.5% CAA-42 3/7/2003 13.02 13.06 0.3% CAA-4 8 3/7/2003 12.26 12.22 0.4% CAA-79 3/13/2003 5455 5342 2% CAA-95 R2 3/13/2003 10.64 10.61 0.3% CAA-32 R4 3/13/2003 102.87 101.06 2% CAA-90 R3 3/13/2003 8.15 8.16 0.2% CAA-23 3/14/2003 3.03 2.99 1.3% CAA-27 R2 3/14/2003 2.64 2.61 0.9% CAA-37 R4 3/14/2003 0.44 0.43 2.3% CAA-47 R2 3/14/2003 1.31 1.32 0.2% CAA-88 R4 3/14/2003 0.42 0.38 9.5% 4.2.1 Performance Evaluation Audit A PE audit was conducted to assess the quality of the reference measurements made in this verification test. For the PE audit, an independent, NIST-traceable, reference material was obtained from a different commercial supplier than the calibration standards and the standard used to prepare the PT and field QCS samples. Accuracy of the reference method was verified by comparing the arsenic concentration measured using the calibration standards to those obtained using the independently-certified PE standard. Relative percent difference as calculated by Equation 3 was used to quantify the accuracy of the results. Agreement of the standard within 10% was required for the measurements to be considered acceptable. As shown in Table 4-4, the PE sample analysis was within the required range. 12 ------- Table 4-4. Reference Method PE Audit Results Measured Arsenic Actual Arsenic Date of Concentration Concentration Percent Sample ID Analysis (ppb) (PPb) Difference PE-1 3/24/03 9.63 10.0 4 4.2.2 Technical Systems Audit An independent Battelle Quality staff conducted a technical systems audit (TSA) on February 24 to ensure that the verification test was being conducted in accordance with the test/QA plan(1) and the AMS Center QMP.(3) A TSA of the reference method performance was conducted by the Battelle Quality Manager on March 5, 2003, when the reference analyses were initiated. As part of the TSA, test procedures were compared to those specified in the test/QA plan, data acquisition and handling procedures were reviewed, and the reference standards and method were reviewed. Observations and findings from the TSA were documented and submitted to the Battelle Verification Test Coordinator for response. None of the findings of the TSA required corrective action. TSA records are permanently stored with the Battelle Quality Manager. 4.2.3 Data Quality Audit At least 10% of the data acquired during the verification test were audited. Battelle's Quality Manager traced the data from the initial acquisition, through reduction and statistical analysis, to final reporting to ensure the integrity of the reported results. All calculations performed on the data undergoing the audit were checked. 4.3 QA/QC Reporting Each audit was documented in accordance with Sections 3.3.4 and 3.3.5 of the QMP for the ETV AMS Center.(4) Once the audit reports were prepared, the Battelle Verification Test Coordinator ensured that a response was provided for each adverse finding or potential problem and imple- mented any necessary follow-up corrective action. The Battelle Quality Manager ensured that follow-up corrective action was taken. The results of the TSA and the data quality audit were submitted to the EPA. 4.4 Data Review Records generated in the verification test received a one-over-one review before these records were used to calculate, evaluate, or report verification results. Table 4-5 summarizes the types of data recorded and reviewed. All data were recorded by an MTI representative. Data were reviewed by a Battelle technical staff member involved in the verification test. The person performing the review added his/her initials and the date to a hard copy of the record being reviewed. 13 ------- Table 4-5. Summary of Data Recording Process Data to be Recorded Where Recorded How Often Recorded Disposition of Data(a) Dates, times of test events ETV field data sheets Start/end of test event Used to organize/check test results; manually incorporated in data spreadsheets as necessary Test parameters (temperature, analyte/ interferant identities, and all PDV 6000 portable analyzer results ^ ETV field data sheets When set or changed, or as needed to document test Used to organize/check test results, manually incorporated in data spreadsheets as necessary Reference method sample analysis, chain of custody, and results Laboratory record books, data sheets, or data acquisition system, as appropriate Throughout sample handling and analysis process Transferred to spreadsheets (a) All activities subsequent to data recording were carried out by Battelle. (b) Most of the PDV 6000 results were also recorded electronically. 14 ------- Chapter 5 Statistical Methods The statistical methods used to evaluate the performance factors listed in Section 3.2 are presented in this chapter. Qualitative observations were also used to evaluate verification test data. 5.1 Accuracy All samples were analyzed by both the PDV 6000 and reference methods. For each sample, accuracy was expressed in terms of a relative bias (B) as calculated from the following equation: where d is the average difference between the reading from the PDV 6000 and those from the reference method, and CR is the average of the reference measurements. 5.2 Precision When possible, the standard deviation (S) of the results for the replicate samples at each concentration was calculated and used as a measure of PDV 6000 precision. Standard deviation was calculated from the following equation: and C is the average concentration of the replicate samples. Precision was reported in terms of the relative standard deviation (RSD) as follows: B = =x 100 (4) (6) where n is the number of replicate samples, Ck is the concentration measured for the kth sample, RSD = = x 100 C (7) 15 ------- 5.3 Linearity Linearity was assessed by performing a linear regression of PDV 6000 results against the reference results, with linearity characterized by the slope, intercept, and correlation coefficient (R). Linearity was tested using the five PT samples over the range 1 ppb to 100 ppb arsenic. Samples with results below the vendor-stated PDV 6000 detection limit were not included in the analysis. Results from both PDV 6000 units were plotted against the corresponding reference concentrations and separate regressions were performed. 5.4 Method Detection Limit The MDL for the PDV 6000 was assessed using results from both units for seven replicate analyses of a sample spiked with 25 ppb arsenic. The standard deviation of the seven replicate samples was calculated using Equation 6. The MDL was calculated using the following equation: MDL = txS (8) where t is the Student's t value for a 99% confidence level and S is the standard deviation of the seven replicate samples. 5.5 Matrix Interference Effects The potential effect of interfering substances on the sensitivity of the PDV 6000 was evaluated by calculating accuracy (expressed as bias) using Equation 4. These results were qualitatively compared with accuracy results for PT samples containing only arsenic to assess whether there was a positive or negative effect due to matrix interferences. 5.6 Inter-Unit Reproducibility Inter-unit reproducibility for the two PDV 6000 units was assessed by performing a linear regression of sample results generated by the two units. The slope, intercept, and correlation coefficient were used to evaluate the degree of inter-unit reproducibility. A paired t-test was also conducted to evaluate whether the two sets of sample results were significantly different at a 95% confidence level. 5.7 Rate of False Positives/False Negatives The rates of false positives and false negatives produced by the PDV 6000 were assessed relative to the 10-ppb target arsenic level. A false positive result is defined as any result reported to be greater than the guidance level (10 ppb) and greater than 125% of the reference value, when the reference value is less than or equal to that guidance level. Similarly, a false negative result is defined as any result reported below or equal to the guidance level and less than 75% of the 16 ------- reference value, when the reference value is greater than that guidance level. The rates of false positives and false negatives were expressed as a percentage of total samples analyzed for each type of sample. 17 ------- Chapter 6 Test Results The results of the verification test of the PDV 6000 portable analyzer are presented in this section. 6.1 QC Samples As described in Section 3.3.1, the QC samples analyzed with the PDV 6000 portable analyzer included RB, QCS, and LFM samples. The RB samples were analyzed at a frequency of 10% and results were used to verify that no arsenic contamination was introduced during sample handling and analysis. RB sample results for the PDV 6000 are presented in Table 6-1. Unique sample identification codes were assigned to each container of ASTM Type I water that was used. The RB samples were analyzed at the required frequency. All RB samples were reported as below the portable analyzer's detection limit. QCS, which were referred to as standards in the vendor's operation manual, were analyzed at the beginning and end of each test period, and after every fifth sample as required. The Application Note for the arsenic in water analysis, provided with the PDV 6000, specified the acceptance criteria and corrective action for the standards. If a standard peak height dropped more than 30% from the original standard peak height, then a new standard was prepared and analyzed. If the new standard was lower than the original standard, then the working electrode was re-plated with a gold film. All QCS (standard) samples were within the acceptance criteria except for two on the last day of testing. After the first set of low standards, the working electrodes on both units were re-plated and the standards were re-run. However, after analysis of the next set of test samples, the standard peaks had dropped again. The MTI representative concluded that the standard peak drops were probably due to interference from organic material or sulfide in the samples (see Section 6.2.5). One LFM sample was prepared from each environmental sample to evaluate potential matrix interferences. The LFM sample results for the PDV 6000 are presented in Table 6-2. The percent recovery associated with each LFM sample was calculated using Equation 2 (Section 4.1). The average percent recoveries ranged from 0% for the Ayer treated water LFM sample to 153% for the Taunton River water LFM sample. Apparent matrix effects can be seen in the results for the Battelle drinking water LFM sample and Ayer treated water LFM sample, with average recoveries of 36% and 0%. These matrices appear to be affecting the recovery of arsenic and the portable analyzer results for these samples may be negatively biased. The high recoveries for the Taunton River water LFM sample may be due to a spiking error; the reference method result for this sample was 18.9 ppb (Table 6-3). 18 ------- Table 6-1. RB Sample Results for the PDV 6000 Arsenic Arsenic Sample ID Replicate Analysis Date Unit #1 (ppb) Unit #2 (ppb) CAA-60 1 2/20/2003 <5 <5 CAA-60 2 2/20/2003 <5 <5 CAA-58 1 2/21/2003 <5 <5 CAA-58 2 2/21/2003 <5 <5 CAA-58 1 2/24/2003 <5 <5 CAA-58 2 2/24/2003 <5 <5 CAA-59 1 2/25/2003 <5 <5 CAA-59 2 2/25/03 <5 <5 Table 6-2. LFM Sample Results for the PDV 6000 Amount Analysis Unspiked (a) Spiked Spiked Percent Description Date (ppb) (ppb) (ppb) Recovery Battelle drinking water LFM Unit # 1 2/20/2003 <5 2.7 10 27% Unit #2 2/20/2003 <5 4.6 10 46% Ayer untreated water LFM Unit # 1 2/20/2003 <5 7.8 10 78% Unit #2 2/20/2003 <5 9.0 10 90% Ayer treated water LFM Unit # 1 2/20/2003 <5 <5 10 0% Unit #2 2/20/2003 <5 <5 10 0% Falmouth Pond LFM Unit # 1 2/21/2003 <5 12.0 10 120% Unit #2 2/21/2003 <5 9.4 10 94% Taunton River LFM Unit # 1 2/24/2003 <5 15.3 10 153% Unit #2 2/24/2003 <5 15.4 10 154% Tal Non-detected results considered zero in the percent recovery calculation. 6.2 PT and Environmental Samples Table 6-3 presents the results for the PT and environmental samples. The table includes the PDV 6000 results for both units and the reference method results. One replicate of the PT sample containing low levels of interfering substances was inadvertently omitted by the analyst. Addi- tionally, the result for one replicate of the PT sample containing high levels of interfering sub- stances was not hand-recorded, and the electronic record of the result was lost when the software failed to operate. 19 ------- Table 6-3. PDV 6000 and Reference Sample Results Unit #1 Unit #2 Reference Description Sample ID Replicate Analysis Date (ppb) (ppb) (ppb) PT - 1 ppb As CAA-81 1 2/24/2003 <5 <5 1.13 CAA-81 2 2/24/2003 <5 <5 1.11 CAA-81 3 2/24/2003 <5 <5 1.13 CAA-81 4 2/24/2003 <5 <5 1.14 PT - 3 ppb As CAA-94 1 2/21/2003 3.1 4.8 3.20 CAA-94 2 2/21/2003 3.5 4.4 3.19 CAA-94 3 2/21/2003 2.5 3.7 3.12 CAA-94 4 2/21/2003 3.0 3.7 3.12 PT - 10 ppb As CAA-95 1 2/21/2003 9.1 12.5 11.3 CAA-95 2 2/21/2003 11.1 12.1 10.6 CAA-95 3 2/21/2003 8.9 11.0 10.8 CAA-95 4 2/21/2003 9.1 8.8 10.7 PT - 30 ppb As CAA-30 1 2/24/2003 32.0 33.0 36.1 CAA-30 2 2/24/2003 31.1 32.9 36.5 CAA-30 3 2/24/2003 33.1 34.5 35.9 CAA-30 4 2/24/2003 22.9 24.7 35.9 PT - 100 ppb CAA-32 1 2/24/2003 78.4 93.9 110.1 As CAA-32 2 2/24/2003 78.2 94.8 105.4 CAA-32 3 2/24/2003 86.9 100.9 103.7 CAA-32 4 2/24/2003 85.5 98.7 102.9 Detection CAA-80 1 2/24/2003 27.5 24.2 Limit CAA-80 2 2/24/2003 21.8 21.1 CAA-80 3 2/24/2003 30.4 26.4 CAA-80 4 2/24/2003 25.0 26.2 27.3 CAA-80 5 2/24/2003 25.0 24.8 CAA-80 6 2/24/2003 24.6 26.0 CAA-80 7 2/24/2003 24.2 25.4 PT - 10 ppb As CAA-34 1 2/25/2003 4.8 6.2 + CAA-34 2 2/25/2003 6.4 6.2 low level CAA-34 3 2/25/2003 6.8 6.9 10.9 interferents not not CAA-34 4 2/25/2003 analyzed analyzed PT - 10 ppb As CAA-36 1 2/25/2003 4.4 4.8 + CAA-36 2 2/25/2003 6.4 Data lost 10.9 high level CAA-36 3 2/25/2003 5.7 6.4 interferents CAA-36 4 2/25/2003 6.8 6.9 Italicized values were measured below the vendor-stated detection limit of the PDV 6000. 20 ------- Table 6-3. PDV 6000 and Reference Sample Results (continued) Sample Analysis Unit #1 Unit #2 Reference Description ID Replicate Date (ppb) (ppb) (ppb) Battelle drinking CAA-88 1 2/20/2003 <5 <5 <0.5 water CAA-88 2 2/20/2003 <5 <5 <0.5 CAA-88 3 2/20/2003 <5 <5 <0.5 CAA-88 4 2/20/2003 <5 <5 <0.5 Battelle drinking water LFM CAA-89 1 2/20/2003 2.7 4.6 10.2 Ayer untreated CAA-90 1 2/20/2003 <5 <5 8.43 water CAA-90 2 2/20/2003 <5 <5 8.06 CAA-90 3 2/20/2003 <5 <5 8.15 CAA-90 4 2/20/2003 <5 <5 7.68 Ayer untreated water LFM CAA-91 1 2/20/2003 7.8 9.0 18.9 Ayer treated water CAA-92 1 2/20/2003 <5 <5 0.95 CAA-92 2 2/20/2003 <5 <5 0.99 CAA-92 3 2/20/2003 <5 <5 1.03 CAA-92 4 2/20/2003 <5 <5 0.95 Ayer treated water LFM CAA-93 1 2/20/2003 <5 <5 12.2 Falmouth Pond CAA-43 1 2/21/2003 <5 <5 <0.5 water CAA-43 2 2/21/2003 <5 <5 <0.5 CAA-43 3 2/21/2003 <5 <5 <0.5 CAA-43 4 2/21/2003 <5 <5 <0.5 Falmouth Pond water LFM CAA-46 1 2/21/2003 12.0 9.4 11.5 Taunton River CAA-47 1 2/24/2003 <5 <5 1.36 water CAA-47 2 2/24/2003 <5 <5 1.31 CAA-47 3 2/24/2003 <5 <5 1.31 CAA-47 4 2/24/2003 <5 <5 1.26 Taunton River water LFM CAA-96 1 2/24/2003 15.3 15.4 18.9 Italicized values were measured below the detection limit of the PDV 6000. For the PDV 6000, samples with no arsenic peak were assigned a value of <5 ppb, which is the vendor-stated detection limit for the analyzer. Several samples had arsenic peaks that were quantified below 5 ppb; these values are reported. The reporting limit for the reference analyses was 0.5 ppb, which corresponds to the lowest calibration standard used. Results for each performance factor are presented below. 21 ------- 6.2.1 Accuracy Table 6-4 presents the accuracy results for the PDV 6000 portable analyzer, expressed as percent bias as calculated by Equation 4 (Section 5.1). Percent bias was not calculated for results below the detection limit (i.e., <5 ppb). The four replicate analyses for each sample were averaged in the calculation of bias. The bias ranged from -74% for the Battelle drinking water LFM sample (unit #1) to 31% for the 3 ppb arsenic PT sample (unit #2). Almost all biases were less than 25% except for the high- and low-level interferent samples, the Battelle drinking water LFM sample, and the Ayer untreated water LFM sample. As noted in Section 6.1, the LFM sample results suggest that the Battelle drinking water and Ayer treated water samples may be matrices that adversely affect the detection of arsenic. The vendor representative stated that the matrix effect was most likely due to copper in the pipes for the drinking water supplies, and generally can be prevented by purging the pipes prior to sampling. Both the PDV 6000 and reference method results for the Battelle drinking water sample were below detection. The PDV 6000 results for the Ayer treated water sample were also below detection (< 5 ppb), which is consistent with the average reference method concentration of 1 ppb. Table 6-4. Accuracy Results for the PDV 6000 Percent Bias Description Unit #1 Unit #2 Performance Test Samples 1 ppb As NA NA 3 ppb As (a) -4% 31% 10 ppb As -12% 2% 30 ppb As -17% -13% 100 ppb As -22% -8% 10 ppb As + low level interferents -45% -51% 10 ppb As + high level interferents -46% -44% Environmental Samples Battelle drinking water NA NA Battelle drinking water LFM -74% -55% Ayer untreated water NA NA Ayer untreated water LFM -58% -52% Ayer treated water NA NA Ayer treated water LFM NA NA Falmouth Pond water NA NA Falmouth Pond water LFM 4% -18% Taunton River water NA NA Taunton River water LFM -19% -18% NA: Below Detection Limit (a) PDV 6000 results for this sample were measured below the detection limit. 22 ------- 6.2.2 Precision Precision results for the PDV 6000 portable analyzer are presented in Table 6-5. The RSD was determined according to Equation 7 (Section 5.2). The RSD was not calculated if any of the results for a set of replicates were below the detection limit. The RSDs ranged from 6% to 16% for unit #1, and from 3% to 15% for unit #2. Table 6-5. Precision Results for the PDV 6000 RSD Description Unit #1 Unit #2 Performance Test Samples 1 ppb As NA NA 3 ppb As 14% 13% 10 ppb As 11% 15% 30 ppb As 16% 14% 100 ppb As 6% 3% Environmental Samples Battelle drinking water NA NA Ayer untreated water NA NA Ayer treated water NA NA Falmouth Pond water NA NA Taunton River water NA NA NA indicates a measurement below detection limit. 6.2.3 Linearity The linearity of the PDV 6000 measurements was assessed by performing a linear regression of the PDV 6000 results against the reference method results for the five PT samples ranging from 1 ppb to 100 ppb arsenic. In these regressions, results reported as below the detection limit by the PDV 6000 (i.e., <5 ppb arsenic) were not used. Figure 6-1 presents the results of the linear regression for the two PDV 6000 units. The slope, intercept, and correlation coefficient for each regression equation are shown on the charts. The plots indicate that unit #2 shows a closer correspondence to reference measurements than unit #1, and that PDV 6000 results were generally lower than reference method results. 6.2.4 Method Detection Limit The MDL was assessed by analyzing seven replicates of a sample spiked at approximately five times the vendor-stated detection limit for the PDV 6000 portable analyzer (i.e., 5 ppb X 5 = 25 ppb arsenic). Table 6-6 provides the standard deviations for the seven replicate samples for the PDV 6000 results, and the calculated MDLs. 23 ------- ~ PDV6000 Unit #1 ¦ PDV6000 Unit #2 - - Linear (PDV6000 Unit #1) - - Linear (PDV6000 Unit #2) y = 0.77X+ 1.22 R = 0.9934 0 20 40 60 80 100 120 Reference concentration (ppb) Figure 6-1. Linearity of PDV 6000 Results Table 6-6. Detection Limit Results for the PDV 6000 Sample ID Replicate Analysis Date Unit #1 (ppb) Unit#2 (ppb) CAA-80 1 2/24/2003 27 24 CAA-80 2 2/24/2003 22 21 CAA-80 3 2/24/2003 30 26 CAA-80 4 2/24/2003 25 26 CAA-80 5 2/24/2003 25 25 CAA-80 6 2/24/2003 25 26 CAA-80 7 2/24/2003 24 25 Standard Deviation 2.74 1.85 Method Detection Limit (ppb) 8.6 5.8 6.2.5 Matrix Interference Effects Matrix interference effects were assessed by comparing the calculated bias for the samples containing low-level and high-level concentrations of interfering substances with the bias reported for the other PT samples (Table 6-4). The biases for the samples with low and high levels of interfering compounds ranged from -44% to -51% for both PDV 6000 units, whereas the biases for the PT samples ranged from -22% to 31% for both units. These results indicate that the interfering substances (iron and/or sulfide) adversely affected the detection of arsenic by the 24 ------- PDV 6000. Detection of arsenic by the reference method was not affected by the interfering substances. 6.2.6 Inter- Unit Reproducibility Inter-unit reproducibility was evaluated by comparing the data for the two PDV 6000 units. All detected results for the PT and environmental samples were included in the analysis. Linear regressions of the data for each instrument are shown in Figure 6-2. These results indicate that unit #2 tended to return higher measurements than unit #1. A paired t-test of the two sets of data indicated that the two PDV 6000 units were significantly different at a 5% significance level; however, they were not significantly different if the 100 ppb arsenic PT samples were excluded from the analysis. 120 100 ~ / R = 0.9954 1:1 line 80 60 £ 40 20 0 10 20 30 40 50 60 70 80 90 100 Unit #1 ~ PDV6000 Linear (PDV6000) Figure 6-2. Comparison of PDV 6000 Test Results for Units #1 and #2 6.2.7 Rate of False Positives/False Negatives Tables 6-7 and 6-8 show the data and results for the rates of false positives and false negatives, respectively, obtained from the PDV 6000. All PT and environmental samples were included in this evaluation. 25 ------- Table 6-7. Rate of False Positives for PDV 6000 False Positive (Y/N) Description Sample ID Replicate Unit #1 Unit #2 PT -1 ppb As CAA-81 1 N N CAA-81 2 N N CAA-81 3 N N CAA-81 4 N N PT - 3 ppb As CAA-94 1 N N CAA-94 2 N N CAA-94 3 N N CAA-94 4 N N Battelle drinking CAA-88 1 N N water CAA-88 2 N N CAA-88 3 N N CAA-88 4 N N Ayer untreated water CAA-90 1 N N CAA-90 2 N N CAA-90 3 N N CAA-90 4 N N Ayer treated water CAA-92 1 N N CAA-92 2 N N CAA-92 3 N N CAA-92 4 N N Falmouth Pond water CAA-43 1 N N CAA-43 2 N N CAA-43 3 N N CAA-43 4 N N Taunton River water CAA-47 1 N N CAA-47 2 N N CAA-47 3 N N CAA-47 4 N N Total number of samples 28 28 Total number of false positives 0 0 Percent false positives 0% 0% 26 ------- Table 6-8. Rate of False Negatives for PDV 6000 False Negative (Y/N) Description Sample ID Replicate Unit #1 Unit #2 PT-10 ppb As CAA-95 1 N N CAA-95 2 N N CAA-95 3 N N CAA-95 4 N N PT-30 ppb As CAA-30 1 N N CAA-30 2 N N CAA-30 3 N N CAA-30 4 N N PT-100 ppb As CAA-32 1 N N CAA-32 2 N N CAA-32 3 N N CAA-32 4 N N Battelle drinking water LFM CAA-89 1 Y Y Ayer untreated water LFM CAA-91 1 Y Y Ayer treated water LFM CAA-93 1 Y Y Falmouth Pond LFM CAA-46 1 N N Taunton River LFM CAA-96 1 N N 10 ppb As + low level CAA-34 1 Y Y interferents CAA-34 2 Y Y CAA-34 3 Y Y 10 ppb As + high CAA-36 1 Y Y level CAA-36 2 Y NA interferents CAA-36 3 Y Y CAA-36 4 Y Y Total number of samples 24 23 Total number of false negatives 10 9 Percent false negatives 42% 38% As shown in Table 6-7, 28 samples had an arsenic concentration at or below 10 ppb as measured by the reference analysis. For these samples, none of the PDV 6000 results were >10 ppb and greater than 125% of the reference measurement, yielding false positive rates of 0% for both units. Twenty four samples had arsenic concentrations above 10 ppb as measured by the reference analysis (Table 6-8) (unit #2 had 23 samples because the data for one sample was lost). For these samples, PDV 6000 results were <10 ppb and less than 75% of the reference measurement for 9 samples analyzed on unit #1 and 8 samples analyzed on unit #2, yielding false negative rates of 42% and 38%, respectively. 27 ------- 6.3 Other Factors During testing activities, the operator was instructed to keep a record of comments on ease of use, reliability, portability, and generation of waste materials. This section summarizes these observations and other comments pertaining to any problems encountered during testing. Cost information is also presented. 6.3.1 Ease of Use The technical and non-technical operators that were originally scheduled to test the PDV 6000 were unable to successfully operate the analyzer with the materials and instructions provided by the vendor. The operators were unable to plate the working electrodes on either unit with a gold film prior to analysis, apparently because the electrodes for both units provided for the test were damaged. Both operators commented that the instructions in the operation manual were difficult to follow and required moving back and forth between chapters to follow sequential instructions. The Battelle Verification Test Coordinator and Battelle Quality Staff had difficulty correlating the activities of the vendor's representative with the instructions on the Application Note for arsenic in water analysis during the TSA because the instructions moved back and forth between the Application Note and the operation manual. Some of the test sample peaks were manually adjusted to obtain the final arsenic concentration, and some professional judgment was required when selecting the appropriate standard to use for test sample quantification. Both of these factors indicate that level of experience in the operation of the PDV 6000 analyzer and VAS software is likely to influence the reliability of the results. The PDV 6000 portable analyzer was readily transported to the Battelle storage shed where environmental samples were tested. The analyzer and associated equipment were easily stored in a durable carrying case. The PDV 6000 and laptop computer would require protection from rain and high winds during outdoor use. 6.3.2 Analysis Time The instrument setup and calibration time prior to sample analysis was approximately one-half hour. The average total analysis time for each sample was about five minutes. 6.3.3 Reliability The PDV 6000 portable analyzer operated reliably with several exceptions. As previously noted, apparent damage to the electrodes prevented the operation of the PDV 6000 analyzer by the Battelle operators. Additionally, a stir motor in one of the units failed and was replaced during the test. The VAS software for one of the units failed to function on one occasion, and the data for one sample were lost. 28 ------- 6.3.4 Waste Material The waste generated by the PDV 6000 portable analyzer was manageable. The electrolyte solution contained dilute hydrochloric acid; therefore, disposal of this waste in an appropriate manner must be taken into consideration. Each sample analysis required 20 mL of electrolyte solution, so the volume of waste was relatively small. 6.3.5 Cost The listed price for PDV 6000, including VAS software, software upgrades, batteries, charger, and carrying case is $7900. 29 ------- Chapter 7 Performance Summary The PDV 6000 portable analyzer was verified by evaluating the following parameters: ¦ Accuracy ¦ Precision ¦ Linearity ¦ MDL ¦ Matrix interference effects ¦ Inter-unit reproducibility, and ¦ Rate of false positives/negatives. The assessment of accuracy indicated that the bias for the PDV 6000 ranged from -74% to 31%. Almost all biases were less than 25% except for the high- and low-level interferent samples, the Battelle drinking water LFM sample, and the Ayer treated water LFM sample. The LFM sample results suggest that the Battelle drinking water and Ayer treated water samples have matrices that adversely affect the detection of arsenic. The reference method results for both of these samples were below the detection limit of the PDV 6000. Precision was assessed by analyzing four replicates of each sample. The RSDs ranged from 6% to 16% for unit #1 and from 3% to 15% for unit #2. RSDs were not calculated for samples with one or more replicate results below the detection limit. The linearity of response was evaluated by plotting the PDV 6000 results against the reference analysis results for the PT samples. PDV 6000 results were generally lower than reference method results. The regression equations are as follows: Unit #1 y = 0.77X+ 1.22, R =0.9934 Unit #2 y=0.91X +0.59, R =0.9955 where x is the reference concentration and>' is the PDV 6000 concentration. The plots indicate that unit #2 showed a closer correspondence to the reference measurements than unit #1. The MDL was assessed by analyzing seven replicates of a sample spiked at a level approximately five times the vendor stated detection limit (i.e., 5 ppb x 5 = 25 ppb spiked sample). The MDLs calculated using the precision data from these replicates were 8.6 ppb for unit #1, and 5.8 ppb for unit #2. 30 ------- Results for samples containing low and high levels of interfering compounds indicated that low and high levels of interferents (iron and/or sulfide) adversely affected the detection of arsenic. Biases for these samples were higher than those calculated for PT samples containing arsenic only. Inter-unit reproducibility was evaluated by comparing the data for the two PDV 6000 units. A linear regression of the two sets of data indicated that unit #2 tended to return higher measurements than unit #1. A paired t-test indicated that the data for the two PDV 6000 units were significantly different at a 5% significance level; however, they were not significantly different if the 100 ppb arsenic PT samples were excluded from the analysis. The regression equation was as follows, where x is unit #1 and >' unit #2: PDV 6000 y = 1.17x- 1.56, R = 0.9954 A false positive was defined as a PDV 6000 result that was greater than 10 ppb and greater than 125% of the reference concentration, when the reference concentration is less than or equal to 10 ppb. None of the PDV 6000 results demonstrated a false positive. A false negative was defined as a PDV 6000 result that was below or equal to 10 ppb and less than 75% of the reference concentration, when the reference concentration was greater than 10 ppb. The false negative rates for PDV 6000 portable analyzers were 42% for unit #1 and 38% for unit #2. The technical and non-technical operator that were originally scheduled to test the PDV 6000 were unable to successfully operate the analyzer with the materials and instructions provided by the vendor. Consequently, all samples were analyzed by a vendor's representative. Battelle staff commented that the instructions in the operation manual were difficult to follow and required moving back and forth between operation manual chapters and the accompanying Application Note for analysis of arsenic in water. Some of the test sample peaks were manually adjusted to obtain the final arsenic concentration, and some professional judgment was required when selecting the appropriate standard to use for test sample quantification. Both of these factors indicate that level of experience in the operation of the PDV 6000 analyzer and VAS software is likely to influence the reliability of the results. The PDV 6000 portable analyzer was readily transported to the Battelle storage shed where environmental samples were tested. The analyzer and associated equipment were easily stored in a durable carrying case. The PDV 6000 and laptop computer would require protection from rain and high winds during outdoor use. The instrument setup and calibration time prior to sample analysis was approximately one-half hour. The average total analysis time for each sample was about five minutes. The PDV 6000 portable analyzer operated reliably during the test except for the failure of a stir motor in one analyzer, and the malfunction of the VAS software for one of the units on one occasion. The listed price for PDV 6000, including VAS software, software upgrades, batteries, charger, and carrying case is $7900. 31 ------- Chapter 8 References 1. Test/QA Plan for Verification of Portable Analyzers, Battelle, Columbus, Ohio, Version 2. December 8, 2000. 2. U.S. EPA Method 200.8, Determination of Trace Elements in Waters and Wastes by Inductively Coupled Plasma Mass Spectrometry, Revision 5.5, October, 1999. 3. Federal Register, Vol. 66 No. 14, January 22, 2001. Part VIII, Environmental Protection Agency. 40 CFR Parts 9, 141, and 142: National Primary Drinking Water Regulations; Arsenic and Clarifications to Compliance and New Source Contaminants Monitoring: Final Rule, http://www.epa.gov/safewater/ars/arsenic finalrule.pdf. 4. Quality Management Plan (QMP) for the ETV Advanced Monitoring Systems Pilot, Version 4, U.S. EPA Environmental Technology Verification Program, Battelle, Columbus, Ohio, December, 2002. 32 ------- |