Environmental Technology Verification Nutrient Reduction Technologies PROTOCOL FOR THE VERIFICATION OF RESIDENTIAL WASTEWATER TREATMENT TECHNOLOGIES FOR NUTRIENT REDUCTION Prepared by: NSF International P. O. Box 130140 Ann Arbor, MI 48113-0140 734-769-8010 800-673-6275 with support from the U.S. Environmental Protection Agency Environmental Technology Verification Program November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 1 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies FOREWORD The U.S. Environmental Protection Agency (EPA) has instituted a program, the Environmental Technology Verification Program - or ETV - to verify the performance characteristics of commercial-ready environmental technologies through the evaluation of objective and quality- assured data. Managed by EPA's Office of Research and Development, ETV was created to substantially accelerate the entrance of innovative environmental technologies into the domestic and international marketplaces. ETV provides purchasers and permitters of technologies with an independent and credible assessment of the technology they are purchasing or permitting. During its five-year pilot phase, EPA will cooperatively manage twelve ETV pilots in conjunction with partner organizations, including states, federal laboratories, associations, and private sector testing and standards organizations. The pilots, which have been phased in over a three-year period, focus on each of the major environmental media and various categories of environmental technologies and are guided by the expertise of a Stakeholder Group. Stakeholder Groups consist of representatives of all verification customer groups for the particular technology sector, including buyers and users of technology, developers and vendors, state and federal regulatory personnel, and consulting engineers. All technology verification activities are based on testing and quality assurance protocols that have been developed with input from the major stakeholder/customer groups. NSF International is an independent, not-for-profit organization, dedicated to public health, safety, and protection of the environment. NSF develops standards, provides educational services, and offers superior third-party conformity assessment services, while representing the interest of all stakeholders. In addition to well-established standards-development and certification programs, NSF specifically responds to and manages research projects, one-time evaluations and special studies. NSF is the verification partner organization for three pilots under EPA's ETV Program: Package Drinking Water Treatment Systems, Wet Weather Flow Technologies, and Source Water Protection Technologies. This Protocol for the Verification of Residential Wastewater Treatment Technologies for Nutrient Reduction was developed under the Source Water Protection Pilot, whose goal is to verify the performance of commercial-ready technologies used to protect ground and surface waters from contamination. Testing conducted under the ETV program using this protocol does not constitute an NSF or EPA Certification of the product tested. Rather, it recognizes that the performance of the equipment has been determined and verified by these organizations. Verification differs from certification in that it employs a broad distribution of test reports and does not use pass/fail criteria. In addition, there are differences in policy issues relative to certification versus verification. Certification, unlike verification, requires auditing of manufacturing facilities, periodic retesting, mandatory review of product changes and use of the NSF Mark. Both processes are similar, however, in regard to having standardized test methods and independent performance evaluations and test result preparation. This protocol is subj ect to revision; please contact NSF to confirm this revision is current. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 2 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies ACKNOWLEDGEMENTS EPA and NSF acknowledge and thank those persons who participated in the preparation and review of this Protocolfor the Verification of Residential Wastewater Treatment Technologies for Nutrient Reduction. Without their hard work and dedication to the project, this document would not have been approved through the process which has been set forth for this ETV proj ect. NSF Staff Thomas Stevens Maren Roush Project Manager, ETV Source Water Protection Pilot Project Coordinator, ETV Source Water Protection Pilot, protocol writer ETV Source Water Protection Pilot Technology Panel Damann Anderson Terry Bounds Sasha Earl Dr. Anish Jantrania Dr. David Lindbo Raymond Peat Joe Pearce Richard Piluk Quay Schappell Dr. George Tchobanoglous Bob Uebler Ayres Associates Orenco Systems, Inc. Loomis Austin, Inc. Virginia Department of Health North Carolina State University BioMicrobics, Inc. North Carolina State Division of Environmental Health Anne Arundel County Health Department Cromaglass Corporation University of California-Davis North Carolina State Division of Environmental Health EvTEC Staff Will Kirksey Jenise Dunn Kristi Dunn Director Project Manager Research Engineer EvTEC Technical Evaluation Panel David Althoff John Blount Larry Hepner Dr. Anish Jantrania Kenneth L. Keil Richard Piluk Richard Rose Dr. Bob Rubin Thomas Stevens Susan K. Weaver Neil Weinstein Allison Wiedeman Duane Wilding Eric Winkler PA Department of Environmental Protection Harris County, Texas Delaware Valley College Virginia Department of Health Parsons Engineering Science Anne Arundel County Health Department New Mexico Environmental Department US Environmental Protection Agency NSF International, Inc. PA Department of Environmental Protection Low Impact Development Center EPA/Chesapeake Bay Program Maryland Environmental Service University of Massachusetts Consultants Stuart L. Rosenthal, P.E. Richard L. Hinkle, P.E. Gilmore & Associates, Inc. Gilmore & Associates, Inc. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 3 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies TABLE OF CONTENTS CHAPTER 1.0 INTRODUCTION 8 1.1 Background 8 1.2 Evaluation Objectives 8 1.3 Scope of Technology Coverage 9 1.4 Responsible Parties and Roles 9 CHAPTER 2.0 TECHNOLOGY VERIFICATION TESTING CONDITIONS 12 2.1 Verification Test Site Characteristics 12 2.2 Technology Evaluation Test Plan 13 CHAPTER 3.0 TECHNOLOGY EVALUATION REQUIREMENTS 14 3.1 Performance Testing/Verification Requirements 14 3.1.1 Duration 14 3.1.2 Analytical Parameters 14 3.1.4 Sampling Requirements 17 3.1.4.1 Location 17 3.1.4.2 Frequency 17 3.1.4.3 Type 18 3.1.5 Sampling Procedures 18 3.1.5.1 Sample Collection Procedures 18 3.1.5.2 Sample Labeling and Designation 18 3.1.5.3 Sample Packing/Shipping Procedures 18 3.1.5.4 Sample Chain of Custody 18 3.1.5.5 Field Records and Documentation 18 3.1.7 Analytical Procedures 20 3.1.7 Additional Performance Evaluations 20 3.1.7.1 Alarm Systems 20 3.1.7.2 Other 20 3.2 Operation and Maintenance Considerations 20 3.2.1 General 20 3.2.2 Mechanical Components 20 3.2.3 Electrical/Instrumentation Components 21 3.2.4 Chemical Feed Components 21 3.2.5 Other Components 21 3.2.6 Byproducts or Residuals 21 3.2.7 Level of Operator Skill and Attention Required 21 3.2.8 Electrical Usage 22 3.2.9 Chemical Usage 22 3.3 Environmental Considerations 22 3.3.1 Noise 22 November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 4 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies 3.3.2 Odors 22 3.4 Miscellaneous 22 3.4.1 Proprietary Issues 22 4.0 QUALITY ASSURANCE / QUALITY CONTROL (QA/QC) 24 4.1 QA/QC Objectives 24 4.2 Intended Uses of Acquired Data 25 4.3 Analytical Quality Levels and Quality Control Levels 25 4.4 Quality Control Indicators 25 4.4.1 Precision 25 4.4.2 Accuracy 26 4.4.3 Representativeness 26 4.4.4 Completeness 27 4.4.5 Comparability 27 4.5 Water Quality and Operational Control Checks 27 4.5.1 Water Quality Data 27 4.5.1.1 Spiked Samples 27 4.5.1.2 Method Blanks 28 4.5.1.3 Field Duplicate Samples 28 4.5.1.4 Performance Evaluation Samples 28 4.5.2 Quality Control for Equipment Operation 28 4.6 Corrective Actions 29 5.0 DATA REDUCTION, EVALUATION, AND REPORTING 30 5.1 Data Reduction 30 5.2 Data Review 30 5.2.1 Level 1 Technical Data Review 30 5.2.2 Level 2 Technical Data Review 31 5.2.3 Level 3 Administrative Data Review 31 5.3 Data Validation 31 5.4 Data Reporting 32 5.5 Project Data Flow and Transfer 32 5.6 Reports 32 5.6.1 Sampling Report 32 5.6.2 Data Summary Report 33 5.6.4 Quality Control and Analytical Report 33 5.7 Use of Existing Data 34 November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 5 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies 6.0 HEALTH AND SAFETY MEASURES 35 Appendix A Environmental Technology Verification Program, Existing Data: Policy and Process November 27, 2000 This document shall not be copied or distributed, in whole or in part, Page 6 of 41 without the written permission of NSF International. ------- Environmental Technology Verification Nutrient Reduction Technologies ACRONYMS BOD biochemical oxygen demand CBOD carbonaceous biochemical oxygen demand COC chain-of-custody DEP data evaluation panel EPA United States Environmental Protection Agency ETV Environmental Technology Verification Program EvTEC Environmental Technology Evaluation Center mg/L milligrams per liter NELAC National Environmental Laboratory Accreditation Council NIST National Institute of Standards and Technology NSF NSF International PQL practical quantitation limit QA quality assurance QAPP quality assurance project plan QC quality control RPD relative percent difference SOP standard operating procedure TKN total Kjeldahl nitrogen November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 7 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies CHAPTER 1.0 INTRODUCTION 1.1 Background Domestic wastewater contains various physical, chemical and bacteriological constituents, which require treatment prior to release to the environment. Various wastewater treatment processes exist which provide for the reduction of oxygen demanding materials, suspended solids and pathogenic organisms. Reduction of nutrients, principally phosphorus and nitrogen, has been practiced since the 1960's at treatment plants where there is a specific need for nutrient reduction to protect the water quality and, hence, the uses of the receiving waters, whether ground water or surface water. The primary reasons for nutrient reduction are to protect water quality for drinking water purposes, as there is a drinking water standard for nitrite and nitrate, and to reduce the potential for eutrophication in nutrient sensitive surface waters by the reduction of nitrogen and/or phosphorus. 1.2 Evaluation Objectives This protocol has been developed to evaluate and verify nutrient reduction associated with wastewater treatment systems capable of treating domestic wastewater from individual homes and having hydraulic capacities up to 1500 gpd. The following objectives apply: • verify that the raw wastewater at the test site is representative of "normal" domestic wastewater for selected key parameters • verify performance of the technology with respect to nutrient reduction while maintaining performance with respect to conventional parameters (i.e., CBOD5, suspended solids, pH) under a specified influent flow pattern • assess operation and maintenance considerations associated with the technology, including an evaluation of the performance and reliability of various components and measurement of the level of required operator attention • measure cost factors associated with the use of the technology • identify and assess environmental inputs and outputs (beyond effluent quality) including chemical usage, energy usage, generation of byproducts or residuals, noise, and odors. • establish and implement strict QA/QC methods and procedures during sampling, field and laboratory analyses, and data handling (data recording, reduction, evaluation and reporting) • assess additional claims by the Vendor, as described in the Test Plan, with respect to the technology performance November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 8 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies 1.3 Scope of Technology Coverage This protocol has been developed to evaluate technologies that are capable of performing nutrient reduction on domestic wastewater from individual homes (with hydraulic loadings up to 1500 gpd). Technologies to be evaluated according to this protocol shall be commercially ready and reproducible. These technologies are commonly known as "package plants", as opposed to field erected wastewater treatment plants. Nutrient reduction technologies may also include certain elements that are field erected/assembled such as tanks, piping, etc. Natural systems involving features such as vegetation, wetlands, free access or buried sand filters, and soil systems may be evaluated using this protocol as long as the system has a single discharge point from which a discreet sample may be taken. 1.4 Responsible Parties and Roles The principal parties involved with an evaluation of a nutrient reduction technology under this protocol may include the following: • Verification Organization • U.S. Environmental Protection Agency (EPA) • Technology Panel • Testing Organization • Vendor The primary roles and responsibilities of each party may include: Verification Organization: • coordinate with Testing Organization and Vendor to identify and secure a site for the technology verification • coordinate with Testing Organization and Vendor to prepare a site-specific and technology-specific Test Plan • review and approve Test Plan prior to commencement of testing • coordinate and oversee the evaluation and laboratory testing associated with each technology verification • review data generated during testing • oversee the development of the Verification Report • print and distribute the final documents (i.e. protocol, verification report) • perform quality assurance (QA) oversight of the sampling and analysis program outlined in this protocol or designate this responsibility to another party • qualify Testing Organization(s)/laboratory(ies). For more information about procedures followed to qualify testing organizations and laboratories under the ETV Source Water Protection Pilot, please refer to Quality Management Plan for the Pilot. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 9 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies U.S. Environmental Protection Agency (EPA) This protocol was developed with financial and quality assurance assistance from the Environmental Technology Verification (ETV) Program, which is overseen by the EPA Office of Research and Development. Any Verification Report developed under the ETV Program using this protocol will be subject to the approval of the ORD laboratory director. Technology Panel: • developed and approved the protocol • review and approve Test Plans prepared in accordance with the protocol (each Test Plan shall be reviewed by a minimum of two Technology Panel members) Testing Organization: • coordinate with Verification Organization and Vendor to identify and secure a site for the technology verification • coordinate with the Verification Organization and Vendor relative to preparing a specific Test Plan • obtain approval of the Test Plan by the Verification Organization prior to commencement of testing • conduct the technology verification in accordance with the Test Plan with oversight by the Verification Organization • coordinate with and report to the Verification Organization during the technology verification process • assume all roles and responsibilities of day-to-day coordination with the laboratory(ies), ensure the laboratory(ies) properly implement the Test Plan, resolve any quality concerns that may be encountered, and report all findings to the Verification Organization • provide analytical results of the technology evaluation to the Verification Organization • if necessary, document changes in plans for testing and analysis, and notify the Verification Organization of any and all such changes before changes are executed Note: The laboratory functions associated with verification testing may be carried out by either an independent commercial laboratory under contract with the Testing Organization or by a laboratory associated with the Testing Organization, in accordance with the specifications of the Verification Organization. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 10 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies Vendor: • assist in preparing a specific Test Plan for the technology verification • coordinate with the Verification Organization and Testing Organization to identify and secure a site for the technology verification • provide a complete field-ready version of the technology of the selected capacity for verification and assist the Testing Organization with installation at the test site • provide start-up services and technical support as required during the period prior to the evaluation • provide technical assistance to the Testing Organization during operation and monitoring of the equipment undergoing verification testing as requested • remove equipment associated with the technology and any discarded items from the test site following termination of the verification evaluation. • provide funding for verification testing November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 11 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies CHAPTER 2.0 TECHNOLOGY VERIFICATION TESTING CONDITIONS 2.1 Verification Test Site Characteristics Minimum requirements for a test site (to be demonstrated prior to the development of a site- specific Test Plan) include: • The wastewater shall be "typical" domestic, relative to key parameters such as BOD5 TSS, TKN and phosphorus. Wastewater of weaker strength due to infiltration/inflow or wastewater of excessive strength due to industrial waste, restaurant wastewater, etc., is not acceptable. It shall be documented that the raw wastewater is domestic. • Raw wastewater characteristics shall be determined based on a minimum of six (6) 24-hour composite samples collected at a minimum interval of one (1) week. The following are suggested guidelines for domestic wastewater. Parameter Biochemical oxygen demand (BOD5, 20°C) Total Suspended Solids (TSS) TKN (as N) Total Phosphorus (as P) pH Alkalinity (as CaCC^) Temperature Concentration Range 100-450 mg/L 100-500 mg/L 25-70 mg/L 3-20 mg/L 6-9 units Greater than 60 mg/L (alkalinity addition may be required) Greater than 10 °C and less than 30°C • The test site shall have a suitable means and location for sampling of raw wastewater and a sampling arrangement to collect representative samples. • The test site shall be capable of controlled dosing to the technology being evaluated to simulate a diurnal flow variation and to allow for stress testing. The test site shall have a sufficient flow of wastewater to accomplish the required controlled dosing pattern. • The test site shall be accessible, relative to operational control and oversight, and secure to prevent tampering by outside parties. • The test site shall have a legal means of wastewater disposal of both the effluent from the testing operation and for any untreated wastewater generated when testing is not occurring. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 12 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies • The test site shall be capable of accommodating the start-up period, testing period, stress testing and any additional testing activities, such as a determination of operations and maintenance requirements. 2.2 Technology Evaluation Test Plan A detailed Test Plan shall be developed for every technology to be evaluated according to this protocol. Both the Vendor and the Testing Organization shall assist in the preparation of the Test Plan. Test Plans shall include the following sections, in addition to other sections specified by the Verification Organization for the evaluation: • Title Page • Forward • Table Of Contents • Executive Summary • Abbreviations and Acronyms • Introduction • Technology Verification Testing Responsibilities • Technology Capacity and Description • Experimental Design • Field Operation Procedures • Quality Assurance Proj ect Plan (QAPP) • Data Management and Analysis • Safety Plan The Vendor shall provide at least the following items in the Test Plan: • A brief statement of the water quality treatment objectives (what are the target nutrients); • A statement of the technology's performance capabilities; • Equipment and process description; • A brief statement of the Test Plan objectives; • Operation and maintenance (O&M) manual(s); and • Health and safety information relating to the equipment and the process. Test Plan requirements are discussed in detail throughout the protocol. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 13 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies CHAPTER 3.0 TECHNOLOGY EVALUATION REQUIREMENTS 3.1 Performance Testing/Verification Requirements 3.1.1 Duration The duration of the evaluation period shall be a minimum of one (1) year following a maximum start-up period of eight (8) weeks. Sampling frequency during the start-up period is determined at the time of Test Plan development, at the discretion of the Vendor, the Testing Organization, and the Verification Organization, and in accordance with the O&M manual for the technology. All data generated during the start-up period shall be reported as such in the Verification Report but will not be included in statistical analyses performed and reported on data generated during the evaluation period. When the technology performance has stabilized during the start-up period, the Vendor shall advise the Testing Organization that the evaluation period can commence. The evaluation period duration of one (1) year will allow for an assessment of the impact of seasonal variations on performance. 3.1.2 Analytical Parameters The analytical parameters of interest for verifying system performance for nutrient reduction are noted in Table I, which includes the requirements for both raw influent and treated effluent samples. If the treatment process involves multiple stages, it may be appropriate to collect samples at intermediate points. The Test Plan shall clearly indicate the sampling points for the technology being evaluated. If the Vendor does not intend to seek verification with respect to reduction of a certain nutrient, then the parameter list and subsequent Verification Report and Statement can be adjusted accordingly. The Vendor may also seek verification with respect to parameters not listed in Table I, as shall be detailed in the specific Test Plan. 3.1.3 Influent Flow Pattern The influent flow shall conform to the following pattern as representative of a typical residence(s) scenario: 6 a.m. - 9 a.m. approximately 35% of total daily flow 11 a.m. - 2 p.m. approximately 25% of total daily flow 5 p.m. - 8 p.m. approximately 40% of total daily flow Total daily flow shall be within 100% ± 10% of the rated capacity of the technology undergoing testing based on a thirty (30) day average. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 14 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies TABLE I: SAMPLING MATRIX PROTOCOL FOR THE VERIFICATION OF WASTEWATER NUTRIENT REDUCTION TECHNOLOGIES SAMPLE LOCATION RAW TREATED TESTING PARAMETER BOD5 CBOD5 Suspended Solids pH Temperature (°C) Alkalinity (as CaCOs) Dissolved Oxygen TKN (as N) Ammonia-N (as N) Nitrite-N (as N) Nitrate-N (as N) Phosphorus, Total (as P) SAMPLE TYPE INFLUENT EFFLUENT LOCATION 24 Hour composite 24 Hour composite 24 Hour composite Grab Grab 24 Hour composite Grab 24 Hour composite 24 Hour composite 24 Hour composite 24 Hour composite 24 Hour composite V V V V V V V V V V V V V V V V V V V Laboratory Laboratory Laboratory Test Site Test Site Laboratory Test Site Laboratory Laboratory Laboratory Laboratory Laboratory Orthophosphate (as P) 24 Hour composite V Laboratory November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 15 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies When necessary to account for dilution by precipitation, such as during the evaluation of a free access sand filter, it may be helpful to add chlorides to the sampling matrix. The Testing Organization shall monitor and record influent flows daily to ensure that the dosing pattern is delivered as specified in the protocol. The Test Plan shall specify the way in which flow rates will be measured (i.e.: totalizer flow meter, rate meter, etc...) One stress test shall be performed following every two months of normal operation during the technology evaluation, so that each of the five stress scenarios is addressed within the twelve (12) month evaluation period. Stress testing shall involve the following simulations: • Wash-day stress • Working parent stress • Low-loading stress • Power/equipment failure stress • Vacation stress Wash-day stress simulation shall consist of three (3) wash-days in a five (5) day period with each wash-day separated by a 24-hour period. During a wash-day, the technology shall receive the normal flow pattern (Section 3.1.3.1); however, during the course of the first two (2) dosing periods per day, the hydraulic loading shall include three (3) wash loads [three (3) wash cycles and six (6) rinse cycles]. Common (readily available to consumers) detergent and non-chlorine bleach shall be added to each wash load at the manufacturer's recommended loading. Working parent stress simulation shall consist of five (5) consecutive days when the technology is subjected to a flow pattern where approximately 40% of the total daily flow is received between 6 a.m. - 9 a.m. and approximately 60% of the total daily flow is received between 5 p.m. and 8 p.m., which shall include one (1) wash load [one (1) wash cycle and two (2) rinse cycles]. Low-loading stress simulation shall consist of testing the technology for 50% of the design flow loading for a period of 21 days. Approximately 35% of the total daily flow is received between 6 a.m. - 11 a.m., approximately 25% of the flow is received between 11 a.m. - 4 p.m., and approximately 40 % of the flow is received between 5 p.m. and 10 p.m. Power/equipment failure stress simulation shall consist of a flow pattern where approximately 40% of the total daily flow is received between 5 p.m. and 8 p.m. on the day when the power/equipment failure stress is initiated. Power to the technology shall then be turned off at 9 p.m. and the flow pattern shall be discontinued for 48 hours. After the 48-hour period, power shall be restored and the technology shall receive November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 16 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies approximately 60% of the total daily flow over a three (3) hour period which shall include one (1) wash load [one (1) wash cycle and two (2) rinse cycles]. Vacation stress simulation shall consist of a flow pattern where approximately 35% of the total daily flow is received between 6 a.m. and 9 a.m. and approximately 25% of the total daily flow is received between 11 a.m. and 2 p.m. on the day that the vacation stress is initiated. The flow pattern shall be discontinued for eight (8) consecutive days with power continuing to be supplied to the technology. Between 5 p.m. and 8 p.m. of the ninth day, the technology shall receive 60% of the total daily flow, which shall include three (3) wash loads [three (3) wash cycles and six (6) rinse cycles], 3.1.4 Sampling Requirements 3.1.4.1 Location Samples shall be collected of the raw influent and treated effluent. It may also be necessary or appropriate to collect samples at intermediate points if the equipment/process involves multiple stages. Effluent samples shall be collected from a location where wastewater is flowing (i.e. from a pipe or equivalent). For technologies with subsurface discharge, a location shall be provided for collecting an effluent sample prior to discharge to the soil system. Given the potential variability in soil characteristics, a wide range of results for nutrient reduction will likely occur if soil systems are taken into account, and it is unlikely that evaluation of the technology will be reproducible. If a particular technology involves the use of a soil system capable of being reproduced from one location to another, then the effluent sample may be collected at a location following the soil system. For such systems, the Test Plan shall provide documentation evidencing the reproducibility of the soil system. All natural systems involving features such as vegetation, wetlands, free access or buried sand filters, and soil systems shall have a single discharge point from which a discreet sample may be taken. 3.1.4.2 Frequency Samples shall be collected at a minimum interval of once per month at all sampling locations. The Test Plan shall indicate the sampling frequency to be performed during verification testing. Samples shall be collected on the day each stress simulation is initiated and when approximately 50% of each stress test has been completed. Twenty-four (24) hours after the completion of wash-day, working-parent, low-loading, and vacation stress scenarios, samples shall be collected for six (6) consecutive days. Forty-eight (48) hours after the completion of the power/equipment failure stress, samples shall be collected for five (5) consecutive days. Samples shall also be collected for five (5) consecutive days at the end of the yearlong evaluation period. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 17 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies 3.1.4.3 Type Sample type (24 hour composite, grab) shall be as indicated in Table I for the various parameters. All composite samples shall be collected proportional to flow or volume. 3.1.5 Sampling Procedures 3.1.5.1 Sample Collection Procedures The Test Plan shall indicate how the following sample collection procedures shall be performed during performance testing: • Locate sample collection points • Set up and place sampling equipment in service to obtain flow proportioned composite samples • Collect grab samples for those parameters requiring a grab sample analysis • Add appropriate preservatives to the sample containers and transport all sample containers in a chilled cooler (4°C) • Document the sample collection points and the sampling event recording all relevant information in the Field Log 3.1.5.2 Sample Labeling and Designation The Test Plan shall establish the means by which samples will be labeled and uniquely identified. 3.1.5.3 Sample Packing/Shipping Procedures All samples collected for laboratory analysis shall be shipped to the laboratory on the day of collection, following proper identification, chain-of-custody, preservation, and packaging procedures as established in the Test Plan. 3.1.5.4 Sample Chain of Custody Test Plans for the evaluation of technologies shall specify the means by which sample chain of custody will be recorded. 3.1.5.5 Field Records and Documentation A Field Log shall be prepared and maintained by the Testing Organization or a qualified designee throughout the course of the evaluation. The Field Log will be turned in to the Verification Organization for copying/filing/tracking when complete. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 18 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies Field Log entries shall be recorded on a permanent medium. If errors are made in any Field Log, chain-of-custody record, or any other field record document, corrections may be made by crossing a single line through the error, entering the correct information, initialing, and dating the correction. All entries in the Field Log shall be legible and contain accurate and inclusive documentation of all project activities. Once completed, the Field Log becomes an accountable document and shall be maintained as part of the project files. The Test Plan shall include the qualifications of all persons involved in Field Log entries, chain-of-custody records or any other field record documentation. All aspects of sample collection and handling, as well as visual observations, shall be documented in the Field Log. All sample collection equipment (where appropriate), field analytical equipment, and equipment used to make physical measurements shall be identified in the Field Log. All calculations, results, and calibration data for field sampling, field analytical, and field physical measurement equipment shall also be recorded in the Field Log, except where these are referenced as being recorded on approved field forms. All field analyses and measurements shall be traceable to the specific piece of field equipment utilized and to the field investigator collecting the sample, making the measurement, or conducting analyses. The Field Log shall be updated as fieldwork progresses. These following minimum information shall be recorded in the Field Log: • Date • Weather conditions (including ambient temperature) • Description of the work performed • List of personnel involved, their position, and respective affiliations • List of equipment on-site • Description of decontamination performed • List of sample I.D. numbers of environmental samples taken, and analyses requested • The uniquely numbered COCs forwarded, and the recipient • Identification of problems encountered and/or deviations from the test plan • Calibrations performed • Problems encountered and corrective actions taken 3.1.6 Waste Management Plan The Test Plan shall describe the procedures to be followed to assure that wastes generated during the verification testing are managed in a manner that is protective of November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 19 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies human health and the environment. The management of wastes includes the containerization, characterization, transportation, and disposal of wastes. 3.1.7 Analytical Procedures The methods for the analysis of the parameters in Table I and any additional parameters to be evaluated during verification testing shall be those contained in 40 CFR Part 136, or alternate test procedures approved pursuant to 40 CFR Part 136. The laboratory shall be qualified by the Verification Organization prior to commencement of the evaluation. The Test Plan shall contain information about the procedures that the approved laboratory will follow during the evaluation process (i.e., SOPs, etc.). For testing to be performed immediately at the test location (i.e., dissolved oxygen, pH, and temperature), the Test Plan shall describe the means by which the test site personnel have been trained and demonstrated proficiency in the use of the test equipment. 3.1.7 Additional Performance Evaluations 3.1.7.1 Alarm Systems The nutrient reduction technology may incorporate certain alarm systems to alert the property owner and/or operator of equipment failure, high liquid level, etc. During the evaluation period, any alarm systems associated with the technology shall be operationally tested and verified at least once per month. The Test Plan shall describe the means by which alarm systems are to be evaluated. 3.1.7.2 Other The Vendor may have additional claims relative to the performance or functioning of the technology to be evaluated during the test period. The Test Plan shall specifically address the means by which additional claims will be verified. 3.2 Operation and Maintenance Considerations 3.2.1 General Installation and operation and maintenance requirements for the technology shall be overseen by the Testing Organization and shall be performed in accordance with the Vendor's written instructions. The Test Plan shall address how the installation requirements and maintenance performed will be documented during the course of verification testing. The Vendor shall not be permitted to perform operation or maintenance tasks without direct supervision by the Testing Organization. 3.2.2 Mechanical Components November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 20 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies Wastewater treatment processes may involve the use of compressors or blowers, mixers, and chemical and wastewater pumps. Performance and reliability of the equipment during the test period shall be observed and documented, including equipment failure rates, replacement rates, and the existence and use of duplicate or standby equipment. The testing period may be extended to a second year of operation to fully evaluate equipment performance, reliability, and durability. This would result in a second verification of the technology, with an increased focus on operation and maintenance issues. 3.2.3 Electrical/Instrumentation Components Electrical components, particularly those that might be adversely affected by the corrosive atmosphere of a wastewater treatment process, and instrumentation and alarm systems shall be monitored for performance and durability during the course of verification testing. The Test Plan shall indicate the means by which these components are to be evaluated. 3.2.4 Chemical Feed Components The Test Plan shall include testing requirements for the verification of the chemical feed delivery rate. Chemical feed systems may involve alkalinity addition to maintain the proper pH level, chemical addition for phosphorus reduction and/or carbon source for denitrification. The Test Plan shall also specify observation of the chemical feed components following completion of the evaluation period. All observations (i.e. corrosion, wear, etc.) shall be noted in the Field Log. 3.2.5 Other Components The Vendor may have additional components relative to the operation and maintenance of the technology to be considered during the test period. The Test Plan shall indicate the means and frequency by which these components are to be evaluated. 3.2.6 Byproducts or Residuals A nutrient reduction process may involve generation of byproducts or residuals, which shall require off-site disposal. Such byproducts or residuals, when generated, may include septage, sludge, ion exchange regenerates/brines, etc.. The quantity and quality of any byproducts or residuals generated during the evaluation process shall be recorded. The volume, mass and other characteristics of the byproducts or residuals (such as TSS, VSS, etc.) shall be recorded with a projection of the required pump out frequency. 3.2.7 Level of Operator Skill and Attention Required November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 21 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies All wastewater treatment plants require periodic operator attention. The Test Plan shall address how the required operation/maintenance tasks, along with an indication of the extent (i.e., hours per month) and level of operator attention required to maintain performance, will be determined and recorded during the verification process. 3.2.8 Electrical Usage The Testing Organization shall record the monthly energy consumption (kilowatt hours) of the technology. This may require a dedicated electric meter. The intent is to provide information on the power source (single or three phase), voltage, and the overall electric usage of the technology. If the Vendor claims an energy recovery benefit, the Test Plan shall address the means by which this claim will be verified. 3.2.9 Chemical Usage Any chemicals added to the technology during verification testing shall be recorded and quantified. The Test Plan shall identify chemicals used with the technology and verification of the chemical shall be noted in the Field Log. 3.3 Environmental Considerations 3.3.1 Noise Noise levels associated with mechanical equipment (particularly compressors and blowers) shall be verified during the evaluation period. A decibel meter shall be used to measure the noise level associated with the technology. Measurements shall be taken one meter from the source(s) at one and a half meters above the ground, at 90° intervals in four (4) directions. Any mitigation measures for noise control provided by the Vendor shall be noted. Noise levels shall be measured once during the evaluation, approximately one month after completion of start-up period. 3.3.2 Odors Monthly observations shall be made by the Testing Organization during the evaluation period with respect to odors generated by the technology. The observation shall be qualitative and shall include odor strength (intensity) and type (attribute). If the treatment system is buried, covered or otherwise has odor containment, the means of ventilating the compartment(s), including any odor treatment systems shall be noted. 3.4 Miscellaneous 3.4.1 Proprietary Issues November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 22 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies The Test Plan shall identify proprietary issues relative to the Vendor's nutrient reduction technology and discuss how they will be addressed during the course of verification testing and reporting. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 23 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies CHAPTER 4.0 QUALITY ASSURANCE / QUALITY CONTROL (QA/QC) 4.1 QA/QC Objectives Quality assurance and quality control of the equipment calibration, equipment operation, process maintenance, and the measured water quality parameters shall be maintained throughout the verification testing program. The Testing Organization shall prepare a Quality Assurance Project Plan (QAPP) for the Verification Testing, to be included in the Test Plan, that specifies procedures to be followed to ensure the validity of test results and their use as the basis for equipment performance verification. The QAPP applies to all organizations involved in the Equipment Verification Testing, including Testing Organizations and laboratories qualified by the Verification Organization. The Testing Organization shall have the primary responsibility for ensuring that all individuals involved in the Technology Verification Testing comply with QA/QC procedures during the course of the evaluation, although the Verification Organization shall qualify the Testing Organization and laboratories prior to initiation of testing. The objective of QA/QC is to ensure strict methods and procedures are followed during testing so that the data obtained are valid for use in the verification of a technology according to this protocol. In addition, QA/QC ensures that the conditions under which data is obtained will be properly recorded so as to be directly linked to the data, should a question arise as to its validity. The following QA/QC measures shall be addressed in the QAPP: • Description of methodology for measurement of accuracy, the materials used, the frequency, the criteria for determining the acceptability of accuracy measurements and the actions to be taken if criteria are not met; • Description of methodology for measurement of precision, the materials used, the frequency, the criteria for determining the acceptability of precision measurements and the actions to be taken if criteria are not met; • Description of the methodology for use of blanks, the materials used, the frequency, the criteria for acceptable method blanks and the actions to be taken if criteria are not met; • Description of any specific procedures appropriate to the analysis of the performance evaluation samples. It has to be clear how these samples are going to be used in the verification testing; • Outline of the procedure for determining samples to be analyzed in duplicate, the frequency and approximate number; • Description of the procedures used to assure that the data are correct; • Definition of data to be reported during the verification testing, in terms of analytical parameter type and frequency; • Listing of techniques an/or equations used to quantify any necessary data quality indicator calculations in the analysis of water quality parameters, microbiological November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 24 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies contaminants or operational conditions (e.g., flow rates, mixer speeds, detention times); • Outline of the frequency, format, and content of self-assessments of the Testing Organization's technical systems; • Description of the means by which the Testing Organization shall ensure that laboratories have properly implemented the Test Plan; • Outline of the frequency, format, and content of assessment reports prepared by the Testing Organization for the Verification Organization; • Development of a corrective action plan responding to audit findings; and • Requirement to provide all QC information, such as calibrations, blanks and reference samples, in an appendix to the report. All raw data shall also be reported in an appendix. 4.2 Intended Uses of Acquired Data The intended uses of the data acquired under this protocol are to determine the degree of treatment a nutrient reduction technology achieves during a site-specific testing period by measuring influent and effluent concentrations of selected parameters. 4.3 Analytical Quality Levels and Quality Control Levels Whether the quality assurance (Q A) obj ectives for the proj ect, as outlined in the QAPP, are met will be determined through the use of quality control (QC) elements assessing precision, accuracy, representativeness, completeness and comparability. Each of the QC elements is discussed in the following section. 4.4 Quality Control Indicators 4.4.1 Precision Precision is defined as the degree of mutual agreement relative to individual measurements of a particular sample. As such, Precision provides an estimate of random error. Precision is evaluated using analysis of field or matrix spiked duplicates. Method precision is demonstrated through the reproducibility of the analytical results. Relative percent difference (RPD) may be used to evaluate Precision by the following formula: RPD=[(Ci- C2) + ((Ci + C2)/2)] x 100% Where: Ci= Concentration of the compound or element in the sample C,2= Concentration of the compound or element in the duplicate The Test Plan shall present the precision methods to be employed in the analysis of data generated under the Verification Testing Program. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 25 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies 4.4.2 Accuracy For water quality analyses, accuracy is defined as the difference between the measured or calculated sample result and the true value for the sample. The closer the numerical value of the measurement comes to the true value or actual concentration, the more accurate the measurement. Loss of accuracy can be caused by errors in standards preparation, equipment calibrations, interferences, and systematic or carryover contamination from one sample to the next. Analytical accuracy may be expressed as the percent recovery of a compound or element that has been added to laboratory reagent water at known concentrations prior to analysis. The following equation is used to calculate percent recovery: Percent Recovery=( Ar-A0 )/Af xl00% Where: Ar= Total amount detected in spiked laboratory reagent water A0= Amount detected in unspiked laboratory reagent water Af= Spike amount added to laboratory reagent water. For parameters which are not routinely spiked during analysis (e.g., BOD, CBOD, TSS, pH, and alkalinity), performance evaluation samples shall be obtained and used to develop control limits for the laboratory. Where appropriate and stable, the same performance evaluation sample may be analyzed over a period of time. Accuracy will be ensured in technology evaluation by maintaining consistent sample collection procedures, including sample locations, sample timing, sample handling, and by executing random spiking procedures for specific target constituent(s). The Test Plan shall discuss methods to determine the accuracy of sampling and analyses. For equipment operating parameters, accuracy refers to the difference between the reported operating condition and the actual operating condition. For operating data, accuracy entails collecting a sufficient quantity of data during operation to be able to detect a change in system operations. As an example, accuracy of flowrate may be the difference between the flow indicated by a flow meter and the flow measured on the basis of volume overtime (with a container of known volume and a stopwatch). Meters and gauges shall be checked at least monthly for accuracy. The Test Plan shall discuss means for determining the accuracy of equipment operating parameters. 4.4.3 Representativeness Representativeness is the degree to which data accurately and precisely represent a characteristic of a population, parameter variations at a sampling point, a process November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 26 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies condition, or an environmental condition. Representativeness is a qualitative parameter relating to the proper design of a sampling program. The Test Plan shall describe the means by which the representativeness of samples collected during the technology evaluation will be ensured. 4.4.4 Completeness Completeness is expressed as the percentage of valid, acceptable data obtained from a measurement process compared to the minimum amount that was needed to draw an accurate conclusion. The Test Plan shall specify the minimum amount of data needed for each of the various testing stages (start-up period, sampling, stress testing, etc.); however, that amount shall not be less that that provided in this protocol. 4.4.5 Comparability Comparability is a qualitative parameter expressing the confidence with which one data set can be compared with another. Analytical results are comparable to results from other laboratories as a result of participation in procedures/programs such as the following: use of instrument standards traceable to National Institute of Standards & Technology (NIST) or EPA sources; use of standard or validated methodology; reporting of results in consistent units; and participation, as appropriate, in inter-laboratory studies to document laboratory performance. By using traceable standards and validated methods, the analytical results can be compared to other laboratories operating similarly. The Test Plan shall describe the means by which the comparability of data sets generated during the technology evaluation will be ensured. 4.5 Water Quality and Operational Control Checks Quality control checks provide a means of measuring the quality of the data obtained. This section describes quality control checks for both water quality analyses and equipment operation. The Testing Organization may not need to use all of the checks identified in this section. The selection of appropriate quality control checks depends on the equipment, the experimental design, and the performance goals. The quality control checks to be used in the evaluation of a technology shall be specified in the Test Plan, in addition to discussion of the corrective action to be taken if the quality control parameters fall outside of the evaluation criteria. 4.5.1 Water Quality Data Following the start-up period, the results of the treatment achieved by the nutrient reduction technology being evaluated are interpreted in terms of water quality. Thus, the quality of the sampling and analysis is important. The QAPP shall emphasize methods to be employed for sampling and analysis QA/QC. Some important aspects to be considered are the following: 4.5.1.1 Spiked Samples November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 27 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies The use of spiked samples will depend on the testing program and the target contaminants. If spiked samples are to be used, the Test Plan shall specify the procedures, frequency, acceptance criteria, and actions if criteria are not met. 4.5.1.2 Method Blanks Method blanks are analyzed for selected water quality parameters to evaluate analytical method-induced contamination, which could cause false-positive results. The Test Plan shall identify the need and procedures for method blanks. 4.5.1.3 Field Duplicate Samples A field duplicate sample is a second sample collected at the same location as the original sample. Duplicate sample results are used to assess precision, including variability associated with both the laboratory analysis and the sample collection process. Duplicate samples are collected simultaneously or in immediate succession, using identical recovery techniques, and treated in an identical manner during storage, transportation, and analysis. The procedure for determining samples to be analyzed in duplicate shall be provided in the Test Plan, with the required frequency of analysis and the approximate number. The Test Plan should also discuss the number of duplicate samples to be provided to the laboratory as "blind duplicates". 4.5.1.4 Performance Evaluation Samples Performance evaluation (PE) samples are samples whose composition is unknown to the analyst. PE samples are submitted with statistics about each sample that have been derived from the analysis of the sample by a number of laboratories using EPA-approved methods. These statistics include a true value of the PE sample, a mean of the laboratory results obtained from the analysis of the PE sample, and an acceptance range for sample values. PE samples shall be analyzed for selected water quality parameters before the analytical laboratory initiates technology evaluation. Control limits for PE samples will be used to evaluate the method performance of the analytical laboratory. An analytical laboratory that does not meet the control limits shall not be used for verification analyses. 4.5.2 Quality Control for Equipment Operation The Test Plan shall explain the methods used to check the accuracy of equipment operating parameters and the frequency at which these checks will be performed. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 28 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies All sampling and analytical instalments to be used at the local test site (i.e., DO meters, dosing system, sampler, etc.) shall be maintained and calibrated by trained test site personnel in accordance with manufacturer's instructions. 4.6 Corrective Actions Each Test Plan shall include a corrective action plan. This plan shall include the predetermined acceptance limits, the corrective action to be initiated whenever such acceptance criteria are not met, and the names of the individuals responsible for implementation. Routine corrective action may result from common monitoring activities, such as: • Performance evaluation audits • Technical systems audits Ultimately, responsibility for project quality assurance/quality control (QA/QC) during implementation of this protocol rests with the Verification Organization, specifically the Verification Organization Project Manager, with appropriate input from the Verification Organization QA/QC Manager. However, immediate QA/QC for individual tasks (e.g. sample collection, handling, preparation, and analysis) rests with the individuals and organization performing the task at hand, as described in this chapter throughout the protocol. The Verification Organization Proj ect Manager will coordinate oversight and/or audits of these tasks with the Testing Organization Project Manager to ensure that the Test Plan is being executed as written, and that nonconformances are appropriately reported and documented. Corrective action shall be taken whenever a nonconformance with the Test Plan occurs. Nonconformances can occur within the realm of sampling procedures, sample receipt, sample storage, sample analysis, data reporting, and computations. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 29 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies CHAPTER 5.0 DATA REDUCTION, EVALUATION, AND REPORTING The analytical data generated by the laboratory shall be reviewed internally prior to submission to Testing Organization and/or the Verification Organization to assure the usability/validity of the reported results. This internal data review process will consist of data generation, reduction, a minimum of three levels of documented review, and reporting. The data generated by on-site tests (dissolved oxygen, pH, temperature), will not be validated by an independent reviewer. Independent data validation will be performed on definitive data collected, i.e., the laboratory. The data reduction, review, reporting, and validation procedures described in this section will ensure that (1) complete documentation is maintained, (2) transcription and data reduction errors are minimized, (3) the data are reviewed and documented, and (4) the reported results are qualified. Laboratory data reduction and verification procedures are required to ensure that the overall objectives of analysis and reporting meet method and project specifications. 5.1 Data Reduction Analytical data are first generated in raw form at the instrument. These data may be in either graphic form or printed in tabular form. Specific data reduction procedures, generation procedures, and calculations, which convert raw results into a form from which conclusions can be drawn regarding equipment performance, shall be detailed in the laboratory SOPs for each analytical method used. Analytical results shall be reported consistently. Data reduction shall be performed by a laboratory QA/QC Chemist, or qualified designee, who is experienced with the particular analysis and knowledgeable of project QA/QC requirements. 5.2 Data Review The technician/analyst who generates the analytical data is responsible for the correctness and completeness of those data. This review process involves evaluation of both the results of the QC data and the professional judgement of the person(s) conducting the review. This application of technical knowledge and experience to the evaluation of data is essential in ensuring that high quality data are generated. The Test Plan shall document the data review procedures which will be followed by laboratory personnel. For example, the data review may be conducted at the laboratory level prior to submittal following this three step process: 5.2.1 Level 1 Technical Data Review In the Level 1 data review process, the analysts review the quality of their work based on an established set of guidelines. The review will ensure at a minimum that appropriate preparation, analysis, and SOPs have been followed; analytical results are correct and complete; QC samples are within established control limits; and that documentation is complete (e.g., any anomalies have been documented). November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 30 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies 5.2.2 Level 2 Technical Data Review This level of review will be performed by a supervisor or data review specialist whose function is to provide an independent review of the data package. This review will also be conducted according to an established set of guidelines (i.e., method requirements and laboratory SOPs). The Level 2 review includes a review of qualitative and quantitative data and review of documented anomalies. 5.2.3 Level 3 Administrative Data Review The final review of the data, prior to submittal, will be performed by the QA/QC Officer or program administrator at the laboratory. This level of review provides a total overview of the data package to ensure its consistency and compliance with project requirements. 5.3 Data Validation The Testing Organization shall verify that the data forms, data acquisition and reduction are complete and accurate. A field supervisor or another technical member of the Testing Organization shall review calculations and inspect logbooks and data sheets. Laboratory operators shall examine calibration and QC records, verify all instrument systems are in proper working order and ensure that QA objectives have been met. Analytical outlier data are defined as those QC data lying outside a specific QC objective window for precision and accuracy for a given analytical method. Should QC data be outside control limits, the laboratory supervisor shall notify the Testing Organization and investigate the cause of the problem. If the cause is an analytical problem, the sample shall be reanalyzed. If the cause can be attributed to the sample matrix, the result shall be flagged with a data qualifier. This data qualifier shall be included and explained in the final analytical report from the laboratory. The following are examples of validation flags that may be applied to the data: U The analyte was analyzed for but was not detected. The associated numerical value is at or below the method detection limit. F The analyte was positively identified, but the numerical value is below the PQL. M A matrix effect was present. B The analyte was found in the associated blank, as well as in the sample. R The data is unusable due to deficiencies in the ability to analyze the sample and meet QC criteria. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 31 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies 5.4 Data Reporting The laboratory(s) analytical reports shall conform to the following minimum reporting requirements: • A table, which matches the contract laboratory sample ID to the QA laboratory split sample ID collected. This table also will identify all duplicates and blanks with their corresponding samples. • A "Cooler Receipt Form" for the purposes of noting problems in sample packaging, chain-of-custody, and sample preservation. • A copy of the chain-of-custody submitted with the samples. • Analytical summaries which report results for all samples, blanks, and QC for each analytical fraction. The detection limits are those established by the methods identified and all analytes will be reported. The referenced analytical methods (including preparation methods), date of sample collection, data of extraction, and the date of analysis, as well as any dilution factor, also are required. • Matrix Duplicates - Relative percent difference (RPD) values will be reported, as well as the project/analyte control limits. • Matrix Spike/Matrix Spike Duplicates - The relative percent difference will be reported for each spiked compound. Concentrations for each spiked compound and the method-specific control limits will be reported. 5.5 Project Data Flow and Transfer Data flow from the laboratory and test site to the Verification Organization shall follow established procedures to ensure that data are properly tracked, reviewed, and validated for use. All test site data and laboratory data packages shall be submitted to the Verification Organization Project Manager. No changes to the laboratory data packages shall be made without approval from the Verification Organization. The Test Plan shall describe the format, schedule and means (i.e., electronic format, tables, etc.) for reporting data to the Verification Organization. 5.6 Reports Reports shall be submitted by the Testing Organization to the Verification Organization during the course of the evaluation to ensure that any problems arising during sampling and analysis are investigated and corrected as quickly as possible. The following sections describe the types of QC reports that shall be submitted. 5.6.1 Sampling Report November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 32 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies The Testing Organization Project Manager or designee shall prepare a report of each sampling event during the evaluation period following all sampling activities. This report shall consist of a brief summary of the major actions performed, any problems encountered since the previous report, and corrective actions taken to correct problems. This information shall be kept in project files along with the COC forms and the Field Log documenting the sampling activities. 5.6.2 Data Summary Report The laboratory shall provide tabulated summaries of the data to the Testing Organization in both electronic and hard copy format. The summaries will show the sample identifiers, the analyses performed, and the measured concentration or effects, including all relevant qualifiers and validation flags. A brief narrative statement on the overall data quality and quantity will also accompany the tabulated summaries. The Testing Organization Project Manager will coordinate with the laboratory project manager to define the format of these data summary reports. All data summary reports shall also be forwarded to the Verification Organization Project Manager following review by the Testing Organization Project Manager. 5.6.3 Operation and Maintenance Report The Testing Organization Project Manager or designee shall prepare a report of the operation and maintenance activities that were performed during the verification testing period. This report shall include a summary of the recommended operation and maintenance activities for the technology and any additional operation or maintenance tasks that were required during the test period. This report shall clearly delineate when the Vendor provided technical assistance to the Testing Organization. 5.6.4 Quality Control and Analytical Report This report shall be used to address the quality control practices employed during the project. It shall also summarize the problems identified in the sampling reports, which are likely to impact the quality of the data. The following required elements represent the minimum items to be included in the report: • A project description, including report organization and background information • Summaries of the sampling procedures, sample packaging, sample transportation, and decontamination procedures. • A summary of the laboratory analytical methods, detection limits, quality control activities, deviations from planned activities, and a summary of the data quality for each analysis and matrix. November 27, 2000 This document shall not be copied or distributed, in whole or in part, Page 33 of 41 without the written permission of NSF International. ------- Environmental Technology Verification Nutrient Reduction Technologies • An assessment of the sampling and analyses techniques, an evaluation of the data quality of each parameter, and an evaluation of the usability of the data. • A summary of the field or analytical procedures that could be changed or modified to better characterize the raw influent and treated effluent in future evaluations. • An overall discussion of the quality of the environmental data collected during the evaluation and whether or not it meets the project objectives. • Identification of the QA samples which were split and sent to the laboratory and to the QA laboratory. • All cooler receipt and COC forms associated with the required sample results. • A laboratory case narrative to be included in the results if nonconformances or other evaluation events affect the sample results. • The portion of the primary field sample results and associated batch QC results, which conform to the QA samples submitted to the QA laboratory. 5.7 Use of Existing Data Existing data may not be used as the sole basis for verification under this protocol. General conditions under which existing data may be used are described in Appendix A, however, the use of existing data shall be at the discretion and determination of the Verification Organization. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 34 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies CHAPTER 6.0 HEALTH and SAFETY MEASURES The safety procedures shall address safety considerations, which relate to the health and safety of personnel required to work on the site of the test equipment and persons visiting the site. Many of these items will be covered by site inspections and construction and operating permits issued by responsible agencies. They will include: • Regulations covering the storage and transport of chemicals. • Site specific spill response plan with respect to wastewater and any chemical usage. • Site specific health and safety plan addressing storage and handling of any chemicals. • Regulations regarding disposal of byproducts. • Conformance with the National Electric Code. • Provision of parking facilities, sanitary facilities and drinking water. • Provision of and access to fire extinguishers. • Regulations covering site security. • Conformance to any building permits requirement such as provision of handicap access or other health and safety requirements. • Ventilation of equipment or of trailers or buildings housing equipment, if gases generated by the equipment could present a safety hazard. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 35 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies APPENDIX A ENVIRONMENTAL TECHNOLOGY VERIFICATION PROGRAM EXISTING DATA: POLICY AND PROCESS Adapted from Appendix C of the Environmental Technology Verification Program Quality and Management Plan for the Pilot Period (1995-2000), National Risk Management Research Laboratory, National Exposure Research Laboratory, Office of Research and Development, U.S. Environmental Protection Agency, Cincinnati, Ohio 45268, EPA Report No. EPA/600/R-98/064. BACKGROUND The Environmental Technology Verification program was established by the U.S. EPA for the purpose of verifying the performance of commercial-ready technologies for their ability to monitor, prevent, control, or clean-up pollution. Verification is accomplished by the evaluation of objectively-collected, quality-assured data which are provided to potential purchasers and permitters as an independent and credible assessment of the performance of a technology. Data are collected and evaluated in partnership with independent third party verification partners chosen from the public sector (such as states), the private sector (such as non-profit research institutions), federal laboratories, and others. During the pilot phase (1994-2000), EPA provides oversight of the verification partner to assure the credibility of the process and data, and keeps the authority for the verification process and decision (except in the case of an independent pilot). After the pilot phase, responsibility and authority revert to the verification partner. The ETV program seeks to identify optimal methods to verify environmental technologies without compromising quality. Stakeholder groups, consisting of representatives of major verification customer groups, advise and assist EPA and the verification partners in this effort. One consistent and urgent request has been that existing data, i.e., data collected prior to the ETV program, be used for ETV verification. This suggestion is reinforced by the programs of individual states, as well as those of other countries, that routinely consider previously-collected data in the verification of Vendor claims for a technology. The purpose of this document is to establish a guideline whereby the ETV program may use these "historical," "existing," or "secondary" data to increase and enhance the scope of individual pilot projects. POLICY Currently, under the U.S. ETV program, the verification partner and the technology developers typically plan and execute tests, which provide the objective and quality-assured data by which the environmental technologies are evaluated. Existing data are used to support test plan development. Measurements and data are collected in a demonstration of the technology by the developer, under the direction of the verification partner, and overseen by EPA. Reports are peer-reviewed and verification statements are issued. In this closely-monitored scenario, the origin and quality of the data upon which the verification statement rests are generally known and documented, and therefore the possibility for verification decision error is minimized. The consequences of a serious verification decision error can include verification of fraudulent November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 36 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies claims, litigation, and loss of credibility for the ETV program, the verification partners, and EPA. Compelling arguments exist for considering using certain qualified existing data to replace some or all of the verification testing for a given technology. Some technologies are time-consuming and expensive to evaluate. Due to resource constraints, demonstrations can, at best, show the performance of the technology under only limited conditions. A test may provide only one small performance snapshot in time as opposed to providing data from several years of performance collected by the developer or his customers under a full range of conditions. Limited resources may require that testing focus on only one component of a technology rather than its full range of capability. Before coming to the commercially viable stage of development, these technologies may have been tested numerous times with acceptably reproducible results. Judicial precedent provides argument for the defensible use of existing data. In Daubert v. Merrill Dow Pharmaceuticals. Inc., the Supreme Court in 1993 adopted a new standard for the admissibility of scientific evidence. The Court there held that Federal Rule of Evidence 702 requires that, when presented with proposed scientific testimony, the district court must make a preliminary assessment of whether the reasoning or methodology underlying the testimony is scientifically valid, and therefore reliable. The Court declined to adopt a definitive checklist or test, but noted several factors a court should consider. Those factors include: (1) does the theory or technique involve testable hypotheses; (2) has the theory or technique been subject to peer review and publication; (3) are there known or potential error rates and are there standards controlling the technique's operation; and (4) is the method or technique generally accepted in the scientific community? The court must also consider the relevance or fit of the proposed testimony by determining if the reasoning and methodology can properly be applied to the facts at issue. The Clean Air Act Credible Evidence Revisions (see Federal Register, Vol. 62, No. 36, February 24, 1997) provide precedent within the Agency for defensible consideration of existing data for verification use. These revisions clarify that data from methods which are not EPA Standard Reference Methods can be used in enforcement actions and for compliance certification. Conversely, emission sources will be able to use any credible evidence (ACE) for contesting allegations of noncompliance in enforcement actions. As the rule states, it "exemplifies EPA's common sense" approach to environmental protection, which encourages smarter, cheaper and more flexible means of achieving environmental goals without compromising the fundamental health and environmental protections provided by federal environmental laws." It follows that if EPA can use ACE for enforcement actions, it can be considered for verification. Other precedent within the Agency exists at the Office of Air Quality Planning and Standards (OAQPS). OAQPS uses secondary data, defined as data that are utilized for a purpose other than that for which they were initially collected, in its regulatory efforts. In order to effectively focus its quality assurance (QA) efforts within the constraints of available resources, OAQPS concentrates its consideration of secondary data according to category of project. The QA activities associated with evaluating secondary data are conducted to assure that the data will be adequate and sufficient for their planned secondary use. November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 37 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies Recognizing therefore that it is neither prudent nor cost-effective to ignore existing data, the ETV program establishes by this document a consistent process to evaluate these data for the extent of their credibility and usability in the verification decision. Data to be considered for use to replace verification testing undergo a rigorous process of evaluation using stringent criteria. The following guidelines are used to qualify existing data for verification purposes (detailed procedures follow in the "process" section of this document): 1. Data are evaluated using qualified reviewers following the data evaluation process established in the "process" section of this document. 2. The documentation of the candidate data is sufficient to allow the reviewers to assess the quality of the data and its usability for verification. 3. The data are evaluated to determine that they meet the same minimum quality acceptance criteria as that collected in a comparable ETV pilot demonstration. 4. All of the data used for a verification must have been obj ectively collected, independently of the Vendor. 5. Only data collected under a well-defined, documented quality system will be considered. Such data sets should contain all the elements required to withstand peer review, and thus be useful for verification. Recognizing that useful data exist which will not qualify for verification under these guidelines, and responding to customer needs, individual pilots may establish individual evaluation criteria by which existing data may be considered. These data may not be used directly for verification, but may be used, for example, to support planning or to augment verification testing. No ETV program-wide guidelines are necessary for the use of existing data for purposes other than for verification. PROCESS Identifying and Qualifying the Data The Vendor proposes the data to be evaluated. EPA and the verification partner shall (with input from the stakeholder group, as applicable) identify for the Vendor the procedures and acceptance criteria used in the pilot demonstrations to evaluate technology performance. These procedures and criteria are the same as that used for other technologies evaluated by the verification partner. The data requirements are developed by EPA, the verification partner, and interested stakeholders for the pilot, and are not specific to the existing data. The Vendor and verification partner perform the initial evaluation. The Vendor shall provide the verification partner with the detailed protocols and test plans used to develop the existing data. The Vendor shall identify those data that he/she believes will meet the acceptance criteria, qualify those data, and submit the data along with detailed evidence that the data meet the requirements of the pilot project. The evidence shall be submitted to the EPA November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 38 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies and verification partner in a detailed report. The report shall show how the data verify the performance of the technology, identify data that were excluded, give an explanation of how and why they were excluded, and address other requirements specific to the pilot project. The Vendor shall be prepared to provide all of the raw data. The verification partner shall review the planning documents to determine whether they meet the requirements of those being used by the verification partner for evaluation tests of other technologies. At a minimum the existing data protocols and test plans shall require the same level of QA/QC, replicate tests, data treatment, and reporting as that required by the verification partner in its technology demonstrations. The verification partner shall conduct a detailed review of the Vendor's data report to determine whether the data adequately evaluate the performance of the technology. The verification partner has access to the raw data and works through a reasonable random sample (suggest 10% of the data). A recommended method for evaluation of data is tracing a random selection of data points from the raw data set to the final report. Minimum General Acceptance Criteria • The technology is based on sound scientific and engineering principles. • The conditions under which the data were collected are clearly defined and were appropriate for the demonstration of the capabilities of the technology. • The data are quality assured. For example, where appropriate, the documentation provides a measure of the bias and precision of the measurements. Where needed, minimum detection limits have been determined and reported. Where applicable, the measurement range of the technology is given. A narrative statement will include a discussion of how well the data represent the capabilities of the technology in its intended environmental application • Sufficient data are supplied to allow the technology to be verified. Sufficiency of the data will be determined by the reviewers. • Vendor-generated data may be reviewed as part of the evaluation process because it is a rich source of knowledge about the technology. Only data collected objectively and independently of the Vendor, however, may be used to replace verification testing. Specific Acceptance Criteria In addition to the general acceptance criteria, the specific pilot project stakeholders may impose specific acceptance criteria which must be as stringent as the acceptance criteria for the data collected during verification testing. Convening the Data Evaluation Panel If the verification partner determines that the report does not adequately evaluate the performance of the technology, the Vendor is notified and no further action is required. If the verification partner determines that the Vendor's report does adequately evaluate the performance of the technology, then a data evaluation panel (DEP) is appointed. The verification partner enlists the services of 3 qualified reviewers to serve on the DEP. During the pilot phase of the ETV program, the DEP will generally consist of one person from EPA, one person from the verification partner, and one person who is an outside expert in the technology being evaluated. The DEP must contain members who are credible, experienced, knowledgeable, and November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 39 of41 ------- Environmental Technology Verification Nutrient Reduction Technologies qualified in the technical areas critical to the technology being evaluated. The members of the DEP must be objective and have no real or perceived conflict of interest with the commercial developer of the technology they are evaluating. DEP members must be independent; they cannot have been involved in the collection of the data being evaluated. Evaluation of the Data by the DEP The DEP reviews and agrees on the acceptance criteria and determines their applicability to the data to be evaluated. The evaluation shall follow the procedures and criteria developed by the verification partner and EPA for other technology verifications conducted in the pilot project. The verification partner provides a written summary of its review to the DEP. When the verification partner submits the data to the DEP, it ceases to be proprietary. The DEP reviews and evaluates the data using the applicable acceptance criteria.. The DEP determines that the data were gathered following appropriate test protocols similar to the protocol used for verification testing. It ensures that the data were gathered following written test plans developed using a similar protocol. Planning must have included specific test objectives, experimental design, criteria for data quality, QA/QC procedures followed and reported, number of samples or frequency of sampling, and sampling and analytical procedures. The DEP must determine that the data quality meets or exceeds the minimum data quality requirements of the verification testing conducted during the pilot. The quality and usability of the existing data shall be evaluated against clearly defined data quality requirements based on the data quality requirements of the ETV pilot project. The data shall be sufficient to evaluate the performance of the technology. Recommendations for Acceptance of Data for Verification Role The DEP shall prepare a report on its findings. At a minimum the report must address the following: • Were the data collected by following the protocol and test plan provided by the Vendor? • Do the data meet the minimum QA/QC requirements of the ETV pilot project demonstrations? • Do the data adequately evaluate the performance of the technology? Are there enough data, and are the data of sufficient quality for the verification partner, the ETV program, and EPA to place their reputations on the line? The DEP provides a written statement of the performance of the technology as provided by the data, a statement of how well the data meet the acceptance criteria, and a data acceptance recommendation. Review and Acceptance of Recommendation by Verification Partner and EPA The EPA reviews the report, determines whether to accept the data acceptance recommendation, and signs the verification statement. It is suggested that testing entities having a quality system which is modeled after the American National Standard Institute/American Society for Quality Control (ANSI/ASQC) Standard E-4- November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 40 of 41 ------- Environmental Technology Verification Nutrient Reduction Technologies 1994, Specifications and Guidelines for Quality Systemsfor Environmental Data Collection and Environmental Technology Programs, or the International Organization for Standardization (ISO) Standard 9000, Quality Management and Quality Assurance Standards: Guidelines for Selection and Use, may have appropriate quality systems. Other similar quality systems may be accepted at the discretion of the reviewers. last revised October 8,1998 November 27, 2000 This document shall not be copied or distributed, in whole or in part, without the written permission of NSF International. Page 41 of 41 ------- |