Office of Research and Development National Exposure Research Laboratory Interim Final Characterization Research Division October 1996 P.O. Box 93478 Las Vegas, NV 89193-3478 TECHNOLOGY A Guidance Manual for the Preparation of Site Characterization and Monitoring Technology Demonstration Plans ^ ^m y *^r 55 o •z- *. ------- A Guidance Manual for the Preparation of Characterization and Monitoring Technology Demonstration Plans Prepared by Stephen Billets, Gary Robertson and Eric Koglin Consortium for Site Characterization Technology Characterization Research Division-Las Vegas National Exposure Research Laboratory Office of Research and Development U.S. Environmental Protection Agency October 31,1996 Interim Final Report Version 5.0 ------- NOTICE The information in this document has been developed by the U.S. Environmental Protection Agency (EPA). It has not been subjected to the Agency's external review process. The policies and procedures set forth here are intended as guidance to Agency and other government and non- government personnel. They do not constitute rule making by the Agency, and may not be relied upon to create a substantive or procedural right enforceable by any other person. The Government may take action that is at variance with the policies and procedures in this document. Mention of trade names or commercial products does not constitute endorsement or recommendation for use. October 31, 1996 ii Interim Final Report Version 5.0 ------- FOREWORD This work represents the technical and editorial contributions from a large number of U.S. Environmental Protection Agency (U.S. EPA) employees and others familiar with or interested in the demonstration and evaluation of innovative site characterization and monitoring technologies. The Characterization Research Division - Las Vegas (CRD-LV) first convened a body of experts—the Consortium Action Team—to define the elements of this guidance document. Subsequent discussions and meetings were held to revise and expand the contents to create this version. EPA staff from each of the ten Regions, the Office of Solid Waste and Emergency Response, and the Office of Research and Development participated in this process. This interdisciplinary, inter-programmatic team was convened to ensure that the demonstration procedures articulated are acceptable across the Agency. This was an important first step for gaining the acceptance of innovative technologies for use in characterizing and monitoring the environment. October 31,1996 iii Interim Final Report Version 5.0 ------- This page intentionally left blank. October 31, 1996 iv Interim Final Report Version 5.0 ------- TABLE OF CONTENTS NOTICE ii FOREWORD iii LIST OF ACRONYMS ix CHAPTER 1 INTRODUCTION 1-1 1.1 Purpose and Content of this Guidance Manual 1-1 1.2 The Consortium for Site Characterization Technology 1-2 1.2.1 Background 1-2 1.2.2 What is the Consortium for Site Characterization Technology? 1-3 1.2.3 Using the Consortium Process 1-5 1.2.4 Roles and Responsibilities 1-6 CHAPTER 2 HOW TO USE THIS GUIDANCE MANUAL 2-1 2.1 Elements of a Demonstration Plan 2-1 2.2 Using Chapter 3 to Build a Demonstration Plan 2-1 CHAPTER 3 ELEMENTS OF A TECHNOLOGY DEMONSTRATION PLAN 3-1 TITLE PAGE 3-1 FOREWORD 3-1 TABLE OF CONTENTS 3-1 EXECUTIVE SUMMARY 3-1 ABBREVIATIONS AND ACRONYMS 3-2 1.0 INTRODUCTION 3-2 1.1 Demonstration Objectives 3-2 1.2 What is the Consortium for Site Characterization Technology? 3-2 1.3 Technology Verification Process 3-3 1.4 Purpose of this Demonstration Plan 3-3 2.0 DEMONSTRATION RESPONSIBILITIES AND COMMUNICATION ........ 3-3 October 31, 1996 v Interim Final Report Version 5.0 ------- 2.1 Demonstration Organization and Participants 3-4 2.2 Organization 3.4 2.3 Responsibilities 3.4 3.0 TECHNOLOGY DESCRIPTION 3-5 3.1 Technology Performance 3-6 3.2 History of the Technology 3-6 3.3 Technology Applications 3-6 3.4 Advantages of the Technology 3-6 3.5 Limits of the Technology 3-7 3.6 Demonstration Performance Goals 3-7 4.0 DEMONSTRATION SITE DESCRIPTIONS 3-8 4.1 Site Name and Location 3-8 4.2 Site History 3-8 4.3 Site Characteristics 3-8 5.0 CONFIRMATORY PROCESS 3-8 5.1 Method Selection 3-9 5.2 Reference Laboratory Selection 3-9 5.3 Contingency Laboratory Selection 3-9 5.4 Special QC Requirements 3-9 5.5 Laboratory Audit 3-10 5.6 Statistical Analysis of Results 3-10 5.6.1 Methods of Data Reduction and Adjustment 3-10 5.6.2 Methods of Statistical Analysis 3-10 6.0 DEMONSTRATION DESIGN 3-11 6.1 Objectives 3-11 6.2 Experimental Design 3-11 6.2.1 Qualitative Factors 3-11 6.2.2 Quantitative Factors 3-12 6.3 Sampling Plan 3-12 6.3.1 Sampling Operations 3-12 6.3.2 Predemonstration Sampling and Analysis 3-13 6.4 Field Data 3-14 6.4.1 Field Audit 3-14 6.5 Demonstration Schedule 3-14 7.0 FIELD OPERATIONS 3-14 7.1 Communication and Documentation 3-15 7.2 Sample Collection Procedures 3-15 7.2.1 Sampling Procedures 3-16 7.2.2 Sampling Locations 3-16 7.2.3 Sample Preparation 3-16 7.2.4 Sample Distribution 3-16 7.2.5 Decontamination and Disposal 3-16 October 31, 1996 vi Interim Final Report Version 5.0 ------- 7.3 Personnel Requirements 3-17 7.4 Technology Logistical Needs 3-17 7.4.1 Special Needs 3-17 8.0 QUALITY ASSURANCE PROJECT PLAN 3-17 8.1 Purpose and Scope 3-17 8.2 Quality Assurance Responsibilities 3-17 8.3 Data Quality Indicators 3-18 8.3.1 Representativeness 3-18 8.3.2 Comparability 3-19 8.3.3 Completeness 3-19 8.3.4 Accuracy 3-20 8.3.5 Precision 3-20 8.4 Calibration Procedures and Quality Control Checks 3-21 8.4.1 Initial Calibration Procedures 3-21 8.4.2 Continuing Calibration Procedures 3-21 8.4.3 Method Blanks 3-22 8.4.4 Spike Samples 3-22 8.4.5 Laboratory Control Samples 3-22 8.4.6 Performance Evaluation Materials 3-22 8.4.7 Duplicate Samples 3-22 8.5 Data Reduction, Validation, and Reporting 3-23 8.5.1 Data Reduction 3-23 8.5.2 Data Validation 3-23 8.5.3 Data Reporting 3-23 8.6 Calculation of Data Quality Indicators 3-24 8.7 Performance and System Audits 3-24 8.7.1 Performance Audit 3-24 8.7.2 On-Site System Audits 3-24 8.8 Quality Assurance Reports 3-24 8.8.1 Status Reports 3-24 8.8.2 Audit Reports 3-25 8.9 Corrective Action 3-25 9.0 DATA MANAGEMENT AND ASSESSMENT 3-25 10.0 HEALTH AND SAFETY PLAN 3-26 10.1 Health and Safety Plan Enforcement 3-26 October 31, 1996 vii Interim Final Report Version 5.0 ------- LIST OF FIGURES Figure 1-1. Technology Demonstration Process 1-4 Figure 2-1. Elements of a Technology Demonstration 2-1 Figure 2-2. Table of Contents from a Typical Technology Demonstration Plan 2-2 Figure 3-1. Typical Table of Contents from a Health and Safety Plan 3-27 APPENDICES APPENDIX A Description of The Environmental Monitoring Management Council A-l APPENDIX B Environmental Monitoring Management Council Method Format B-l APPENDIX C Office of Solid Waste Method Requirements C-l APPENDIX D Representative Demonstration Schedule D-l APPENDIX E Guidance for Addressing Quality Assurance/Quality Control Requirements in a Demonstration Plan E-l October 31, 1996 viii Interim Final Report Version 5.0 ------- LIST OF ACRONYMS ADQ Audit of Data Quality AOAC Association of Official Analytical Chemists ASTM American Society for Testing and Materials CAS Chemical Abstract Service CERCLA Comprehensive Environmental Response, Compensation, and Liability Act (a.k.a. Superfund) CRD-LV Characterization Research Division-Las Vegas CSCT Consortium for Site Characterization Technology DoD U.S. Department of Defense DOE U.S. Department of Energy EMMC Environmental Monitoring Management Council ETV Environmental Technology Verification Program EPA U.S. Environmental Protection Agency ETT Environmental Technology Initiative FPXRF Field portable X-ray fluorescence spectrometer GC/MS Gas chromatography/mass spectrometer HASP Health and Safety Plan ITVR Innovative technology verification report LCS Laboratory Control Sample NERL National Exposure Research Laboratory ORD Office of Research and Development OSW Office of Solid Waste OSWER Office of Solid Waste and Emergency Response PEA Performance Evaluation Audit PE Performance Evaluation QA Quality assurance QAPP Quality Assurance Project Plan QC Quality Control RA Regional Administrator RCI Rapid Commercialization Initiative RCRA Resource Conservation and Recovery Act RPD Relative Percent Deviation RSD Relative Standard Deviation SOP Standard Operating Procedure SPC Science Policy Council SW-846 Test Methods for Evaluating Solid Waste, EPA Publication SW-846, Third Edition TER Technology Evaluation Report TSA Technical System Audit October 31, 1996 IX Interim Final Report Version 5.0 ------- This page intentionally left blank. October 31, 1996 x Interim Final Report Version 5.0 ------- CHAPTER 1 INTRODUCTION 1.1 Purpose and Content of this Guidance Manual The purpose of this manual is to provide guidance to technology developers in preparing a demonstration plan for the performance testing of characterization, monitoring, measurement, and sampling technologies. The purpose of the demonstration plan is to assure that demonstrations will be performed in a manner that generates the data needed to verify the performance of the technology. Another purpose of the plan is to assure that all appropriate health, safety, regulatory, quality assurance, and environmental concerns related to the demonstration are addressed. This guidance document is divided into three chapters. Chapter 1 (this chapter) provides an overview of the purpose of the Consortium for Site Characterization Technology (CSCT) and a description of the technology demonstration process. Chapter 2 contains a description of how to use this guidance manual, and an introduction to Chapter 3. Chapter 3 provides detailed guidance on how to prepare a technology demonstration plan. The guidance is in the form of an annotated outline. Each section of the plan is identified as a subsection of the chapter. Each subsection contains a short description of the information that should be included in the demonstration plan. The use of this standard structure will facilitate document preparation and reduce the amount of review time required for plan approval. This approach will also help the EPA point of contact or technical expert provide timely assistance to the demonstration plan authors. There are five appendices to this document. Appendix A provides a description of the Environmental Monitoring Management Council (EMMC) and its proposed role in helping the Consortium achieve Agency-wide approval. Appendix B describes the standard EMMC format which is used to document EPA's environmental monitoring methods. This format is used to standardize the method used during the demonstration and for subsequent Agency approval, if appropriate. Appendix C provides an overview of the expectations that the EPA's Office of Solid Waste has regarding the acceptance of a method into S W-846, Test Methods for Evaluating Solid Waste. A single demonstration may not produce sufficient information for approval. This appendix describes additional information which may be needed to support method approval. Appendix D provides a representative schedule of key events in the demonstration process. Appendix E is a condensation of EPA document Preparation Aids for the Development of Category II Quality Assurance Project Plans. This condensed version contains those portions that are applicable to the technology demonstration process. Site characterization and monitoring instruments can include a diverse assortment of technologies. These can range from immunoassay test kits to field portable instrumentation (for example, x-ray fluorescence spectrometers). Most, if not all, of the demonstration plan elements described in Chapter 3 will be applicable to all types of technologies. However, there are often special conditions and concerns that are unique to a particular type of technology. These can be addressed on a case by case basis following the framework presented in Chapter 3. October 31, 1996 1-1 Interim Final Report Version 5,0 ------- 1.2 The Consortium for Site Characterization Technology (CSCT) 1.2.1 Background The quick acceptance and application of innovative monitoring and site characterization technologies are goals shared by many. Potential users of innovative approaches must be persuaded that new technologies perform as well as or better than conventional methods. This is particularly important when considering that data are being collected to support important decisions (for example, protection of human health and the environment, remedy selection, enforcement, or litigation). Currently, most information about the performance of innovative technologies comes from the vendor or developer. However, a user's confidence and willingness to apply an innovative technology is more likely following EPA recognition and acceptance. The user community looks to the EPA, because of its regulatory mission, to evaluate innovations that will improve the way the Nation manages its environmental problems. Potential users may find new technologies appealing, but without government acceptance of such technologies, users will continue to rely on already accepted approaches, whether or not they are the most appropriate or cost-effective. In 1995, on the 25th anniversary of the first Earth Day, the President announced a new environmental technology strategy, Bridge to a Sustainable Future. This government-wide strategy recognizes that industry is the primary creator of new technology and the main engine of sustained economic growth. It assigns to government a catalytic role in promoting the development of new technologies across a range of environmental sectors. The Consortium plays a major part in the fulfillment of three of the 15 initiatives found in the Strategy: • The Environmental Technology Initiative (ETI) seeks to accelerate environmental protection, strengthen America's industrial base, and increase exports of U.S. technologies and expertise. The Consortium's primary source of funding comes from ETI. • The Environmental Technology Verification Program (ETV) is a new program to verify the performance and cost of new technologies through partnership with public and private third-party testing organizations. EPA sponsored verification of performance, as new technologies reach commercialization stage, is expected to speed the adoption of innovative methods both at home and abroad. The Consortium is the first often pilots to come on line under the ETV program. The Rapid Commercialization Initiative (RCI) is a partnership between EPA, the Departments of Commerce, Defense, and Energy, and several multi-State organizations to combine facilitated siting, performance verification, and expedited permitting to speed environmental technology into use. The Consortium participates in the verification of characterization and monitoring technologies in this program. October 31, 1996 1-2 Interim Final Report Version 5.0 ------- 1.2.2 What is the Consortium for Site Characterization Technology? As noted above, the Consortium for Site Characterization Technology (CSCT) is one of the ETV pilots being supported with ETI resources. The Consortium is a partnership between, at present, the EPA, and the Departments of Energy and Defense (DOE and DoD). DOE and DoD have established programs and facilities (testing venues) for testing, demonstrating, and evaluating the performance of monitoring, measurement, and site characterization technologies. Our primary partner, DOE's Oak Ridge National Laboratories, Oak Ridge, Tennessee, is serving as a technology verification organization. A technology verification organization conducts such activities as: assist EPA in formulating generic guidance; assist developers in designing demonstration plans; oversee actual testing of technologies; conduct quality assurance/quality control (QA/QC) activities; and submit reports to EPA on technology performance and cost. As a partnership, the Consortium offers valuable technical expertise to support the demonstration and verification of the performance of new and emerging technologies and offers access to a wide array of testing venues. A goal of the Consortium is to facilitate the acceptance and use of cost-effective technologies applicable to a wide range of environmental problems. The Consortium will meet this goal by working with technology developers and other agencies in planning and conducting demonstrations, evaluating data generated in demonstrations, and managing and disseminating information. The Consortium is not intended to become another technology testing organization that must touch every technology, but rather it is designed to support existing demonstration efforts and developer-driven demonstrations. The Consortium does not offer any financial support to those desiring to conduct a technology demonstration. The developer is expected to secure the appropriate resources to support their part of the technology verification process. These responsibilities are described in Section 1.2.4. The Consortium provides developers with a clearly defined technology verification pathway to follow, from demonstration planning to data evaluation and verification (Figure 1-1). The technology demonstration/verification process established by the Consortium is intended to serve as a template for conducting technology demonstrations that will generate high-quality data needed by the Agency to verify technology performance. The Consortium verification process is a model process that can help in moving innovative site characterization and monitoring technologies into routine use more quickly. The Consortium provides cost and performance data to technology users and developers. A major product of the Consortium is the preparation of reports that contain the data generated for each technology demonstration and an assessment of the technology's performance. This important product should increase the users' confidence in employing innovative technologies for site characterization and monitoring. A key part of the technology verification report is a verification statement. The verification statement is a summary of the findings of the demonstration which describes how well the technology achieved the performance goals the demonstration was designed to test. It also provides a discussion of the advantages and limitations of the technology and defines representative conditions under which the technology can be applied. October 31, 1996 1-3 Interim Final Report Version 5.0 ------- ENVIRONMENTAL TECHNOLOGY DEMONSTRATION AND VERIFICATION PROCESS ! Demonstration ; Plan Developer with Assistance from a Verification Organization or Independent Third Party (with EPA Review and Approval) * Demonstration i -> Developer with Assistance from a Verification Organization or Independent Third Party (with EPA Oversight) r i Verification > Ml Report Verification Organization (with EPA Oversight and Approval) i Verification Statement Prepared by EPA and sent to: Developer EPA Regions Program Officers Other Federal Agencies States Public Figure 1-1. Technology Demonstration Process The Consortium is not a panacea for developers and users of new and emerging monitoring, measurement, and site characterization technologies trying to gain acceptance or commercialize a technology. However, the Consortium attempts to fill many technical and institutional needs. These include: • Providing a sound scientific basis for demonstrating and evaluating technology performance; • Facilitating acceptance of innovative technologies by state, local, and Federal regulators; • Supporting the implementation and use of verified technologies; • Identifying and meeting changing user needs; • Increasing the number and commercial availability of innovative technologies; • Providing a mechanism to "pull" technologies being developed by DoD, DOE, and other public and private entities into routine use at a faster rate; • Providing an incentive for developers to push the state of the technology beyond present capabilities; • Leveraging resources and expertise among Federal agencies, the private sector, and academia. Although the Agency is interested in any and all innovative site characterization and monitoring technologies, the Consortium resources, and those of the verification organization, are limited. Therefore, a major role of the Consortium is to identify the technology and data gaps that impede cost-effective and efficient environmental problem-solving and to communicate them to the developer community. This assessment will allow us to prioritize those technology October 31, 1996 1-4 Interim Final Report Version 5.0 ------- demonstrations that address the most pressing needs and gaps. The information that supports the assessment will be gathered from within EPA, other Federal agencies, states, tribes, and the user industries. 1.2.3 Using the Consortium Process The Consortium conducts field demonstrations of technologies that are determined to be of appropriate maturity and of significant benefit in environmental problem-solving. The Consortium asks the developer to provide a profile of the technology which will be used to determine if the technology will be tested and evaluated or demonstrated and verified. That determination will be based on two factors: the level of maturity and the attributes of the technology. The Consortium has defined three levels of maturity: »• Level I - Laboratory prototype which will be field tested for the first time or is in the earliest stage of development f Level 2 - Pre-production prototype of a technology that has been field tested but is not yet commercially available. > Level 3 - Commercial production unit The attributes used to assess maturity include: Commercial history (number of units sold, years on the market.); • Technology performance (accuracy, sensitivity, selectivity, stability, etc.); • Technology applications (under what conditions has the technology been used, the type(s) of site it was used at); • Costs of technology acquisition, installation, and operation; Field experience (list of field uses, i.e., number of sites, locations, customers, etc.). If the technology is acceptable, an EPA contact (a Regional representative or technical expert) is assigned to facilitate the demonstration process. The developer is also be assigned a verification organization which assists in developing the demonstration plan, staging the demonstration, and which prepares the technology verification report. The demonstration/verification process applies only to those technologies that fit into Level 3. We feel that it is important, if not imperative, for the CSCT to nurture emerging technologies as well as those in or near commercial production. Level 1 and Level 2 technologies are clearly not ready for a rigorous demonstration and verification, as the developer is usually still in the process of exploring the potential of their innovation. However, developers of Level 1 and Level 2 technologies are faced with a need to try their technologies in the field and to have a credible body, particularly the EPA, participate with and assist them in testing and evaluation. Therefore, the CSCT also administers a scaled-down version of the demonstration/verification process - a testing and evaluation process - that helps the Level 1 or Level 2 developer with access to sites; provides limited review of their test plan; provides limited oversight of their field activities, and October 31, 1996 1-5 Interim Final Report Version 5.0 ------- that serves as a distribution point for the information about the technology. Under a Level 1 demonstration, the developer is responsible for a self-evaluation of their performance. The CSCT does not provide an independent evaluation. In a Level 2 demonstration, a more formal procedure is followed and the EPA or its verification organization will review the performance of the technology and issue an evaluation report. 1.2.4 Roles and Responsibilities Consortium: The Agency and its verification organizations are responsible for the following activities: Defining the readiness of a technology for demonstration; • Assisting the developer with the preparation of the demonstration plan (in accordance with the guidelines presented in this document); • Assisting in selecting the demonstration site; • Approving the demonstration plan prior to the demonstration; • Participating in the demonstration to monitor the implementation of the demonstration plan; • Evaluating and verifying the data and information generated in the demonstration; • Preparing the technology verification report and issuing a verification statement. Developer: The developer is responsible for: • Contributing to the development of the demonstration plan; • Articulating the performance goals of the technology; Mobilizing the technology to the demonstration site; • Operating the technology during the demonstration; • Collecting data in accordance with the demonstration plan; • Contributing to the cost of referee analysis (optional); Providing the data and information generated during the demonstration to the Agency or the verification organization in the prescribed format. October 31, 1996 I-6 Interim Final Report Version 5.0 ------- CHAPTER 2 HOW TO USE THIS GUIDANCE MANUAL 2.1 Elements of a Demonstration Plan The purpose of the demonstration process is to generate the data and information necessary to fairly and thoroughly evaluate the performance of a technology. Figure 2-1 depicts the steps in a technology demonstration and some of the activities associated with each. This guidance manual focuses primarily on the activities associated with the planning and demonstration steps. Planning j ; Demonstration ! Data Review & Evaluation * Quality Assurance * Sample Collection * Data/Information Reduction * Experimental Design * Sample Analysis (field) * Data Interpretation * Sampling & Analysis Plan * Referee Analysis (laboratory) * Report Findings * Articulate Performance Goals * Field Data Management » Method or SOP * QA/QC Materials * Selection of Demonstration * Photodocumentation Site(s) * Data Management Plan * Coord. Regional Participation * Predemonstration Testing Figure 2-1. Elements of a Technology Demonstration The demonstration plan serves a number of purposes. First, it provides a "roadmap" for the demonstration. It contains detailed guidance for those executing the demonstration on how data need to be generated and collected to support an objective performance evaluation of the technology. Second, it is an important reference for those who choose to review the steps used by the developer in executing the demonstration to assess the validity of the process. Finally, it can serve as a useful reference for other technology developers in building future demonstration plans involving related technologies. 2.2 Using Chapter 3 to Build a Demonstration Plan The section order and content specified in Chapter 3 of this manual should be followed. Figure 2-2 is a typical Table of Contents for a demonstration plan. It is derived from the section headings in Chapter 3. The preparer of the demonstration plan should be working with the assigned verification organization and/or an EPA contact. This is required so that all demonstration activities under the Consortium are coordinated and the appropriate oversight provided. The user of Chapter 3 must be aware of the font appearance conventions used to distinguish text that is intended to provide guidance from text that can be directly included in the plan. The portions of the manual with the font having a normal appearance (the words in this sentence are normal) are intended to be included directly into the demonstration plan, assuming the narrative is appropriate to the demonstration. In places where there is a name, date, single word, or phrase to be inserted, it will appear in bold. Finally, the text in italics is intended to serve as October 31, 1996 2-1 Interim Final Report Version 5.0 ------- guidance to the user in how to prepare the specific section. In an effort to make the demonstration/verification process as expedient as possible, an electronic copy of Chapter 3 can be provided. The electronic copy contains all the headings, sub-headings, etc., identified in Figure 2-2, and the necessary language that is common to all demonstrations. The developer will need to fill in the technology-specific information and the EPA or the verification organization will provide those portions identified in Chapter 3. Variations in the content of a technology-specific demonstration plan are expected, since different technologies have different characteristics and needs. For example, some field analytical technologies will have a directly corresponding laboratory method for referee analysis, while others, such as down-hole geophysical measurements, may require other methods of confirming performance. Typically, demonstration plans are 50 - 100 pages in length, plus appendices, as appropriate. Figure 2-2. Table of Contents from a Typical Technology Demonstration Plan TITLE PAGE FOREWORD TABLE OF CONTENTS EXECUTIVE SUMMARY ABBREVIATIONS AND ACRONYMS 1.0 INTRODUCTION 1.1 Demonstration Objectives 1.2 What is the Consortium for Site Characterization Technology? 1.3 Technology Verification Process 1.4 Purpose of this Demonstration Plan 2.0 DEMONSTRATION RESPONSIBILITIES AND COMMUNICATION 2.1 Demonstration Organization and Participants 2.2 Organization 2.3 Responsibilities 3.0 TECHNOLOGY DESCRIPTION 3.1 Technology Performance 3.2 History of the Technology 3.3 Technology Applications 3.4 Advantages of the Technology 3.5 Limits of the Technology 3.6 Demonstration Performance Goals 4.0 DEMONSTRATION SITE DESCRIPTIONS 4.1 Site Name and Location 4.2 Site History 4.3 Site Characteristics October 31, 1996 2-2 Interim Final Report Version 5.0 ------- Figure 2-2. Table of Contents from a Typical Technology Demonstration Plan Cont'd 5.0 CONFIRMATORY PROCESS 5.1 Method Selection 5.2 Reference Laboratory Selection 5.3 Contingency Laboratory Selection 5.4 Special QC Requirements 5.5 Laboratory Audit 5.6 Statistical Analysis of Results 5.6.1 Methods of Data Reduction and Adjustment 5.6.2 Methods of Statistical Analysis 6.0 DEMONSTRATION DESIGN 6.1 Objectives 6.2 Experimental Design 6.2.1 Qualitative Factors 6.2.2 Quantitative Factors 6.3 Sampling Plan 6.3.1 Sampling Operations 6.3.2 Predemonstration Sampling and Analysis 6.4 Field Data 6.4.1 Field Audit 6.5 Demonstration Schedule 7.0 FIELD OPERATIONS 7.1 Communication and Documentation 7.2 Sample Collection Procedures 7.2.1 Sampling Procedures 7.2.2 Sampling Locations 7.2.3 Sample Preparation 7.2.4 Sample Distribution 7.2.5 Decontamination and Disposal 7.3 Personnel Requirements 7.4 Technology Logistical Needs 7.4.1 Special Needs 8.0 QUALITY ASSURANCE PROJECT PLAN 8.1 Purpose and Scope 8.2 Quality Assurance Responsibilities 8.3 Data Quality Indicators 8.3.1 Representativeness 8.3.2 Comparability 8.3.3 Completeness 8.3.4 Accuracy 8.3.5 Precision 8.4 Calibration Procedures and Quality Control Checks 8.4.1 Initial Calibration Procedures October 31, 1996 2-3 Interim Final Report Version 5.0 ------- Figure 2-2. Table of Contents from a Typical Technology Demonstration Plan Cont'd 8.4.2 Continuing Calibration Procedures 8.4.3 Method Blanks 8.4.4 Spike Samples 8.4.5 Laboratory Control Samples 8.4.6 Performance Evaluation Materials 8.4.7 Duplicate Samples 8.5 Data Reduction, Validation, and Reporting 8.5.1 Data Reduction 8.5.2 Data Validation 8.5.3 Data Reporting 8.6 Calculation of Data Quality Indicators 8.7 Performance and System Audits 8.7.1 Performance Audit 8.7.2 On-Site System Audits 8.8 Quality Assurance Reports 8.8.1 Status Reports 8.8.2 Audit Reports 8.9 Corrective Action 9.0 DATA MANAGEMENT AND ASSESSMENT 10.0 HEALTH AND SAFETY PLAN 10.1 Health and Safety Plan Enforcement October 31, 1996 2-4 Interim Final Report Version 5.0 ------- CHAPTER 3 ELEMENTS OF A TECHNOLOGY DEMONSTRATION PLAN After a technology has been accepted into the CSCT Demonstration Program, a letter agreement will be signed by the developer and the CSCT. The purpose of the agreement is to specify a framework of responsibilities for conducting the demonstration and evaluating the technology. Several activities need to be conducted by the EPA, the developer, the verification organization, and other demonstration participants before the demonstration begins. Response to the following issues will be required during the preparation of the demonstration plan: • Identification of demonstration sites that provide the appropriate analytes in the desired environmental sample media or matrices (contaminants must be present in concentrations amenable to the technology being evaluated) Definition of the roles of appropriate demonstration participants Arranging appropriate analytical support for reference laboratory testing • Supplying standard operating procedures (SOPs), analytical methodologies (in the Environmental Monitoring Management Council format - Appendix B), and other relevant protocols It is important to note that the entire demonstration plan, incorporating a Quality Assurance Project Plan (QAPP), must be approved by the developer, verification organization, and EPA before the demonstration can proceed. Note: The section numbers shown in this chapter are used to indicate the order of these headings in the actual demonstration plan. TITLE PAGE FOREWORD TABLE OF CONTENTS Provide a Table of Contents for the Demonstration Plan. It should include the headings provided in this manual although they may be modified as appropriate for a particular technology demonstration. An example Table of Contents is provided as Figure 2-2. EXECUTIVE SUMMARY Describe the contents of the demonstration plan (not to exceed two pages). Include a summary description of the technology and the performance goals which will be verified during the demonstration, the demonstration sites, a schedule, and a list of participants. October 31, 1996 3-1 Interim Final Report Version 5.0 ------- ABBREVIATIONS AND ACRONYMS Provide a list of the abbreviations and acronyms used in the demonstration plan. 1.0 INTRODUCTION This chapter discusses the purpose of the demonstration and the demonstration plan, describes the elements of the demonstration plan and provides an overview of the Consortium for Site Characterization Technology (CSCT) and the technology verification process. 1.1 Demonstration Objectives Discuss the reasons why this demonstration will be performed. Reasons may include: • Generate field data appropriate for verifying the performance of the technology • Evaluate new advances in specific technology (e.g., field portable X-ray fluorescence spectrometer (FPXRF)) Specifically, this plan defines the following elements of the demonstration: • Roles and responsibilities of demonstration participants; • Procedures governing demonstration activities such as sample collection, preparation, analysis, data collection, and interpretation; • Experimental design of the demonstration; Quality assurance (QA) and quality control (QC) procedures for conducting the demonstration and for assessing the quality of the data generated from the demonstration (See Appendix E); and, • Health and safety requirements for performing work at hazardous waste sites. 1.2 What is the Consortium for Site Characterization Technology? The Consortium for Site Characterization Technology (CSCT) is a partnership between the EPA, and the Departments of Energy and Defense. DoD and DOE have established programs and facilities (testing venues) for testing, demonstrating, and evaluating the performance of monitoring, measurement and site characterization technologies, among other technologies. As a partnership, the Consortium will offer valuable technical expertise to support the demonstration and verification of the performance of new and emerging technologies and will offer access to a wide array of testing venues. A goal of the Consortium is to facilitate the acceptance and use of cost-effective technologies applicable to a wide range of environmental problems. The Consortium will meet this goal by working with technology developers and other agencies in planning and conducting October 31, 1996 3-2 Interim Final Report Version 5.0 ------- demonstrations, evaluating data generated in demonstrations and managing and disseminating information. The Consortium is not intended to become another technology testing organization that must touch every technology, but rather it is designed to support existing demonstration efforts or developer-driven demonstrations. The Consortium does not offer any financial support to those desiring to conduct a technology demonstration. The developer is expected to secure the appropriate resources to support their part of the technology verification process. 1.3 Technology Verification Process The technology verification process established by the Consortium is intended to serve as a template for conducting technology demonstrations that will generate high quality data that the Agency can use to verify technology performance. The Consortium verification process is a model process that can help in moving innovative site characterization and monitoring technologies into routine use more quickly. After the completion of the selection process, the verification of a technology's performance involves five steps: 1. Development of a demonstration/test plan. 2. Execution of the demonstration. 3. Data reduction, analysis, and cost verification. 4. Report preparation 5. Information transfer. Although the Agency is interested in any and all innovative site characterization and monitoring technologies, the Consortium resources, and those of the verification organization, are limited. Therefore, a major role of the Consortium is to identify the technology and data gaps that impede cost-effective and efficient environmental problem-solving and to communicate them to the developer community. This assessment identifies those technologies that meet the most pressing needs. The information that supports the assessment will be gathered from within EPA, other Federal agencies, states, tribes, and the user industries to ensure that the most pressing needs and gaps are addressed first. 1.4 Purpose of this Demonstration Plan The purpose of the demonstration plan is to describe the procedures that will be used to verify the performance goals of a technology. This document incorporates the QA/QC elements needed to provide data of appropriate quality sufficient to reach a defensible position regarding the technology performance. This is not a method validation study, nor does it represent every environmental situation which may be acceptable for this technology. But it will provide data of sufficient quality to make a judgement about the application of the technology under conditions similar to those encountered in the field demonstration. 2.0 DEMONSTRATION RESPONSIBILITIES AND COMMUNICATION This section identifies the organizations involved in this demonstration and describes the primary responsibilities of each organization. It also describes the methods and frequency of communication that will be used in coordinating the demonstration activities. October 31, 1996 3-3 ' Interim Final Report Version 5.0 ------- 2.1 Demonstration Organization and Participants This demonstration is being conducted by the developer with support by Verification Organization under the direction of the U.S. Environmental Protection Agency's (EPA) Office of Research and Development, National Exposure Research Laboratory, Characterization Research Division - Las Vegas, Nevada (CRD-LV). The CRD-LV's role is to administer the CSCT Demonstration Program. Verification Organization's role is to provide technical and administrative leadership and support in conducting the demonstration. Participants in this demonstration are listed in Table -. The specific responsibilities of each demonstration participant are discussed in Section 2.3 Provide a table which includes the name, affiliation, and mailing address of each demonstration participant, a point of contact, their role, and telephone, fax and email address. 2.2 Organization The organizational structure for the demonstration showing lines of communication is to be provided. See Figure I, Appendix Efor an example organization chart. 2.3 Responsibilities The responsibilities of the developer will vary depending on the type of demonstration. When a multiple developer demonstration is held many of the duties given here will be performed by the Verification Organization as they would be common to all of the developers. The Verification Organization will inform the developer of the specific sections of the demonstration plan they must provide. For a single developer demonstration, the developer is responsible for all of the listed duties. The Developer, in consultation with Verification Organization and the EPA technical lead, is responsible for the following elements of this demonstration: • Designing and preparing all elements of the demonstration plan; • Developing a quality assurance project plan (QAPP) (Section 8 of the demonstration plan); The QAPP is an integral part of the demonstration plan, not a separate document. • A health and safety plan (HASP) (Section 10 of the demonstration plan) for the demonstration activities; The HASP could either be included in the demonstration plan as Section 10 or provided as a separate document and referenced in Section 10. Acquiring the necessary reference analysis data; October 31, 1996 3-4 Interim Final Report Version 5.0 ------- Detailed procedures for using the technology; This could either be a standard operating procedure or an analytical method provided in the EMMC method format, which is included in Appendix B of this document • Complete, field-ready technology for demonstration; • Operating and monitoring the technology during the demonstration; • Documenting the experimental methodology and operation of the technology during the demonstration; • Data reduction and interpretation support, as required; and, • Logistical, and other support, as required. Verification Organization and the EPA have coordination and oversight responsibilities for: Providing needed logistical support, establishing a communication network, and scheduling and coordinating the activities of all demonstration participants; • Ensuring that appropriate sites are selected consistent with the objectives of the demonstration (developer may furnish a demonstration site(s)); • Performing on-site sampling activities including collecting and homogenizing samples, dividing them into replicates, and bottling, labeling, and shipping them where appropriate; • Managing, evaluating, interpreting, and reporting on data generated by the demonstration; and, • Evaluating and reporting on the performance of the technologies. The site owners and the EPA will provide the following support: • Site access; • Characterization information for the site; • Health and safety information for the site; and, • Other logistical information and support needed to coordinate access to the site for the field portion of the demonstration. 3.0 TECHNOLOGY DESCRIPTION This section provides a description of the innovative technology including the scientific principles, components and application history of the technology. Technology performance goals October 31, 1996 3-5 Interim Final Report Version 5.0 ------- are provided, both those specific to the demonstration and those relevant to general applications. The performance range through which the technology is applicable and advantages and limitations of the technology are also included. 3.1 Technology Performance Describe the technology to be demonstrated. This should include: • A brief introduction and discussion of the scientific principles on which the technology is based • A brief description of the physical construction/components of the technology. Include general environmental requirements and limitations, -weight, transportability, ruggedness, power and other consumables needed, etc. • Identify the parameters or analytes the technology is designed to measure • Identify the matrices for which the technology is applicable, e.g., soil, water, sludge, etc. • Cost of the technology (purchase or lease and typical operational costs) • Typical operator training requirements and sample handling or preparation requirements. • Define the performance range of the technology and verification requirements of the demonstration • Identify any special licensing requirements associated with the operation of the technology 3.2 History of the Technology This section should outline prior applications and experience with the innovative technology. 3.3 Technology Applications Identify and discuss the environmental measurement problem that the technology is designed to address, how the technology will solve the problem, and the potential users of the technology. 3.4 Advantages of the Technology Describe the applications of the technology and what advantages it provides over existing technology. Provide comparisons in such areas as: initial cost, cost per analysis, speed of analysis, precision and accuracy of the data, usable or linear operating range, field versus laboratory operation, solvent use, durability, potential for waste minimization, etc. October 31, 1996 3-6 Interim Final Report Version 5.0 ------- 3.5 Limits of the Technology Discuss the known limitations of the technology. Include such items as detection limits in various matrices (as appropriate), interferences, environmental limits (temperature, vibration, light, dust, power requirements, water needs, etc.), upper concentration limits, linear range, operator training, and experience requirements, etc. 3.6 Demonstration Performance Goals This section discusses factors that will be considered in the design and implementation of the demonstration. These factors include comparability, precision, portability, ease of operation, ruggedness and instrument reliability, health and safety issues, sample throughput, and sample matrix effects. This section is established to summarize the advantages of the technology and to state formally the results required to demonstrate these advantages. The presentation should address several issues including: (1) the criterion for a successful demonstration in terms of the quality parameters listed in Section 8.3 should be described in detail; (2) the range of applications over which the technology can be applied should be specified; and (3) a summary of the technological advantages and disadvantages of the innovative technology should be specified. The Developer must provide performance features of the technology which can be verified in the demonstration. These elements must be specific and be verifiable by a statistical analysis of the data. The performance goals should be within the capabilities of the technology but challenging to it. Goals that are too easily met may not be of interest to the potential user, while those that are overstated may not be achievable. This section forms the basis of the entire demonstration and must be carefully written. The experimental design must include a sufficient number and kinds of samples to permit the statistical verification of performance. The performance goal must be specific and be verifiable by a statistical analysis of the data. An example of satisfactory goal would be: "The technology will detect 95 percent of all analytes detected by the reference laboratory at concentrations greater than 50 parts per billion. The technology will produce results that are within 30 percent of the reference laboratory on 90 percent of the samples. The relative standard deviation on replicate samples will be less than 25 percent." Statements such as: The technology will provide better precision than the referee method; the technology has low detection limits; are not acceptable. Goals will differ from demonstration to demonstration, depending on the nature and maturity of the technology. The developers 'performance goals should specify levels of accuracy, precision, and completeness (see Section 8.3 for detailed definitions) that the demonstration will test and verify. October 31, 1996 3-7 Interim Final Report Version 5.0 ------- Accuracy will, in general, be the most important of the performance goals. If the technology is not accurate, then the degree to which samples are precise or complete loses importance. If results from the innovative technology are accurate, then their precision and completeness become important. Precision and completeness provide a basis for comparing the number of samples required to characterize the site using one technology to that of another (and hence for comparing the costs of information for the site characterization). Clearly, these quality parameters are of interest to potential users of the technology. Here the developer should specify goals regarding accuracy, precision and completeness. Section 10.1 of Appendix E provides a discussion of some of the possible measures of accuracy and precision and Section 8.3.3 provides the definition of completeness. 4.0 DEMONSTRATION SITE DESCRIPTIONS This section discusses the history and characteristics of the demonstration site(s). 4.1 Site Name and Location Provide the site names and locations where appropriate, area and location maps should be included. In most cases, the technology will be demonstrated at more than one site. The technology should be tested under different geologic, dimatologic, and waste environments. Information on the site history and site characteristics should be available through the Verification Organization or EPA contact unless the developer is making their own arrangements for the demonstration sites. 4.2 Site History Insert the site history. Include history of ownership and uses, especially those relating to the contamination found at the site. 4.3 Site Characteristics Provide a description of the site. This should include a description of the buildings, etc. at the site and the adjoining property. This description should also include a geological description of the site including soil types, etc. Provide a list of the kno\vn contaminants at the site, including the distribution and concentrations. Use tables and maps as appropriate. 5.0 CONFIRMATORY PROCESS The verification process is based on the presence of a statistically validated data set against which the performance goals of the technology may be compared. The choice of an appropriate reference method and referee laboratory are critical to the success of the demonstration. The developer, verification organization, and EPA must work closely to select the best combination of method and laboratory. In certain cases, modification of existing methods, special QC requirements, or laboratory training may be required. October 31,1996 3-8 Interim Final Report Version 5.0 ------- 5.1 Method Selection The reference analytical method should be chosen from standard methods approved by EPA or another recognized body, such as American Society for Testing and Materials (ASTM) or Association of Official Analytical Chemists (AOAC). The method selected should generate data similar in quality and type expected to be derived from the technology being demonstrated. A justification for selecting a specific method must be provided. The selection process may identify a non-standard method as providing the best data match. Since many of the field technologies offer qualitative data, rigorous quantitative laboratory methods may make direct comparisons unreasonable. Although it is not the purpose of this program to develop or validate new methods; some modification of existing methods may be required to insure that an appropriate method is used for comparison. 5.2 Reference Laboratory Selection To assess the performance of the insert technology name technology, the data obtained using the technology will be compared to data obtained using conventional analytical methods. Developer will propose a reference laboratory in consultation with Verification Organization and the EPA. This decision will be based on the experience of prospective laboratories with QA procedures, reporting requirements, and data quality parameters consistent with the goals of the Program. Describe how the laboratory was chosen and demonstrate that there is no conflict of interest with the developer. The Laboratory must demonstrate past proficiency with the method selected and may be asked to participate in a review of the experimental design. Laboratory management should be briefed on the nature and purpose of the demonstration and may suggest enhancements to the proposed procedure. Approval of the method and procedure will reside with the EPA. 5.3 Contingency Laboratory Selection A contingency laboratory would be used to support the data from the referee laboratory if preliminary results differ significantly from those obtained by the technology in the field. This section should outline the circumstances when a contingency laboratory would be used and the type ofQA/QC which would be employed. The degree to which a contingency laboratory will be used should also be discussed. 5.4 Special QC Requirements In order to increase the likelihood that high quality data will be obtained, an enhanced QC strategy may be required. The use of standard reference materials, double blind standards and special performance evaluation materials should be considered. The nature and use of these materials will depend on the technology being tested, the nature of the analytical method, and the performance of the laboratory. This section should describe the use and benefit of such materials. Cost and availability of the appropriate materials should also be considered. October 31,1996 3-9 Interim Final Report Version 5.0 ------- 5.5 Laboratory Audit The Verification Organization or the EPA will conduct an on-site audit of the reference laboratory both before and during sample analysis. The preliminary audit will identify laboratory history with samples of the type to be analyzed and the reporting requirements. The use of performance evaluation samples may be considered if insufficient history is evident. Before such analyses are required, the selection process should be reviewed and reaffirmed. An audit of the laboratory will be scheduled during the time period in which the field samples are being analyzed. The audit will address the QC procedures and document any changes to the analysis process. This section of the demonstration plan will highlight key elements of the audit process and any special procedures which must be addressed. 5.6 Statistical Analysis of Results The discussion in this section relate to the analysis of the results. This section should provide methods of data reduction and an example showing one possible specific method of analysis. General methods of statistical analysis differ from application to application too extensively to be provided here. 5.6.1 Methods of Data Reduction and A djustment Before any formal statistical analysis of the data, it is important to determine which data adjustments and which transformations are appropriate. Likely adjustments include the identification and removal of outlying values and the transformation of results so that they more closely satisfy statistical requirements. Statistical analyses of concentration data are often more accurately performed based on concentration percentages rather than the raw data results. Proposed methods of data reduction and adjustment should be specified here. 5.6.2 Methods of Statistical Analysis The purpose of this section is to describe the statistical methods that might be used to evaluate the developers goals. Appropriate statistical procedures depend on several issues described next. • The nature of the developers' performance goals —for example if a goal of the developers is as simple as a required percentage error then this analysis is reduced to a simple computation. More sophisticated goals, like demonstrating better results with a (specified) high probability, could involve statistical tests. • The type of data (qualitative vs. quantitative, for example) will dictate different statistical methodology. • Assumptions associated with the data -- distributional assumptions, for example, can be requirements for specific statistical tests. Tests or methods of estimation may have to be October 31, 1996 3-10 Interim Final Report Version 5.0 ------- proposed where their eventual use is contingent on verification of assumptions through data analysis. • The treatment of data from the reference laboratory. Because of these dependencies it is unreasonable to try to list all possible statistical procedures that may be appropriate for a specific demonstration. This section should list methods proposed for statistical analysis of demonstration results. 6.0 DEMONSTRATION DESIGN This section discusses the objectives of the demonstration, factors that must be considered to meet the performance objectives, and the information that Verification Organization will use to evaluate the results of the demonstration. 6.1 Objectives The primary objectives of this demonstration are to evaluate insert technology name in the following areas: (1) how well it performs relative to conventional analytical methods, (2) the impacts of sample matrix variations on performance, and (3) the logistical and economic resources necessary to operate the technology. Secondary objectives for this demonstration are to evaluate insert technology name for its reliability, ruggedness, cost, and range of usefulness, data quality, and ease of operation. The performance will be compared to the performance of conventional analytical methods used in performing similar site characterization activities. The verification process will evaluate the performance of the technology against the developer performance goals as stated in Section 3.6. 6.2 Experimental Design This section discusses factors that will be considered in the design and implementation of the demonstration. These factors include comparability, precision, portability, ease of operation, ruggedness and instrument reliability, health and safety issues, sample throughput, and sample matrix effects. Demonstration procedures will simulate routine field conditions as much as possible by testing the performance characteristics of the technology which were defined in Section 5.2. 6.2.1 Qualitative Factors Some factors, while important, are difficult or impossible to quantify. These are considered qualitative factors. Typical factors to be discussed are listed below. Discuss those that are appropriate to your technology, others may be added: Reliability or susceptibility to environmental conditions Effect of operator experience on results October 31, 1996 3-11 Interim Final Report Version 5.0 ------- »• Portability of the technology *• Special logistical requirements Add additional factors as appropriate for the technology. 6.2.2 Quantitative Factors Many factors in this demonstration can be quantified by various means. Some can be measured while others, such as the presence of interferents, cannot be controlled. The demonstration should be designed to distinguish between these factors and to report on the results of each. Typical factors to be discussed are listed below. Discuss those that are appropriate to your technology, others may be added. *• Weight and size of the technology >• Power and environmental requirements *• Cost of operation, expendables, and waste disposal Add additional factors as appropriate for the technology These quantitative factors will be used to assess technology performance by comparison to reference laboratory data. Where practical, 100 percent of the samples will be submitted for referee analysis. Common reference materials will be supplied to both the field technology and laboratory so that direct comparisons can be made. 6.3 Sampling Plan The sampling plan for this demonstration specifies the procedures that will be used to ensure the consistency and integrity of the samples. In addition, this plan will outline the sample collection procedures necessary to meet the demonstration purpose and objectives. Careful adherence to these procedures will ensure that sample data analyzed by the reference laboratory using conventional analytical methods will be directly comparable to the data obtained from the field technology. A table should be used to summarize the sampling and analysis strategy, giving the numbers and types of samples needed. An example table is provided in Appendix E. Table E-l. 6.3.1 Sampling Operations Provide an overview of the sample collection activities. Discuss specific aspects of the field sampling effort. These aspects include: > The matrix to be sampled, the requirement for different sampling techniques, equipment, etc. depends on whether the target matrix is a single-phase system, such as soil, sediment, air, water, or a multimedia sample involving multiple phases. October 31, 1996 3-12 Interim Final Report Version 5.0 ------- *• A discussion of the sampling design and the physical locations of the samples to be collected. Selection of the sampling equipment. Discuss considerations necessary in choosing a proper sampling method. Some of these aspects include: sampler compatibility in terms of physical and chemical characteristics of the tool; matrix effects resulting from the use of the tool as related to the need for disturbed versus undisturbed sample; whether the volume capacity is sufficient to obtain the needed sample amount in a single grab or whether multiple grabs will be needed; the physical requirements necessary to operate/use the tool, such as the need for an external power source, the weight and size of the tool as they relate to its transport and use; the ease of operation of the equipment (whether there is a need to train personnel on using the tool); whether the tool requires multiple operators or just a single person; the decontamination and reuse potential in terms of the ease, success, and time required to decontaminate; cost of the equipment; the usability of the equipment for various sample types such as soil or sediment and ground or surface water, etc. Provide or include an SOP for the sampling activities. The SOP should include such items as equipment and supply needs, transport to the field, and reserves/backups required should equipment failures occur. Guidance for performing some of these tasks may be found in Test Methods for Evaluating Solid Waste. EPA publication number SW-846, Third Edition. The verification organization and/or an EPA representative can make this reference available. 6.3.2 Predemonstration Sampling and Analysis A predemonstration sampling and analysis event is required to establish that the samples from the proposed site are appropriate for the technology and the reference method. The technology developer and the referee laboratory should each analyze replicate portions of one or more samples from the site. An objective of this activity should be to better anticipate the numbers of samples and replicates that will be required to meet statistical analysis requirements. This predemonstration sampling will allow the technology developers to refine their technologies and revise their operating instructions, if necessary. This sampling will also allow an evaluation of matrix effects or interferences that may affect the demonstration. Information generated through this predemonstration sampling and analysis event may be used to revise the demonstration sampling and analysis plan, if necessary. A failure to meet the performance goals at this point could indicate a lack of maturity of the technology and the demonstration would be canceled. Verification Organization will provide the predemonstration samples for this portion of the demonstration. This sampling requirement has the following objectives: > To provide or verify data on the extent of contamination at each site, and locate sampling areas for the demonstration October 31, 1996 3-13 Interim Final Report Version 5.0 ------- *• To allow the developers to analyze samples from the demonstration sites in advance, and, if necessary, refine and calibrate their technologies and revise their operating instructions *• To allow an evaluation of any unanticipated matrix effects or interferences that may occur during the demonstration Results of the predemonstration sample analyses by the developers must be made available to the Verification Organization two weeks after the receipt of the samples. This will allow time for analysis of the data and notification if problems exist. The verification organization will discuss the details of the predemonstration sampling and analysis activities. Discuss the sample collection, handling, and analysis procedures which were used. Samples are to be analyzed by both the developer and a reference laboratory. The use of performance evaluation samples should also be included in the analysis process. 6.4 Field Data The technology will be operated by the Developer, who will provide the results to the EPA or Verification Organization. The Developer will be responsible for reducing the raw data into a presentation format consistent with the evaluation requirements. The Developer will submit all QC data and a description of how this data may be used to validate the field data. 6.4.1 Field Audit The EPA or Verification Organization will conduct an audit of all field activities. This activity will document any deviations from the demonstration plan, use of QC materials, operational details, and other factors associated with an evaluation of the field technology. This audit report will be included as part of the Quality Assurance Project Plan (Section 8.0). 6.5 Demonstration Schedule Predemonstration sampling for this demonstration is planned for insert date. Samples collected at this time will be used by the developers to test and possibly recalibrate their technologies using the site-specific samples. The predemonstration data will be used to select demonstration sampling areas and verify the magnitude and distribution of contaminants at each site. Demonstration activities, including sampling and analysis, are scheduled to occur on insert date. Details of the sampling activities are provided in the next section. A typical demonstration schedule is provide in Appendix D. 7.0 FIELD OPERATIONS This section will describe the logistical requirements associated with sample collection and technology operation. This phase of the demonstration requires close communication between the October 31, 1996 3-14 Interim Final Report Version 5.0 ------- Developer, Verification Organization and the EPA. Preliminary testing and personnel training may be required before the actual field study. Successful field operations require detailed planning and extensive communication. The implementation of the demonstration must be consistent with the requirements of the study and routine operation of the technology. 7.1 Communication and Documentation The Verification Organization will communicate regularly with the demonstration participants to coordinate all field activities associated with this demonstration and to resolve any logistical, technical, or QA issues that may arise as the demonstration progresses. The successful implementation of the demonstration will require detailed coordination and constant communication between all demonstration participants. All Developer/Verification Organization field activities will be thoroughly documented. Field documentation will include field logbooks, photographs, field data sheets, and chain-of- custody forms. The Verification Organization field team leader will be responsible for maintaining all field documentation. Field notes will be kept in a bound logbook. Each page will be sequentially numbered and labeled with the project name and number. Completed pages will be signed and dated by the individual responsible for the entries. Errors will have one line drawn through them and this line will be initialed and dated. All photographs will be logged in the field logbook. These entries will include the time, date, direction, subject of the photograph, and the identity of the photographer. Specific notes about each sample collected will be written on sample field sheets and in the field logbook. Any deviations from the approved final demonstration plan will be thoroughly documented in the field logbook and provided to the verification organization. Developer will obtain all equipment needed for field work associated with this demonstration. Provide a list of all equipment to be used in the demonstration. A table format is suggested. 7.2 Sample Collection Procedures The Verification Organization will collect and prepare samples using the procedures described below. All field activities will conform with requirements of the health and safety plan (HASP) and with all requirements in this demonstration plan. If unanticipated or unusual situations are encountered that may alter the sampling design, sampling location, or data quality, the situation will be discussed with the EPA technical lead. Any deviations from the approved final demonstration plan will be thoroughly documented. The sampling design should be in this section. Items to include are: The number of sites and the criteria used for selection; the number of samples and why; how the samples will be distributed by location, matrix and analyte concentration. For certain types of technologies, e.g. geophysical, October 31, 1996 3-15 Interim Final Report Version 5.0 ------- where direct sample collection is not applicable, a sample would be defined as a reading or set of readings at a particular location. 7.2.1 Sampling Procedures Describe the detailed sample collection procedures. Sufficient detail must be provided to direct the step by step sample collection process. Identify the tools and collection procedures, contamination prevention, and decontamination procedures. 7.2.2 Sampling Locations Identify the sampling locations and how specific sampling locations within the site will be selected. Considerations would include such things as analytes, concentration, soil type, sampling depth, etc. 7.2.3 Sample Preparation Describe the procedures used to preserve or homogenize the sample. Cite differences bet\veen field analysis and requirements for reference laboratory analysis. Justify any differences bet\veen the standard method and the field sample preparation requirements 7.2.4 Sample Distribution The Verification Organization will be responsible for sampling and sample distribution. The procedures which ensure homogeneity and integrity must be described. 7.2.4.1 Laboratory Samples Describe the sample storage, packaging and shipping procedures. Include information on allowable holding times, chain of custody, etc. for the referee samples. 7.2.4.2 Field Samples Describe the requirements for sample size, format, and short term storage appropriate to the technology. 7.2.4.3 Archive Samples Whenever possible, test samples should be preserved and stored for future use. Describe the procedures and a suitable location for long term storage. 7.2.5 Decontamination and Disposal Describe the decontamination and/or disposal of all contaminated items. This includes sampling equipment, protective clothing, and other items. Specify a procedure for separating hazardous waste from non-hazardous waste. October 31, 1996 3-16 Interim Final Report Version 5.0 ------- 7.3 Personnel Requirements List any special training or requirements needed to support the operation of the technology. 7.4 Technology Logistical Needs Provide a list of all normal support equipment needed to operate the technology in the field. 7.4.1 Special Needs List any special equipment needed to support the requirements of this demonstration. 8.0 QUALITY ASSURANCE PROJECT PLAN (QAPP) The QAPP for this demonstration specifies procedures that will be used to ensure data quality and integrity. Careful adherence to these procedures will ensure that data generated from the demonstration will meet the desired performance objectives and will provide sound analytical results. Detailed guidance is provided in Appendix E, Quality Assurance/Quality Control (QA/QC) Requirements. It is strongly recommended that Appendix E be thoroughly reviewed prior to the preparation of this section. This appendix is intended to supplement the material provided in this section. The guidance contained in this section supersedes all other instructions. 8.1 Purpose and Scope The primary purpose of this section is to outline steps that will be taken by operators of the insert technology name technology and by the reference laboratory to ensure that data resulting from this demonstration is of known quality and that a sufficient number of critical measurements are taken. The EPA considers the demonstration to be classified as a Category II project. This section of the demonstration plan addresses the key elements that are required for Category II projects prepared according to guidelines in the EPA guidance documents "Preparation Aids for the Development of Category II Quality Assurance Project Plans" (Simes 1991) (The relevant parts of this document are in Appendix E), "Preparing Perfect Project Plans (1989), and the Interim Guidelines and Specifications for Preparing Quality Assurance Project Plans" (Stanley and Verner 1983). 8.2 Quality Assurance Responsibilities The Developer project manager is responsible for coordinating the preparation of the QAPP for this demonstration and for its approval by the EPA project manager and the Verification Organization. The Developer project manager will ensure that the QAPP is implemented during all demonstration activities. The Developer QA manager for the demonstration will review and approve the QAPP and will provide QA oversight of all demonstration activities. The QA audit function will be the responsibility of the EPA. October 31, 1996 3-17 Interim Final Report Version 5.0 ------- The entire demonstration plan including the QAPP must be approved by the Developer, Verification Organization, and EPA before the demonstration can proceed. Samples will be collected and analyzed on site by the insert technology name technology and off site by the reference laboratory using EPA-approved methods (see Appendix E, Section 6.1). Primary responsibility for ensuring that sampling activities comply with the requirements of the sampling collection procedures (Section 7.2 will rest with the EPA technical lead or Verification Organization field team leader. QA/QC activities for the insert technology name technology will include those activities recommended by Developer and those required by the EPA or Verification Organization to assure the demonstration will provide data of the necessary quality. QA/QC activities for the reference laboratory analysis of samples will be the responsibility of the reference laboratory supervisor. If problems arise or any data appear unusual, they will be thoroughly documented and corrective actions will be implemented as specified in this section. The QA/QC measurements made by the reference laboratory are dictated by the analytical methods being used. 8.3 Data Quality Indicators The data obtained during the demonstration must be of sufficient quality for conclusions to be drawn on the insert technology name technology. For all measurement and monitoring activities conducted for EPA, the Agency requires that data quality parameters be established based on the proposed end uses of the data. Data quality parameters include five indicators of data quality: representativeness, completeness, comparability, accuracy, and precision. Data generated by the insert technology name technology will be compared to the data generated from Fill in details on the reference method(s). High quality, well documented reference laboratory results are essential for meeting the purpose and objectives of this demonstration. Therefore, the following indicators of data quality will be closely evaluated to determine the performance of the technology when measured against data generated by the reference laboratory. For examples of discussing data quality parameters, including suggested tabular formats, see Appendix E, Section 4. The referee data must be of sufficient quality to allow comparison to the technology. This may require tighter QC limits than are normally used by the reference method. 8.3.1 Representativeness Representative samples, in general, are samples that contain a reasonable cross-section of the "population" over which they are to be used to make inferences. The population for demonstrations analyzed as part of this project includes a variety of media (groundwater, soil, soil-gas) and contaminants that the innovative technologies are developed to accommodate. In order to test the innovative technology across this range, care must be taken in site selection, experimental design and sample planning and in sample collection, preparation, storage and shipping. In preparing the demonstration plan one will consider the following: October 31, 1996 3-18 Interim Final Report Version 5.0 ------- 1) The design for the set of demonstrations should consider the media and contaminants at each site, and the site selection process should be performed to try to include the entire range of these materials for which the technology is applicable. When this is not possible, the demonstration plan should specify -why the selected demonstration sites provide a fair indication of the technologies' capabilities. 2) The experimental design and sampling plan should guarantee that the site is being fully utilized (for its purpose as determined above in 1). For example, a site containing more than one soil type should utilize a design and sample plan that -will very likely include these materials. 3) The procedures for collection, preparation, storage and transport of the samples at the site and the reference laboratory must be specified to assure that when the samples are analyzed at the laboratory, they are representative of the samples analyzed in the field. 8.3.2 Comparability Comparability is a quality parameter determined for the most part in the planning stages of the demonstration, often on the basis of prior knowledge of the innovative technologies' performance capabilities. First, the innovative technology must be comparable in some way to a reference or baseline method for the demonstration to be worthwhile. Given this, questions arise as to how the technologies should be compared. Two issues that must be addressed in this section are: 1) should the comparison be made quantitatively, semi-quantitatively or qualitatively? and 2) how will differences be addressed? The first of these questions refers to whether the sample results will be qualified as detect/non- detect by the t\vo technologies (innovative and reference) or whether numerical estimates of contamination levels will be compared. Intermediate cases (semi-quantitative) are possible, for example, samples could be classified as clean, low, or highly contaminated. The second question addresses the issue of which method is correct when discrepancies occur between results of the f\vo methods. For example, will results of the referee laboratory be treated as the true value, or will imprecision in analytical results be considered in the comparison. 8.3.3 Completeness Completeness refers to the amount of data collected from a measurement process expressed as a percentage of the data that would be obtained using an ideal process under ideal conditions. This subsection should include a listing of the factors that might contribute to incomplete data along with an estimate of the percentage complete for the demonstration. These factors and estimates should be provided for both the innovative and the reference technologies. For the innovative technology these estimates should be included in Section 3.6 with the developers performance goals and the factors listed in Section 3.5 with limitations of the innovative technology. For the reference technology, estimates should be included with the developers performance goals while the factors may be included in Section 3.4, advantages of the innovative technology. October 31, 1996 3-19 Interim Final Report Version 5.0 ------- 8.3.4 Accuracy Accuracy is a measure of how close, on average, values of the innovative technology are to the true values. Inaccuracies or biases are the result of systematic differences between these values. When comparing the innovative technology to a reference technology difficulties can arise. In some cases biases can be attributed to the innovative technology. These biases are often the result of poor calibration. Other possible sources of bias include systematic errors in standards preparation, biases introduced in the sample extraction, storage and shipping processes and biases resulting from setup-related differences at the reference laboratory. Only the former of these sources is likely to be incurred by users of the innovative technologies. Most of the remaining sources represent inaccuracy that might be avoided through use of the innovative technology. Consequently every effort should be made by the verification organization, the developers and the reference laboratory to identify specific sources of accuracy. The design of blanks, duplicates and performance assessment samples should provide substantiating evidence to support this partitioning of sources of inaccuracy when results become available. Differences that appear as inaccuracies in a single demonstration may reflect imprecision in calibration procedures that might lead to different biases on another day and under different environmental conditions (and hence become a precision issue on a larger scale). Since there will be too few demonstrations to analyze variability associated with the calibration procedures, care should be taken by the developers in making claims concerning this portion of the process. Accuracy in environmental applications is often best specified as a percentage of the true contaminant concentration. Several possible measures of accuracy are listed in Section 10.1 of Appendix E. Accuracy is clearly among the most important aspects of the innovative technology to be analyzed in the demonstration. Special care should be given to define the appropriate performance goal for this factor. 8.3.5 Precision Precision, in general, refers to the degree of mutual agreement among measurements of the same materials and contaminants. Environmental applications often involve situations where "measurements of the same materials" can take on a number of interpretations. In environmental applications, precision is often best specified as a percentage of contaminant concentration. Several possible measures of precision are listed in Section 10.1 of Appendix E. The following lists several possible interpretations of precision for environmental applications. 1) The precision involved in repeated measurements of the same sample without adjusting the test equipment. 2) The precision involved in repeated measurements of the same sample after reset, repositioning, or re-calibration of the test equipment or when using different equipment of the same technology. 3) The precision involved in measurements of materials taken from adjacent locations. October 31, 1996 3-20 Interim Final Report Version 5.0 ------- 4) The precision characteristics of a specific technology in determining contamination at a specific site or at an arbitrary site. In general, users of the technology will want to be assured that imprecision in 1) and 2) is small. The interpretation of precision described in 3) is likely to be too site specific to be of general interest. The imprecision discussed in 4) is perhaps of most interest as it includes imprecision resulting from possible differences in the design activities and effects of environmental conditions such as temperature that would vary from one site characterization to another as well as site and technology specific sources. If available, this information would provide the potential user with an estimate of how close a site characterization using this technology would come to providing the true site contaminate levels. Unfortunately, it is unlikely that the demonstrations will be extensive enough to provide much information on how this estimate would be provided. Different components of precision may be important for different technologies or for different applications of the same technology. The developers claims of precision should be specified in the developers performance goals, Section 3.6, and should distinguish clearly bet\veen different interpretations of precision. Methods for evaluating these claims should be specified in the section on statistical methods for evaluating demonstration results. 8.4 Calibration Procedures and Quality Control Checks This section describes the calibration procedures and method-specific QC requirements that apply to both the technology and the referee analyses. It also contains a discussion of the corrective action to be taken if the QC parameters fall outside of the evaluation criteria. The quality control checks provide a means of measuring the quality of data produced. The developer may not need to use all the ones identified in this section. The selection of the appropriate quality control checks depends on the technology, the experimental design and the performance goals. The selection of quality control checks will be based on discussions among the developer, the EPA technical lead, and the verification organization. 8.4.1 Initial Calibration Procedures Describe the initial calibration procedure for the technology. Initial calibration consists of those procedures used to establish the qualitative and quantitative performance of the technology for the analytes of interest prior to making measurements. Include the frequency and acceptance criteria for initial calibration and the actions taken if criteria are not met. The initial calibration procedures for the reference method may be incorporated by reference. 8.4.2 Continuing Calibration Procedures Describe the continuing calibration procedures. Include the frequency and acceptance criteria and action if criteria are not met. Continuing calibrations are those procedures used to assure the technology maintains the calibration that was established in the initial calibration. The continuing calibration for the reference method may be included by reference. October 31, 1996 3-21 _ Interim Final Report Version 5.0 ------- 8.4.3 Method Blanks Method blanks are used to evaluate technology-induced contamination, which may cause false positive results. Describe the use of method blanks, the materials used, the frequency, the criteria for acceptable method blanks and the actions if criteria are not met. Include this information for both the technology and the reference method. 8.4.4 Spike Samples The use of spiked samples will depend on the technology. If spiked samples are to be used specify the procedure, frequency, acceptance criteria, and actions if criteria are not met. 8.4.5 Laboratory Control Samples Laboratory control samples are samples of known composition that are analyzed periodically to assure that the analytical system is in control. These are analyzed just like a regular sample. These may or may not be appropriate for the technology. They may be required for the reference method. If laboratory control samples are used, their composition should be described in this section. 8.4.6 Performance Evaluation Materials Performance evaluation materials are samples whose composition is unknown to the analyst that are used to evaluate system performance. PE samples will be submitted to the reference laboratory and to the technology for analysis. The control limits for the PE samples will be used to evaluate the technology and reference laboratory's method performance. PE samples come with statistics about each sample which have been derived from the analysis of the sample by a number of laboratories using EPA-approved methods. These statistics include a true value of the PE sample, a mean of the laboratory results obtained from the analysis of the PE sample, and an acceptance range for sample values. The reference laboratory is expected to provide results from the analysis of the PE samples that meet the performance objectives of the demonstration. Describe any specific procedures appropriate to the analysis of the PE materials. It has to be clear how these samples are going to be used in the demonstration. One use ofPE materials is in the conduct of a performance audit. 8.4.7 Duplicate Samples Duplicate samples must be analyzed by the technology to determine the precision of the analysis. These same samples should be submitted in duplicate for analysis by the reference method so the precision of the methods can be compared. Precision often represents one of the key performance claims for the technology, therefore careful selection of the type and number of duplicate samples is critical to the success of the demonstration. Provide the procedure for determining samples to be analyzed in duplicate, the frequency and approximate number. October 31,1996 3-22 Interim Final Report Version 5.0 ------- 8.5 Data Reduction, Validation, and Reporting To maintain good data quality, specific procedures will be followed during data reduction, validation, and reporting. These procedures are detailed below. Also see Appendix E, Section 7 for additional guidance. 8.5.1 Data Reduction Data reduction refers to the process of converting the raw results from the technology into a concentration or other data format which will be used in the comparison. The procedures to be used will be technology dependent. The purpose of this step is to provide data which will be used to verify the performance claims. These data will be obtained from logbooks, instrument outputs, and computer outputs as appropriate. Describe what will be provided and the procedures used to assure that the data are correct. The actual comparisons will be performed by the Verification Organization or EPA. 8.5.2 Data Validation The operator will verify the completeness of the appropriate data forms and the completeness and correctness of data acquisition and reduction. The reference laboratory or field team supervisor will review calculations and inspect laboratory logbooks and data sheets to verify accuracy, completeness, and adherence to the specific analytical method protocols. Calibration and QC data will be examined by the individual operators and the laboratory supervisor. Laboratory project managers and QA managers will verify that all instrument systems are in control and that QA objectives for accuracy, completeness, and method detection limits have been met. Analytical outlier data are defined as those QC data lying outside a specific QC objective window for precision and accuracy for a given analytical method. Should QC data be outside of control limits, the reference laboratory or field team supervisor will investigate the cause of the problem. If the problem involves an analytical problem, the sample will be reanalyzed. If the problem can be attributed to the sample matrix, the result will be flagged with a data qualifier. This data qualifier will be included and explained in the final analytical report. 8.5.3 Data Reporting This section contains a list of the data to be reported by both the technology and the reference method. At a minimum, the data tabulation will list the results for each sample and include detection limits, reporting units, sample numbers, results, and data qualifiers. All QC information such as calibrations, blanks and reference samples are to be included. All raw analytical data should also be reported. All data should be reported in hardcopy and electronically in a common spreadsheet or database format. October 31, 1996 3-23 Interim Final Report Version 5.0 ------- 8.6 Calculation of Data Quality Indicators Provide the equations for any data quality indicator calculations that may be used. These include: precision, relative percent deviation, standard deviation, accuracy, and completeness. For additional information refer to Appendix E, Section 10. 8.7 Performance and System Audits The following audits will be performed during this demonstration. These audits will determine if this demonstration plan is being implemented as intended. Guidance on conducting audits is available in Appendix E, Section 9 8.7.1 Perform an ceA udit Performance evaluation (PE) samples will be submitted to the reference laboratory and to the insert technology name technology for analysis. The control limits for the PE samples will be used to evaluate the insert technology name technology and the reference laboratory's method performance. The insert technology name technology will analyze the PE samples periodically during the demonstration. Specify control limits and the corrective measures which will be used if the results exceed the specified acceptance criteria. 8.7.2 On-Site System Audits On-site system audits for sampling activities, field operations, and laboratories will be conducted as requested by the EPA project manager. These audits will be performed by the EPA Project Manager or the Verification Organization. 8.8 Quality Assurance Reports QA reports provide the necessary information to monitor data quality effectively. It is anticipated that the following types of QA reports will be prepared as part of this demonstration. See Appendix E, Section II for more information. 8.8.1 Status Reports The developer and verification organization project managers will prepare periodic reports for the EPA project manager. These reports should discuss project progress, problems and associated corrective actions, and future scheduled activities associated with the demonstration. When problems occur, the developer and verification organization project managers will discuss them with the EPA project manager or EPA technical lead, estimate the type and degree of impact, and October 31, 1996 3-24 Interim Final Report Version 5.0 ------- describe the corrective actions taken to mitigate the impact and to prevent a recurrence of the problems. Provide the frequency, format, and content of these reports. 8.8.2 Audit Reports Any QA audits or inspections that take place in the field or at the reference laboratory while the demonstration is being conducted will be formally reported by the auditors to the EPA project manager who will forward them to the developer, Verification Organization QC Manager, and the Verification Organization project manager for appropriate actions. 8.9 Corrective Action Each demonstration plan must incorporate a corrective action plan. This plan must include the predetermined acceptance limits, the corrective action to be initiated -whenever such acceptance criteria are not met, and the names of the individuals responsible for implementation. Routine corrective action may result from common monitoring activities, such as: • Performance evaluation audits • Technical systems audits • Calibration procedures 9.0 DATA MANAGEMENT AND ASSESSMENT The developer, Verification Organization, and EPA each have distinct responsibilities for managing and analyzing demonstration data. The verification organization is responsible for managing all the data and information generated during the demonstration. The developer is responsible for furnishing those records generated by the technology operator. EPA and the verification organization are responsible for analysis and verification of the data. There are a variety of pieces of data and information that will be generated during a demonstration. Each piece of data or information identified for collection in the demonstration plan will need to be provided to EPA or the Verification Organization. This section should describe what types of data and information needs to be collected and managed. It should also describe how the data will be reported to Verification Organization for evaluation. Innovative Technology Data: The developer is responsible for obtaining, reducing, interpreting, validating, and reporting the data associated with his technology's performance. These data should be reported in hard copy and electronic format (e.g., spreadsheet). A data dictionary should be included as a text file on the same diskette containing the data. Reference Laboratory Analyses: The raw data and the validated data must be provided to the Verification Organization. These data should be provided in hard copy and in electronic format. As with the data generated by the innovative technology, the electronic copy of the laboratory data should be provided in a spreadsheet, and a data dictionary should be provided. In addition to the sample results, all QA/QC summary forms for the referee analyses must be provided. October 31,1996 3-25 Interim Final Report Version 5.0 ------- Other items that must be provided include: • field notebooks; • photographs, slides and videotapes (copies); • results from the use of other field analytical methods; • Hthologic logs; • drilling logs, and; • cone penetrometer traces. 10.0 HEALTH AND SAFETY PLAN The Health and Safety Plan (HASP) is a very important part of the demonstration plan. It should be an adaptation or appendix to the existing site HASP with any additions that are specific to the demonstration. A copy of the site HASP should be available from the site manager or through the verification organization or EPA contact. Figure 3-1 contains a representative list of topics that should be addressed in the HASP. The verification organization can provide assistance in preparing this section of the demonstration plan. 10.1 Health and Safety Plan Enforcement The Verification Organization project manager, field site supervisor, and site health and safety officer will be responsible for implementing and enforcing the health and safety plan. The Verification Organization project manager will ultimately be responsible for ensuring that all demonstration participants abide by the requirements of this HASP. The Verification Organization field site supervisor will oversee and direct field activities and is responsible for ensuring compliance with this HASP. October 31, 1996 3-26 Interim Final Report Version 5.0 ------- Figure 3-1. Typical Table of Contents from a Health and Safety Plan. Health and Safety Plan - XYZ, Inc. Site Introduction Purpose and Policy Health and Safety Plan Enforcement for the XYZ, Inc. Site Project Manager and Field Site Supervisor Health and Safety Director Site Health and Safety Officer Visitors Site Background Demonstration-Specific Hazard Evaluation Exposure Pathways Inhalation Dermal Contact Ingestion Health Effects Physical Hazards Fire Heat Stress Mechanical Unstable/Uneven Terrain Insect and Other Animal Stings and Bites Noise Electrical Inclement Weather Training Requirements Personal Protection Levels of Protection Protective Equipment and Clothing Limitations of Protective Clothing Duration of Work Tasks Respirator Selection, Use, and Maintenance Medical Surveillance Health Monitoring Requirements Documentation and Record keeping Requirements Medical Support and Follow-up Requirements Environmental Surveillance Initial Air Monitoring Periodic Air Monitoring Monitoring Parameters October 31, 1996 3-27 Interim Final Report Version 5.0 ------- Figure 3-1. Typical Table of Contents from a Health and Safety Plan Cont'd. Use and Maintenance of Survey Equipment Heat Stress Monitoring Site Control Site Control Zones Safe Work Practices Health and Safety Plan Enforcement Complaints Decontamination Personnel Decontamination Equipment Decontamination Emergency Contingency Planning Injury in the Exclusion or Contamination Reduction Zones Injury in the Support Zone Fire or Explosion Protective Equipment Failure Emergency Information Telephone Numbers Hospital Route Directions October 31, 1996 3-28 Interim Final Report Version 5.0 ------- APPENDIX A DESCRIPTION OF THE ENVIRONMENTAL MONITORING MANAGEMENT COUNCIL (EMMC) A primary goal of this demonstration program is to gain Agency-wide recognition and acceptance of the results. The Environmental Monitoring Management Council is a key, Agency- wide organization that represents the Program and Regional Offices regarding the standardization of various policies and practices. This brief overview highlights the principle mission and purpose of this body. October 31, 1996 A-l Interim Final Report Version 5.0 ------- This page intentionally left blank. October 31, 1996 A-2 Interim Final Report Version 5.0 ------- DESCRIPTION OF THE ENVIRONMENTAL MONITORING MANAGEMENT COUNCIL The Environmental Monitoring and Management Council (EMMC) was established in March of 1990 to: a) Coordinate Agency-wide policies concerning environmental monitoring issues, especially in the areas of analytical methods integration, laboratory accreditation, and quality assurance b) Address Congressional concern over our ability to make national environmental assessments c) Respond to the needs of the Administrator/Deputy Administrator to make decisions based on credible scientific data. The EMMC Policy Council is made up of Assistant Administrator (AA), and Regional Administrator (RA) level personnel and is chaired by the Office of Research and Development (ORD) AA, and the U.S. EPA Region III RA. The Steering Committee is comprised of Office Directors and Division Directors. Scientific and engineering program and regional staff provide direction to the panels and work groups. The Deputy Administrator reviewed EMMC activities on March 4, 1994, and made the following decisions: a) Designated EMMC to be the focal point for internal/external policy on monitoring information activities b) Endorsed EMMC Methods Format and Framework for Methods Development c) Encouraged EMMC to continue developing a national program for laboratory accreditation d) Directed EMMC to brief the Science Policy Council (SPC) on its activities including: • the laboratory accreditation activities and associated near term resource needs • options and recommendations for an EPA-wide approach for applying performance-based methods to monitoring activities. e) Requested Assistant Administrators and Regional Administrators to continue support of EMMC activities. EMMC activities are important to establish better science/credibility for data comparability and information-sharing. EMMC is working to simplify lab procedures and to effect cost reductions by eliminating duplication of methods development efforts, avoiding duplication of field, laboratory and QC efforts, and conducting fewer lab evaluation studies. October 31, 1996 A-3 Interim Final Report Version 5.0 ------- EMMC issues that will be brought to the Science Policy Council in the future include: a) The use of EMMC as a model for interaction with Science Committees in the Agency; b) Procedures to implement a performance-based approach to method development; c) Elimination of process barriers that prevent use of innovative technology. This last point is of primary interest to advancing the goals of the Consortium. The EMMC serves as a forum to determine which methods need integration, provides for consensus of all offices and serves as a cross-program vehicle for assuring documentation of methodology and data and for comparability of data. Integration of four monitoring methods have been completed. These are: Graphite, Furnace Atomic Absorption, Inductively Coupled Plasma Atomic Emissions Spectrometry, Hot Acid Extraction for Elemental Analyses (as part of above), and Determination of Purgeable Organic Compounds by Capillary Column Gas Chromatography. The EMMC recommends a variety of subjects for Agency-wide acceptance to the Science Policy Council. The EMMC is currently working on assessment/design activities to ensure environmental measurement/quality assurance are built into the regulatory process. The EMMC acts to review specific products and to provide formal EPA responses in the areas that are most important to achieving nationwide recognition. These activities enhance the value of the demonstration program by offering a standard process which will receive Agency-wide acceptance and approval. October 31 1996 A-4 Interim Final Report Version 5.0 ------- APPENDIX B ENVIRONMENTAL MONITORING MANAGEMENT COUNCIL METHOD FORMAT (EMMC) This appendix describes the standard format that will be used to document EPA's environmental monitoring methods. The EMMC Method Format was developed to provide a uniform system for the integrated review of methods by the Work Groups of the Environmental Monitoring Management Council's Ad Hoc Panel on Methods Integration. The outline form of the Methods Format was approved by the EMMC Methods Integration Panel and Chairs of the respective Work Groups on January 24, 1992. While the format originally was intended for use in documenting methods for use across the various EPA Program Offices, some of the specifics in this description relate to the special problems of documenting a performance-based method. Technologies that require data collection using measurement procedures must include a detailed method. The Consortium will require the use of the EMMC standard method format for all methods used in a technology demonstration. After the demonstration is completed, a revised EMMC method will be prepared as part of the Technology Evaluation Report. October 31, 1996 B-1 Interim Final Report Version 5.0 ------- This page intentionally left blank. October 31,1996 B-2 Interim Final Report Version 5.0 ------- Environmental Monitoring Management Council (EMMC) Methods Format 1.0 Scope and Application Use a tabular format whenever possible for: • Analyte list(s) • Chemical Abstract Service (CAS) numbers • Matrices • Method Sensitivity (expressed as mass and as concentration with a specific sample size) Include a list of analytes (by common name) and their CAS registry numbers, the matrices to which the method applies, a generic description of method sensitivity (expressed both as the mass of analyte that can be quantified and as the concentration for a specific sample volume or size), and the data quality objectives which the method is designed to meet. Much of this material may be presented in a tabular format. 2.0 Summary of Method Sample volume requirements Extraction • Digestion • Concentration, and other preparation steps employed • Analytical instrumentation and detector system(s), and • Techniques used for quantitative determinations Summarize the method in a few paragraphs. The purpose of the summary is to provide a succinct overview of the technique to aid the reviewer or data user in evaluating the method and the data. List sample volume, extraction, digestion, concentration, other preparation steps employed, the analytical instrumentation and detector system(s), and the techniques used for quantitative determinations. 3.0 Definitions Include the definitions of all method-specific terms here. For extensive lists of definitions, this section may simply refer to a glossary attached at the end of the method document. October 31, 1996 B-3 Interim Final Report Version 5.0 ------- 4.0 Interferences This section should discuss any known interferences, especially those that are specific to the performance-based method. If known interferences in the reference method are not interferences in the performance-based method, this should be clearly stated. 5.0 Safety Above and beyond good laboratory practices Disclaimer statement (look at ASTM disclaimer) • Special precautions • Specific toxicity of target analytes or reagents • Not appropriate for general safety statements This section should discuss only those safety issues specific to the method and beyond the scope of routine laboratory practices. Target analytes or reagents that pose specific toxicity or safety issues should be addressed in this section. 6.0 Equipment and Supplies Use generic language wherever possible. However, for specific equipment such as GC (gas chromatograph) columns, do not assume equivalency of equipment that was not specifically evaluated, and clearly state what equipment and supplies were tested. 7.0 Reagents and Standards Provide sufficient details on the concentration and preparation of reagents and standards to allow the work to be duplicated, but avoid lengthy discussions of common procedures. 8.0 Sample Collection, Preservation and Storage • Provide information on sample collection, preservation, shipment, and storage conditions. Holding times, if evaluated " If effects of holding time were specifically evaluated, provide reference to relevant data, otherwise, do not establish specific holding times. 9.0 Quality Control Describe specific quality control steps, including such procedures as method blanks, laboratory control samples, QC check samples, instrument checks, etc., defining all terms in Section 3.0. Include frequencies for each such QC operation. October 31, 1996 B-4 Interim Final Report Version 5.0 ------- 10.0 Calibration and Standardization Discuss initial calibration procedures here. Indicate frequency of such calibrations, rcie to performance specifications, and indicate corrective actions that must be taken when performance specifications are not met. This section may also include procedures for calibration verification or continuing calibration, or these steps may be included in Section 11.0. 11.0 Procedure Provide a general description of the sample processing and instrumental analysis steps. Discuss those steps that are essential to the process, and avoid unnecessarily restrictive instructions. 12.0 Data Analysis and Calculations Describe qualitative and quantitative aspects of the method. List identification criteria used. Provide equations used to derive final sample results from typical instrument data. Provide discussion of estimating detection limits, if appropriate. 13.0 Method Performance A precision/bias statement should be incorporated in the section, including: detection limits • source/limitations of data Provide detailed description of method performance, including data on precision, bias, detection limits (including the method by which they were determined and matrices to which they apply), statistical procedures used to develop performance specifications, etc. Where performance is tested relative to the reference method, provide a side-by-side comparison of performance versus reference method specifications. 14.0 Pollution Prevention Describe aspects of this method that minimize or prevent pollution that may be attributable to the reference method. 15.0 Waste Management Cite how waste and samples are minimized and properly disposed. 16.0 References • Source documents Publications October 31, 1996 B-5 Interim Final Report Version 5.0 ------- 17.0 Tables, Diagrams, Flowcharts and Validation Data Additional information may be presented at the end of the method. Lengthy tables may be included here and referred to elsewhere in the text by number. Diagrams should only include new or unusual equipment or aspects of the method. October 31, 1996 B-6 Interim Final Report Version 5.0 ------- APPENDIX C OFFICE OF SOLID WASTE METHOD REQUIREMENTS EPA acceptance of a method associated with an innovative technology can be based, in part, on the results of a demonstration conducted under the auspices of the Consortium. This appendix provides an overview of the expectations that the EPA's Office of Solid Waste (OSW) has regarding the acceptance of a method into Publication SW-846, Test Methods for Evaluating Solid Waste. The following is a copy of a standard letter that is provided to those individuals expressing an interest in submitting analytical methods for inclusion in SW-846. The Consortium will provide guidance to the developer in preparing the appropriate data package for the OSW Methods Review Panel. Participation in the demonstration program does not guarantee acceptance by any EPA Program Office of the analytical method. The Consortium serves to verify technology performance as described in the demonstration plan. October 31, 1996 C-1 Interim Final Report Version 5.0 ------- This page intentionally left blank. October 31,1996 C-2 Interim Final Report Version 5.0 ------- OFFICE OF SOLID WASTE METHOD REQUIREMENTS Dear Colleague: The Methods Section of the Office of Solid Waste is responsible for the promulgation of rugged and reliable analytical techniques in support of the Resource Conservation and Recovery Act (RCRA) Program. The methods published in Test Methods for Evaluating Solid Waste, SW-846, are used to measure the concentration of specific pollutants or to establish whether a waste stream demonstrates a hazardous characteristic (e.g., ignitabiliry, corrosivity, reactivity or toxicity). SW-846 currently provides reliable and sensitive laboratory methods for the analysis of Appendix VIII analytes. However, some of these methods may be too costly or require too much analysis time for some applications. The Methods Section also recognizes the savings that could be achieved by sending only contaminated samples to analytical laboratories for quantitative analysis. Therefore, the Methods Section has recognized the need for more rapid, less expensive field screening procedures. A number of sources have developed reliable, reproducible and cost-effective field or screening procedures with potential application for the RCRA Program. This letter provides developers with a description of the type of performance data that is required for an effective initial evaluation of screening or field procedures. If a developer's data supports adoption of a new method, the Methods Section will work through the SW-846 Work Group process to promulgate it. This letter odes not supersede or replace the more rigorous requirement described in Test Method Equivalency Petitions, EPA/530-SW-87-008, OSWER Policy Directive No. 9433.00-2 (2/87). That document provides the requirements for a method equivalency petition which may be used to promulgate a method outside of the Work Group process. While screening procedures need not be fully quantitative, they should measure the presence or absence of target analytes at or below regulatory action levels. Therefore, initial demonstration of method performance involves measuring the percentage of false negatives and false positives generated using the procedure for a single sample matrix. Data should be submitted for split samples analyzed using the developer's technique and an approved SW- 846 quantitative methods. A candidate procedure should ideally produce no false negatives. Definition of a false negative is a negative response for a sample that contains up to two times the stated detection level of the target analyte(s). A candidate procedure should produce no more than 10% false positives. Definition of a false positive is a positive response for a sample that contains analytes at one half the detection level. Between 20 and 50 samples spiked at twice the detection level should be tested to establish the percentage of false negatives. It is recommended that a sufficient volume of each spiked sample be prepared to complete each test with one lot of material. Sufficient randomly selected aliquots of each spiked matrix should be analyzed by appropriate SW-846 methods to October 31, 1996 C-3 Interim Final Report Version 5.0 ------- demonstrate sample homogeneity and to characterize the sample in terms of target analytes and potential interferences. A separate study should also be conducted to establish the effect of non-target interferences. A screening procedure should produce no more than 10% false positives for a set of 20 samples that contains a 100 fold excess of interferences. Positive interferences should be selected that are chemically related to the target analytes and are environmentally relevant. Negative interferences (i.e., masking agents) should also be investigated whenever they are suspected. Developers should also analyze three different types of samples to provide matrix- specific performance data. These samples should either be characterized reference materials or spiked matrices containing known amounts of target analytes. In either case, bulk samples should be carefully homogenized to reduce sub-sampling errors. The sample matrices should be selected to represent what is regulated under RCRA (e.g., soil, oil waste or waste waters), not to provide the best performance data. Blanks should be analyzed with each set of samples. Matrix-specific performance data, including detection limits and dynamic range, are gathered by analyzing ten replicate aliquots of three different sample mattocks spiked at two concentrations. If spiked samples are used, suggested spiking levels are the matrix-specific detection limit and 50 times the detection limit. Positive or negative results should be reported for the low concentration samples. Results for high concentration samples should be reported as either semi-quantitative results or as positive/negative with the dilution factor used for the samples. Specific spiking concentrations are provided for selected target analytes in the attachments to this letter. The low values are normal reporting limits for routine analyses, and the high value is 50 times the low concentrations. The Methods Section recognizes that it may not be appropriate to spike all of the target analytes listed within a chemical class. If the developer has field data, the Methods Section would welcome the opportunity to compare the results obtained using the screening procedure with sample concentrations determined in a laboratory using SW-846 methods. To summarize, the Methods Section does not require an unreasonable body of data for the initial evaluation of new techniques. Data will need to be submitted on the percentage of false negatives, percentage of false positives, sensitivity to method interferences, and matrix-specific performance data in order to complete the table below. In addition to these data, the developer should also provide a description of the procedure and copy of any instructions provided with the test kits. October 31, 1996 C-4 Interim Final Report Version 5.0 ------- As part of the peer review comments oh a previous version of this document, Oliver Fordham, Manager of the Inorganic Methods Program, Office of Solid Waste, asked that the following material be included as part of this appendix. This information is printed as it was provided by Mr. Fordham. When reviewing a new monitoring technology, the first question is, "How well does it work with respect to measurement methodology." How well a technology works is related to such properties as: What is the lowest level of the property that the technology can measure in the types of materials to be tested? This property is commonly referred to as Method Sensitivity. In many monitoring situations, while the technology yields a measurement, the measured value is not a valid or accurate indication of the true value of the property of the sample. In most environmental monitoring situations, where accuracy is a problem, the methodology tends to "miss" some of the constituents for which it is being analyzed. In some cases, the measured value is less than what is actually in the sample. The difference between the measured value and the true value (i.e., the actual amount of the substance in the sample being determined) is termed the bias. While it is generally more common for measurement methods to give a low bias, in some cases, measurement methods indicate higher levels of a substance than are actually present and thus show a high bias. How reproducible is the method? This information is needed to determine what the minimum difference between samples or between the sample property and some action or regulatory level needs to be in order for the method to distinguish between the two within an acceptable level of confidence. This property is commonly referred to as Method Precision. For what types of materials or matrices is the technology suitable? What are the characteristics of the sample that are needed in order to use this method? Such characteristics include: physical state (e.g., liquid, gas, solid), concentration range of the analyte of interest (e.g., percent level, ppm level, trace levels), sample temperature, matrix composition (e.g.,organic, aqueous, inorganic, high salinity). Accurately knowing what matrix the method is suitable for is critical to a user trying to select an appropriate technique. Even when a method is generally suitable for use on a specific type of material, one often finds that the presence of certain other materials in the sample will interfere with obtaining reliable results. This property, termed interferences, needs to be determined. October 31, 1996 C-5 Interim Final Report Version 5.0 ------- Therefore, the minimum information that EPA requires when evaluating a new technology is: What substances or properties can be measured using this technology? • How accurate is the technique? • How precise? • How sensitive? What matrices is it suitable for use on? • What will interfere with obtaining accurate results? • What is its operating range? A number of sources have developed screening procedures for use in determining whether or not a sample possesses a property or analyte concentration above some specified level. For such tests, the properties of accuracy and precision need to be looked at differently. While such screening procedures need not be fully quantitative, they should measure the presence or absence of the target analytes at or above the regulatory or other action level with a high degree of confidence. Therefore, initial demonstration of method performance involves measuring the percentage of false negatives (i.e., no test response at an analyte concentration in the sample above the stated method sensitivity) and false positive (i.e, test responds to the sample when the sample contains less than the stated sensitivity). A candidate method should produce no more than 10% false positives. The Agency will consider a result to be a false positive if a positive test response is produced with samples containing the analytes of interest one half of the stated test detection level. We will now examine each of these attributes in terms of the specific data requirements involved. Applicable Analytes or Properties The documentation should describe the specific analytes or properties that the technology is applicable to. For each analyte listed, data shall be supplied that documents the performance of the technology on appropriate samples containing that analyte. If the technology is well enough understood, then claim of applicability can be made on the basis that a substance of interest has the same properties as a substance that has already been demonstrated to perform satisfactorily with the technology. However, for such a claim to be accepted, the developer must be able to describe the boundary conditions for the method' being examined. The boundary conditions are those points in the spectrum of sample/analyte properties where the technology ceases to perform well enough for the intended monitoring purposes. Bias Bias is given as the difference between the measured value and the true value for the property or analyte. Accuracy for methods that measure the concentration of a substance is generally given as the percent recovery range using laboratory spikes, or preferably, October 31, 1996 C-6 Interim Final Report Version 5.0 ------- reference samples having analyte concentrations determined by using methods of known accuracy. Precision Precision refers to the variability in measurements that are made on sample that have the same level of the property or analyte of interest. Precision for all methods except pH are presented as relative percent difference (RPD) of replicate analyses. Precision of pH measurements are listed in pH units. In general, data from at least five different materials, representing the variety of samples to be analyzed using this method, should be tested. For each material, both single laboratory precision based on triplicate analysis and multi- laboratory analysis using five or more laboratories shall be reported. Sensitivity Sensitivity or method detection limit (MDL) refers to the minimum concentration or other measure of the property that can confidently be determined by the method. The MDL is that concentration of the analyte that must be present in the sample for there to be less than a 5% probability that the analyte is not present in the sample. In addition to the MDL, the submitter shall describe the minimum concentration that can reliably be measured and ascribed a value to. This level is generally referred to as the Minimum Quantitation Limit (MQL). There are no requirements established by EPA for technology MDL and MQL. The required sensitivity is a function of the application and the corresponding regulatory or other action levels. It is the responsibility of the developer to identify the monitoring application for which method evaluation is being requested and to ensure that the submitted method possesses the requisite sensitivity. Matrices The submitter needs to determine what type of samples the methodology is suitable and demonstrate the claims using appropriate test materials. Listed in order of preference are appropriate test materials. Performance Evaluation (PE) samples which are samples whose matrix properties are nearly identical to those for which the method is claimed to be suitable for and contains measurable levels of the analytes of interest. The composition should be unknown to the analyst for it to be suitable for evaluating system performance. PE samples should come with statistics about each sample which have been derived from the analysis of the sample by a number of laboratories using well accepted reference methods. These statistics should include a true value of the property and a mean of laboratory results obtained from the analysis of the PE Sample. October 31, 1996 C-7 Interim Final Report Version 5.0 ------- Where PE materials having a matrix close to that for which the method is being evaluated are not available, laboratory spiked samples of the appropriate matrix composition should be used. A spike sample is a sample of the matrix to which a known amount of the analyte to be determined has been added. The successful use of spiked samples for method evaluation however depends on several factors. First a procedure for introducing the analyte of interest into the matrix in such a manner that the spiked sample will behave analytically like a "real world" sample must be available. For example, if a method is to be used to determine the concentration of lead in an incinerator ash that has been subjected to temperatures high enough to glassify the ash, then a spike sample prepared by adding a solution of a soluble lead salt to incinerator ash, then a spike sample prepared by adding a solution of a soluble lead salt to incinerator ash, stirring to coat the ash particles with the lead containing solution and then low temperature drying to remove water would not yield an acceptable method evaluation material. In the aforementioned glassified ash, the lead is incorporated into the glass matrix. Such lead is very difficult to solubilize and then measure. In a spike sample prepared as described above, the lead is all on the surface and readily available for extraction. Thus a method that is not very aggressive relative to its ability to "digest" the ash matrix would be a poor tool to measure lead concentration in the real world samples but show excellent results on the artificial sample. Where PE materials having a matrix identical are not available to that of those for which the method is to be used, and preparing laboratory spike samples is not feasible, then PE samples using a matrix that would be expected to pose a sufficient challenge to the method should be employed. An example of this situation is a procedure that is to be used for monitoring the concentration of pesticides in milk from nursing mothers. If reference samples of contaminated mother's milk is not available, and introducing the pesticide into the fat component of the milk might cause other problems, reference samples of contaminated cow's milk could be used. When submitting an application for EPA evaluation of an innovative monitoring technology, it is the submitter's responsibility to select the materials that are used in evaluating the method and to explain why the particular materials are appropriate. Interferences For any new technology, a study should be conducted to establish the effect of non- target analytes on the measurement process. The effect of substances that might yield positive interferences (i.e., measured values are higher than actual values) or negative interferences (i.e., measured values are lower than the actual sample concentration), also known as masking agents, should be investigated in order to determine the methodology's limitations. When evaluating the effect of potential interferents, samples with both low concentrations (1 to 10 times the MDL) and high concentrations (50 to 100 times the MDL) of the target analytes should be employed. October 31 1996 C-8 Interim Final Report Version 5.0 ------- Operating Range While its sensitivity is a primary of the Agency when evaluating a new technology, some methods can only be used on samples with a narrow concentration range. Also, some methods require controlled environmental conditions for satisfactory operation. Any such limitations should be described in the application. October 31, 1996 C-9 Interim Final Report Version 5.0 ------- This page intentionally left blank. October 31, 1996 C-10 Interim Final Report Version 5.0 ------- APPENDIX D REPRESENTATIVE DEMONSTRATION SCHEDULE The following identifies key events in a typical demonstration process. The analysis of results from the predemonstration study is the critical step in deciding to go forward with the demonstration. This is a dress rehearsal of events and is designed to remedy flaws in the experimented design, sampling process and reference laboratory performance. The developer also has an opportunity to resolve any matrix effects related to the site. October 31, 1996 D-l Interim Final Report Version 5.0 ------- This page intentionally left blank. October 31,1996 D-2 Interim Final Report Version 5.0 ------- REPRESENTATIVE DEMONSTRATION SCHEDULE CHARACTERIZATION AND MONITORING PROGRAM Task Completed Weekl Week 4 Weeks 4-6 TASK 1) Publication of a CBD announcement and solicitation of other likely sources for developer participation TASK 2) Developers' Conference - Time and place will be noted in announcement for Task 1. Developer describes technology and learns about the demonstration process. TASK 3) Review and Select Technologies - Separate into relevant categories to define the scope of the demonstration. Six to nine technologies with some common features or requirements. Notify developers of selection. TASK 4) Submission of proposals or letters of intent from the Weeks 6-8 developers. A requirement of the Guidance Manual which aids in the preparation of the demonstration plan. TASKS)* Preparation and Review of Demonstration Plan - Weeks 7-14 Based on the Guidance Manual for the Preparation of Characterization Technology Demonstration Plan. TASK 6) Selection of appropriate field sites- Concurrent with Task 5. Weeks 10-14 TASK 7) Selection and audit of reference laboratory - Concurrent with Weeks 10-14 Task 5. Predemonstration study in Task 9 confirms selection. TASK 8)* Review and approve draft demonstration plan by all participants. Weeks 14-16 TASK 9) Predemonstration Sampling - Critical element in final Weeks 15-16 selection of technologies and sites. Final test of reference laboratory performance. TASK 10)* Data review from Task 9. Receipt of laboratory data is the Weeks 16-22 major time factor. TASK 11) Revise and approve final demonstration plan - Weeks 21-23 Critical step in process. Need concurrences from all participants. Based on data from Tasks 9 and 10. TASK 12) Conduct field demonstration - Six to nine days at each site. Visitors Day - Travel and logistics. Weeks 24-27 October 31, 1996 D-3 Interim Final Report Version 5.0 ------- TASK 13)* Complete audit reports/receive reference laboratory data/ Weeks 27-33 report preliminary findings. TASK 14)* Prepare Innovative Technology Evaluation Reports - Internal Weeks 29-36 EPA review. TASK 15) Developer Review. Include draft verification statement. Weeks 36-38 TASK 16)* External Peer Review. Weeks 38-42 TASK 17)* Complete report - issue verification statements Weeks 43-46 * Indicates tasks where less time may be required NOTE: Given the specific nature of the technologies and other factors associated with a demonstration, it may be possible to shorten the project completion period by 4 to 6 weeks. October 31, 1996 D-4 Interim Final Report Version 5.0 ------- APPENDIX E GUIDANCE FOR ADDRESSING QUALITY ASSURANCE/QUALITY CONTROL (QA/QC) REQUIREMENTS IN A DEMONSTRATION PLAN This material was abstracted from the report Preparation Aids for the Development of Category II Quality Assurance Project Plans, EPA/600/8-91/004. The report was prepared by Guy F. Simes, National Risk Management Research Laboratory (formerly the Risk Reduction Engineering Laboratory), Cincinnati, Ohio, Office of Research and Development, U.S. Environmental Protection Agency. The Agency has determined that the level of quality assurance required in Category II is the most appropriate for planning and executing a technology demonstration that leads to the verification of cost and performance. This report contains guidance that should be useful to a developer or whoever is responsible for preparing demonstration plans. The technology demonstration plan is expected to include the elements described in this Appendix. The body of the guidance manual (Chapter 3) often refers to the appropriate sections of this appendix for additional detail or clarification. Since quality control measures are viewed as an integral part of a demonstration, the Consortium requires a single document which combines the technology description and quality control requirements. This document, as it was originally prepared, is intended to provide the reader with exhaustive guidance for preparing a stand-alone quality assurance project plan. As mentioned previously, the Consortium prefers to have all the necessary elements of QA/QC integrated into a single document - a demonstration plan. Therefore, the tone of the original document has been modified to provide the reader with useful information for preparing the QA/QC portions of the demonstration plan. From the foreword to the subject guidance document... Perhaps the greatest benefit to the users of this document comes from its emphasis on up- front planning—do the right thing, right, the first time. While this sounds straightforward, managers know that determining the right course in a complex and uncertain situation is anything but simple. Determining customer requirements up-front, and then having the processes and procedures in place to accomplish them, averts costly mistakes. Resources are conserved in two ways: by avoiding rework to correct efforts that do not initially meet management or customer specifications; and by performing to the specifications required and not beyond them. In these ways, this "Preparation Aids" document can help management achieve its mission with more effective utilization of diminishing resources. E. Timothy Oppelt, Director National Risk Management Research Laboratory Cincinnati, Ohio October 31,1996 E-1 Interim Final Report Version 5.0 ------- Table of Contents 1.0 INTRODUCTION E-5 1.1 Purpose of the Quality Assurance Project Plan (QAPP) E-5 1.2 Contents of a QA Project Plan E-5 2.0 PROJECT DESCRIPTION £-7 2.1 General Overview £.7 2.2 The Technology, Site, Facility, or System E-7 2.3 Statement of Project Objectives E-7 2.4 Experimental Design E-8 2.5 Schedule E-8 3.0 PROJECT ORGANIZATION AND RESPONSIBILITIES E-10 4.0 QUALITY ASSURANCE OBJECTIVES E-12 4.1 Determining QA Objectives E-13 4.2 Quantitative QA Objectives: Precision, Accuracy, Method Detection Limit, and Completeness E-13 4.3 Qualitative QA Objectives: Comparability and Representativeness E-14 4.4 What If QA Objectives Are Not Met? E-15 4.5 Other QA Objectives E-15 5.0 SITE SELECTION AND SAMPLING PROCEDURES E-18 5.1 Sampling Site Selection E-18 5.2 Sampling Site Description E-19 5.3 Sampling Procedures E-19 5.4 Sample Custody E-20 6.0 ANALYTICAL PROCEDURES AND CALIBRATION E-25 6.1 EPA-Approved or Other Validated Standard Methods E-25 6.2 Nonstandard or Modified Methods E-27 6.3 Calibration Procedures and Frequency E-27 7.0 DATA REDUCTION, VALIDATION, AND REPORTING E-28 7.2 Data Validation E-28 7.3 Data Reporting E-31 8.0 INTERNAL QUALITY CONTROL CHECKS E-32 8.1 Types of QC Checks E-32 9.0 PERFORMANCE AND SYSTEMS AUDITS E-34 10.0 CALCULATION OF DATA QUALITY INDICATORS E-36 10.1 Common Data Quality Indicators E-36 10.1.1 Precision E-36 October 31, 1996 E-2 Interim Final Report Version 5.0 ------- 10.1.2 Accuracy £.37 10.1.3 Completeness E-37 10.1.4 Method Detection Limit (MDL) E-38 11.0 QUALITY CONTROL REPORTS TO MANAGEMENT E-38 12.0 REFERENCES E-38 LIST OF TABLES Table E-1. Summary of Planned Analyses (Including QC) for Chemical Treatment of Water E-9 Table E-2. Expanded Sample Summary forNon-QC and QC Samples E-10 Table E-3. QA Objectives for Precision, Accuracy, and Method Detection Limits (MDL) . E-16 Table E-4. QA Objectives for Precision, Accuracy, and Detection Limits - Alternate Form E-17 Table E-5. Required Detection Limits for Volatile Chlorinated Organic Compounds E-18 Table E-6. Summary of Number of Samples Required for Hypothetical Incineration Test . E-22 Table E-7. Summary of Laboratory Analyses and Sample Quantity Requirement E-23 Table E-8. Required Containers, Preservation Techniques, and Holding Times E-24 Table E-9. Typical Summary Table of Standard Methods and Procedures E-26 Table E-10. Summary Table of Calibration Requirements for Process Measurements E-31 Table E-l 1. Scheduled QC and Calibration E-35 LIST OF FIGURES Figure 1. Typical project organization flowchart E-l 1 Figure 2. Example of a data reduction, validation and reporting scheme E-30 October 31, 1996 E-3 Interim Final Report Version 5.0 ------- This page intentionally left blank. October 31,1996 E-4 Interim Final Report Version 5.0 ------- 1.0 INTRODUCTION 1.1 Purpose of the Quality Assurance Project Plan (QAPP) The purpose of a QA Project Plan is to relate project objectives to specific measurements required to achieve those objectives. For purposes of a demonstration, those objectives are usually defined in terms of verifying the performance claims of the technology. The QA Project Plan must provide sufficient detail to demonstrate the following: • Intended measurements are appropriate for achieving project objectives; Quality control procedures are sufficient for obtaining data of known and adequate quality; and • Such data will be defensible if challenged technically. Technology demonstrations require the coordinated efforts of numerous individuals, including developers, managers, engineers, scientists, and statisticians. The QA Project Plan must integrate the requirements of everyone involved, in a form that permits an easy and rapid review. It must also provide unambiguous instructions to the sampling team, the analytical laboratory, and any other parties responsible for data generation. Finally, the QA Project Plan must provide sufficient detail to allow a thorough, detailed review by an independent party not involved in the demonstration. Because the end use of the data determines the degree of quality assurance that is required, the Consortium has determined that demonstrations will be conducted as Category II projects. Category II projects produce results that complement other inputs. These projects are of sufficient scope and substance that their results could be combined with those from other projects of similar scope to produce information for making rules, regulations, or policies. In addition, projects that do not fit this pattern, but have high visibility, are also included in this category. The technology demonstrations are usually assigned to this category. 1.2 Contents of a QA Project Plan A QA Project Plan must cover the following topics: Technology being tested and the objectives of the demonstration (i.e., the hypothesis to be tested, for purposes of a demonstration, this is technology performance.); Type and number of measurements that will be taken to achieve those objectives; Quality of data that will be required and how that quality will be obtained; and • How the data will be recorded, calculated, reviewed, and reported. Be sure that the final demonstration plan also includes: October 31, 1996 E-5 _ Interim Final Report Version 5.0 ------- Any technology-specific information necessary for the sampling team, analytical laboratory, data reduction team, and other project participants. However, assume these parties are familiar with standard methods. Any deviations from standard methods or procedures, or clarification of such changes whenever there are ambiguities. The demonstration plan is an integrated document which contains the key elements of a QAPP and should not be repetitious. For example: • Do not discuss the same subject matter more than once in the demonstration plan. Although the required elements of the QA Project Plan may appear to overlap in some cases, repetition leads to confusion and to documents too long for effective communication. If you are unsure where a subject should be treated, discuss it once and cross-reference that discussion in the other sections. • Do not repeat material from Section 6.0 (Sampling Plan) of the demonstration plan; reference the section and page number in the demonstration plan as needed. • Do not repeat material from standard methods that are available from the EPA or any other official source. Provide detailed citations and assume that the analyst and reviewer have these methods on-hand. 1.3 Category II Format Notes As a Category II level activity, demonstration plans must incorporate the following: • Title page. • Technology Demonstration Plan Approval Form (Signature Page).It is important that the project principals understand and agree on the experimental approach. For this reason, the Technology Demonstration Plan Approval Form must be signed by all key project personnel. These signatures, which must be obtained before the demonstration is started, indicate that the key personnel have read the appropriate sections of the demonstration plan and are committed to full implementation. Table of Contents Document control format. Section number, revision, date, and page should be recorded in an upper corner of each page. This format requirement is illustrated below: Section No. 1.0 Revision: 0 Date: December 1, 1994 Page: 3 of 5 October 31, 1996 E-6 Interim Final Report Version 5.0 ------- 2.0 PROJECT DESCRIPTION The information in this section should be useful for preparing the project description. It describes the technology or environmental system that is to be tested, the project objectives, a summary of the experimental design, and the proposed project schedule. 2.1 General Overview This section provides a brief synopsis of the overall demonstration. The purpose(s) of the study should be described, along with the decisions that are to be made and the hypothesis to be tested. Other anticipated uses of the data should also be noted. The type of technology or environmental system that is to be tested should also be described. The discussion should also include results of any pertinent or relevant preliminary investigations or data generated in the pre-demonstration study. Example: Preliminary investigations of the test site indicated an average pentachlorophenol concentration of 150 mg/kg, with a range of 10 to 500 mg/kg. 2.2 The Technology, Site, Facility, or System Provide a description of the technology, facility, or environmental system that will be tested. Include flow diagrams, maps, charts, etc., as needed. Approximate sampling mass or volumetric flow rates should be indicated to permit proper evaluation of the sampling or process monitoring procedures. Additional diagrams are often included to unambiguously describe sampling points. The discussion should contain enough material to permit a technical reviewer who is unfamiliar with the specific project to assess the technology and sampling strategy. 2.3 Statement of Project Objectives The project objectives should be summarized and stated clearly. Avoid scattering statements of project objectives throughout the sampling and analytical sections of the demonstration plan. Demonstration objectives should be stated in numerical terms whenever possible: Poor Statement: The Tri-Corder will be characterized with respect to its ability to measure semivolatile organic compounds. Better Statement: The major objective is to demonstrate a Tri-Corder for an accuracy of 95 percent for the compounds listed in Table... Best Statement: The objective is to demonstrate a Tri-Corder for an accuracy of 90 percent or higher at a confidence level of 95 percent for the compounds listed in Table... It is common to rank project objectives according to importance (e.g., into primary and secondary objectives). Although this ranking is not essential, it does help focus effort on the primary goals of the demonstration. October 31, 1996 E-7 Interim Final Report Version 5.0 ------- 2.4 Experimental Design List all measurements that will be made during the demonstration, then classify them as critical or noncritical measurements. Critical measurements are those that are necessary to achieve project objectives; they may include either on-site physical or chemical measurements. Noncritical measurements are those used for process control or background information. Summarize in tabular form all measurements that are planned for each sample. Ideally, this table should indicate the total number of samples for each sample point, including quality control and check samples, as illustrated in Table E-l for a hypothetical chemical treatment project. Relate the individual samples to the sampling points shown in a process diagram, if applicable. For projects involving a large number of samples or analyses, it may not be possible to include all QC and check samples in a single table. For such cases, two or more tables may be necessary, the first to summarize the primary (non-QC) samples, and the others to show which QC samples are associated with which analyses, as is show in Table E-2. Information on the total number of all sample types is needed both to perform a comprehensive review and to estimate. 2.5 Schedule Indicate start-up and ending dates, including those for preliminary field studies and laboratory activities. In developing a demonstration timeline, be sure to: • Allow adequate time for document review and revision. • Be realistic regarding a possible disruption in field activities due to weather conditions. Coordinate these activities with the appropriate members of the technology demonstration. October 31 1996 E-8 Interim Final Report Version 5.0 ------- Table E-1. Summary of Planned Analyses (Including QC) for Chemical Treatment of Water Semivolatile Organic Compounds (SVOC)" Non-QC (Primary) Field Sampling Blankc Field Duplicates Laboratory Duplicates Matrix Spikes (MSs) Matrix Spike Duplicates (MSDs) Spare Samples" Independent Check Standard Preliminar y Samples 3 1 0 0 1 1 2 0 Number of Tests Performed* Influent Water Sample Points: C1 60 1 0 0 3 3 10 1 Effluent Water Sample Points: E2 60 0 0 0 10 10 10 0 Sludge Sample Points: S3 10 0 5 0 2 2 0 0 Grand Total Metalsc Non-QC (Primary) Field Sampling Blank6 Field Duplicates Laboratory Duplicates Matrix Spikes (MSs) Matrix Spike Duplicates (MSDs) Independent Check Standard Spare Samples" 3 1 0 1 1 0 0 2 60 1 0 3 3 0 1 10 60 0 0 10 10 0 0 10 10 0 5 2 2 0 0 0 Grand Total Tota I 133 2 5 0 16 16 22 1 195 133 2 5 16 16 0 1 22 195 a Samples will be divided evenly among the ten anticipated runs. More QC samples are planned for the effluent than for the influent, since the former samples are expected to exhibit more variability. A matrix spike/matrix spike duplicate or a laboratory duplicate/matrix spike of the effluent stream will be determined for each treatment condition, because the effluent matrix may vary significantly with each treatment b Refers to the twelve compounds listed in Table c Trip blanks will be collected but not analyzed unless field blanks indicate a contamination problem. d Not analyzed unless required. October 31, 1996 E-9 Interim Final Report Version 5.0 ------- This table is provided to illustrate a typical format Table E-2. Expanded Sample Summary for Non-QC and QC Samples Measurement TCLP-Metals SVOCs-Total • Type Non-QC Samples Leachate Duplicate Leachate Blank Reference Material Matrix Spike Non-QC Samples Sample Duplicate Sample Equipment Blank Method Blank Independent Check Std Matrix Spike Matrix Spike Duplicate Raw Soil 9 1 1 1 0 9 3 1 1 1 1 1 Treated Soil (28 Days) 9 1 1 1 1 9 0 1 1 1 1 1 Reagent Mix 1 0 0 0 0 1 0 0 0 0 0 0 Long-Term Treated Soils 15 3 3 3 0 0 0 0 0 0 0 0 Total 34 5 5 5 1 19 3 2 2 2 2 2 This table is provided to illustrate a typical format 3.0 PROJECT ORGANIZATION AND RESPONSIBILITIES The demonstration plan must show that the project organization is adequate to accomplish demonstration goals, and that all responsibilities have been assigned. This will establish proper communication links between key personnel. Information flow regarding schedules, meetings and other commitments regarding the technology demonstration must be defined. Providing a table or chart illustrating project organization and lines of authority needed to conduct the technology demonstration is recommended. Identify by name all key personnel for each organizational entity and give their geographic locations and phone numbers. The organizational chart should also include all third party participants and their points of contact. The organizational chart should identify all QA Managers and should illustrate the relationship to other project personnel. The QA Managers should be organizationally independent of the project management so that the risk of conflict of interest is minimized. Figure 1 illustrates a typical project organizational chart. There should also be a description of the responsibilities of all project participants, including QA Managers. Be sure to indicate responsibility for each type of analysis, physical measurement, or technology operation. This summary should designate responsibility for planning, coordination, sample collection, sample custody, analysis, review, and report preparation. This document must identify the person responsible for making final recommendations to the EPA technical lead and the verification entity. A conflict resolution process may be required to reach a final recommendation. The frequency and mechanisms of communications among the developer, the QA Manager, the EPA Project Manager, as well as other participants must be discussed. Provide a regular October 31, 1996 E-10 Interim Final Report Version 5.0 ------- schedule for progress reports, site visits, and teleconferences, and describe any special occurrences would trigger additional communication. In developing this information, be sure to: Demonstrate that the QA Manager is independent of project organization. Since each participant may have a QA responsibility, a QA Representative must be identified for each organization. • Provide names, locations, affiliations, and telephone numbers for key personnel (those who have authority to make final recommendations to the EPA technical lead). • Show independence of the confirmatory laboratory from the developer and verification entity. • Describe who has responsibility for sample collection and chain-of-custody control procedures. Individuals who have access to the demonstration site must be clearly identified. • Recognize the technology demonstration timeline when establishing a schedule for meetings and project deliverables. October 31,1996 E-11 . Interim Final Report Version 5.0 ------- EPA PROGRAM MANAGER EPA TECHNICAL LEAD VERIFICATION ORGANIZATION DEVELOPER QUALITY ASSURANCE MANAGER SITE MANAGER (EPA/OWNER) LABORATORY MANAGER SAMPLING COORDINATION DEVELOPER DEMONSTRATION MANAGER Figure 1. Typical project organization flowchart 4.0 QUALITY ASSURANCE OBJECTIVES Quality assurance objectives are specifications that measurements must meet in order to achieve project objectives. For example, in ordering a pump for a chemical plant, factors such as capacity, pressure, and materials of construction must be specified. Similarly, precision, accuracy, detection limits, and completeness must be specified for physical or chemical measurements. Additional analytical requirements may be described qualitatively in terms of representativeness and comparability. Quality assurance objectives are needed for all critical measurements and for each type of sample matrix (soil, water, biota, etc.). Be sure to include quality assurance objectives for physical as well as chemical measurements. Once the demonstration performance requirements and QA objectives are established, the measurement systems must be designed to meet them. The project manager, analytical chemists, and other principals must agree on the feasibility and appropriate value of these performance objectives. October 31,1996 E-12 Interim Final Report Version 5.0 ------- 4.1 Determining QA Objectives QA objectives must be defined in terms of demonstration requirements, and not in terms of the capabilities of the intended test methods. Of course, the QA objectives must be achievable by available methods, and for this reason it is important that the laboratory review the QA objectives. When QA objectives exceed the capabilities of available methods, either the methods must be modified or the test plan must compensate for these deficiencies. Modifications may often be as simple as collecting a larger sample. However, if nonstandard or significantly modified test methods are required, Section 7 (QA Project Plan) of the demonstration plan must include laboratory validation data to prove that the method is capable of achieving the desired performance. The following are examples of how QA objectives can be determined: Example 1: Biological treatment ofpentachlorophenol (PCP). Previous experience -with this treatment process has indicated that the output concentration typically varies by a factor of two under stable operating conditions with uniform feed. To avoid contributing additional error, the precision of the analytical methods should be significant less than this value or approximately ±30 percent or less. Example 2: Determination of destruction removal efficiency (DEE) for incineration of a Dinoseb formulation containing 20 percent wA> Dinoseb. The purpose of this test is to demonstrate a DRE of 99.99 percent or better. Dinoseb in stack gases will be collected on a Modified Method 5 sampling train. Extracts of all components of this train will be combined and reduced to 0.5-mL volume. To allow a reasonable margin of error, the detection limit required for the Dinoseb determination will be 40 ug/MM5 train. Previous experience with the determination of Dinoseb on XAD resin has shown that this detection limit can be achieved routinely with the method attached as Appendix B. Previous experience has also shown that Dinoseb can be recovered routinely from a MM5 train with a recovery 2 50 percent and a precision of 230 percent relative percent difference (RPD), and deviations beyond these ranges indicate analytical problems. Results of a methods validation study performed in our laboratory are attached as Appendix C. 4.2 Quantitative QA Objectives: Precision, Accuracy, Method Detection Limit, and Completeness QA objectives for precision, accuracy, method detection limit, and completeness should be presented in a QA objectives table similar to those shown in Tables E-3, E-4, and E-5. Be sure to include QA objectives for all matrix types and to indicate the units in which these QA objectives are given. Summary tables are very helpful to the confirmatory laboratory which must meet these objectives. Because precision and accuracy can be measured in various ways, explain the method to be used. If precision, for instance, is to be determined by duplicates, explain whether sample October 31,1996 E-13 - Interim Final Report Version 5.0 ------- splitting will occur in the laboratory, during sampling, or at some other stage. Then summarize all such information in either text or tabular format. Be sure to include the number and type of samples in the sampling plan. The following statements are examples of descriptions for precision, accuracy, method detection limits and completeness: Precision objectives for all the listed methods except pH are presented as relative percent difference (RPD) of field duplicates. Precision objectives for pH are listed in pH units and expressed as limits for field duplicates. Accuracy objectives for organic compounds and metals are given as percent recovery range of laboratory matrix spikes. Accuracy objectives for temperature measurements are absolute deviations in °C. Detection limits are defined as the method detection limit (MDL) multiplied by the dilution factor required to analyze the sample. MDLs will be determined by replicate extraction and analysis of seven identical spiked samples ofXAD resin. Completeness is defined as the number of measurements judged valid compared to the number of measurements needed to achieve a specified level of confidence in decision making. List QA objectives according to compound type, as was done for the semivolatile detection limits in Table E-3. In other cases, where detection limits are derived from applicable regulations, list detection limits for individual compounds, as in Table E-5. Finally, it is important to explain how the QA objectives are to be interpreted in a statistical sense. QA objectives are often interpreted in a sense that all data must fall within these goals; for such projects any data that fail to satisfy the QA objectives are rejected and corrective action is undertaken. However, other interpretations are possible. For example, the project requirements may be satisfied if the average recovery is within the objectives; that is, excursions beyond the objectives might be permitted, but the average recovery would have to satisfy the goals. Whatever the case, it is important to describe in this section how tabulated QA objectives will be interpreted. 4.3 Qualitative QA Objectives: Comparability and Representativeness Comparability is the degree to which one data set can be compared to another. For instance, to evaluate an environmental cleanup process, analyses of the feed and discharge streams must be comparable. Similarly, to perform a nationwide environmental survey, methods used at different locations must be comparable. Comparability is achieved by the use of consistent methods and by traceability of standards to a reliable source. The choice of a comparative method is a critical element in the demonstration. A discussion of the selection process will comprise a separate section in the demonstration plan. The laboratory conducting the confirmatory analysis must be shown to be independent of the technology developer. The laboratory must also demonstrate knowledge and experience with the selected confirmatory method. October 31, 1996 E-14 Interim Final Report Version 5.0 ------- Representativeness is the degree to which a sample or group of samples is indicative of the population being studied. An environmental sample is representative for a parameter of interest when the average value obtained from a group of such samples tends towards the true value of that parameter in the actual environment, as the number of representative samples is increased. Representativeness is normally achieved by collecting and analyzing a sufficiently large number of unbiased samples. A critical element in this process is often the cost per analysis. A choice must be made between conducting many simple analysis, which are often less informative, versus fewer complex, more expensive, but data rich analysis. The economics of conducting a demonstration must always be considered in deciding how to design the confirmatory studies. It should also be understood that the confirmatory method and laboratory will have certain precision and accuracy limitations. A critical review of laboratory procedures will be required to insure that a fair evaluation of the demonstration technology can be made. 4.4 What If QA Objectives Are Not Met? There should be some discussion concerning the impact of not meeting one or more QA objectives. Will the demonstration be a complete loss? Will some, but not all, of the demonstration goals still be realized? Will the statistical confidence level be reduced? Are there legal or regulatory ramifications? Answers to such questions will help provide a critical perspective on the QA program being applied to this demonstration and the overall value of the demonstration. 4.5 Other QA Objectives Some demonstrations may require additional QA objectives, requirements for such QA measurements should be stated. This may also include a discussion of additional confirmatory analysis or the use of a referee laboratory. October 31, 1996 E-15 - Interim Final Report Version 5.0 ------- Table E-3. QA Objectives for Precision, Accuracy, and Method Detection Limits (MDL) Critical Measurement Semivolatile organic compounds" (SVOC) Volatile chlorinated organic compounds Metals As Ba Cd Cr Pb Flow rate Temperature Matrix Water Soil Water Soil/water Soil/water Soil/water Soil/water Soil/water Water/air Water/air Method 8270 8270 601 7060 (GFAA) 7080 (FAA) 7131 (GFAA) 7190 (FAA) 7421 (GFAA) Rotameter Thermocouple Reporting Units M9/L ug/kG ug/L ug/L ug/L ug/L ug/L pg/L L/min °C MDL 20 600 (e) 5' 1000' 0.5' 600' 1' . - Precision (RPD)' 30 40 50 35 35 35 35 35 . - Accuracy (% Recovery)" 50-150 50-150 50-150 80-120 80-120 80-120 80-120 80-120 ±5g 2h Completeness' 200 200 300 200 200 200 200 200 100 100 * Given as Relative Percent Difference of laboratory duplicates, unless otherwise indicated. b As percent recovery of matrix spike, unless otherwise indicated. c Based on the number of measurements needed to achieve % level of confidence in decision making permit reanalysis, as required.) d All of the Principal Organic Hazardous Compounds (POHCs) of interest to this project will be required to compounds determined by this method need not satisfy these objectives. • MDLs are given in Table . (See example Table 3-3) ' MDL for water samples. MDL for soils is 100 times greater. Reporting units for soils will be mg/kg. 0 Maximum permitted deviation against volumetric standard. h Absolute deviation against ASTM thermometer. (Extra samples will be collected to meet these objectives. Other This table is provided to illustrate a typical format ------- Table E-4. QA Objectives for Precision, Accuracy, and Detection Limits - Alternate Form Data Quality Parameter Method of Determination Frequency Required Objective*-"' Semivolatile Organic Compounds Precision -Water - TCLP leachates Accuracy -Water& TCLP leachates - Water & TCLP leachates - Water - MM5 train Detection limits - Water - MM5 train Field duplicate Duplicate leaching of laboratory - split sample Laboratory matrix spike Surrogate NIST standard reference materials Spike of XAD resin 7 analyses of spiked clean water 3 analyses of spiked XAD resin® 1/test condition 1/test condition 1 water/test condition 1 leachate/test condition All samples 1 /project 3/project 1 before project 1 before project RPD <50 RPD<60 Recovery = 50-150% (b) Recovery = 50-150% Recovery = 50-150% MDL*10 ug/l_neutrals MDL s20 |jg/L,phenols MDL s100 pg/L, bases MDL s20 pg * RPD = relative percent difference. MDL = method detection limit. b As specified in Method 8270. Note (1) Objectives must be met for all critical measurements. (See Section 1.2 of this document.) Note (2) MDLs must be calculated according to Equation (8), Section 9.0 of this document, which takes into account the number of low-level samples analyzed. This table is meant to illustrate a typical format October 31,1996 E-17 - Interim Final Report Version 5.0 ------- Table E-5. Required Detection Limits for Volatile Chlorinated Organic Compounds Compound 1,1,1 -Trichloroethane 1,1-Dichloroethane 1,1-Dichloroeth8ne Vinyl chloride 1 ,2-Dichloroethane Perchloroethylene Regulatory Threshold (pg/L) 5 5 5 2 1 5 Required MDL (pg/L) 0.5 0.5 0.5 i 0.2 0.1 0.5 1 Method detection limits for these compounds are critical and will be determined experimentally by the laboratory before sample collection is started. MDLs must be well below the Regulatory Threshold in order to demonstrate that treated waters meet discharge limits. This table is provided to illustrate a typical format 5.0 SITE SELECTION AND SAMPLING PROCEDURES Provide a discussion which describes a plan for site selection and sampling which is responsive to the demonstration objectives. A detailed discussion of the sampling plan development process is presented in Volume 2 of SW-846, Test Methods for Evaluating Solid Waste (3rd Ed.). Since SW-846 will likely undergo revisions, be sure to refer to the latest revision or edition. The narrative should explain the overall sampling strategy, the specific sampling procedures that will be employed, sample custody, record keeping, and shipping rec'.'irements. 5.1 Sampling Site Selection For most technology demonstration projects, the sampling sites will have been identified and preliminary data will be available. Explain how specific sampling locations will be selected for the technology demonstration. The following information should be provided: • A qualitative statement of the sample population to be represented by the samples; A description of the statistical method or scientific rationale to be used in selecting sample locations; October 31,1996 E-18 Interim Final Report Version 5.0 ------- Descriptions of the type of sampling strategy (e.g., simple, stratified, systematic random sampling); • A description of the sample types (air, water, soil, biota); A qualitative statement regarding potential sources of sample contamination; and, • A discussion of the extent to which site selection will affect the validity of the resulting data and demonstration objectives. 5.2 Sampling Site Description Provide relevant charts, maps, sampling grids, or tables specifying the exact sampling locations. Note any site modifications or additions that will be needed prior to sampling and describe all locations and access points for critical process measurements such as flow rates, pressures, or temperatures. Discuss any site-specific factors that may affect sampling procedures. For each analyte and each sampling point, list the frequency of sample collection and the total number of samples collected. This summary can best be prepared in tabular form, as shown in Table E-6. Note that the numbers given in this table may differ from the number of analyses, since some samples may be analyzed in replicate, while others may be analyzed only on a contingency basis. Explain the statistical basis for the sampling scheme, as necessary. 5.3 Sampling Procedures Describe the specific procedures that will be used for collecting and preserving samples. • Discuss each sampling procedure that will be employed. For EPA-approved procedures, a reference is sufficient; other sampling procedures should be summarized hi the text, and additional details should be provided in an Appendix. Prepare a list of analytes, sample volumes to be collected, and the amount of sample that is required for each analysis (see Table E-7). Note whether the required amount is intended for matrix spike/matrix spike duplicate determinations or for a single determination. Be sure to review this table to ensure that the sample volume or mass is sufficient for all intended confirmatory analyses. Describe any compositing or sample splitting procedures that will be employed in the field or laboratory. October 31,1996 E-19 Jnterim Final Report Version 5.0 ------- • Describe any sampling equipment that will be used, and how this equipment will be calibrated. • Explain how sample containers will be cleaned to prevent sample contamination, and how new sample containers will be checked for contaminants. • Describe the containers used for sample collection, transport, and storage for each sample type. Include sample preservation methods, noting specific reagents, equipment, supplies, etc., required for sample preservation, and the specific time requirements for shipping samples to the laboratory. Note refrigeration conditions and holding times that will be employed. (See, for example, Table E-8.) • Describe the procedures used to record sample history, sampling conditions, and any other pertinent information; include examples of forms that will be employed. Describe the numbering sequence to ensure that each sample will be assigned a unique number. • Include an example of the sample label to be used. 5.4 Sample Custody Occasionally samples are spilled, contaminated, accidentally evaporated to dryness, or otherwise compromised before or during sampling and analysis. Sample custody allows detection of such problems should they occur and minimizes such occurrences by assigning responsibility for all stages of sample handling. Sample custody is maintained when the samples are in a secure area or are in the view of, or under the control of, a particular individual. Records of everyone handling samples are maintained so that a sample history can be reconstructed later, should the need arise. In developing appropriate procedures, be sure to: • Give the names of all sample custodians in the field and in each laboratory. • Give examples of forms that will be used to maintain sample custody in the field, during shipping, and in the laboratory. Seal shipping containers with chain-of-custody seals. Give an example of the seal that will be used. • Describe procedures that will be used to maintain chain of custody during transfer from the field to the laboratory and within the laboratory, October 31,1996 E-20 Interim Final Report Version 5.0 ------- Provide for archiving of all shipping documents and other paperwork received at the laboratory with the samples. Chapter 9 of SW-846 (3rd Ed.) provides additional discussion of shipping and custody procedures. Since SW-846 will likely undergo revisions, be sure to refer to the latest version. Table E-6. Summary of Number of Samples Required for Hypothetical Incineration Test Description/Use SVOCs1 Non-QC Samples + MS + MSD" Sample Duplicates Sample Blanks Spare Samples' Modified Method 5 Non-QC Samples Sample Blanks (see text) VOCs1 Non-QC Samples Sample Blanks Trip Blanks" Spare Samples0 Slurry Feed (F1) 10 3 1 5 0 0 20 1 1 20 Scrubbe r Sump (S1) 10 0 1 5 0 0 20 1 1 20 Scrubber Blowdow n (S2) 10 0 0 5 0 0 20 1 1 20 Ash (S3) 5 0 0 5 0 0 10 1 1 10 Stack (S4) 0 0 0 0 4 1 0 0 0 0 Total 35 3 2 20 4 1 70 4 4 70 * As designated in Section . " Each sample will provide enough material for an original sample plus a matrix spike and a matrix spike duplicate. c Not analyzed unless first analysis fails. " Not analyzed unless sample blank is contaminated. This table is meant to illustrate a typical format October 31,1996 E-21 Interim Final Report Version 5.0 ------- Table E-7. Summary of Laboratory Analyses and Sample Quantity Requirement Stream Test Soil Treated soil Scrubber makeup scrubber liquor Scrubber solids Stack Sampling Method Shelby tubes ASTM-C172/ composite'2' Grab Thief Method 5 Midget M-5(2) Metals train® Modified Method 5 NIOSH-IOOa® Analysis Parameter Semivolatiles Volatiles Metals scan Dioxins/furans EP toxicity TCLP Higher heating value Chlonne Moisture Semivolatiles Volatiles Metals scan Dioxina/furans TCLP Semivolatiles Volatiles Metals scan Dioxins/furans Semivolatiles Volatiles Metals scan Dioxin/furans Partioilate HCI Metals Semivolatiles Dioxins/furans Volatiles Container Size 250 mL 250 ml 250 mL 250 mL 250 mL 250 mL 250 mL 250 mL 250 mL 250 mL 250 mL 250 mL 250 mL 250 mL 1 L 40 mL VGA vial 1 L 1 L 250 mL 250 mL 250 mL 250 mL Sample Quantity Required for Analysis 50 g 50 g 100 g 50 g 250 g 500 g 250 g 250 g 100g 50 g 50 g 100g 50 g 500 g 1 L 40mLVOAvial 1L 1L 50 g 50 g 100g 50 g 900 L 900L 900L 3000 L 3000 L 20 L Note (1) Indicate whether the listed quantity suffices for replicates, spikes, and other QC purposes, or if it is sufficient for only a single analysis. In the latter case, indicate how extra material will be provided for QC purposes. Note (2) Copies of these methods would have to be appended because they are not readily available. This table is meant to illustrate a typical format October 31, 1996 E-22 Interim Final Report Version 5.0 ------- Table E-8. Required Containers, Preservation Techniques, and Holding Times Measurement Extractable organics Pesticides, PCBs Metals (except mercury and chromium VI) Mercury Chromium VI pH Residue Organic carbon, total Sulfide Type- G Teflon-lined septum G Teflon-lined septum P.G P.G P,G P,G P.G P.G P.G Preservation" Cool to 4°C, protect from light Cool to 4°C, pH 5-9 HN03 to pH <2 HN03 to pH < 2 Cool, 4°C None required Cool, 4°C Cool, 4°C, HCI or H2S04topH<2 Cool to 4°C; add zinc acetate plus NaOH to pH<9 Maximum Holding Times0 7 days until extraction, 40 days after extraction 7 days until extraction, 40 days after extraction 6 months 28 days 24 hours Analyze water immediately (on site), none specified for soil 7 days 28 days 7 days a Polyethylene (P) or glass (G). b Sample will be preserved immediately upon sample collection. c Samples will be analyzed as soon as possible after collection. The times listed are the maximum times that samples will be held before analysis and still be considered valid. All data obtained beyond the maximum holding times will be flagged. This table is meant to illustrate a typical format October 31, 1996 E-23 Interim Final Report Version 5.0 ------- 6.0 ANALYTICAL PROCEDURES AND CALIBRATION The information in this section can be used in preparing Sections 6 and 7 of the demonstration plan. It describes all of the field and laboratory procedures used for both chemical and physical measurements. Sample preparation methods and cleanup procedures (such as extraction, digestion, column cleaning) should also be included. All methods must be appropriate for their intended use and be described in sufficient detail. When coupled with appropriate QC procedures this should provide enough detail to permit the analytical chemists or other measurement specialists to carry out their procedures. Most confirmatory analysis will rely on EPA-approved methods that have been validated for environmental samples. Standardized procedures from other organizations, such as the ASTM and the American Public Health Association, are also commonly employed when approved or validated EPA methods are unavailable or inappropriate. When used, nonstandard methods must be described in detail. It is essential to consult the analytical laboratory and other measurement specialists for guidance because they know the intricacies and limitations of the methods. Close coordination with the EPA technical lead will also be required since they must approve selection of the confirmatory method. Demonstrations often involve many different kinds of measurements. A separate table must be provided for each type of test. Specify the parameter to be measured, the sample type, the method number (when available), the title, the method type, and a reference (See Table E-9). Note the revision number of the method or the edition number of a publication, since nominally identical methods may be modified by these updates. Provide additional project-specific information as necessary in the demonstration plan. 6.1 EPA-Approved or Other Validated Standard Methods EPA-approved or similar validated methods can be incorporated by reference. Once a method is cited, do not repeat information that is already found in the method. Some additional information is almost always required, however, to assure that demonstration-specific requirements will be met. The following considerations apply: • Some EPA-promulgated methods contain only general procedure descriptions and lack specific QC requirements or applicable validation data. For example, EPA Method 18 provides general requirements for GC analysis of stack gases, but contains no QC requirements. For such methods, validation data pertinent to the specific project must be appended. A preliminary method validation can be specified as a subtask of the demonstration; in that case specific procedures and acceptance criteria for method validation must also be included as part of the demonstration plan. • Other EPA-approved standard methods, such as those found in Standard Methods for the Examination of Water and Wastewater, give operating procedures but omit most QC and calibration requirements. This information must be provided in this October 31, 1996 E-24 Interim Final Report Version 5.0 ------- section of the demonstration plan. Be sure to specify the frequency, acceptance criteria, and corrective action plans for all QC procedures and calibrations. • It is not necessary to include copies of methods or sections of methods from SW-846, the Code of Federal Regulations, or Methods for Chemical Analysis of Water and Waste because these sources are readily available. However, do append ASTM and NIOSH (National Institutes for Occupational Safety and Health) procedures because they may not be so readily available to other project principals or reviewers. • EPA-approved or similarly validated methods that are significantly modified are considered to be invalid and are treated as described in the next section. • Certain EPA methods such as those found in SW-846 specify most operating details, including quality control and calibration requirements. Such procedures, however, frequently allow the user to specify certain other options to satisfy project objectives. For example, for multianalyte methods such as GC/MS, the user will typically specify the target compound list that is required for the project. Matrix spike compounds are chosen from the compounds of interest to the particular project, and are not necessarily those recommended in the method. List the project-specific target compounds to be used as calibration check compounds and matrix spike compounds. Specify the acceptance criteria for matrix spike compounds. In certain cases, isotopically labeled forms of the project analytes may be included as surrogates. Table E-9. Typical Summary Table of Standard Methods and Procedures Parameter Polychlorinated Biphenyls (Leaching Tests) Sample Type' L/LTL U/RM/T 3520/3540 Extracts 3520/3540 Method Number EPA Method 3520 EPA Method 3540/3620 EPA Method 8080 EPA Method 680 (Backup) Method Title Continuous Liquid- Liquid Extraction Soxhlet Extraction/ Florisil Column Cleanup Organochlorine Pesticides and PCBs Determination of Pesticide PCBs in Water and Soil/ Sediment by GC/MS Method Type Extraction Extraction/ Cleanup GC/ECD GC/MS Source SW-846 SW-846 SW-846 EPA aS = Treated waste, while still a slurry U = Untreated waste RM = Reagent mix T .= Treated water LT = Treated water, long-term monitoring L = Leachate LTL = Leachate, long-term monitoring This table is meant to illustrate a typical format October 31, 1996 E-25 Interim Final Report Version 5.0 ------- The following is an example of how one might specify various options allowed by Method 8080, a validated method from SW-846 which contains QC and calibration requirements: Example: Method 8080. This method will be employed to determine the 19 pesticides listed in Table _-_ of this QA Project Plan. Although PCBs can also be determined by this method, the GC will not be calibrated for these products unless they are observed The matrix spike compounds will be the six critical pesticides listed previously in this QA Project Plan. The surrogates will be tetrachlorometaxylene (not dibutylchlorendate). Detection will be by electron capture detection (ECD). Quantitation will be by external calibration. Acceptance criteria for surrogate recovery will not be determined by control charts, but must be within a 50-150 percent range. Acceptance criteria for matrix spike matrix spike duplicates are those stated in Table _- of this QA Project Plan Extraction and cleanup procedures are described below... 6.2 Nonstandard or Modified Methods Any nonstandard procedure must be described in detail in the format of an EMMC method and appended to the demonstration plan. Validation data applicable to the expected samples must also be included. Validation data must demonstrate that the analytes of interest can be determined without interferences in the expected matrices, and that precision, accuracy, and detection limits will be adequate for the intended use of the data. The method is validated only for samples expected from the demonstration, and not for general environmental usage. Once the SOP is written and the validation data are accepted, the method is validated only for use on that specific project and on the specific sample matrix associated with that demonstration. This validation process for a project specific analysis does not constitute EPA approval for other projects or matrix types. All nonstandard methods must be validated prior to approval of the demonstration plan. 6.3 Calibration Procedures and Frequency Discuss the calibration procedures for each analytical or measurement system used to obtain data for critical measurements. Each description should include the specific calibration procedure to be used and the frequency of calibration verification. In the case of standard EPA-approved methods that include calibration procedures, a reference to those methods suffices. Simply list the required frequency and acceptance criteria in a summary table, such as that shown in Table E-10. For nonstandard methods, reference can be made to the SOP. Describe all other calibration procedures in detail. • A list of calibration standards, including source, traceability, and verification of purity, must be included. October 31, 1996 E-26 Interim Final Report Version 5.0 ------- • For process measurements (e.g., flow, mass, etc.) and for chemical or physical analyses, specify the planned frequency of initial and continuing calibration checks and the acceptance criteria for all calibration measurements. For routine, scheduled calibrations, include the calibration frequency and acceptance criteria in, along with a summary of QC requirements. For physical measurements such as temperature and pressure, calibration statements can be quite brief. Example: All thermocouples intended for use in the range 50" C to 350° C will be calibrated versus an NIST-traceable thermometer at a minimum of two temperatures, by placing both in an oven simultaneously and noting any difference between readings. The thermocouple readout must be within 2" C of the corrected mercury thermometer or the thermocouple device will be replaced or corrected. 7.0 DATA REDUCTION, VALIDATION, AND REPORTING Describe how data will be reduced, validated and reported. Deliverables that will be required from the analytical laboratory must also be specified. This material is provided for information purposes only, since the EPA will be responsible for this element of the demonstration process. Provide an overall schematic of data flow, such as shown in Figure 2. This flow chart indicates the entire process of data handling, collection, transfer, storage, recovery, and review for both field and laboratory operations. 7.1 Data Reduction • Name the individuals responsible for data reduction. For a technology demonstration, this will be the EPA or a verification entity. • Summarize the data reduction procedures that are specific to this demonstration. Data reduction procedures that are part of standard methods or SOPs should not be repeated here other than to note any deviations. • Summarize the planned statistical approach including formulas, units, and definition of terms. Do not simply reference a "standard text." • Explain how results from blanks will be treated in calculations. 7.2 Data Validation • Name the individuals responsible for data validation at each stage of data reduction. This will be a responsibility of the EPA or a verification entity. Describe the procedures that will be used for determining outliers. Describe the guidelines that will be employed for flagging or validating data. October 31,1996 E-27 Interim Final Report Version 5.0 ------- SAMPLE RECEIPT SAMPLE PREPARATION DAT/ DATA DATA 1 » SAMPLE ANALYSIS ) 1 I DATA ACQUISITION AND REDUCTION V RAW DATA ANALYSIS BY LAB ANALYSTS k APPROVED? , YES ANALYTICAL/QC DATA REVIEW BY LAB SUPERVISOR APPROVED? „ YES FINAL DATA REVIEW BY PROJECT AND Q.A. MANAGERS APPROVED? T YES REPORT PREPARATION I FINAL REPORT REVIEW BY PROJECT MANAGER 1 L. 1 j 1 i i 1 REVIEW RAW DATA, NO „ RFTAMAI V7ET IA/UCDC' INDICATED A 1 1 NO REVIEW DATA, TAKE h PnDDCfTIV/C AfTIOM * OwKrvCU 1 IVt f\\s 1 IVJN, WHERE INDICATED T i i i i i REVIEW REPORT, NO TAKE CORRECTIVE ACTION REANALYZE WHERE INDICATED REPORT APPROVED? YES V RELEASE REPORT Figure 2. Example of a data reduction, validation and reporting scheme. October 31, 1996 E-28 Interim Final Report Version 5.0 ------- Table E-10. Summary Table of Calibration Requirements for Measurements Process Parameter Flow rate of dopant feed Secondary volume standard Secondary time standard Temperature (50-300°C) Temperature (300-800°C) Flow rate in stack Measurement Classification Critical Critical Critical Noncritical Critical Critical Device Mass flow meter Dry test meter Stopwatch Thermometer Thermocouple Pitot tube and manometer Calibration Procedure Compare to calibrated dry test meter NIST-traceable spirometer NIST time base Comparison to certified thermometer Comparison to NIST-calibrated thermocouple Measure pitot orifice with NIST-traceable micrometer; compare manometer markings to NIST-calibrated meter stick Frequency Before/after field test, weekly Before/after field test Before field test Before field test Before field test Before field test Acceptance* Criteria %5 2% .05 sec/min 5°C 5°C 1% 3 Maximum allowable deviation from standard. This table is meant to illustrate a typical format 7.3 Data Reporting • Name the individuals responsible for the various types of reports. This will be a responsibility of the EPA or a verification entity. • Indicate the units for each measurement and each matrix, and whether data will be reported on a wet, dry, or some other reduced basis. If this requirement has been covered in other sections, do not repeat it here. Indicate data storage requirements that will be expected of the laboratory once the project is complete. Will the laboratory need to maintain a complete set of raw data for six months? one year? Five years? Also indicate how long the actual samples will be stored, in case reanalysis is required. List the deliverables that will be expected from the laboratory and from the field operations. Will the data package include all raw data sufficient to recalculate any October 31, 1996 E-29 Interim Final Report Version 5.0 ------- result, if need be? What type of QC data will be reported? Reporting requirements may vary, depending on the intended use of the data. • Summarize the data that will be included in the final report. What QC data will be included? Will analytical and other measurement data be partially reduced before they are reported, or will all individual measurements be reported? Example: The final report will contain the folio-wing analytical data: All analytical results from all primary samples. Data judged to be outliers will be included, along with a justification for excluding this information from further interpretation. All individual results from standard reference materials, independent check samples, replicates, and matrix spikes, including initial and final concentrations. Data from the continuous emission monitors will be reported as 15-minute averages and, when appropriate, will be summarized graphically. For Category II projects, this section should contain a statement that the final report will include a QA section that documents QA/QC activities and results. These results must be readily correlated to the primary data and must clearly indicate the limitations of the data and the range of validity of the conclusions. The final report should also include a summary of the original QA objectives, and a statement regarding whether or these objectives were met. If QA objectives were not met, include an explanation of the impact of not meeting the project's QA objectives. 8.0 INTERNAL QUALITY CONTROL CHECKS This section describes all internal quality control (QC) checks that will be used throughout the project, including field and laboratory activities of all organizations involved. The QC procedures that are specified should follow from the QA objectives previously stated. This section describes how the QA specifications will be met. 8.1 Types of QC Checks Examples of QC checks that should be considered include the following: • Samples - Collocated, split, replicate Spikes - Matrix spikes and matrix spice duplicates - Spiked blanks - Surrogates and internal standards Blanks October 31,1996 E-30 Interim Final Report Version 5.0 ------- - Sampling, field, trip, method, reagent, instrument - Zero and span gases Others - Standard reference materials (complex natural materials, pure solutions) - Mass tuning for mass analysis - Confirmation on second column for gas chromatographic analyses - Control charts - Independent check standard - Determinations of detection limits - Calibration standards - Proficiency testing of analysts - Any additional checks required by the special needs of your project Include QC checks for process measurements as well. Most of this information can be summarized in a table, as shown in Table E-l 1. This table should designate the types of QC procedures, required frequencies, associated acceptance criteria, and corrective action that will occur if the acceptance criteria are not met. When QC procedures are referenced to a standard method that describes an exact procedure, additional discussion is not normally needed. However, standard methods lacking exact QC procedures or nonstandard methods require a detailed explanation, either in this section of the QA Project Plan or in an appended Standard Operating Procedure. The tabular format shown in Table E-l 1 is also convenient for summarizing routine and ongoing calibration requirements. If routine calibration is summarized in this section, a reference to that effect should be included in the section on analytical procedures. The accompanying text must assure that there are no ambiguities. Particularly troublesome terms are "duplicate" or "replicate." The text should explain precisely how and when replicates are taken. Do replicate samples, for instance, refer to samples collected simultaneously or sequentially in the field; to samples collected at the same sample point but at different times; to samples that are split upon receipt in the laboratory? The term "QC check sample" must also be carefully defined. Indicate at which point matrix spiking occurs. Exact procedures for preparing the numerous kinds of blanks must be described fully in the text. Never assume that a term such as "field blank" will mean the same to the sampling team or to a reviewer that it does to you. Specify the compounds or elements that will be employed as matrix spikes and surrogates. Some standard methods recommend surrogate and matrix spike compounds, which may be incorporated by reference when appropriate. Typically, some of the matrix spike compounds must be selected on a project specific basis. In some cases, it may also be necessary to provide additional discussion of potential problems that might be expected with certain QC procedures, along with the proposed solutions. For example, spiking samples in the field is frequently less reliable and more difficult than October.31, 1996 E-31 Interim Final Report Version 5.0 ------- spiking in the laboratory, due to contamination and less controlled conditions. If field spiking is required, then a discussion of procedures that minimize such problems is also required. Because standard methods often include extensive QC requirements, it is natural to ask why such QC procedures must be summarized in this section. Why not simply state that QC will be performed as required in the method? In some cases, EPA standard methods are sufficiently complete for this approach, but frequently they are not. Many EPA-approved methods do not include specific QC procedures. The more complete EPA methods allow options, such as the choice of matrix spike compounds, or the use of either control charts or fixed acceptance limits. The analytical and measurement component of a demonstration will require specific QC guidelines to document technology performance. Be sure to: • Identify the stage at which replication and spiking occur. Avoid using terms such as "sample replicate" without explaining how such replication will be performed. • Explain exactly how blanks will be prepared. 9.0 PERFORMANCE AND SYSTEMS AUDITS The demonstration plan should describe the QA audits planned for monitoring any system used for obtaining critical measurements. A schedule of all audits should be included, along with identification of responsible personnel. It should indicate what audit reports will be generated and who will receive them. If no audits are planned, include an explanation. A Technical Systems Audit (TSA) is a qualitative evaluation of all components of the total measurement system, including technical personnel and QA management. This type of audit includes a careful evaluation of both field and laboratory QC procedures. TSAs are normally performed before, or shortly after, measurement systems are operational; they should also be performed on a regularly scheduled basis throughout the project. After measurement systems are operational and begin generating data, Performance Evaluation Audits (PEA) are conducted periodically to determine the bias of the total measurement system(s). As part of a PEA, the laboratory analyzes a performance evaluation sample. Long-term projects should provide for regularly scheduled PEAs. Audits of Data Quality (ADQ) are retrospective evaluations of data. Typically, a representative portion of the results in an analytical report is reviewed in detail, starting with raw data and chromatograms, and proceeding through the calculation of final results. ADQs are often used to resolve specific questions regarding the quality of a data set. October 31, 1996 E-32 Interim Final Report Version 5.0 ------- Table E-11. Scheduled QC and Calibration Section Number in Method SVOCs (EPA 8270) 7.3.1 7.3.1 7.3.3 7.3.4 7.3.4 7.4 8.1,8.6 8.6 8.9 8.2,8.6 8.5 Procedure DFTPP tune Inertness for DDT 5-pt ICAI Continuing calibration Matrix spike Replicate spike Surrogate recovery Method blank Proficiency Test Frequency ofQC Procedure 12 hours 12 hours Initially and as needed 12 hours Each batch s20 Each batch s20 Each sample Before any samples; each blank Each new analyst Acceptance Criteria Table 3/8270 DDE.DDD<20%ofDDT RSD of CCC compounds <30% RRTs within 0.06 RRF for all SPCCs * .05 RF for all SPCCS 2 .05 RF for all CCCs within 30% of ICAL RT of IS within 30 sec of last CCC Area of IS within factor of two of last CCC See Table 6 None for duplicates From control charts Default = Table 8 No significant interference to target target analytes See Table 6 Corrective Action Re-run before sample analysis Repair before sample analysis Re-calibrate Correct before analysis, otherwise repeat ICAL Run independent QC reference standard If reference standard okay, accept data, otherwise reject data. None Repeat analysis Find and remove interference Re-do before samples are analyzed. This table is provided to illustrate a typical format ------- 10.0 CALCULATION OF DATA QUALITY INDICATORS The following discussion describes how data quality indicators should be calculated and reported. As a minimum, equations must be provided for precision, accuracy, completeness, and method detection limits. In addition, equations must be given for other project-specific calculations, such as mass balance, confidence ranges, etc. Listed below are general guidelines for calculating the more common data quality indicators. 10.1 Common Data Quality Indicators 10.1.1 Precision If calculated from duplicate measurements, relative percent difference is the normal measure of precision: (C.-C,) * 100% RPD - - - C2)/2 (1) where: RPD = relative percent difference C, = larger of the two observed values C2 = smaller of the two observed values. If calculated from three or more replicates, use relative standard deviation rather than RPD: RSD = (S/Y) x 100% (2) where: RSD = relative standard deviation S= standard deviation 7= mean of replicate analyses. Standard deviation is defined as follows: 5 = ^ n-1 where: S = standard deviation y-t = measured value of the ith replicate y = mean of replicate measurements n = number of replicates. October 31, 1996 E-34 Interim Final Report Version 5.0 ------- For measurements, such as pH, where the absolute variation is more appropriate, precision is usually reported as the absolute range, D, of duplicate measurements: D = |m, - m2| (4) where: D = absolute range m, = first measurement m2 = second measurement The standard deviation, s, given above, can also be used. 10.1.2 Accuracy For measurements where matrix spikes are used, calculate the percent recovery as follows: S-U %R = 100% x C (5) where: %R = percent recovery S = measured concentration in spiked aliquot U- measured concentration in unspiked aliquot CM = actual concentration of spike added When a standard reference material (SRM) is used: %R = 100% x C srm (6) where: %R = percent recovery Cm = measured concentration of SRM Cjn,, = actual concentration of SRM. 10.1.3 Completeness Completeness is defined as follows for all measurements: %C = 100% x f —\ (7) V n) where: %C = percent completeness V= number of measurements judged valid n = total number of measurements necessary to achieve a specified level of confidence in decision making. Note: This more rigorous definition of completeness is an improvement of the conventional definition in which "n" is replaced by "T," the total number of measurements. October 31,1996 E-35 Interim Final Report Version 5.0 ------- 10.1.4 Method Detection Limit (MDL) MDL is defined as follows for all measurements: MDL = t. , x s (8) (n~i) v ' where: MDL = method detection limit s = standard deviation of the replicate analyses t^ = students' t-value for a one-sided 99% confidence level and a standard deviation estimate with n-l degrees of freedom 11.0 QUALITY CONTROL REPORTS TO MANAGEMENT Part of the discussion in Section 7 of the demonstration plan should identify the individuals responsible for preparing QA reports, and should describe the type and frequency of reports (weekly oral presentations and discussions, monthly written reports, etc.) that will be used to keep project management informed. As a minimum, such reports include: • Changes in the demonstration plan; • Summary of QA/QC programs and accomplishments • Results of technical systems and performance evaluation audits • Significant QA/QC problems, recommended solutions, and results of corrective actions Data quality assessment in terms of precision, accuracy, representativeness, completeness, comparability, and method detection limits • Discussion of whether the QA objectives were met and the resulting impact on decision making Limitations on use of the data obtained from the technology. 12.0 REFERENCES References, if any, can be included in the body of the text, as footnotes, or collected in a • separate section of the demonstration plan. References must uniquely identify the cited material. In particular, when citing various compendia of standard methods published by the EPA, ASTM, American Public Health Association, etc., be sure to include the edition number, since such methods can change substantially from one edition to the next. October 31,1996 E-36 Interim Final Report Version 5.0 ------- End of Document October 31, 1996 E-37 Interim Final Report Version 5.0 ------- |