ElV CPC Concurrent Technologies Corporation ENVIRONMENTAL TECHNOLOGY VERIFICATION PROGRAM METAL FINISHING TECHNOLOGIES QUALITY MANAGEMENT PLAN For U. S. Environmental Protection Agency By Concurrent Technologies Corporation Concurrent Technologies Corporation is the Verification Partner for the EPA ET\r MF Technologies Center under EPA Cooperative Agreement No. CR826492-01-0. Revision 1 March 26, 2001 ------- Revision 1 03/26/01 TITLE: ENVIRONMENTAL TECHNOLOGY VERIFICATION PROGRAM METAL FINISHING TECHNOLOGIES QUALITY MANAGEMENT PLAN ISSUE DATE: March 26, 2001 DOCUMENT CONTROL This document will be maintained by Concurrent Technologies Corporation in accordance with the EPA Environmental Technology Verification Program Quality and Management Plan for the Pilot Period 1995-2000 (EPA/600/R-98/064). Document control elements include unique issue numbers, document identification, numbered pages, document distribution records, tracking of revisions, a document MASTER filing and retrieval system, and a document archiving system. ACKNOWLEDGMENT This is to acknowledge Jim Totter and Valerie Whitman for their help in preparing this document. Concurrent Technologies Corporation is the Verification Partner for the EPA ETV Metal Finishing Technologies Center under EPA Cooperative Agreement No. CR826492-01-0. l ------- Revision 1 03/26/01 Environmental Technology Verification Program Metal Finishing Technologies Quality Management Plan Prepared By: Clinton Twilley CTC QA Manager 7 Date 'cf Approvals: Dorm W. Brown ' Date CTC ETV-MF Program Manager /}£(^ £. Z3t ZOO/ Alva Edwards Daniels Date EPA ETV Center Manger 11 ------- Revision 1 03/26/01 TABLE OF CONTENTS 1.0 INTRODUCTION 1 1.1 Purpose and Content 3 1.2 Scope and Applicability 3 1.3 Document Organization 3 1.4 Review and Update 4 PART A: MANAGEMENT SYSTEMS 5 2.0 MANAGEMENT SYSTEMS (PART A) 6 2.1 Management and Organization 6 2.1.1 Program Team Responsibilities 6 2.1.2 Authority to Stop Work for Safety and Quality Considerations 9 2.2 Quality System Description 10 2.2.1 Quality System Documents 10 2.2.2 Quality Controls 11 2.2.3 Management System Review 12 2.3 Personnel Qualifications and Training 12 2.4 Procurement 13 2.5 Documents and Records 14 2.5.1 Laboratory Data and Reports 15 2.5.2 Protocols and Test/QA Plans 15 2.5.3 Document Control and Distribution 19 2.5.4 Verification Reports and Verification Statements 19 2.6 Computer Hardware and Software 19 2.6.1 Corporate Computing 19 2.6.2 Process Control 20 2.6.3 Data Collection 20 2.6.4 ETV Website 20 2.7 Planning 21 2.7.1 PI anning Meetings 21 2.7.2 Generic Verification Protocols 21 2.7.3 Test/QA Plans 22 2.7.4 Data Quality Objectives (DQO) 23 in ------- Revision 1 03/26/01 2.7.5 Standard Operating Procedures (SOPs) 23 2.8 Implementation of Work Processes 24 2.9 Assessment and Response 24 2.9.1 Numbers and Types of Assessments 25 2.9.2 Management Assessments 26 2.9.3 Technical Assessments 27 2.9.4 Data Quality Assessments 27 2.9.5 Assessment Procedures 28 2.9.6 Assessment Personnel Qualifications, Responsibility, and Authority 28 2.9.7 Response 28 2.10 Quality Improvement 28 2.10.1 Detecting and Correcting Quality System Problems 29 2.10.2 Cause and Effect Relationship 29 2.10.3 Root Cause 30 2.10.4 Quality Improvement Action 30 PART B: COLLECTION AND EVALUATION OF ENVIRONMENTAL DATA 31 3.0 COLLECTION AND EVALUATION OF ENVIRONMENTAL DATA (PART B) 32 3.1 Planning and Scoping 32 3.1.1 Planning Personnel 32 3.1.2 Purpose, Scope, and Objectives 33 3.1.3 Collected Data and Design of Experiment 33 3.1.4 Documentation and Reporting 33 3.1.5 Assessments 33 3.1.6 Constraints and Suspension of Work 33 3.1.7 Waste Minimization and Disposal 33 3.2 Design of Technology Verification Tests 34 3.2.1 Test/QA Plans 34 3.2.2 Intended Use of the Data 34 3.2.3 Data Quality Indicators 34 3.2.4 Performance Characteristics 35 3.2.5 Specification 35 3.3 Implementation of Planned Operations 35 iv ------- Revision 1 03/26/01 3.3.1 Technology Provider Equipment 36 3.3.2 Analytical Laboratories 36 3.3.3 Sample Handling Procedures 37 3.3.4 Laboratory Data Handling, Validation, Reporting, and Archiving 37 3.3.5 Control of SOPs, Test/QA Plans, and Methods 38 3.3.6 Work Instructions 38 3.3.7 Verification of Equipment Operation Prior to Testing 39 3.3.8 Procedures for Data Validation 39 3.4 Assessment and Response 39 3.4.1 Maintenance and Calibration Frequency 40 3.4.2 Analysis of Known Samples/Standards 40 3.4.3 Internal and External Audits 40 3.4.4 Assessment of Data Usability 41 3.5 Reporting 41 PART C: DESIGN, CONSTRUCTION AND OPERATION OF ENVIRONMENTAL TECHNOLOGY 43 4.0 DESIGN, CONSTRUCTION AND OPERATION OF ENVIRONMENTAL TECHNOLOGY (PART C) 44 4.1 Planning 44 4.2 Design of System 44 4.3 Construction/Fabrication of Systems and Components 45 4.4 Operation of Systems 45 4.4.1 Management of Consumables 45 4.4.2 Operational Controls 45 4.4.3 Maintenance and Calibration Frequency 46 4.5 Assessment and Response 46 4.6 Verification and Acceptance of Systems 46 5.0 DISTRIBUTION 47 6.0 REFERENCES 48 v ------- Revision 1 03/26/01 LIST OF FIGURES Figure 1. ETV-MF Verification Process 2 Figure 2. ETV-MF Program Management 8 LIST OF TABLES Table 1. Records Management Scheme 18 Table 2. Assessments 25 LIST OF APPENDICES APPENDIX A: Concurrent Technologies Corporation Quality Policy Statement, Environmental, Health, and Safety Policy Statement A-l APPENDIX B: ETV-MF Quality Planning Checklist B-l APPENDIX C: CTC ISO/EPA ETV Documentation Nomenclature Matrix C-1 vi ------- Revision 1 03/26/01 DOCUMENTS AND GENERAL TERMS CTC Concurrent Technologies Corporation (C7U) is the verification partner for this Metal Finishing Technologies Center Program. The Program will be coordinated through the CTC Office in Largo, Florida. CTC ETV-MF Program Manager The person designated by CTC to manage the center and serve as the chief point of contact (POC) with EPA is Donn Brown. CTC Leadership Team Upper tier management group that guides the corporation. CTC Management Team Group of managers that are heads of sections and directly oversee personnel. CTC QA Manager The person designated by CTC to manage quality assurance for the center on behalf of the CTC ETV-MF PM is Clinton Twilley. CTC Review/Audit Reports The "quality records" developed by CTC as a result of conducting assessments during ETV-MF implementation. Data Quality Objectives (DQO) Data quality objectives are the qualitative and quantitative outputs from a strategic planning process that identifies and defines the type, quality and quantity of data needed to satisfy a specified use. E4 Standard The American National Standards Institute/American Society for Quality (ANSI/ASQ) national consensus standard that is applicable to quality systems for pollution control technology development. The standard is entitled, Specifications and Guidance for Quality Systems for Environmental Data Collection and Environmental Technology Program, ANSI/ASQC E4-1994. EPA Line Management The management structure to whom each EPA Center Manager reports; i.e., branch chief, division director, laboratory director. EPA ETV Center Manager The EPA person designated by EPA line management to serve as the lead for an individual ETV Center. Alva Edwards Daniels is the EPA Center Manager for the Metal Finishing Technologies Center. vii ------- Revision 1 03/26/01 EPA Quality Assurance (QA) Manager The EPA Quality Assurance person designated by EPA line management to manage quality assurance efforts for this Metal Finishing Center program is Lauren Drees. EPA Review/Audit Reports The "quality records" developed by EPA as a result of conducting assessments during ETV implementation. ETV-MF Team CTC employees and subcontractors actively working on the ETV-MF Program; CTC engineers and quality personnel, and subcontracted Metal Finishing industry consultants. ETV-MF Program Environmental Technology Verification for Metal Finishing Pollution Prevention Technologies Program. ETV-MF Project Manager ETV-MF team member assigned by the CTC ETV-MF Program Manager to manage a specific verification project. ETV Program Environmental Technology Verification Program established by EPA to accelerate the development or commercialization of environmental technologies through third party verification of performance. ETV Quality and Management Plan for the Pilot Period 1995-2000 (EPA/600/R-98/064) The specific policies and procedures set forth by EPA for management of quality-related activities in the ETV program. ETV Test Objective The stated objective(s) of each technology test. CTC uses the data quality objectives process to establish test objectives and test measurement quality criteria. ETV Verification Report The report of the result of an individual ETV technology test. ETV Verification Statement An executive summary developed by CTC and approved by the EPA Center Manager, which reports individual technology performance. Focus Area A critical area within the Metal Finishing industry requiring innovative pollution prevention solutions. viii ------- Revision 1 03/26/01 Generic Verification Protocols Those protocols developed, modified, or selected to promote uniform testing cf a single focus area for the ETV-MF Program operated by CTC. Procedure A document that reflects the practice defined in the Quality Manual. QA/G-6 Guidance for Development of Standard Operating Procedures (SOPs) for Quality Related Documents (EPA/600/R-96/027). QA/G-9 Guidance for Data Quality Assessment (EPA/600/R-96/084). QA/G-9D Data Quality Evaluation Statistical Toolbox (DataQUEST) User's Guide (EPA/600/R-96/085). Raw Data All data and information recorded in support of analytical and process measurements made during planning, testing, and assessing environmental technology, including support records such as: computer printouts, instrument run charts, standards preparation records, field log records, technology operation logs, and monitoring records. ETV and CTC test files (all records including raw data) and technical data and associated quality control data which support the data summarized and the conclusions made in each ETV-MF verification report. Records All books, papers, maps, photographs, machine-readable materials, or other documentary materials, regardless of physical form or characteristics, made or received by EPA or CTC or their designated representative for the ETV-MF Program. Stakeholder Advisory Group The group established for the ETV-MF Program consisting of representatives of any or all of the following verification customer groups: buyers and users of Metal Finishing equipment, technology developers and vendors, members of technical associations, and Federal, state and local government. Standard Operating Procedures (SOP) Procedures describing verification activities such as process equipment operation, sample collection and analytical testing. Technology Providers Pollution control equipment manufacturers and vendors. Technology Provider Agreements Agreements between technology providers and CTC to collaborate on verification testing. IX ------- Revision 1 03/26/01 Test Measurement Those critical measurements that must be made during the course of a Metal Finishing technology test to evaluate achievement of the ETV-MF test objective. Test/QA Plan The plan developed by CTC for each individual test of a technology or technology class. Therefore, the Test/QA Plan may include more than one technology. The Test/QA Plan provides the experimental approach with clearly stated test objectives and associated quality objectives for the related measurements. The Test/QA Plan may incorporate or reference Generic Verification Protocols. User Industry Buyers and users of Metal Finishing equipment. Verification Establishing or providing the truth of the performance of a technology under specific, predetermined criteria or protocols and adequate data quality assurance procedures. Verification Partner The public or private sector organization selected by EPA to implement the ETV program. CTC is the verification partner for this ETV-MF Program. Work Instruction A document that provides specific requirements on how to perform or carry out associated duties. Work Plan Revisions A task-specific budget which estimates the costs of completing a verification. x ------- Revision 1 03/26/01 ABBREVIATIONS AND ACRONYMS A2LA American Association for Laboratory Accreditation ANSI American National Standards Institute ASQ or ASQC American Society for Quality ASTM American Society for Testing and Materials CTC Concurrent Technologies Corporation DQO Data Quality Objective E4 Standard ANSI/ASQC E4-1994 EHS Environmental, Health and Safety EPA U.S. Environmental Protection Agency EMS Environmental Management System ETV Environmental Technology Verification ETV-MF Environmental Technology Verification for Metal Finishing Pollution Prevention Technologies Program ETV QMP Environmental Technology Verification Program Quality and Management Plan for the Pilot Period (1995 - 2000) ISO International Organization for Standardization MIL-SPEC Military Specifications MSDS Material Safety and Data Sheets NELAC National Environmental Laboratory Accreditation Council NELAP National Environmental Laboratory Accreditation Program NIST National Institute of Standards and Technology OSHA Occupational Safety and Health Administration P2 Pollution Prevention PC Personal Computer POC Point of Contact PR Procurement Requisition QA Quality Assurance QC Quality Control QMP Quality Management Plan XI ------- Revision 1 03/26/01 ABBREVIATIONS AND ACRONYMS (continued) RAB Registrar Accreditation Board RCRA Resource Conservation and Recovery Act RFQ Request for Quotation RFT Request for Technologies RvA Raad voor de Accreditatie SAE Society of Automotive Engineers SARA Superfund Amendments and Reauthorization Act SOP Standard Operating Procedure SOW Statement of Work VP Verification Partner Xll ------- Revision 1 03/26/01 1.0 INTRODUCTION The Environmental Technology Verification for Metal Finishing Pollution Prevention Technologies (ETV-MF) Program is a partnering program under a cooperative agreement between Concurrent Technologies Corporation (CTC) and the U.S. Environmental Protection Agency (EPA). The purpose of the program is to provide verification testing of environmental technologies intended for reducing pollution in the Metal Finishing industry. This is one of the ETVs established as part of a broader program administered by EPA. The ETV-MF verification process is shown in block flow format in Figure 1. Examples of technologies targeted for verification under this program may include: • Emissions and risk characterization from plating operations • Methods to reduce cyanide concentrations in wastewater and solids • In-process recovery of metals, acids, and cleaners (Pollution Prevention) • Off-site recovery of metals, acids, and cleaners • Reduction and replacement of the use of chlorinated solvents for cleaning The goal of this Quality Management Plan (QMP) is to document quality procedures followed within the ETV-MF for meeting Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs, ANSI/ASQC E4- 1994 (E4 Standard). This standard has been established as the standard for all the ETV Programs along with the EPA-developed Environmental Technology Verification Program Quality and Management Plan for the Pilot Period 1995-2000, EPA/600/R-98/064 (ETV QMP), which is in conformance with the E4 Standard. This QMP details the system of management and team infrastructure to ensure that data resulting from the execution of this program is 1) valid and defensible and 2) meets project quality and environmental quality objectives. First tier documentation of quality procedures pertaining to this ETV-MF Program is the QMP (this document). As second tier documentation, a Generic Verification Protocol has been developed for all focus areas [Ref. 1], If the composition of a test plan varies significantly, the Generic Verification Protocol will be supplemented or additional protocols will be developed. These Generic Verification Protocols will be supported by third tier Test/Quality Assurance (QA) Plans for evaluation of each individual technology to be verified under the auspices of this ETV Program. Each technology-specific Test/QA Plan will identify the objectives of testing, design of the experiment, and critical and non-critical measurements required. Fourth tier documentation, if necessary, is recorded in the Standard Operating Procedures (SOPs). 1 ------- Revision 1 03/26/01 A Metal Finishing Focus Area may be a finishing process, a type of equipment, an application, or some other category of Metal Finishing technology for pollution prevention. Focus areas are identified based on industry needs; these focus areas are evaluated and prioritized by the Stakeholder Advisory Group. A Request for Technologies (RFT) is issued for pollution prevention technologies that address the focus areas. The technology suppliers provide time and funding, in addition to the technology to be tested. They team with a user industry facility having a process within the focus area chosen. The users include production Metal Finishing shop specialists who are willing to host the verification testing. Responses to the RFT are reviewed for relevant information, technology eligibility, teaming arrangement(s), and the levels of support to be provided by the technology supplier and industry user. Final selection of technologies for verification is based on guidelines from the Stakeholder Advisory Group. Test plans are developed for verification of the technology's ability to satisfy Metal Finishing and environmental requirements. These plans are agreed to by all parties, and will become the definitive requirement for verification. Verification testing is performed on the process, product, and/or equipment, in the user industry Metal Finishing production shop or facility. All verification activities are conducted by the ETV-MF team. A verification report is prepared for EPA detailing the results of the verification test. The technology supplier is given an opportunity to review the report before final approval. Verification reports are posted on the EPA ETV and ETV-MF Internet websites. A verification statement, which is a summary of the report, is distributed via the websites and other information channels after approval. Figure 1. ETV-MF Verification Process 2 ------- Revision 1 03/26/01 1.1 Purpose and Content This ETV Program establishes a unique organization that requires a detailed system of quality management to fulfill its verification and data collection mission. The purpose of this document is to serve as an overall guide to the quality system and the management organization areas that have key roles in the oversight of this ETV-MF Program. This ETV-MF QMP contains information concerning the planning, implementation, and assessment of the quality system. • Personnel directly involved with the execution of the program have the responsibility and authority for quality control and quality assurance activities. • Oversight of the project is the responsibility of the CTC ETV-MF Program Manager, Donn Brown. • The person designated to manage quality assurance for the ETV-MF Program is the CTC QA Manager, Clinton Twilley. • The CTC ETV-MF Program Manager and CTC QA Manager are independent entities, both reporting through the CTC International Organization for Standardization (ISO) management system. 1.2 Scope and Applicability Management and data collection activities of this program are for performance assessment of environmental technologies used for pollution prevention (P2). The QA and Quality Control (QC) functions are a vital part of the ETV operations and are applicable to all tiers of the project. All tiers of project personnel are individually responsible for QA and QC functions. The CTC QA Manager, independent of the CTC ETV-MF Program Manager, is responsible for managing and addressing QA activities. Review of project data and reports occurs across multiple tiers of management. 1.3 Document Organization The remaining sections of this QMP are organized according to the quality requirements found in the E4 Standard. Appropriate references are also made to EPA's ETV QMP and CTCs ISO 9001 and 14001 management systems. The sections of the E4 Standard that apply to this ETV-MF Program are: Management Systems (Part A), Collection and Evaluation of Environmental Data (Part B), and the operational portions of Design, Construction, and Operation of Environmental Technologies (Part C). The applicable subsection requirements are listed below: Management Systems (Part A') • Management and Organization • Quality System Description • Personnel Qualifications and Training • Procurement • Documents and Records • Computer Hardware and Software 3 ------- Revision 1 03/26/01 • Planning • Implementation of Work Processes • Assessment and Response • Quality Improvement Collection and Evaluation of Environmental Data (Part B) • Planning and Scoping • Design of Technology Verification Tests • Implementation of Planned Operations • Assessment and Response • Reporting Design. Construction, and Operation of Environmental Technologies (Part C) • Planning • Design of System • Construction/Fabrication of Systems and Components • Operation of Systems • Assessment and Response • Verification and Acceptance of Systems 1.4 Review and Update This document is reviewed and approved by the CTC ETV-MF Program Manager, CTC Leadership Team and EPA. Input and guidance on key points are provided by EPA. This document will be reviewed annually by project personnel. Corrections and additions will be submitted to ETV Program Management by the staff associated with this ETV-MF Program. Major revisions will require the preparation of an updated QMP. The updated plan will be submitted to EPA for review and approval as required. 4 ------- Revision 1 03/26/01 PART A: MANAGEMENT SYSTEMS Part A of this ETV-MF Quality Management Plan contains the specifications and guidelines applicable to common or routine quality management functions and activities necessary to effectively manage the ETV-MF Program. 5 ------- Revision 1 03/26/01 2.0 MANAGEMENT SYSTEMS (Part A) This section of this ETV-MF QMP contains the specifications and guidelines applicable to common or routine quality management functions and activities necessary to effectively manage the ETV-MF Program. The goal of the ETV-MF Program is to test and verify the performance of innovative technologies that have environmental benefits. This ETV-MF Program is to become a self-sustaining, independent entity providing an unbiased evaluation of these technologies to Metal Finishing companies, technology providers, and other interested stakeholders. To this end, high standards of quality must be maintained throughout the company and along program lines. CTC s Johnstown, Pennsylvania ISO 9001 Quality Management System (QMS) and ISO 14001 Environmental Management System have been certified as compliant by KEMA Registered Quality, Inc., a third party registrar. KEMA Registered Quality Inc. of Chalfont, Pennsylvania, is an ISO 9000, QS-9000 and ISO 14000 registrar, accredited by the Registrar Accreditation Board (RAB) and Raad voor de Accreditatie (RvA). The CTC Regional Offices, including the Largo, Florida, office, have not yet been certified. CTC s certified ISO 9001 QMS and ISO 14001 Environmental Management System comprise the overall management system that will be used for this program. The specific requirements of the E4 Standard and ETV QMP will be addressed in this QMP as well as references to specific CTC ISO quality requirements as appropriate. CTC s ISO 9001 Quality Management System starts with the Quality Policy, which is contained in Appendix A This appendix also contains the CTC Environmental, Health and Safety Policy Statement. It is the responsibility of each individual in the Company to comply with the provisions of both these policies in support of this program. 2.1 Management and Organization CTCs Quality Manual, section 4.1 Management Responsibility, addresses the overall management organization. Specifics for this program are shown in Figure 2. The CTC ETV-MF Program Manager is responsible for ensuring proper communication throughout all program lines and maintaining a cohesive team structure. 2.1.1 Program Team Responsibilities The CTC ETV-MF Program Manager has overall responsibility for the ETV-MF Program. He will be the sole point of contact (POC) with EPA for all Program activities. The CTC ETV-MF Program Manager will be one source of information on the technical activities associated with each verification project. Another source of project technical information will be periodic reviews with ETV-MF Team members, the CTC ETV-MF Program Manager, and the CTC QA Manager. 6 ------- Revision 1 03/26/01 EPA will be invited to attend all project review meetings. Figure 2 illustrates the management organization of the Program Team and its functional relationships, respectively. The CTC QA Manager is responsible for ensuring that the applicable requirements of the Environmental Technology Verification Program Quality and Management Plan for the Pilot Period 1995 - 2000, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs (ANSI/ASQC E4-1994), and CTC s Johnstown, Pennsylvania, ISO 9001 Quality Management System and ISO 14001 Environmental Management System are addressed in the ETV-MF QMP and followed. In addition, the CTC QA Manager is directly responsible for performing the QA assessments identified in Table 2. The CTC QA Manager will report the results of these assessments to the CTC ETV-MF Program Manager, who then reports the results to the EPA Center Manager. Similarly, revisions to CTC's Johnstown, Pennsylvania, ISO 9001 Quality Management System and ISO 14001 Environmental Management System will flow down from the CTC QA Manager to the CTC ETV-MF Program Manager for incorporation into the ETV-MF Program. ETV-MF team members report directly to the CTC ETV-MF Program Manager. Team members are comprised of CTC engineers and quality personnel, and subcontracted Metal Finishing industry consultants. Subcontractors are selected on the basis of their experience in the Metal Finishing industry and their capability for meeting the testing and quality assurance requirements. Subcontractors will help identify potential stakeholders and initial focus areas, develop Generic Verification Protocols, identify Metal Finishing shops and technology providers, and oversee verification testing and analysis of results. 7 ------- Revision 1 03/26/01 Figure 2. ETV-MF Program Management 8 ------- Revision 1 03/26/01 Members of the ETV-MF Team are responsible for accomplishing all of the technical activities associated with the ETV-MF Program. These include the activities identified in Part B: Collection and Evaluation of Environmental Data and Part C: Design, Construction and Operation of Environmental Technology. Members of the ETV-MF Team, as designated by the CTC ETV-MF Program Manager, will act as ETV-MF Project Managers for specific verification testing projects. Progress on project technical activities will be reported to the CTC ETV- MF Program Manager and to EPA through periodic project reviews. Stakeholders are defined as parties who have a reason to be interested in the results of the verification process. These stakeholders will be drawn from buyers and users of Metal Finishing equipment, treatment system equipment manufacturers, members of technical associations, and Federal, state and local government regulatory agencies. EPA and CTC recognize that the quality of the Stakeholder Advisory Group (i.e., their knowledge of markets and technologies, and their willingness to share that knowledge by giving of their time and energy) will be key to the success of this program. As shown in Figure 2, Metal Finishing shops will be production facility test sites for the verification processes and implementation of the technologies. Verification will evaluate how well tie technology supplied by vendors performs as part of the overall manufacturing process, and the technology's ability to reduce or eliminate multimedia emissions from operations performed by the Metal Finishing industry. Testing conducted outside CTC facilities may involve the use of additional subcontractors. These will be selected and used according to the required work and their capability of meeting the testing and quality assurance requirements (note section 2.4). The CTC ETV-MF Program Manager will manage subcontractors. 2.1.2 Authority to Stop Work for Safety and Quality Considerations CTC shall stop unsafe work and work of inadequate quality in the following manner: • CTC will review all planned verification tests and ensure environmental, safety and health issues are addressed. • The CTC ETV-MF Program Manager, CTC QA Manager, and ETV MF Team members acting as Project Managers will issue a Stop Work Order in the event that unsafe work, work of inadequate quality, or work that has the potential for harm to the environment is identified. • The EPA ETV Center Manager, Alva Edwards Daniels, and the EPA QA Manager, Lauren Drees, shall contact the CTC authorized individual(s) in the event that work of inadequate quality is discovered. 9 ------- Revision 1 03/26/01 2.2 Quality System Description A quality system is herein planned and will be established, documented, implemented and assessed as an integral part of the ETV-MF management system as defined by ETV quality policy. The authority requiring an appropriate quality system for ETV-MF is the U.S. Code of Federal Regulations (CFR), 40 CFR Part 30.54. This plan complies with the E4 Standard, which is comparable to the ISO 9000 standard series. This ETV-MF quality system addresses each applicable specification provided in the E4 Standard, using the policies and procedures in this plan as appropriate. 2.2.1 Quality System Documents CTC has standardized on ISO 9001 as the Quality Management System (QMS) model, using ISO 9001:1994, Quality systems - Model for QA in design, development, production, installation and servicing for guidance. This system is a four-tier system consisting of a Quality Manual (Tier 1), Procedures (Tier 2), Work Instructions (Tier 3), and Records and Forms (Tier 4). All documentation is available on-line through the CTC computer network and can be accessed by each employee from a workstation. Appendix C contains a matrix that correlates with CTCs ISO documentation with the EPA's ETV documentation nomenclature. Since initiating this quality effort, CTC has added to the quality goal of "meeting customer needs" by including the requirement that it occur in an environmentally responsible, healthy, and safe manner. To ensure this added requirement is consistently met, CTC has established an environmental, health and safety (EHS) management system built on the framework of the ISO 14001 Environmental Management System (EMS) standard, and including health and safety aspects as well. The EMS has been built on and integrated with the ISO 9001 system described above. This has resulted in a seamless, integrated management system that provides the highest possible value to CTC, our staff, and customers. The ISO 9001 Quality and ISO 14001 Environmental Manuals describe policies and activities that support and maintain CTC's QMS and EMS Management System. They are designed to meet the 20 target items of the ISO 9001 QMS standard and 17 target items of the ISO 14001 EMS standard. Management has reviewed and endorsed these manuals and Tier 2 Procedures. The Management Systems Representative for CTC reports to a Senior Vice President who participates on the Quality Committee. This committee consists of two General Managers, three Directors, and the Management Systems Representative. The committee is responsible for implementation of the CTC management system. Quality Representatives reporting to the Management Systems Representative are responsible for maintaining the documentation, investigating root cause, documenting and implementing corrective action, and maintaining records. In addition, the QMS requires a member of the CTC leadership team to review and approve contract-required deliverables prior to submittal to EPA. 10 ------- Revision 1 03/26/01 Tier 2 procedures will be used where directly applicable to ETV-MF. Tier 3 and 4 documents will be used directly or serve as guides to develop ETV-MF specific documents (i.e., Test/QA Plans and SOPs for testing or process equipment operation). Procedures, Tier 2 documents, provide direction on how to meet the requirements of the quality and environmental system. These procedures provide guidance to personnel for major program planning through the Quality Planning Checklist, preparation and delivery of documents, purchase of goods and services, scheduling, training, calibration and maintenance of equipment, inspection and testing of products, and assessments. CTC s Demonstration Test PI an/Work Order Request System, ISO Procedure 0902, requires the use of test protocols (Generic Verification Protocols and Test/QA Plans) and Demonstration Test Plans (equipment and process operation SOPs), which are specified in this QMP. Work Instructions, Tier 3, provide operating, process, and laboratory personnel with step-by-step instructions on performing a job function. Forms, Tier 4, are used to capture the information required to document the quality system and test results. When forms are completed, they become some of the quality records used to establish documentation that the system is being operated correctly. Tier 4 documents are equivalent to SOPs specified in EPA's ETV QMP. Data required by Test/QA Plans will use existing forms when possible or new ones developed specifically for the ETV-MF Program. 2.2.2 Quality Controls Internal audits are conducted approximately three times a year in Johnstown, Pennsylvania, to evaluate the CTC ISO Quality Management System. Qualified staff members who have lead auditor status conduct these audits. Additional staff members who have received internal training assist in the internal audit process. In addition, third party surveillance audits are conducted every six months, at CTC in Johnstown, Pennsylvania. The surveillance audits are conducted by KEMA Registered Quality, Inc. This QMP, with reference to CTC's certified ISO 9001 and 14001 Management System, offers an overall guide to the management system of the ETV-MF Program. As the quality and environmental management system for CTC, the Quality Manual, Environmental Manual, ISO Procedures, and Work Instructions serve as the basis for project planning, work implementation, and assessment of the ETV-MF Program. 11 ------- Revision 1 03/26/01 2.2.3 Management System Review As part of CTC s Quality and Environmental Management System requirements, the management system is reviewed by executive management for continued effectiveness and suitability (ISO Procedure 0401). The management system also includes project reviews at different tiers on a monthly and quarterly basis (ISO Procedure 1001). These reviews are both technical and management in nature. Details of these reviews are also discussed in section 2.9 of this QMP. 2.3 Personnel Qualifications and Training CTC uses a matrix management system where personnel of various tiers of education, experience, and capability are available to a project as a manager, consultant, or team member. Certain management personnel possess MBA degrees or are licensed professionals. Numerous personnel maintain a membership in a professional society. This allows for access to experts in key areas. CTC also maintains a network of academic, military, and private company contacts who are available as consultants. Only qualified personnel will be used on the ETV-MF Program. Qualifications include work experience in the Metal Finishing field, sample collection and analysis, and equipment setup and operation. The qualifications of CTC and subcontractor employees participating in the ETV-MF Program are documented in resumes that are maintained on file at CTC for reference as necessary. Generic Verification Protocols and Test/QA Plans will identify training requirements to ensure that CTC and subcontractor program participants are trained in appropriate courses prior to verification testing. Identification of specific training requirements for this ETV-MF Program is the responsibility of the CTC ETV-MF Program Manager. The CTC ETV-MF Program Manager, along with line management, monitors changes in program personnel and ensures that staff members are qualified and trained. These training requirements range from technical competency to awareness training cn the environmental, safety and health aspects of an employee's or contractor's job. As stated in CTC ISO Procedure 4420, training will be implemented and documented as required. Training documentation will be maintained at the Largo, Florida, CTC office and at CTC Corporate Human Resources in Johnstown, Pennsylvania, and will be available for review by EPA. Conferences, seminars, and vendor-provided training will be utilized when identified and deemed necessary for program execution. Records are maintained for each employee who receives training from internal and external sources. 12 ------- Revision 1 03/26/01 2.4 Procurement Procurement is implemented through CTC s Contract Resources per ISO Procurement Procedure 0301. Contract Resources uses the CTC Procurement Manual, which contains all procedures necessary for the purchase of items and services for the program. Contract Resources not only has responsibility for ensuring that all requirements of a contract are met, but also initiates the purchasing activity subsequent to receiving input from program personnel. Procurement of equipment and services is initiated using a Procurement Requisition (PR). The PR includes the following information: • Date PR is prepared • Date item is required • Account number to which the item will be charged • Contract number for the program • PR number for tracking • Item number • Description of item • Vendor's identification number • Quantity desired • Unit cost and total price • Vendor name, address, and phone number • Signature of person preparing the PR • Signature of Program Manager • Signature of Line Manager • Appropriate approvals Project personnel are responsible for providing information concerning the specifications of equipment and supplies to procurement personnel. Additional requirements or specifications, such as quality or technical requirements, may be included as an attachment to the PR. By completing the appropriate section on the PR, an inspection of the item or service for conformance to quality and technical requirements will be performed at the time it is received. This inspection is documented on a Material Inspection Receiving Report or a similar document at the Metal Finishing shop where the equipment or supplies will be used. The PR is inspected by a procurement representative to ensure that all necessary information has been provided. The representative will contact the person who initiated the PR if additional information is required. Major purchases must be made through a competitive bid unless a sole provider can be justified. The bid process is initiated by checking Request for Quotation (RFQ) on the PR. Procurement personnel, along with project personnel, will select vendors and suppliers based on their ability to deliver the appropriate quality and quantity of material or equipment that meets program requirements. Procurement personnel may make these decisions based on past history, performance and reliability history, self-assessment, on-site visit, or product inspection. 13 ------- Revision 1 03/26/01 At a minimum, all material will be inspected at the time of receipt to ensure that the correct item and quantity has been received. Shipping and Receiving personnel will reject unacceptable or damaged material upon receipt. In the event that unacceptable goods or services are received, Contract Resources will act as the agent for negotiating replacement of parts or services, rework, or refund. Any equipment purchase may be assessed through an acceptance test to ensure proper operation. Subcontractors will be evaluated according to their ability to provide the appropriate quality of service that meets program requirements. An evaluation may be made based upon self-assessment, onsite visit, past performance and reliability history, or by contacting references. For laboratories the review will also include evaluation of their quality control plans and SOPs for all proposed non-standard analytical procedures. In addition to this review, a determination will be made as to the laboratory's ability to meet the requirements specified in this QMP, particularly sections 2.5.1, 3.3, and 3.4. 2.5 Documents and Records CTC procedures are established, controlled, and maintained for identifying, preparing, reviewing, approving, revising, collecting, indexing, filing, storing, maintaining, retrieving, distributing, and disposing of pertinent quality documents and records. All forms of documents and records, including both printed and electronic media, are subject to these procedures. Original records, developed on behalf of this ETV-MF Program and required to demonstrate the quality of information and data provided in ETV-MF verification reports, are specifically included in these procedures. In addition, records requiring document control are identified and properly controlled. Contract Resources will issue document numbers and maintain a chronological log of all memoranda, letters and reports issued to EPA. Contract Resources, along with the CTC ETV-MF Program Manager, is responsible for ensuring that required documents are prepared and delivered as scheduled. The following document types may be generated and submitted to EPA as part of ETV-MF activities: • ETV-MF QMP (this document) • Generic Verification Protocols • Test/QA Plans • Standard Operating Procedures (SOPs) • Engineering Drawings • Resumes • Quarterly Status Reports (QSRs) • Raw Data • Stakehol der Memb er Li st • Stakeholder Meeting Summaries • Technology Provider Agreements • Verification Reports and Verification Statements • Management and Quality System Audit reports • Non-Conformance and Preventive/Corrective Action Reports 14 ------- Revision 1 03/26/01 • Work PI an Revi si ons 2.5.1 Laboratory Data and Reports Copies of all quality records, process records, bench sheets and data printouts will be maintained by the laboratories and be available for inspection by CTC upon request. All data, calculations, and records are identified by verification trail and the specific technology evaluated. Documents and records generated by analytical laboratories shall be retained by the laboratory in accordance with the laboratory QMP. Laboratory final reports are held for three years after final payment on the Cooperative Agreement #CR826492-01-0, per 40 CFR30.53. 2.5.2 Protocols and Test/QA Plans Generic Verification Protocols and technology-specific Test/QA Plans are prepared, reviewed, approved, and distributed as shown in Table 1, Records Management Scheme. These documents are held for three years after the final payment on the Cooperative Agreement #CR826492-01-0, per 40 CFR30.53. 15 ------- Revision 1 03/26/01 Record Type Preparation/Updating Review Approval Finals Distributed To: ETV-MF Quality Management Plan CTC QA Manager CTC ETV-MF Program Manager CTC Leadership Team EPA ETV Center Manager EPA QA Manager CTC QA Manager CTC ETV-MF Program Manager EPA ETV Center Manager CTC QA Manager CTC ETV-MF Program Manager EPA ETV Center Manager EPA QA Manager CTC Contract Resources Generic Verification Protocols ETV-MF Project Manager CTC ETV-MF Program Manager CTC QA Manager CTC Leadership Team EPA ETV Center Manager EPA QA Manager Stakeholders CTC QA Manager CTC ETV-MF Program Manager EPA ETV Center Manager CTC QA Manager CTC ETV-MF Program Manager EPA ETV Center Manager EPA QA Manager CTC Contract Resources EPA ETV Webmaster Test/QA Plans ETV-MF Project Manager CTC ETV-MF Program Manager CTC QA Manager CTC Leadership Team EPA ETV Center Manager EPA QA Manager Stakeholders CTC QA Manager CTC ETV-MF Program Manager EPA ETV Center Manager Technology Providers User industry ETV-MF Project Manager CTC QA Manager CTC ETV-MF Program Manager ETV MF Team EPA ETV Center Manager EPA QA Manager CTC Contract Resources EPA ETV Webmaster Technology Providers User Industry Laboratory Manager (LM) Standard Operating Procedures Technology Provider User Industry ETV-MF Project Manager Technology Provider User Industry ETV-MF Project Manager Technology Provider User Industry ETV-MF Project Manager Engineering Drawings Technology Provider User Industry ETV-MF Project Manager Technology Provider User Industry ETV-MF Project Manager Technology Provider User Industry ETV-MF Project Manager Resumes (Employee Qualifications) ETV-MF Team CTC ETV-MF Program Manager CTC Leadership Team CTC ETV-MF Program Manager CTC Leadership Team CTC ETV-MF Program Manager Quarterly Status Reports CTC ETV-MF Program Manager CTC Leadership Team CTC Line Manager CTC Line Manager CTC Leadership Team CTC ETV-MF Program Manager EPA ETV Center Manager CTC Contract Resources 16 ------- Revision 1 03/26/01 Record Type Preparation/Updating Review Approval Finals Distributed To: Raw Data and Engineering Calculations ETV-MF Project Manager CTC QA Manager CTC ETV-MF Program Manager ETV MF Team CTC ETV-MF Program Manager CTC QA Manager CTC ETV-MF Program Manager EPA ETV Center Manager Stakeholder Member List CTC ETV-MF Program Manager EPA ETV Center Manager EPA ETV Center Manager CTC ETV-MF Program Manager EPA ETV Center Manager EPA ETV Webmaster CTC QA Reviews and Audits See Section 2.9 Section Section 2.9 See Section 2.9 See Section 2.9 Stakeholder Meeting Summaries CTC ETV-MF Program Manager EPA ETV Center Manager ETV-MF Team CTC Leadership Team CTC Leadership Team EPA ETV Center Manager CTC ETV-MF Program Manager EPA ETV Center Manager EPA ETV Webmaster Technology Provider Agreements CTC Contract Resources CTC ETV-MF Program Manager CTC Leadership Team Technology Provider CTC Leadership Te am Technology Provider CTC ETV-MF Program Manager CTC Contract Resources Verification Reports ETV-MF Project Manager Technology Provider CTC ETV-MF Program Manager CTC Leadership Team EPA QA Manager EPA ETV Center Manager CTC QA Manager CTC ETV-MF Program Manager CTC Leadership Team EPA ETV Center Manager EPA ETV Program Manager Technology Provider CTC ETV-MF Program Manager EPA ETV Center Manager EPA ETV Webmaster CTC Contract Resources Verification Statements ETV-MF Project Manager Technology Provider CTC ETV-MF Program Manager CTC Leadership Team EPA QA Manager EPA ETV Center Manager CTC QA Manager CTC ETV-MF Program Manager CTC Leadership Team EPA ETV Center Manager EPA Laboratory Director EPA ETV Program Manager EPA ETV Webmaste CTC ETV-MF Program Manager EPA ETV Center Manager CTC Contract Resources Work Plan Revisions CTC ETV-MF Program Manager CTC Leadership Team EPA ETV Center Manager CTC ETV-MF Program Manager CTC Leadership Team EPA ETV Center Manager CTC ETV-MF Program Manager EPA ETV Center Manager CTC Contract Resources ETV Verification Strategy ETV Coordinator CTC ETV-MF Program Manager ETV Team EPA Laboratory Directors ORD Deputy Assistant Administrator EPA ETV Webmaster 17 ------- Revision 1 03/26/01 Record Type Preparation/Updating Review Approval Finals Distributed To: ETV Quality and Management Plan ETV Directors of Quality Assurance CTC ETV-MF Program Manager EPA QA Manager ETV Team ETV Coordinator EPA Laboratory Directors EPA Division Directors EPA ETV Webmaster COAG/IAG Records CTC ETV-MF Program Manager EPA ETV Center Manager Not Applicable Not Applicable Not Applicable Annual ETV Progress Report Evaluation Contractor CTC ETV-MF Program Manager Stakeholders EPA Team ETV Coordinator EPA Laboratory Directors EPA Reviews/Audit Reports EPA QA Manager ETV Directors of Quality Assurance EPA ETV Center Manager ETV Coordinator Not Applicable EPA Laboratory Directors CTC ETV-MF Program Manager CTC QA Manager Table 1. Records Management Scheme 18 ------- Revision 1 03/26/01 2.5.3 Document Control and Distribution Documents and records will be managed in accordance with Table \ to include preparation/updating, review, approval, and final distribution of ETV-MF related documents. The CTC ETV-MF Program Manager assigns the appropriate personnel for document preparation. Documents are issued and reviewed according to an established distribution list. Reviewers return the document with comments, initials, and date of revision. An approval page is affixed for signatures after revisions are made. The latest versions of documents for this ETV-MF Program will be maintained at CTC offices in Largo, Florida. Copies of both electronic and printed documents will be saved. The CTC QA Manager is responsible for ensuring that all project personnel are using the most recent and approved document. Contract Resources issues the final approved document to EPA. Printed copies of reports are maintained in Contract Resources' offices in Johnstown, Pennsylvania. Copies are filed according to contract number, letter number, and document number. Copies are held for a period of three years after final payment on the Cooperative Agreement #CR826492-01-0, per 40 CFR 30.53. 2.5.4 Verification Reports and Verification Statements Results of verification testing will be documented in a verification report. The report will include all data collected from both process and laboratory testing and engineering calculations used in data reporting. The report will include a QA section that documents data quality indicators, deviations from the approved Test/QA Plan, and limits of the data. In addition, a Verification Statement will be issued that summarizes the testing performed and the results. The persons responsible for preparation, review, approval, and final distribution are shown in Table 1, Records Management Scheme. 2.6 Computer Hardware and Software CTC maintains adequate computer resources to provide computing capability to the program. Computer hardware, software, and networking capabilities are geared toward three major areas. These are corporate computing, process control, and data collection. 2.6.1 Corporate Computing Corporate computing is based upon advanced computing, software, and networking technologies that allow personnel to solve a broad range of industrial problems. Readily available and proven standard technologies, reasonably priced yet geared toward current trends and technological advances, are used. The 19 ------- Revision 1 03/26/01 company has established guidelines to ensure a uniform yet flexible system that is able to meet the daily demands of the contracts it supports. Computing is based on a workgroup approach with each workgroup or contract having its own server. The system requires password login from desktop personal computers (PCs) for security. Desktop computers are connected to the server to provide applications such as word processing, spreadsheet, presentation graphics, forms, printing, filing, network access, and electronic mail. Backup procedures are performed at the end of each workday in order to preserve information stored in network files. 2.6.2 Process Control Computerized process control, if required for the ETV-MF Program, will be performed by the technology provider or user industry. Proper operation of all monitoring devices will be confirmed according to a regular maintenance and calibration schedule specified by the manufacturer. Whenever possible, industry recognized standards will be used for the calibration of computerized process control devices. 2.6.3 Data Collection Computers and electronic readouts are used to collect data where computers are interfaced directly to instruments. Vendor-supplied software is employed for this type of data collection. Software allows for multiple point calibrations and automatic readout of sample value. Calibration data, raw data, and final results are assessed by laboratory management to ensure proper equipment operation and data accuracy. 2.6.4 ETV Website The ETV Program website is established and maintained by EPA to post ETV information and documents for availability to the public. The following documents will be submitted by the CTC ETV-MF Program Manager to the EPA ETV Webmaster: • Stakeholder lists and updates • Meeting announcements and summaries • Generic Verification Protocols • Test/QA Plans • ETV Verification Statements • ETV Verification Reports • News of upcoming meetings/speeches/announcements • Commerce Business Daily Announcements 20 ------- Revision 1 03/26/01 2.7 Planning At the beginning of the ETV-MF Program and after significant changes in scope, budget, or time constraints, the CTC ETV-MF Program Manager will ensure completion of the Quality Planning Checklist (Form 3027 - Appendix B). This checklist is used to ensure that the important aspects of the program have been considered, and it identifies and references the applicable management system procedure. These include Design of Equipment/Facilities, appropriate and sufficient personnel, use of proper methods, processes, materials, and equipment, use of appropriate inspection/assessment/validation criteria, health and safety, waste disposal, training, cost, and deliverable schedules. The CTC ETV-MF Program Manager is also responsible for identifying any legal or regulatory requirements that apply to execution of program activities (i.e., air permits or monitoring). 2.7.1 Planning Meetings The CTC ETV-MF Program Manager will regularly schedule a project review meeting to include all applicable project personnel. The EPA ETV Center Manager will be invited to attend this meeting and will receive a project report. The meetings will typically address the following: • Project schedule • Project performance • Product delivery • Personnel requirements • Identification and justification of the specific Metal Finishing technologies to be evaluated and the performance criteria that will be examined • Identification of the technical and quality goals in order to appropriately evaluate the technology • Translation of the technical and quality goals into a test plan that will adequately validate the technology • Development of a test plan that fits within the cost and schedule constraints • Design of an experiment, with acceptance criteria included, that will adequately verify the technology CTC has established a Stakeholder Advisory Group composed of representatives from buyers and users of Metal Finishing equipment, technology developers and vendors, members of technical associations, and Federal, state and local government. This group, assembled with the guidance and approval of EPA, will assist in identifying and selecting technologies for verification. 2.7.2 Generic Verification Protocols After all of the technologies within a focus area have been tested, a Generic Verification Protocol will be developed for that focus area. This protocol will contain a wide range of test parameters that apply to the focus area. Included in the protocol will be all testing required to gather sufficient data for verification of 21 ------- Revision 1 03/26/01 technologies within a focus area. The issues that may be addressed in the Generic Verification Protocol include: • General description of the ETV-MF Center • Responsibilities of all involved parties • Experimental design • Equipment capabilities and description • Description and use of test sites • Description and use of laboratory test sites • QA/QC • Data handling • Requirements for other documents • Environmental, health, and safety regulatory requirements • References The QA/QC section of the Generic Verification Protocol will describe activities that verify the quality and consistency of the work. Preparation and use of appropriate QA procedures (such as field duplicates, blanks, spiked samples, and performance evaluation samples) will be described. Frequency of calibrations, the QC checks and the rationale for them will be described. Procedures for reporting QC data and results will be identified. Who or what organization is responsible for each QA activity, and who has the responsibility for identifying and taking corrective action, will be identified. Variations between tests will be described in the Test/QA Plan. The protocol will cite documents that explain, extend and/or enhance the protocol, such as related procedures, published literature, or methods manuals. The specific location of any reference, not readily available from a full citation in the reference section (as in a facility-specific standard operating procedure) will be given or attached to the protocol. 2.7.3 Test/QA Plans The requirements for verification testing are included in the Test/QA Plan prepared for each Metal Finishing technology to be evaluated. As stated previously in this QMP, Test/QA Plans will be prepared for each technology to be verified, and will contain the following elements where applicable: • Title and approval sheet • Table of contents, distribution list • Test description, test objectives • Identification of the critical measurements, data quality objectives, and data quality indicator goals • Test (including QA) organization and responsibilities • Documentation and records 22 ------- Revision 1 03/26/01 • Experimental design • Sampling procedures • Sample handling and custody • Analytical procedures (including review of SOPs for non-standard methods) • Test-specific procedures for assessing data quality indicators • Instrument calibration and frequency • Data acquisition and data management procedures • Technical Systems Audits • Corrective action procedures (response actions to audit findings) • Data reduction, data review, data validation, and data reporting • ETV-MF Operation Planning Checklist (required by the Job Training Analysis Plan [Ref. 2]) • Environmental, health, and safety requirements • Test/QA Plan Modification Procedure The draft Test/QA Plans are prepared by the ETV-MF Project Manager and CTC engineers, and reviewed by the CTC QA Manager, CTC ETV-MF Program Manager, and CTC Leadership team member for accuracy according to CTC's ISO 0905 Procedure, Deliverable Documents. 2.7.4 Data Quality Objectives (DQO) CTC uses the DQO process to establish test objectives and test measurement quality criteria. The DQO process will involve systematic and strategic planning, based on scientific methods, to identify and define the type, quality, and quantity of data needed to establish test objectives. The key elements of the DQO process are: • Concisely defining the problem • Identifying the decision to be made • Identifying the key inputs to that decision • Defining the boundaries of the study • Developing the decision rule • Specifying tolerable limits on potential decision errors • Selecting the most resource-efficient data collection design Based upon the qualitative and quantitative statements derived from this process, test objectives and test measurement quality criteria will be established. 2.7.5 Standard Operating Procedures (SOPs) If another level of detail is required for describing test activities (e.g., operation of instrumentation, equipment or a system), an SOP will be written to support the Test/QA Plan as Tier 4 documentation. The following topics, from EPA's Guidance for Development of Standard Operating Procedures (SOPs) for Quality 23 ------- Revision 1 03/26/01 Related Documents, (QA/G-6^, may be included (or a reference provided) in the SOP where applicable: • Scope and applicability • Summary of procedures • Definitions (acronyms, abbreviations, etc.) • Personnel qualifications • Health and safety warnings (warning of activities that could result in possible personal injury) • Cautions (warning of activities that could damage equipment, degrade samples, or invalidate results) • Apparatus and materials • Calibration • Sample collection, sample labeling, and sample tracking • Handling and preservation of samples • Interferences • Sample preparation and analysis • Data acquisition, calculations, and data reduction • Requirements for computer hardware and software used in data reduction and reporting • Data management and records management CTC's existing procedures and work instruction documents will be used to meet the requirements of SOPs as they contain the information listed above. 2.8 Implementation of Work Processes Work will be performed according to approved planning and technical documents. The planning for the implementation of technology performance verifications is contained in Part B of this QMP, sections 3.1 and 3.3. All technology verification work will occur according to Test/QA Plans, and SOPs developed and agreed upon by EPA, CTC, user industry, and the technology provider. The approved Test/QA Plans and SOPs will be present on test site, and the work will be implemented in accordance with them. During the work phase, modifications to plans and procedures will be documented, and the modifications will be incorporated into the final protocols, Test/QA Plans, and SOPs. CTC will implement all work processes in accordance with this QMP. 2.9 Assessment and Response Regular assessments of both the management and technical aspects of the project are planned and scheduled. Assessments are conducted both internally through self- assessment and externally by KEMA Registered Quality, Inc. Results are documented in audit reports and reviewed by appropriate management. Project Management tracks program status through any of CTC's assessment tools such as: 24 ------- Revision 1 03/26/01 • Cost Tracking System • Matrix Management System • Software Tools 2.9.1 Numbers and Types of Assessments Assessments will be planned, scheduled, and conducted to measure the effectiveness of the implemented management and quality systems. The following assessments will be performed in accordance with the frequency noted in Table 2. Assessment Assessor(s) Basis for Assessment Reason for Assessment Frequency Management Systems Review CTC QA Manager ISO 9001, ISO 14001, & ETV- MF QMP Assess quality of management practices Annual Technical Systems Audits Self: CTC QA Mgr Independent: EPA Quality Assurance Mgr Test/QA Plans Assess quality of verification test Self- the first test in a Focus Area and twice thereafter if the technology is significantly different Independent - Twice during project Performance Evaluation Audits Self: CTC ETV Program Mgr Independent: EPA Quality Assurance Mgr Contract Requirements, ETV-MF QMP, Test/QA Plans, & Verification Reports Assess and measure performance in meeting project goals and objectives Self - Monthly Independent - As applicable Audits of Data Quality Self: CTC QA Mgr Independent: EPA Quality Assurance Mgr Raw and Summary Data Assess data calculations and reporting Self - Verily all data and perform statistical assessments for at least 10 percent of data sets Independent - As applicable Table 2. Assessments Any corrective actions required, based on these assessment activities noted above, will be handled through CTCs ISO Procedure 1401 as outlined in section 2.9.7. The CTC ETV-MF Program Manager will be responsible for sending the requests for corrective actions to program subcontractors. 25 ------- Revision 1 03/26/01 2.9.2 Management Assessments Management system assessments will consist of Performance Evaluation Audits of the ETV-MF Program, Management Systems Audits of the CTC system-wide ISO management systems, and a specific Management System Review for this QMP and its interaction with the overall ISO 9001 and 14001 management systems. Performance Evaluation Audits - The CTC ETV-MF Program Manager will regularly conduct quality assessments to ensure that the quality objectives of the program are being met. These internal reviews will be documented and reported to CTC management in a monthly Project Report that documents the cost, schedule, and technical performance of the project. Management Systems Audits - Internal audits of CTC s ISO 9001 and ISO 14001 management system (under which this ETV-MF QMP operates) are conducted periodically to assess compliance with the written and approved Quality Manual and Procedures. CTC has a highly qualified staff to guide and direct this effort. The staff includes assessors certified by the RAB, quality engineers certified by the ASQ, as well as quality professionals with expertise in Quality Improvement Programs, ISO 9000 Systems Development and Auditing, and Industrial Problem Solving. Audits of the QMS and the Environmental Management System are conducted according to CTC's established ISO Procedure 1701. Results of these internal audits are reported to CTC's Quality Committee, chaired by the Management Systems Representative, who reports to a Senior Vice President within the organization. The Quality Committee is responsible for guidance, direction, and oversight of the management system, as well as providing the necessary resources for successful realignment of the QMS and EMS system to meet the ISO requirements. An audit of the management system is also conducted by an independent organization (KEMA Registered Quality, Inc.) every six months as part of the surveillance requirement for ISO 9001 and ISO 14001 certifications. These audits are conducted to ensure that CTC s policies, practices, and procedures are sufficient to provide results of the type and quality required. Results of these audit activities (internal and third party) are reported to CTC s Executive Management. Management System Reviews - CTC QA Manager, using information from the Performance Evaluation Audits and the Management Systems Audits, will perform an audit of the ETV-MF Program by auditing the project at the Largo, Florida, office of CTC. Audit procedures will follow CTCs ISO Procedure 1701. All elements that impact process and product quality and those elements that have the potential for significant EHS impact will be internally audited. Results of the audit will be reported to the CTC Management Systems Representative, the Quality 26 ------- Revision 1 03/26/01 Committee, the CTC ETV-MF Program Manager, and the EPA ETV Center Manager. 2.9.3 Technical Assessments A Test/QA Plan is prepared for each Metal Finishing technology evaluated. The Test/QA Plan contains the specific requirements for each technology evaluated. Project personnel are responsible for implementing the Test/QA Plans. CTC or subcontractor personnel are present during testing and data collection to assess project technical quality. These personnel document adherence to specified procedures and have the authority to suspend work when unsafe or unacceptable quality conditions arise. Technical System Audits (TSAs) are planned at the frequency specified in Table 2. These audits will be conducted during system operation & testing by the CTC QA Manager to ensure that testing and data collection are performed according to the Test/QA Plan requirements. Results of these audits will be reported to the EPA ETV Center Manager, CTC ETV-MF Program Manager, and ETV-MF Project Manager. Independent TSAs will also be conducted by EPA. In addition to these technical audits, all Verification Reports will be reviewed by the CTC ETV-MF Program Manager, CTC QA Manager, technology providers, and EPA to ensure that conclusions are technically sound. 2.9.4 Data Quality Assessments Reviews will include specific data quality indicators and procedures specified in individual Test/QA Plans prepared for each technology assessed. The CTC QA Manager conducts validations for all of the testing performed to determine whether the quality objectives specified in the Test/QA Plan are met. These validations include an evaluation to determine whether the data collected fulfills the test objectives and design of experiment established for the technology. Data quality (statistical) assessments, which require review of a minimum of 10 percent of the data sets (all data for the same parameter analyzed by the same test method), will be conducted according to the procedures specified in the Test/QA Plan prepared for specific technology using EPA's Guidance for Data Quality Assessment, EPA/600/R-96/084 (QA/G-9J as a guideline. These reviews will be selected from data sets where complex methods are used, non-standard methods are used, or where method deviations are required. These assessments are the responsibility of the CTC QA Manager. Results of these reviews are recorded and are reported to the EPA ETV Center Manager, CTC ETV-MF Program Manager, and ETV-MF Project Manager. 27 ------- Revision 1 03/26/01 2.9.5 Assessment Procedures Assessments will be performed according to written and approved procedures, based on careful planning of the scope of the assessment and the information needed. All assessments are based on interviews, on the physical examination of objective evidence, and on the examination of the documentation of past performance. Results are documented in audit reports and reviewed by the appropriate parties shown in Table 1. 2.9.6 Assessment Personnel Qualifications, Responsibility, and Authority Personnel conducting assessments will have the appropriate technical and/or management skills necessary to perform the assigned assessment. CTC will determine and document the tier of competence, experience, and training of audit personnel during hiring and periodic performance reviews. Qualified audit personnel will have access to the appropriate management personnel and documents required in the performance of their audit duties. Audit personnel will have the responsibility and authority to: • Identify and document problems that affect quality of verification results • Propose recommendations for resolving problems that affect quality of verification processes or results • Independently confirm implementation and effectiveness of solutions If assessment personnel identify a severe problem affecting verification quality or endangering the health and safety of personnel, they have the responsibility to bring it to the immediate attention of the ETV-MF Project Manager and the CTC ETV-MF Program Manager. 2.9.7 Response If the recommendations and conclusions from the findings of an assessment are adverse, corrections will be implemented in accordance with CTC s Preventive and Corrective Action Procedure 1401, ISO 9001 and ISO 14001 management system. A Preventive/Corrective Action Request is submitted to a QA Representative. The QA Representative assigns the responsibility for corrective action to the ETV-MF Project Manager. Corrective actions will be identified within 10 days of receiving the assessment or audit report. The QA Representative verifies effectiveness of the corrective action, closes out the Preventive/Corrective Action Request, and provides the status to the Quality Committee as input to management reviews as needed. 2.10 Quality Improvement A quality improvement process will be established and implemented to continuously develop and improve the ETV-MF Program. As Test/QA Plans are developed, testing is 28 ------- Revision 1 03/26/01 performed, and reports are generated, CTC will provide feedback on the QMP's competency based on input received and lessons learned. 2.10.1 Detecting and Correcting Quality System Problems Procedures are established and implemented to address problems identified within the quality system. CTC's ETV-MF Program Manager and QA Manager will remain alert for problems in the following areas: • Adequacy of the ETV-MF Quality Management Plan • Consistency of the quality system • Implementation of the quality system • Correction of quality system procedures • Quality of data • Completeness of documented information • Quality of planning documents • Implementation of the work process CTC Leadership Team meetings and Management Team meetings are held regularly to evaluate and gauge quality management systems. Important project requirements are covered and appropriate briefings are held to communicate findings. Any deficiencies are documented using a Preventive/Corrective Action Request. The QA Representative is responsible for conducting an investigation of the root cause of the deficiency and documenting the findings. 2.10.2 Cause and Effect Relationship CTC ISO Procedure 1401 requires root cause analysis for preventive and corrective actions. When a significant problem is discovered, the CTC QA Manager determines and documents the relationship between cause and effect that is evidenced by the problem and, when possible, determines and documents root cause. The CTC QA Manager communicates this information to the CTC ETV- MF Program Manager so corrective action can be authorized and implemented. A significant problem is identified as any problem requiring: • A change in testing protocol, or • A management system change, or • A quality system change within the ETV-MF Program CTC's QA Manager, in accordance with CTC s ISO 9001 and ISO 14001 management system, will continually review and assess the ETV-MF Program for conformance with internal management system documents. 29 ------- Revision 1 03/26/01 2.10.3 Root Cause The CTC ETV-MF Program Manager ensures that root causes are determined, whenever possible, before permanent preventive measures are planned and/or implemented. 2.10.4 Quality Improvement Action The CTC ETV-MF Program Manager, the CTC QA Manager, and process personnel devise and implement the appropriate corrective action. Effectiveness of the corrective action is verified and documented by the CTC QA Manager. Procedures and instructions are revised as necessary to reflect the implemented changes. 30 ------- Revision 1 03/26/01 PART B: COLLECTION AND EVALUATION OF ENVIRONMENTAL DATA Part B of this ETV-MF Quality Management Plan contains the specifications and guidelines applicable to test-specific environmental activities involving the generation, collection, analysis, evaluation, and reporting of test data. 31 ------- Revision 1 03/26/01 3.0 COLLECTION AND EVALUATION OF ENVIRONMENTAL DATA (Part B) This section contains the specifications and guidelines applicable to test-specific environmental activities involving the generation, collection, analysis, evaluation, and reporting of test data. Evaluation and verification of environmental technologies for Metal Finishing will be carried out at commercial Metal Finishing shops. CTC and subcontractor personnel will be responsible for the performance or supervision of this testing. All activities will be carried out according to the Test/QA Plan approved for the technology to be verified. Important aspects of the collection and evaluation of environmental data are: • Planning and Scoping - sufficient data collection to ensure objectives of the testing are adequately addressed • Design of the Technology Verification Tests - identification of the important experimental parameters to be collected and development of a design of experiment • Implementation of Planned Operations - execution of the designed experiments and planned operations • Assessment and Response - assessment of activities and correction of non-conformances • Assessment and Verification of Data Usability - assessment of data and documentation of reporting limitations 3.1 Planning and Scoping The work of the ETV-MF Program is to verify the performance of commercial-ready technologies. CTC implements a systematic process to plan the individual tests. The following actions will be performed: • Joint identification of test objectives, critical measurements, and non-critical measurements by technology developers, industry consultants, and CTC • Review and/or development of analytical methods, sample collection techniques, holding times, and QA objectives by the analytical laboratory, industry consultant, and CTC • Preparation of Test/QA Plans for the acquisition of data to verify performance of the vendors' technologies • Review of Test/QA Plans by EPA, technology provider, test site, CTC QA Manager, and CTC ETV-MF Program Manager • Completion of an ETV-MF Operation Planning Checklist prior to testing 3.1.1 Planning Personnel CTC will coordinate test planning with the participating organizations, including EPA, Stakeholders, technology providers, and any testing organizations or laboratories participating in the test. CTC will identify the planning roles of the participants and conduct planning activities by shared communication via teleconferencing, video conferencing, and/or face-to-face meetings, as appropriate. 32 ------- Revision 1 03/26/01 3.1.2 Purpose, Scope, and Objectives The purpose of this testing is to verify the performance of commercial-ready Metal Finishing technologies and to establish an effective method for future testing of similar technologies. These objectives are met through the preparation of Test/QA Plans and Generic Verification Protocols. 3.1.3 Collected Data and Design of Experiment During the test planning phase, the method of collecting process data, environmental data, laboratory data, and QA data are identified. Testing organizations, test personnel, skill levels, methods, procedures, and equipment are also identified. 3.1.4 Documentation and Reporting Records generated during the tests are listed in section 2.5. Records consist of both paper and electronic data. 3.1.5 Assessments Assessments and frequencies are identified in section 2.9. 3.1.6 Constraints and Suspension of Work CTC is working under the constraints of time and resources as identified by EPA. When constraints are determined by CTC to affect quality, resolution of the problem will proceed as described in EPA's ETV QMP, section 1.5. Circumstances under which work can be suspended are discussed in section 2.1.2 of this QMP. 3.1.7 Waste Minimization and Disposal CTC has implemented ISO 14001 into its EHS operations. As such, EHS is responsible for compliance to Federal, state, and local regulations, including Superfund Amendments and Reauthorization Act (SARA), Occupational Safety and Health Administration (OSHA), Resource Conservation and Recovery Act (RCRA), and emission permitting. EHS conducts employee training in Hazardous Communication, Safe Work Practices, and Worker-Right-to-Know. EHS personnel will work with the CTC ETV-MF Program Manager to ensure applicable environmental, health and safety issues are identified and resolved prior to verification testing. The QMP requires the CTC ETV-MF Program Manager to identify any legal requirements and environmental aspects and impacts of project operations during the project planning phase. Environmental issues are also addressed on the Operation Planning Checklist during the job training analysis 33 ------- Revision 1 03/26/01 [Ref. 2] conducted prior to each test. Regulatory requirements will be specified in each Test/QA Plan. 3.2 Design of Technology Verification Tests Test/QA Plans are used to convey the objectives and requirements of the verification testing to be conducted for each technology. 3.2.1 Test/QA Plans Technology verification design and data collection requirements, for a specific technology to be verified, are included in the Test/QA Plan. The Test/QA Plan is prepared with input from tie CTC ETV-MF Program Manager, CTC QA Manager, technical staff, and EHS, as well as user industry and technology providers. With respect to design of technology verification tests, the Test/QA Plan will address the requirements identified in section 2.7.3, where applicable. The Test/QA Plan is reviewed by appropriate parties listed in Table 1 to ensure it is technically sound and that the analytical parameters and collection of data are sufficient to meet the objectives of the testing. The Test/QA Plan is reviewed by all program participants to ensure a clear understanding of the work to be performed. Comments and technical points are incorporated into the final Test/QA Plan prior to the initiation of any work. 3.2.2 Intended Use of the Data The data collected may be used not only to verify the performance of an environmentally friendly technology, but also to verify pollution prevention potential of the technology. The data may also be used to identify utility and operating costs and compliance to regulatory standards. Therefore, the data collected must be of sufficient quality to allow these types of analyses with reasonable certainty. 3.2.3 Data Quality Indicators The data quality indicators required for each verification test will be identified in the technology-specific Test/QA Plan. The values for data quality indicators such as sample bottle type and size, sample preservative, holding time, quantity, identification number, collection location, date and time, and analysis method will be identified on a Chain of Custody (COC) form. Information to be delivered to CTC by the laboratory performing the work for each sample include: identification number; results; reporting units; analysis date, time, and method detection limit (MDL); method quantification limit (MQL); precision; accuracy; blanks; duplicates; spikes; and percent recovery as applicable. Values greater than the MDL but less than the MQL will be reported as estimated. 34 ------- Revision 1 03/26/01 3.2.4 Performance Characteristics All critical and non-critical parameters will be listed in the Test/QA Plan, along with the required method, performance characteristics (accuracy, precision, etc.), and calibration requirements. Parameters will be analyzed using approved methods such as American Society for Testing Material (ASTM), EPA, Military Specifications (MIL-SPEC), or Society of Automotive Engineers (SAE) whenever applicable. Field records and laboratory records for calibration will include supplier, lot number, expiration date, and date of usage of standards. Program personnel are responsible for assessing the equipment prior to operation or measurement during verification testing. Laboratory personnel are responsible for assessing the proper operation of the analytical instrumentation prior to testing. This validation will be done using nationally recognized performance standards whenever possible. In some instances, no commonly accepted method may exist due to the lack of a suitable calibration material or the proprietary or innovative nature of the technology. The performance of these non-standard methods will be evaluated based on the manufacturer's specifications or on in-house developed and documented protocols. Deviation from approved standard methods or non- standard method SOPs must be approved by the CTC QA Manager or CTC ETV- MF Program Manager. Requirements and results of spikes, duplicates, blanks, and reference standards will be fully documented. 3.2.5 Specification Trained and qualified personnel will carry out all sampling and data collection activities. Sampling methods and requirements will be listed in the Test/QA Plan. These methods will be taken from the appropriate source such as EPA air and wastewater regulations promulgated in the Code of Federal Regulations, or the National Institute of Occupational Safety and Health (NIOSH). Testing occurs according to the methods and procedures specified in the Test/QA Plan. Any modification of or deviation from an approved method is documented on process Data Collection Sheets and test plan modification form provided in each Test/QA Plan. 3.3 Implementation of Planned Operations Technology performance verifications will be implemented according to Test/QA Plans, which are prepared during the planning process. Test personnel will have access to the approved planning documents, approved changes to these documents, and all referenced documents. All implementation activities are traceable to the planning documents and to test personnel. 35 ------- Revision 1 03/26/01 Only qualified services and items will be used in the performance of verification operations. Acceptance will be noted on the items themselves and/or in documents traceable to the items. If additional level of detail is required for describing a test activity (e.g., operation of instrumentation, equipment, or a system), an SOP is generated. This SOP provides the operating personnel with all necessary setup parameters such as pressure, temperature, flows, etc., of process baths, ovens, equipment, etc. All testing and data collection activities to be performed both during and after processing are listed in the SOP or similar document. A checklist is used to document the settings, parts, and equipment actually used. Process Data Collection Sheets are used to document conditions during processing. These are prepared in response to the critical and non-critical parameters listed in the Test/QA Plan for the specific verification conducted. Any deviations from the methods and parameters specified in the Test/QA Plan and SOP are recorded and reported to the CTC ETV-MF Program Manager. 3.3.1 Technology Provider Equipment The technology provider is responsible for calibrating equipment, associated with the technology, used to record process conditions. Gauges and monitoring devices used for process monitoring and data collection will be maintained and calibrated at the manufacturer's specified interval. Nationally recognized performance standards will be used whenever possible. Records of the calibration data generated during calibration will be collected as part of the verification test data. 3.3.2 Analytical Laboratories Whenever possible, analytical laboratories used by the ETV-MF Program will be accredited by the National Environmental Laboratory Accreditation Program (NELAP) or the American Association for Laboratory Accreditation (A2LA). NELAP accredits laboratories to standards developed by the National Environmental Laboratory Accreditation Conference (NELAC), a cooperative association of state and Federal agencies (including EPA) formed to establish and promote mutually acceptable performance standards for the operation of environmental laboratories. The basis of the NELAC system is ISO Guide 17025, "General Requirements for the Competence of Calibration and Testing Laboratories." The A2LA is a nonprofit, professional membership society that administers a broad spectrum, nationwide laboratory accreditation system, which, like NELAC, uses ISO Guide 17025. Proper operation of all analytical equipment will be verified by the laboratory according to the laboratory QMP prior to testing any samples. This will be done using samples of known value or, when available, purchased standard reference material. A standard is analyzed with each batch of samples to verify proper equipment function. Duplicates and spikes are also analyzed and precision and accuracy data generated. Re-calibration, maintenance, or repair is indicated when 36 ------- Revision 1 03/26/01 unacceptable recovery of data quality indicators occurs. For example, balances will be verified using traceable weights, and ovens will be checked using thermometers calibrated against a National Institute of Standards and Technology (NIST)-traceable thermometer. Proper operation of equipment using a computer software program to calibrate, collect data, and calculate results is verified by the laboratory according to their QMP prior to analysis of samples. A typical scenario uses a minimum of three standards plus a blank to establish a calibration curve. This calibration is verified using a "check" standard from a different supply source. If the check standard is acceptable, the curve is verified by checking the highest standard that must meet the method requirements (typically a 5 percent to 10 percent error). To completely verify the performance of the method and sample preparation steps, a method blank, blank spike, and, when available, a known sample/standard from a third source are analyzed. These checks and spikes must meet method requirements for recovery before analysis of samples can begin. The data quality indicators (precision, accuracy, and recovery) must meet method requirements for the data to be acceptable. Finally, the calibration blank, highest standard, and check standard are again analyzed to verify that the calibration is still accurate and no drift has occurred. Other instrument checks include spectral interference samples, internal standards, or surrogates. Only after the above requirements are met is data reported; otherwise, the instrument is re-calibrated and re-analysis is performed. 3.3.3 Sample Handling Procedures Samples are collected at the location and frequency specified by the design of experiment detailed in the Test/QA Plan. Samples are preserved at the time of, or as soon as possible after, collection. Samples are clearly labeled with the date, time, sample identification (ID), location, and process at the time of collection. Samples transported by common carrier will use security tape to ensure sample integrity upon delivery to the analytical laboratory. Upon arrival at the analytical laboratory, samples are handled in accordance with the laboratory QMP. Equipment usage and calibration logs are used to indicate the standards, samples, and reagents used during analysis of samples. A log is used to document any repair or maintenance performed on the equipment. 3.3.4 Laboratory Data Handling, Validation, Reporting, and Archiving Data collected in the Metal Finishing shops will be placed on Process Data Collection Sheets. This information is reviewed by the CTC QA Manager and forwarded to the CTC ETV-MF Program Manager. Data collected during laboratory testing is processed in accordance with the laboratory QMP. This includes, at a minimum, the parameter and method used for testing, date, analyst initials, identification number of samples and standards, 37 ------- Revision 1 03/26/01 equipment ID number, calibration number, dilution, instrument reading, and final result. Results for the analysis of duplicate, standard, blank, and spike samples are transferred onto a QA/QC Summary. A member of laboratory management will review all of the data on these sheets to ensure that the proper samples, method, standards, reagents, and equipment have been used and that all program requirements are met. Calculations, dilution, and data quality indicators are verified as well as percent recovery of standard and spike samples. The results from the analysis of duplicate, standard, and spike samples are compared to the method requirements and values stated in the Test/QA Plan. The reviewer will initial and date all of the sheets after review to show that all information has been verified. The data is forwarded for entry onto the laboratory report form. The CTC ETV-MF Program Manager may request interim results for preliminary evaluation of the data. In these instances, the data is clearly marked to show results are preliminary. Restrictions on the use of data are indicated in the comment section of the laboratory report. This may occur when matrix interference results in poor recovery of spike standards and the sample is diluted to remove the interference. 3.3.5 Control of SOPs, Test/QA Plans, and Methods SOPs and Test/QA Plans are clearly marked "DRAFT" to indicate that they are preliminary, in the review process, and are not to be used to conduct testing. After the SOP or Test/QA Plan is approved, the document is given a revision number. Revision 0 is used to indicate the first level of revision. Any change to an approved document will require another review and approval step. The changed document will become Revision 1, etc. The date is also imprinted on each page of the document to indicate the date approved. Project personnel will be required to use the current revision with the approval date. Laboratory management is responsible for ensuring that the most recent document is available to laboratory personnel. 3.3.6 Work Instructions The operation of equipment is documented in Work Instructions or SOPs. Engineering staff members, responsible for the equipment, write the Work Instruction. The engineer prepares the instruction based on information provided from equipment manuals, vendor training, EHS staff, and experience of the personnel responsible for the operation of the equipment. 38 ------- Revision 1 03/26/01 The ETV-MF Program primarily specifies EPA methods and Standard Methods for the Examination of Water and Wastewater as sources for environmental sample analysis. ASTM, MIL-SPEC, SAE, or industry specific protocols are followed for finish quality testing. 3.3.7 Verification of Equipment Operation Prior to Testing Prior to verification testing, equipment and instruments will be checked to ensure proper operation and function. The requirements will be specified in the Test/QA Plan. 3.3.8 Procedures for Data Validation Data reduction will be performed according to the requirements and calculations listed in the technology-specific Test/QA Plans. An assessment and statistical analysis of the data will be conducted to determine if the data meets the Design of Experiment and objectives specified in the Test/QA Plan. Data assessment will be conducted according to the EPA QA/G-9 guidance document. A Quality Assurance Summary will be written to document all qualitative and quantitative quality measurements such as precision, accuracy, and problems encountered during processing and testing. The QA Summary will be included in the Verification Report. Instructions for data management (i.e., transmittal, storage, validation, assessment, processing, and retrieval) are discussed in other sections of this document. 3.4 Assessment and Response The project personnel are responsible for evaluating the performance of testing and the collection of data during verification testing. The collected data is assessed by CTC after the testing is completed. CTC engineers check data collected from both in-line monitoring and laboratory testing to confirm adequate calibration, correct method usage, data transcription, and that all data quality indicators are met. The evaluation is based upon the requirements of the Test/QA Plan and the defined data quality objectives. Sample analysis involves the use of blanks, duplicates, spiked samples, or standards of known value. Transcription errors are confirmed with the analyst prior to any correction of data. A request for re-analysis is completed and issued to the analyst when data quality indicators are not met. Re-analysis is performed when an inadequate calibration or an inappropriate method is used. Laboratory management reviews final data packages to make sure all results conform to chemical principles, mass balances, or charge balances. Statistical methods will be used to evaluate the collected data. This may involve using control charts and/or identifying outliers in the data quality indicators. Deviations are reported to the CTC ETV-MF Program Manager. 39 ------- Revision 1 03/26/01 3.4.1 Maintenance and Calibration Frequency Preventive maintenance and calibration of CTC equipment and instruments are performed on a regular basis. Maintenance and calibration schedules for equipment are maintained using calibration-scheduling software. Each piece of equipment requiring manufacturer calibration is labeled with a unique identification number and logged into the software. The software provides a complete history and traceability for calibration performed. Use, calibration, and maintenance activities are traced for each piece of equipment. A monthly schedule is issued for maintenance and calibration activities to be performed on equipment. A Preventive/Corrective Action Request is completed and forwarded to the QA Representative when a piece of equipment is found to be out of calibration, when improper methods or procedures are used, or equipment, supplies, and instruments do not meet quality requirements. The QA Representative is responsible for determining the root cause and implementing corrective action. In the event that equipment or instruments have been out of calibration during testing and results have been reported, the Calibration Technician issues an Out-of-Calibration Assessment. Supervisory personnel are responsible for determining the impact on the quality of the reported data using the equipment. The Out-of-Calibration Assessment is forwarded to the QA Representative, who is responsible for follow- up and corrective action. The findings are reported to management. Technology providers and the user industries will be required to have a similar maintenance and calibration system for equipment used in the ETV-MF Program. 3.4.2 Analysis of Known Samples/Standards Proper operation and calibration of equipment are verified prior to testing as stated in the specific Test/QA Plan or SOP. Nationally recognized performance standards will be used, when available, for this type of calibration. Calibration of analytical laboratory equipment shall be in accordance with the laboratory QMP. 3.4.3 Internal and External Audits As part of CTCs management system, the facilities in Johnstown, Pennsylvania, undergo quarterly internal (self) audits as described in sections 2.2 and 2.9 of this QMP. As part of CTC's ISO system registration process, the facilities in Johnstown, Pennsylvania, are audited every six months by an RAB-accredited registrar, KEMA Registered Quality, Inc. CTC's Quality Committee, chaired by a Management Systems Representative, reports to a Senior Vice President within the organization. This committee is responsible for guidance, direction, and oversight of the management system, and for providing the necessary resources for successful realignment of the QMS and EMS system in meeting the ISO requirements. A report of the audit results is posted electronically and provided to management. A Preventive/Corrective 40 ------- Revision 1 03/26/01 Action Request is used to document findings of these audits and to initiate the corrective action process. The QA Representative is responsible for determining the root cause, documenting the action to be taken, and ensuring implementation of the corrective action. As indicated in section 2.9.2, the CTC QA Manager will conduct an annual audit of the management system at the Largo office. Data from the ISO audits in Johnstown along with the project Performance Audits will be used in the audit planning. Effectiveness of corrective actions related to any area of the program will be verified as applicable. Representatives from EPA may also conduct assessments of the program and perform inspections prior to and during verification testing. 3.4.4 Assessment of Data Usability After the execution of a SOP (test or equipment operating procedure), an evaluation is conducted to identify any problems encountered during processing or collection of data. The ETV-MF team assesses the usability of the data after laboratory analysis is completed. This evaluation involves the ETV-MF Project Manager, laboratory management, quality assurance, and CTC engineers. Evaluation is based upon the requirements specified in the Test/QA Plan and will include any limitations (such as ranges or confidence intervals) on data parameters. A statistical analysis may be performed on the data using DataQuest, MiniTab 12, or Excel software programs. Measurements such as mean, median, standard deviation, t-test, and f-test will be generated to describe the results and the confidence tier. EPA may also conduct an assessment of the data. After the evaluation is conducted, the CTC QA Manager is responsible for generating a report to document the findings. This report will identify any problems encountered during testing, calibration, analysis, or data reduction and provide the corrective action steps required for resolving the issues. The impact on the test objective and verification results will be provided in the QA section of the verification report. 3.5 Reporting Upon completion of verification testing in accordance with the Test/QA Plan, a report will be prepared for EPA that details the results of the verification testing. The report will comprehensively and accurately document the results of the verification testing of the product against the specific criteria detailed in the Test/QA Plan. The draft reports are prepared by the ETV-MF Project Manager and CTC engineers, and reviewed by the CTC QA Manager, CTC ETV-MF Program Manager, and Leadership team member for accuracy according to CTC's ISO 0905 Procedure, Deliverable Documents. Technology providers will have the opportunity to review the verification data after EPA approval. 41 ------- Revision 1 03/26/01 The report and statement will be submitted to EPA. To maximize the ETV-MF Program's exposure to the Metal Finishing industry, the verification statement will be made available over the Internet on the ETV website (http: www.epa.gov e/v) as well as through other sources as approved by EPA. Should the technology provider not agree with the verification results, the test report will still be submitted to EPA; however, no verification statement will be issued. EPA and the CTC ETV-MF Program Manager will keep a copy of all reports on file. Verification Reports will be prepared for each verification conducted. Verification Reports will include: • Verification Statement • ETV Overview • Technology and Application • Test Methods and Procedures • Results and Discussion • Quality Assurance Narrative • Summary and Conclusions Reports and presentations of data will be reviewed independently, by others than those who produced the data or report, to ensure that these items are technically sound and of sufficient quality to meet the objectives of the task. 42 ------- Revision 1 03/26/01 PART C: DESIGN, CONSTRUCTION AND OPERATION OF ENVIRONMENTAL TECHNOLOGY Part C of this ETV-MF Quality Management Plan describes the quality system elements pertaining to the design, construction, installation, and operation of the treatment system technology to be verified. 43 ------- Revision 1 03/26/01 4.0 DESIGN, CONSTRUCTION AND OPERATION OF ENVIRONMENTAL TECHNOLOGY (Part C) Part C applies only if the technology is to be installed only for the purposes of verification testing and will be removed after testing. Part C does not apply if installation is occurring under an agreement (sale or evaluation) between the technology provider and user industry only, with verification testing to occur after the technology is released for unrestricted use. The elements must be used in conjunction with Parts A and B to provide an adequate quality system for the ETV-MF Program. There are large numbers of purportedly low-emission MF technologies that are commercially ready. These technologies will be reviewed to identify candidates for this verification program. The program will result in the collection of environmental, technical (operational), and cost data for dissemination to industry. This section of the ETV-MF QMP contains the following elements: • Planning • Design of Systems • Construction and Fabrication of Systems and Components • Operation of Systems • Assessment and Response • Verification and Acceptance of Systems 4.1 Planning Planning activities will be coordinated between CTC, EPA, the user industries in which the treatment systems will be installed, and the technology providers using the CTC Quality Planning Checklist (Form 3027). The checklist addresses the key items identified in section 4.1.1 of the E4 Standard and includes such items as: • Delivery, handling, and installation requirements • Notification of organizations that have been identified previously to participate in the project and their role in planning, design, construction/fabrication, operation, and assessment activities • Personnel, equipment, and other resources required • Program technical reviews, peer reviews, surveillances, technical and QA audits, readiness reviews, and other assessment processes • Project and QA records required • Technical, performance, regulatory and quality standards, criteria, and objectives Documentation of project and activity planning will consist of meeting minutes, Test/QA Plans, schedules, and installation drawings. 4.2 Design of System The design of the technology to be verified is the responsibility of the technology provider. Designing the process interface with the technology is the responsibility of the 44 ------- Revision 1 03/26/01 user industry. Prior to installation, the designs will be reviewed by the ETV-MF Project Manager to ensure that, to the best of their knowledge, the technology is designed using sound engineering/scientific principles and appropriate standards. The review will address: • The ability of the components and system to perform under expected conditions of use • Delivery documentation requirements • Compliance with regulatory requirements • Safety and maintenance requirements The results of the design review, including installation requirements, will be documented in a letter to the technology provider and user industry. A readiness review will be performed and documented by the user industry, technology provider, and ETV-MF Project Manager prior to operation of the technology. 4.3 Construction/Fabrication of Systems and Components The construction required during installation of the technology will be performed according to drawings and specifications identified during the design phase. Periodic preventive and corrective maintenance of equipment used during installation will be performed according to the manufacturer's specifications. Measurement and test equipment requiring calibration will be calibrated according to the frequency and specifications stated by the manufacturer. Traceability to nationally recognized performance standards will be maintained when they are used. Documentation of construction equipment requirements will be specified and maintained by the user industry and technology provider. Construction personnel qualification requirements will be specified and maintained by the user industry and technology provider. 4.4 Operation of Systems Documented operation of the various technologies within established parameters is essential to ensure valid performance verification. Key elements necessary to control operation of systems are described in the following sections. 4.4.1 Management of Consumables Any materials that could directly affect system operations such as auxiliary materials, utilities, consumables (e.g., water, compressed air, electric power, and chemical feedstocks) will be identified in the SOPs and monitored appropriately. 4.4.2 Operational Controls As stated in section 3.3 of this QMP, Work Instructions or SOPs will be developed for operation of process equipment. The development process will include the 45 ------- Revision 1 03/26/01 identification of status indicators with tolerance limits in order to ensure that the system operates within regulatory and safety parameters. Parameters to be monitored will also include those required to validate that the system is operating within applicable environmental permit conditions. For high-risk operations, the SOPs will address special requirements such as emergency shutdown procedures. 4.4.3 Maintenance and Calibration Frequency Measurement and test equipment requiring calibration will be calibrated according to the frequency and specifications stated by the manufacturer. Traceability to nationally recognized performance standards will be maintained when they are used. Documentation of system requirements will be maintained by the user industry and technology provider. 4.5 Assessment and Response CTCs ISO Procedures for project management, internal audits, and corrective actions, previously identified in section 3.4 of this QMP for environmental data collection, will also be used for monitoring of process parameters during operation. 4.6 Verification and Acceptance of Systems The performance of environmental technology will be verified according to the applicable Test/QA Plan as stated in Part B of this QMP. Start-up testing will be performed as specified by the technology provider. When acceptance criteria are not met, deficiencies will be resolved and reassessments conducted as necessary. If minimum operational criteria are not met, the situation will be handled through section 4.5. 46 ------- Revision 1 03/26/01 5.0 DISTRIBUTION Alva Edwards Daniels, EPA ETV Center Manager Lauren Drees, EPA QA Manager Clinton Twilley, CTC QA Manager James E. Voytko, CTC Director, Engineering & Manufacturing Jacqueline Carr, CTC Contract Resources Donn W. Brown, CTC ETV-MF Program Manager (5) Chris Start, MMTC Peter Gallerani, ITI George Cushnie, CAI Resources Gus Eskamani, CAMP, Inc. Analytical Laboratories 47 ------- Revision 1 03/26/01 6.0 REFERENCES 1. Environmental Technology Verification Program Metal Finishing Pollution Prevention Technologies Pilot Generic Verification Protocol, Revision 0, May 10, 1999. 2. Environmental Technology Verification Program Metal Finishing Pollution Prevention Technologies Pilot Job Training Analysis Plan, Revision 0, May 10, 1999. 48 ------- Revision 1 03/26/01 APPENDIX A Concurrent Technologies Corporation Quality Policy Statement, Environmental, Health, and Safety Policy Statement ------- Revision 1 03/26/01 CTC Quality Policy Statement Revised: January 10, 2001 Concurrent Technologies Corporation (CTC) provides management and technology- based solutions to a wide array of clients representing Federal and state government organizations as well as the private sector, nationally and internationally. As such, we are committed to: • Providing quality products and services that meet or exceed the requirements of our internal and external clients, on-time and every time • Seeking and achieving continuous improvement in the quality and value added of our products and services • Improving efficiency and reducing waste in all of our business processes It is the responsibility of each individual in the Company to comply with the provisions of this Policy. Daniel R. DeVos President & Chief Executive Officer A-l ------- Revision 1 03/26/01 CTC Environmental, Health, & Safety Policy Statement Revised: January 10, 2001 Concurrent Technologies Corporation (CTC) is committed to protecting and improving the environment, and safeguarding the health and safety of its employees, clients, contractors, and the public. CTC integrates environmental, health, and safety (EHS) consciousness throughout all facets of its activities. Through a company-wide commitment to continuous improvement, CTC strives to achieve excellence in EHS performance and environmental stewardship. In addition, CTC exerts beneficial EHS influence far beyond its own operations. CTC demonstrates world-class leadership in the development and deployment of advanced EHS technologies and management practices throughout Federal and state government organizations as well as the private sector, nationally and internationally. As such, we are committed to: • Regulatory Compliance: Meeting or exceeding the requirements of applicable EHS legislation and regulations. • Safe Facilities: Building, operating, and maintaining safe facilities in a manner that uniformly protects the environment and the health and safety of our employees and the public. • Workplace Health and Safety: Providing a hazard-free workplace to prevent occupational injuries and illnesses. • Pollution Prevention: Preventing pollution in our operations and activities. • Resource Conservation: Minimizing our energy and natural resource consumption in fulfilling our mission. Daniel R. DeVos President & Chief Executive Officer A-2 ------- APPENDIX B ETV-MF Quality Planning Checklist ------- QUALITY PLANNING CHECKLIST The purpose of this Quality Planning Checklist is to aid in project planning by providing a structured and guided approach to the various aspects of the Quality and EHS Management System. The Checklist, therefore, serves as a guideline and is an integral part of the overall quality management process. The checklist has been designed to walk PMts through the necessary procedures and supporting documentation necessary to comply with the QMS/EMS system requirements. The completed Checklist shall be retained by the Project Manager and appropriately filed with project documents. B-l ------- QUALITY PLANNING CHECKLIST ETV-MF Program Donn W. Brown (Project) (PMt) U.S. Environmental Protection Agency 1/5/99 11/20/00 (Client) (Original Date) (Revised Date) QUESTION YES NO N/A PROCEDURE REFERENCE DOCUMENTATION required during project execution COMMENTS I. Contract Review A. General 1. Has proposal and/or contract review been performed and documented? (i.e.. Is the statement of work clear, task discrete, and acceptance criteria clearly defined?) X See CTC 0301 or 0302 Notes/documentation that review was conducted prior to acceptance (NCEMT review documentation controlled by Program Manager) Contracts & Business Analyst reviewed supplemental budget request 2. Are the financial resources adequate to cover applicable CTC costs for the level of quality required? X See CTC 0301 or 0302 Notes/documentation that review was conducted prior to acceptance (NCEMT financial documentation controlled by Program Manager) Additional funding requirements being developed for review by EPA B. Personnel 3. Have personnel issues been addressed: • Does adequate staffing exist with the appropriate skill set? X See CTC 0301 or 0302 Notes/documentation that review was conducted prior to acceptance One additional Engineer hired and will start 11/27/00 • Is additional training required for any personnel associated with the project to supplement or improve the skill set currently available? X Training requirements identified in QMP, test plans, and performance appraisals • Will additional personnel (new hires, consultants) need to be recruited to support project requirements? X B-2 ------- QUALITY PLANNING CHECKLIST QUESTION YES NO N/A PROCEDURE REFERENCE DOCUMENTATION required during project execution COMMENTS C. Equipment/Tooling Capability 4. Have equipment capacity requirements been appropriately identified? X See CTC 0301 or 0302 Notes/documentation that review was conducted prior to acceptance 5. Does the equipment have the capability to meet applicable precision, repeatability, or tooling requirements? X See CTC 0301 or 0302 Notes/documentation that review was conducted prior to acceptance 6. Does a new piece of equipment or tooling have to be purchased or can existing equipment or tooling be upgraded or rebuilt? X See CTC 0301 or 0302 Notes/documentation that review was conducted prior to acceptance 7. Have acceptance criteria been identified and documented for new equipment or tooling? X See CTC 0301 or 0302 Notes/documentation that review was conducted prior to acceptance D. Test Capability 8. Is testing capacity sufficient? X See CTC 0406 Notes/documentation that review was conducted prior to acceptance 9. Is new measuring and test equipment required? X See CTC 0406 Notes/documentation that review was conducted prior to acceptance New equipment will be reviewed, purchased or loaned from within CTC and listed in test plan II. Project Planning A. General 10. Has a project plan been developed for this project? X See CTC 0406 Approved Project Plan Approved project plan in file 11. Does the project involve software related products or tasks developed under the National Security Division? X See CTC 2002 Documentation in accordance with CTC 2000 series procedures 12. Will equipment, outside services X See CTC 0601 PRs (Form 201), Pos, MIRRs and Materials & services B-3 ------- QUALITY PLANNING CHECKLIST (subcontractors), and/or materials be purchased? Note: Subcontractor services may include design services, materials, equipment, testing, or other processing services. subcontractor agreements will be procured through contracts or purchasing 13. Are standardized test methods (i.e., NIST or ASTM) required? X See CTC 0403 and 0406 List of test requirements on work orders Methods identified in test plans QUESTION YES NO N/A PROCEDURE REFERENCE DOCUMENTATION required during project execution COMMENTS 14. Are there cost, schedule or technical risks associated with the perfomance of this work? X See CTC 0406 Risk assessment in project plan Risk assessment in Project Plan 15. Have appropriate facility servicing or installation requirements been identified, such as equipment being vented outside the building, etc.? X See CTC 0903 See CJC 4472 FSRS work order should be maintained by Facility Services Operational Planning Checklist (form 308) 16. Are the resources accounted for in the schedule or project budget to implement new equipment or tools? X See CTC 0406 Breakdown of cost elements in Project Plan See program budget and schedule 17. Have you prepared a WBS with resource assignments? X See CTC 0406 WBS in Project Plan See WBS in Project Plan 18. Has funding been allocated for project support and management including: X See CTC 0406 Breakdown of cost elements in Project Plan See Program Budget a. Travel X b. Briefings X c. Computer and network support X d. Technical Writers X e. Graphics X f. Report production X g. Long distance phone calls X h. Overnight mail services X i. Calibration of equipment X j. Project review X k. ISO compliance X 1. Audits X B-4 ------- QUALITY PLANNING CHECKLIST 19. Training: a. Have training requirements been identified? (EHS training) X See CTC 4420 Documentation of identification of training needs Will be identified in each project JTA b. Has training time been allocated in the project schedule? X Schedule in Project Plan Training done prior to the first day of each test c. Have fimds been allocated in training? X Breakdown of cost elements in Project Plan Training budgeted in each test project cost 20. Is configuration management required? (i.e. version control) X See CTC 0401 & 0402 Documentation of version control, etc. Document versions are controlled 21. Have adequate lead times been accounted for: X See CTC 0406 Schedule in Project Plan See individual project schedules QUESTION YES NO N/A PROCEDURE REFERENCE DOCUMENTATION required during project execution COMMENTS a. Subcontracted services X b. Engineering/CAD X c. Purchases X d. Equipment installation X e. Software development/support X f. Support services X g. Client deliverables preparation and review (including peer review) X h. Calibration of new equipment X 22. Does the project involve any client-supplied products, equipment or samples? X See C7T0701 and 1003 Documentation indicating verification and tracking 23. Have product packaging and delivery requirements been considered? X See CTC 1503 & 1504 DD Form 1149, Shipping & Receiving, Property Records 24. Have you planned to survey client satisfaction at project completion? X See CJC 0104 Letter, fax or e-mail requesting completion of Form 3030 Form 3030 sent to EPA annually 25. Will servicing after delivery of product be required? X See CTC 1901 Documentation verifying service activities B-5 ------- QUALITY PLANNING CHECKLIST B. Personnel 26. Have the labor inputs been negotiated with line management and made to MMS? X See CTC 0406 Documentation of MMS input Personnel shall be allocated per MMS a. Have administrative support personnel been allocated? (Library research, deliverable prep, AP support, etc.) X Requirements discussed with line manager b. Have technical support personnel been allocated? (Quality engineering, laboratory, machine shop, test validation, receiving inspection, software, pubs/graphics, computer and network support, etc.) X c. Have engineering support personnel been allocated? (CAD/CAM, drafting, floor layout, tooling, etc.) X C. EHS Planning QUESTION YES NO N/A PROCEDURE REFERENCE DOCUMENTATION required during project execution COMMENTS 27. Does the project involve any potential EHS hazards, including special physical hazards (confined spaces, robots, lasers)? X See CTC 4472 Form 308, documentation and/or written instructions Hazards shall be identified in test plan. Training will be provided using JTA 28. Is proper and sufficient quantity of personal protective equipment available? X See CTC 0906 & 4472 Form 308, MSDSs and/or JSAs Shall be identified in test plans as required 29. Have the environmental aspects and impacts been identified for the project? X See CTC 4310 Completed Form 3013 (EHS aspect/impact checklist) Shall be identified in JTA & test plans as required 30. Have waste disposal issues been addressed? X See CTC 0904 Waste container request form Shall be identified in JTA & test plan. 31. Will chemical products be ordered/used? X See CTC 0906 See CTC 4472 MSDSs Operation Planning Checklist (form 308) Shall be identified in JTA & test plan B-6 ------- QUALITY PLANNING CHECKLIST 32. Is EHS monitoring required to determine potential releases to the environment or exposure to personnel? X See CTC 4510, 4320, & 1602 Form 3014 (legal & other requirements checklist), monitoring/measurement data, records Shall be identified in JTA & test plan D. Design 33. Have customer requirements been addressed in the design of your activity, product or service? X See CTC 0403 Requirements document, design document Shall be identified in test plan and reports 34. Does the design require new materials? X See CTC 0406 Risk assessment in Project Plan 35. Are acceptance criteria clearly defined and understood? X See CTC 0403 Requirements document, design document, design review meeting minutes EPA provides acceptance criteria in review comments 36. Have all design inputs and outputs been identified/documented? X See CTC 0403 Documentation of design inputs and outputs Shall identify in test plan 37. Are design reviews required, if so have the reviews been scheduled at appropriate stages of the design? X See CTC 0406 Design review requirement noted in the Project Plan Document reviews in project file 38. Have appropriate customer approvals been obtained? X See CTC 0406 Documentation of client/ customer approval Customer approval maintained in project file III. Project Implementation A. Process Control QUESTION YES NO N/A PROCEDURE REFERENCE DOCUMENTATION required during project execution COMMENTS 39. Have all additional requirements (customer specifications, industry standards, regulatory requirements, Mil-Stds, etc.) been identified and controlled? X See CTC 4320 Completed Fomi 3014 (legal/other reqts checklist), master list of applicable standards/revision status Shall be identified in test plans and reports 40. Have Facility Support Services been contacted for maintenance and operating X See CTC 0903 Facility Support Request System (FSRS) B-7 ------- QUALITY PLANNING CHECKLIST support services for new equipment? 41. Have material handling requirements been identified? X See CTC 0902 & 1501 Demo test plan/work order request reqts. and/or PO instructions Identified in agreements, JTA, & test plans 42. Have you addressed the handling/controlling of nonconforming products? X See CTC 1301 Form 3004 (non-conforming material report) 43. Are the use of Statistical Techniques such as Design of Experiments needed? X See CTC 2001 Documentation of statistical techniques used Identified in Generic Verification Protocol and test plans 44. Is product traceability and identification required? X See CTC 0801 & 1201 Material tags, markings, etc., documentation of inspection/test status 45. Are project reviews scheduled at appropriate stages of development? X See CTC 1004 Documentation of project review (e.g., forms, reports, etc.) Project reviews conducted weekly and monthly 46. Have you budgeted additional time for senior management review? X See CTC 1004 Documentation of project review (e.g., forms, reports, etc.) Budgeted in MMS and documented on Form 908 47. Have you allowed time in your schedule for internal quality and EMS audit staff? X See CTC 1701 Internal Audit Reports Audits documented in QMP & schedule 48. Have you established a contingency budget to address possible corrective and preventive action requrests (PCARs)? X See CTC 1401 Preventive and Corrective Action Request Form 3026 Resources allocated in budget 49. Has final inspection been addressed? X See CTC 0905 & 1001 Form 908 or contract-specific approval form, or other documentation of inspection Shall be identified in test plan & ETV-MF QMP 50. Have you planned to maintain appropriate records (i.e., project plans, test plans, design review documentation, etc.) X See CTC 1602 Controlled records as required Requirements identified in QMP 51. Are any engineering models or drawings (internal or customer supplied) associated with this project? X See CTC 0505 Controlled drawings or models B-8 ------- QUALITY PLANNING CHECKLIST QUESTION YES NO N/A PROCEDURE REFERENCE DOCUMENTATION required during project execution COMMENTS 52. Have operating, maintenance, safety, and/or calibration instructions been generated for each piece of equipment? If new equipment has been purchased, follow procedure 1101, Control of Measuring and Test Equipment, to have the initial calibration performed and establish a future calibration schedule. X See CTC 0501 & CJC1101 Controlled work instructions Identified in test plans. Equipment calibration scheduled by CTC 53. Do existing procedures/work instructions need to be updated to account for changes in methodology? X See CTC 0504 Form 3002 (document change request form) 54. Have maintenance requirements been considered to ensure on-going suitability of process and /or demonstration factory equipment? X See CTC 0907 PSRS requests, maintenance records, etc. 55. Are shipping/receiving acceptance instructions necessary for the receipt of purchased materials? X See CTC 1003 Acceptance instructions B. Validation/Test 56. Have appropriate validation/inspection tests been identified and scheduled? X See CTC 0403 Verification plans, validation plans, test plans, etc. Identified in test plan 57. Are there any design requirements specified that cannot be evaluated using known validation/inspection techniques? X See CTC 0403 Verification plans, validation plans, test plans, etc. 58. Have various stages of in-process inspection/validation been identified? X See CTC 1201 Schedule of testing/inspection Identified in test plan 59. Have appropriate work orders or test plans with applicable process parameters been developed to conduct experiments? X See CTC 0902 Demonstration test plans or work order requests Test plan to be developed by subcontractor and finalized by CTC 60. Will calibration of measuring and test equipment be required? X See CTC 1101 Calibration schedules, stickers, certificates, etc. Field calibration identified in test plan. B-9 ------- QUALITY PLANNING CHECKLIST Mfg calibration scheduled by CTC 61. Do personnel in the review process understand the client acceptance criteria? X See CTC 0404 Verification of qualified, independent reviewers EPA review comments distributed to subcontractors 62. Are engineering performance testing requirements identified? X See CTC 0403 Verification plans, validation plans, test plans, etc. Identified in test plan 63. Does the client require any in-process validations or tests conducted in their presence? X See CTC 0403 Verification plans, validation plans, test plans, etc. B-10 ------- ------- APPENDIX C CTC ISO/EPA ETV Documentation Nomenclature Matrix ------- CTC ISO/EPA ETV DOCUMENTATION NOMENCLATURE MATRIX Tier CTC ISO Documentation EPA ETV Documentation 1 ISO Quality Manual ETV-MF QMP 2 Procedures Generic Verification Protocol 3 Work Instructions Test/QA Plan 4 Records and Forms Standard Operating Procedures C-l ------- |