ElV

CPC

Concurrent

Technologies

Corporation

ENVIRONMENTAL TECHNOLOGY VERIFICATION PROGRAM
METAL FINISHING TECHNOLOGIES
QUALITY MANAGEMENT PLAN

For U. S. Environmental Protection Agency
By Concurrent Technologies Corporation

Concurrent Technologies Corporation is the Verification Partner for the EPA ET\r MF
Technologies Center under EPA Cooperative Agreement No. CR826492-01-0.

Revision 1

March 26, 2001


-------
Revision 1 03/26/01

TITLE: ENVIRONMENTAL TECHNOLOGY VERIFICATION PROGRAM
METAL FINISHING TECHNOLOGIES QUALITY MANAGEMENT
PLAN

ISSUE DATE: March 26, 2001

DOCUMENT CONTROL

This document will be maintained by Concurrent Technologies Corporation in accordance with
the EPA Environmental Technology Verification Program Quality and Management Plan for the
Pilot Period 1995-2000 (EPA/600/R-98/064). Document control elements include unique issue
numbers, document identification, numbered pages, document distribution records, tracking of
revisions, a document MASTER filing and retrieval system, and a document archiving system.

ACKNOWLEDGMENT

This is to acknowledge Jim Totter and Valerie Whitman for their help in preparing this document.

Concurrent Technologies Corporation is the Verification Partner for the EPA ETV Metal
Finishing Technologies Center under EPA Cooperative Agreement No. CR826492-01-0.

l


-------
Revision 1 03/26/01

Environmental Technology Verification Program
Metal Finishing Technologies
Quality Management Plan

Prepared By:

Clinton Twilley
CTC QA Manager

7

Date

'cf

Approvals:

Dorm W. Brown	' Date

CTC ETV-MF Program Manager

/}£(^ £.	Z3t ZOO/

Alva Edwards Daniels	Date

EPA ETV Center Manger

11


-------
Revision 1 03/26/01

TABLE OF CONTENTS

1.0 INTRODUCTION	1

1.1	Purpose and Content	3

1.2	Scope and Applicability	3

1.3	Document Organization	3

1.4	Review and Update	4

PART A: MANAGEMENT SYSTEMS	5

2.0 MANAGEMENT SYSTEMS (PART A)	6

2.1	Management and Organization	6

2.1.1	Program Team Responsibilities	6

2.1.2	Authority to Stop Work for Safety and Quality Considerations	9

2.2	Quality System Description	10

2.2.1	Quality System Documents	10

2.2.2	Quality Controls	11

2.2.3	Management System Review	12

2.3	Personnel Qualifications and Training	12

2.4	Procurement	13

2.5	Documents and Records	14

2.5.1	Laboratory Data and Reports	15

2.5.2	Protocols and Test/QA Plans	15

2.5.3	Document Control and Distribution	19

2.5.4	Verification Reports and Verification Statements	19

2.6	Computer Hardware and Software	19

2.6.1	Corporate Computing	19

2.6.2	Process Control	20

2.6.3	Data Collection	20

2.6.4	ETV Website	20

2.7	Planning	21

2.7.1	PI anning Meetings	21

2.7.2	Generic Verification Protocols	21

2.7.3	Test/QA Plans	22

2.7.4	Data Quality Objectives (DQO)	23

in


-------
Revision 1 03/26/01

2.7.5 Standard Operating Procedures (SOPs)	23

2.8	Implementation of Work Processes	24

2.9	Assessment and Response	24

2.9.1	Numbers and Types of Assessments	25

2.9.2	Management Assessments	26

2.9.3	Technical Assessments	27

2.9.4	Data Quality Assessments	27

2.9.5	Assessment Procedures	28

2.9.6	Assessment Personnel Qualifications, Responsibility, and Authority	28

2.9.7	Response	28

2.10	Quality Improvement	28

2.10.1	Detecting and Correcting Quality System Problems	29

2.10.2	Cause and Effect Relationship	29

2.10.3	Root Cause	30

2.10.4	Quality Improvement Action	30

PART B: COLLECTION AND EVALUATION OF ENVIRONMENTAL DATA	31

3.0 COLLECTION AND EVALUATION OF ENVIRONMENTAL DATA

(PART B)	32

3.1	Planning and Scoping	32

3.1.1	Planning Personnel	32

3.1.2	Purpose, Scope, and Objectives	33

3.1.3	Collected Data and Design of Experiment	33

3.1.4	Documentation and Reporting	33

3.1.5	Assessments	33

3.1.6	Constraints and Suspension of Work	33

3.1.7	Waste Minimization and Disposal	33

3.2	Design of Technology Verification Tests	34

3.2.1	Test/QA Plans	34

3.2.2	Intended Use of the Data	34

3.2.3	Data Quality Indicators	34

3.2.4	Performance Characteristics	35

3.2.5	Specification	35

3.3	Implementation of Planned Operations	35

iv


-------
Revision 1 03/26/01

3.3.1	Technology Provider Equipment	36

3.3.2	Analytical Laboratories	36

3.3.3	Sample Handling Procedures	37

3.3.4	Laboratory Data Handling, Validation, Reporting, and Archiving	37

3.3.5	Control of SOPs, Test/QA Plans, and Methods	38

3.3.6	Work Instructions	38

3.3.7	Verification of Equipment Operation Prior to Testing	39

3.3.8	Procedures for Data Validation	39

3.4	Assessment and Response	39

3.4.1	Maintenance and Calibration Frequency	40

3.4.2	Analysis of Known Samples/Standards	40

3.4.3	Internal and External Audits	40

3.4.4	Assessment of Data Usability	41

3.5	Reporting	41

PART C: DESIGN, CONSTRUCTION AND OPERATION OF

ENVIRONMENTAL TECHNOLOGY	43

4.0 DESIGN, CONSTRUCTION AND OPERATION OF ENVIRONMENTAL

TECHNOLOGY (PART C)	44

4.1	Planning	44

4.2	Design of System	44

4.3	Construction/Fabrication of Systems and Components	45

4.4	Operation of Systems	45

4.4.1	Management of Consumables	45

4.4.2	Operational Controls	45

4.4.3	Maintenance and Calibration Frequency	46

4.5	Assessment and Response	46

4.6	Verification and Acceptance of Systems	46

5.0 DISTRIBUTION	47

6.0 REFERENCES	48

v


-------
Revision 1 03/26/01

LIST OF FIGURES

Figure 1. ETV-MF Verification Process	2

Figure 2. ETV-MF Program Management	8

LIST OF TABLES

Table 1. Records Management Scheme	18

Table 2. Assessments	25

LIST OF APPENDICES

APPENDIX A:	Concurrent Technologies Corporation Quality Policy Statement,

Environmental, Health, and Safety Policy Statement	 A-l

APPENDIX B:	ETV-MF Quality Planning Checklist	 B-l

APPENDIX C:	CTC ISO/EPA ETV Documentation Nomenclature Matrix	 C-1

vi


-------
Revision 1 03/26/01

DOCUMENTS AND GENERAL TERMS

CTC

Concurrent Technologies Corporation (C7U) is the verification partner for this Metal Finishing
Technologies Center Program. The Program will be coordinated through the CTC Office in
Largo, Florida.

CTC ETV-MF Program Manager The person designated by CTC to manage the center and
serve as the chief point of contact (POC) with EPA is Donn Brown.

CTC Leadership Team

Upper tier management group that guides the corporation.

CTC Management Team

Group of managers that are heads of sections and directly oversee personnel.

CTC QA Manager

The person designated by CTC to manage quality assurance for the center on behalf of the CTC
ETV-MF PM is Clinton Twilley.

CTC Review/Audit Reports

The "quality records" developed by CTC as a result of conducting assessments during ETV-MF
implementation.

Data Quality Objectives (DQO)

Data quality objectives are the qualitative and quantitative outputs from a strategic planning
process that identifies and defines the type, quality and quantity of data needed to satisfy a
specified use.

E4 Standard

The American National Standards Institute/American Society for Quality (ANSI/ASQ) national
consensus standard that is applicable to quality systems for pollution control technology
development. The standard is entitled, Specifications and Guidance for Quality Systems for
Environmental Data Collection and Environmental Technology Program, ANSI/ASQC E4-1994.

EPA Line Management

The management structure to whom each EPA Center Manager reports; i.e., branch chief,
division director, laboratory director.

EPA ETV Center Manager

The EPA person designated by EPA line management to serve as the lead for an individual ETV
Center. Alva Edwards Daniels is the EPA Center Manager for the Metal Finishing Technologies
Center.

vii


-------
Revision 1 03/26/01

EPA Quality Assurance (QA) Manager

The EPA Quality Assurance person designated by EPA line management to manage quality
assurance efforts for this Metal Finishing Center program is Lauren Drees.

EPA Review/Audit Reports

The "quality records" developed by EPA as a result of conducting assessments during ETV
implementation.

ETV-MF Team

CTC employees and subcontractors actively working on the ETV-MF Program; CTC engineers
and quality personnel, and subcontracted Metal Finishing industry consultants.

ETV-MF Program

Environmental Technology Verification for Metal Finishing Pollution Prevention Technologies
Program.

ETV-MF Project Manager

ETV-MF team member assigned by the CTC ETV-MF Program Manager to manage a specific
verification project.

ETV Program

Environmental Technology Verification Program established by EPA to accelerate the
development or commercialization of environmental technologies through third party
verification of performance.

ETV Quality and Management Plan for the Pilot Period 1995-2000 (EPA/600/R-98/064)

The specific policies and procedures set forth by EPA for management of quality-related
activities in the ETV program.

ETV Test Objective

The stated objective(s) of each technology test. CTC uses the data quality objectives process to
establish test objectives and test measurement quality criteria.

ETV Verification Report

The report of the result of an individual ETV technology test.

ETV Verification Statement

An executive summary developed by CTC and approved by the EPA Center Manager, which
reports individual technology performance.

Focus Area

A critical area within the Metal Finishing industry requiring innovative pollution prevention
solutions.

viii


-------
Revision 1 03/26/01

Generic Verification Protocols

Those protocols developed, modified, or selected to promote uniform testing cf a single focus
area for the ETV-MF Program operated by CTC.

Procedure

A document that reflects the practice defined in the Quality Manual.

QA/G-6

Guidance for Development of Standard Operating Procedures (SOPs) for Quality Related
Documents (EPA/600/R-96/027).

QA/G-9

Guidance for Data Quality Assessment (EPA/600/R-96/084).

QA/G-9D

Data Quality Evaluation Statistical Toolbox (DataQUEST) User's Guide (EPA/600/R-96/085).
Raw Data

All data and information recorded in support of analytical and process measurements made
during planning, testing, and assessing environmental technology, including support records such
as: computer printouts, instrument run charts, standards preparation records, field log records,
technology operation logs, and monitoring records. ETV and CTC test files (all records
including raw data) and technical data and associated quality control data which support the data
summarized and the conclusions made in each ETV-MF verification report.

Records

All books, papers, maps, photographs, machine-readable materials, or other documentary
materials, regardless of physical form or characteristics, made or received by EPA or CTC or
their designated representative for the ETV-MF Program.

Stakeholder Advisory Group

The group established for the ETV-MF Program consisting of representatives of any or all of the
following verification customer groups: buyers and users of Metal Finishing equipment,
technology developers and vendors, members of technical associations, and Federal, state and
local government.

Standard Operating Procedures (SOP)

Procedures describing verification activities such as process equipment operation, sample
collection and analytical testing.

Technology Providers

Pollution control equipment manufacturers and vendors.

Technology Provider Agreements

Agreements between technology providers and CTC to collaborate on verification testing.

IX


-------
Revision 1 03/26/01

Test Measurement

Those critical measurements that must be made during the course of a Metal Finishing
technology test to evaluate achievement of the ETV-MF test objective.

Test/QA Plan

The plan developed by CTC for each individual test of a technology or technology class.
Therefore, the Test/QA Plan may include more than one technology. The Test/QA Plan provides
the experimental approach with clearly stated test objectives and associated quality objectives for
the related measurements. The Test/QA Plan may incorporate or reference Generic Verification
Protocols.

User Industry

Buyers and users of Metal Finishing equipment.

Verification

Establishing or providing the truth of the performance of a technology under specific,
predetermined criteria or protocols and adequate data quality assurance procedures.

Verification Partner

The public or private sector organization selected by EPA to implement the ETV program. CTC
is the verification partner for this ETV-MF Program.

Work Instruction

A document that provides specific requirements on how to perform or carry out associated
duties.

Work Plan Revisions

A task-specific budget which estimates the costs of completing a verification.

x


-------
Revision 1 03/26/01

ABBREVIATIONS AND ACRONYMS

A2LA	American Association for Laboratory Accreditation

ANSI	American National Standards Institute

ASQ or ASQC	American Society for Quality

ASTM	American Society for Testing and Materials

CTC	Concurrent Technologies Corporation

DQO	Data Quality Objective

E4 Standard	ANSI/ASQC E4-1994

EHS	Environmental, Health and Safety

EPA	U.S. Environmental Protection Agency

EMS	Environmental Management System

ETV	Environmental Technology Verification

ETV-MF	Environmental Technology Verification for Metal Finishing

Pollution Prevention Technologies Program

ETV QMP	Environmental Technology Verification Program Quality and

Management Plan for the Pilot Period (1995 - 2000)

ISO	International Organization for Standardization

MIL-SPEC	Military Specifications

MSDS	Material Safety and Data Sheets

NELAC	National Environmental Laboratory Accreditation Council

NELAP	National Environmental Laboratory Accreditation Program

NIST	National Institute of Standards and Technology

OSHA	Occupational Safety and Health Administration

P2	Pollution Prevention

PC	Personal Computer

POC	Point of Contact

PR	Procurement Requisition

QA	Quality Assurance

QC	Quality Control

QMP	Quality Management Plan

XI


-------
Revision 1 03/26/01

ABBREVIATIONS AND ACRONYMS (continued)

RAB

Registrar Accreditation Board

RCRA

Resource Conservation and Recovery Act

RFQ

Request for Quotation

RFT

Request for Technologies

RvA

Raad voor de Accreditatie

SAE

Society of Automotive Engineers

SARA

Superfund Amendments and Reauthorization Act

SOP

Standard Operating Procedure

SOW

Statement of Work

VP

Verification Partner

Xll


-------
Revision 1 03/26/01

1.0 INTRODUCTION

The Environmental Technology Verification for Metal Finishing Pollution Prevention
Technologies (ETV-MF) Program is a partnering program under a cooperative agreement
between Concurrent Technologies Corporation (CTC) and the U.S. Environmental Protection
Agency (EPA). The purpose of the program is to provide verification testing of environmental
technologies intended for reducing pollution in the Metal Finishing industry. This is one of the
ETVs established as part of a broader program administered by EPA. The ETV-MF verification
process is shown in block flow format in Figure 1.

Examples of technologies targeted for verification under this program may include:

•	Emissions and risk characterization from plating operations

•	Methods to reduce cyanide concentrations in wastewater and solids

•	In-process recovery of metals, acids, and cleaners (Pollution Prevention)

•	Off-site recovery of metals, acids, and cleaners

•	Reduction and replacement of the use of chlorinated solvents for cleaning

The goal of this Quality Management Plan (QMP) is to document quality procedures followed
within the ETV-MF for meeting Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs, ANSI/ASQC E4-
1994 (E4 Standard). This standard has been established as the standard for all the ETV
Programs along with the EPA-developed Environmental Technology Verification Program
Quality and Management Plan for the Pilot Period 1995-2000, EPA/600/R-98/064 (ETV QMP),
which is in conformance with the E4 Standard. This QMP details the system of management and
team infrastructure to ensure that data resulting from the execution of this program is 1) valid
and defensible and 2) meets project quality and environmental quality objectives.

First tier documentation of quality procedures pertaining to this ETV-MF Program is the QMP
(this document). As second tier documentation, a Generic Verification Protocol has been
developed for all focus areas [Ref. 1], If the composition of a test plan varies significantly, the
Generic Verification Protocol will be supplemented or additional protocols will be developed.
These Generic Verification Protocols will be supported by third tier Test/Quality Assurance
(QA) Plans for evaluation of each individual technology to be verified under the auspices of this
ETV Program. Each technology-specific Test/QA Plan will identify the objectives of testing,
design of the experiment, and critical and non-critical measurements required. Fourth tier
documentation, if necessary, is recorded in the Standard Operating Procedures (SOPs).

1


-------
Revision 1 03/26/01

A Metal Finishing Focus Area may be a finishing process, a type of
equipment, an application, or some other category of Metal Finishing
technology for pollution prevention. Focus areas are identified based on
industry needs; these focus areas are evaluated and prioritized by the
Stakeholder Advisory Group.

A Request for Technologies (RFT) is issued for pollution prevention
technologies that address the focus areas. The technology suppliers provide
time and funding, in addition to the technology to be tested. They team with
a user industry facility having a process within the focus area chosen. The
users include production Metal Finishing shop specialists who are willing to
host the verification testing.

Responses to the RFT are reviewed for relevant information, technology
eligibility, teaming arrangement(s), and the levels of support to be provided
by the technology supplier and industry user. Final selection of technologies
for verification is based on guidelines from the Stakeholder Advisory Group.

Test plans are developed for verification of the technology's ability to satisfy
Metal Finishing and environmental requirements. These plans are agreed to
by all parties, and will become the definitive requirement for verification.

Verification testing is performed on the process, product, and/or equipment,
in the user industry Metal Finishing production shop or facility. All
verification activities are conducted by the ETV-MF team.

A verification report is prepared for EPA detailing the results of the
verification test. The technology supplier is given an opportunity to review
the report before final approval. Verification reports are posted on the EPA
ETV and ETV-MF Internet websites. A verification statement, which is a
summary of the report, is distributed via the websites and other information
channels after approval.

Figure 1. ETV-MF Verification Process

2


-------
Revision 1 03/26/01

1.1	Purpose and Content

This ETV Program establishes a unique organization that requires a detailed system of
quality management to fulfill its verification and data collection mission. The purpose of
this document is to serve as an overall guide to the quality system and the management
organization areas that have key roles in the oversight of this ETV-MF Program. This
ETV-MF QMP contains information concerning the planning, implementation, and
assessment of the quality system.

•	Personnel directly involved with the execution of the program have the responsibility
and authority for quality control and quality assurance activities.

•	Oversight of the project is the responsibility of the CTC ETV-MF Program Manager,
Donn Brown.

•	The person designated to manage quality assurance for the ETV-MF Program is the
CTC QA Manager, Clinton Twilley.

•	The CTC ETV-MF Program Manager and CTC QA Manager are independent entities,
both reporting through the CTC International Organization for Standardization (ISO)
management system.

1.2	Scope and Applicability

Management and data collection activities of this program are for performance assessment
of environmental technologies used for pollution prevention (P2). The QA and Quality
Control (QC) functions are a vital part of the ETV operations and are applicable to all tiers
of the project. All tiers of project personnel are individually responsible for QA and QC
functions. The CTC QA Manager, independent of the CTC ETV-MF Program Manager,
is responsible for managing and addressing QA activities. Review of project data and
reports occurs across multiple tiers of management.

1.3	Document Organization

The remaining sections of this QMP are organized according to the quality requirements
found in the E4 Standard. Appropriate references are also made to EPA's ETV QMP and
CTCs ISO 9001 and 14001 management systems. The sections of the E4 Standard that
apply to this ETV-MF Program are: Management Systems (Part A), Collection and
Evaluation of Environmental Data (Part B), and the operational portions of Design,
Construction, and Operation of Environmental Technologies (Part C). The applicable
subsection requirements are listed below:

Management Systems (Part A')

•	Management and Organization

•	Quality System Description

•	Personnel Qualifications and Training

•	Procurement

•	Documents and Records

•	Computer Hardware and Software

3


-------
Revision 1 03/26/01

•	Planning

•	Implementation of Work Processes

•	Assessment and Response

•	Quality Improvement

Collection and Evaluation of Environmental Data (Part B)

•	Planning and Scoping

•	Design of Technology Verification Tests

•	Implementation of Planned Operations

•	Assessment and Response

•	Reporting

Design. Construction, and Operation of Environmental Technologies (Part C)

•	Planning

•	Design of System

•	Construction/Fabrication of Systems and Components

•	Operation of Systems

•	Assessment and Response

•	Verification and Acceptance of Systems

1.4 Review and Update

This document is reviewed and approved by the CTC ETV-MF Program Manager, CTC
Leadership Team and EPA. Input and guidance on key points are provided by EPA. This
document will be reviewed annually by project personnel. Corrections and additions will
be submitted to ETV Program Management by the staff associated with this ETV-MF
Program. Major revisions will require the preparation of an updated QMP. The updated
plan will be submitted to EPA for review and approval as required.

4


-------
Revision 1 03/26/01

PART A: MANAGEMENT SYSTEMS

Part A of this ETV-MF Quality Management Plan contains the specifications and guidelines
applicable to common or routine quality management functions and activities necessary to
effectively manage the ETV-MF Program.

5


-------
Revision 1 03/26/01

2.0 MANAGEMENT SYSTEMS (Part A)

This section of this ETV-MF QMP contains the specifications and guidelines applicable to
common or routine quality management functions and activities necessary to effectively manage
the ETV-MF Program.

The goal of the ETV-MF Program is to test and verify the performance of innovative technologies
that have environmental benefits. This ETV-MF Program is to become a self-sustaining,
independent entity providing an unbiased evaluation of these technologies to Metal Finishing
companies, technology providers, and other interested stakeholders. To this end, high standards
of quality must be maintained throughout the company and along program lines.

CTC s Johnstown, Pennsylvania ISO 9001 Quality Management System (QMS) and ISO 14001
Environmental Management System have been certified as compliant by KEMA Registered
Quality, Inc., a third party registrar. KEMA Registered Quality Inc. of Chalfont, Pennsylvania, is
an ISO 9000, QS-9000 and ISO 14000 registrar, accredited by the Registrar Accreditation Board
(RAB) and Raad voor de Accreditatie (RvA). The CTC Regional Offices, including the Largo,
Florida, office, have not yet been certified.

CTC s certified ISO 9001 QMS and ISO 14001 Environmental Management System comprise the
overall management system that will be used for this program. The specific requirements of the
E4 Standard and ETV QMP will be addressed in this QMP as well as references to specific CTC
ISO quality requirements as appropriate.

CTC s ISO 9001 Quality Management System starts with the Quality Policy, which is contained
in Appendix A This appendix also contains the CTC Environmental, Health and Safety Policy
Statement.

It is the responsibility of each individual in the Company to comply with the provisions of both
these policies in support of this program.

2.1 Management and Organization

CTCs Quality Manual, section 4.1 Management Responsibility, addresses the overall
management organization. Specifics for this program are shown in Figure 2. The CTC
ETV-MF Program Manager is responsible for ensuring proper communication throughout
all program lines and maintaining a cohesive team structure.

2.1.1 Program Team Responsibilities

The CTC ETV-MF Program Manager has overall responsibility for the ETV-MF
Program. He will be the sole point of contact (POC) with EPA for all Program
activities. The CTC ETV-MF Program Manager will be one source of information
on the technical activities associated with each verification project. Another
source of project technical information will be periodic reviews with ETV-MF
Team members, the CTC ETV-MF Program Manager, and the CTC QA Manager.

6


-------
Revision 1 03/26/01

EPA will be invited to attend all project review meetings. Figure 2 illustrates the
management organization of the Program Team and its functional relationships,
respectively.

The CTC QA Manager is responsible for ensuring that the applicable requirements
of the Environmental Technology Verification Program Quality and Management
Plan for the Pilot Period 1995 - 2000, Specifications and Guidelines for Quality
Systems for Environmental Data Collection and Environmental Technology
Programs (ANSI/ASQC E4-1994), and CTC s Johnstown, Pennsylvania, ISO 9001
Quality Management System and ISO 14001 Environmental Management System
are addressed in the ETV-MF QMP and followed. In addition, the CTC QA
Manager is directly responsible for performing the QA assessments identified in
Table 2. The CTC QA Manager will report the results of these assessments to the
CTC ETV-MF Program Manager, who then reports the results to the EPA Center
Manager. Similarly, revisions to CTC's Johnstown, Pennsylvania, ISO 9001
Quality Management System and ISO 14001 Environmental Management System
will flow down from the CTC QA Manager to the CTC ETV-MF Program
Manager for incorporation into the ETV-MF Program.

ETV-MF team members report directly to the CTC ETV-MF Program Manager.
Team members are comprised of CTC engineers and quality personnel, and
subcontracted Metal Finishing industry consultants. Subcontractors are selected
on the basis of their experience in the Metal Finishing industry and their capability
for meeting the testing and quality assurance requirements. Subcontractors will
help identify potential stakeholders and initial focus areas, develop Generic
Verification Protocols, identify Metal Finishing shops and technology providers,
and oversee verification testing and analysis of results.

7


-------
Revision 1 03/26/01

Figure 2. ETV-MF Program Management

8


-------
Revision 1 03/26/01

Members of the ETV-MF Team are responsible for accomplishing all of the
technical activities associated with the ETV-MF Program. These include the
activities identified in Part B: Collection and Evaluation of Environmental Data
and Part C: Design, Construction and Operation of Environmental Technology.
Members of the ETV-MF Team, as designated by the CTC ETV-MF Program
Manager, will act as ETV-MF Project Managers for specific verification testing
projects. Progress on project technical activities will be reported to the CTC ETV-
MF Program Manager and to EPA through periodic project reviews.

Stakeholders are defined as parties who have a reason to be interested in the results
of the verification process. These stakeholders will be drawn from buyers and
users of Metal Finishing equipment, treatment system equipment manufacturers,
members of technical associations, and Federal, state and local government
regulatory agencies. EPA and CTC recognize that the quality of the Stakeholder
Advisory Group (i.e., their knowledge of markets and technologies, and their
willingness to share that knowledge by giving of their time and energy) will be key
to the success of this program.

As shown in Figure 2, Metal Finishing shops will be production facility test sites
for the verification processes and implementation of the technologies. Verification
will evaluate how well tie technology supplied by vendors performs as part of the
overall manufacturing process, and the technology's ability to reduce or eliminate
multimedia emissions from operations performed by the Metal Finishing industry.

Testing conducted outside CTC facilities may involve the use of additional
subcontractors. These will be selected and used according to the required work
and their capability of meeting the testing and quality assurance requirements (note
section 2.4). The CTC ETV-MF Program Manager will manage subcontractors.

2.1.2 Authority to Stop Work for Safety and Quality Considerations

CTC shall stop unsafe work and work of inadequate quality in the following
manner:

•	CTC will review all planned verification tests and ensure environmental, safety
and health issues are addressed.

•	The CTC ETV-MF Program Manager, CTC QA Manager, and ETV MF Team
members acting as Project Managers will issue a Stop Work Order in the event
that unsafe work, work of inadequate quality, or work that has the potential for
harm to the environment is identified.

•	The EPA ETV Center Manager, Alva Edwards Daniels, and the EPA QA
Manager, Lauren Drees, shall contact the CTC authorized individual(s) in the
event that work of inadequate quality is discovered.

9


-------
Revision 1 03/26/01

2.2 Quality System Description

A quality system is herein planned and will be established, documented, implemented and
assessed as an integral part of the ETV-MF management system as defined by ETV
quality policy. The authority requiring an appropriate quality system for ETV-MF is the
U.S. Code of Federal Regulations (CFR), 40 CFR Part 30.54.

This plan complies with the E4 Standard, which is comparable to the ISO 9000 standard
series. This ETV-MF quality system addresses each applicable specification provided in
the E4 Standard, using the policies and procedures in this plan as appropriate.

2.2.1 Quality System Documents

CTC has standardized on ISO 9001 as the Quality Management System (QMS)
model, using ISO 9001:1994, Quality systems - Model for QA in design,
development, production, installation and servicing for guidance. This system is a
four-tier system consisting of a Quality Manual (Tier 1), Procedures (Tier 2),
Work Instructions (Tier 3), and Records and Forms (Tier 4). All documentation is
available on-line through the CTC computer network and can be accessed by each
employee from a workstation. Appendix C contains a matrix that correlates with
CTCs ISO documentation with the EPA's ETV documentation nomenclature.

Since initiating this quality effort, CTC has added to the quality goal of "meeting
customer needs" by including the requirement that it occur in an environmentally
responsible, healthy, and safe manner. To ensure this added requirement is
consistently met, CTC has established an environmental, health and safety (EHS)
management system built on the framework of the ISO 14001 Environmental
Management System (EMS) standard, and including health and safety aspects as
well. The EMS has been built on and integrated with the ISO 9001 system
described above. This has resulted in a seamless, integrated management system
that provides the highest possible value to CTC, our staff, and customers.

The ISO 9001 Quality and ISO 14001 Environmental Manuals describe policies
and activities that support and maintain CTC's QMS and EMS Management
System. They are designed to meet the 20 target items of the ISO 9001 QMS
standard and 17 target items of the ISO 14001 EMS standard. Management has
reviewed and endorsed these manuals and Tier 2 Procedures. The Management
Systems Representative for CTC reports to a Senior Vice President who
participates on the Quality Committee. This committee consists of two General
Managers, three Directors, and the Management Systems Representative. The
committee is responsible for implementation of the CTC management system.
Quality Representatives reporting to the Management Systems Representative are
responsible for maintaining the documentation, investigating root cause,
documenting and implementing corrective action, and maintaining records. In
addition, the QMS requires a member of the CTC leadership team to review and
approve contract-required deliverables prior to submittal to EPA.

10


-------
Revision 1 03/26/01

Tier 2 procedures will be used where directly applicable to ETV-MF. Tier 3 and 4
documents will be used directly or serve as guides to develop ETV-MF specific
documents (i.e., Test/QA Plans and SOPs for testing or process equipment
operation).

Procedures, Tier 2 documents, provide direction on how to meet the requirements
of the quality and environmental system. These procedures provide guidance to
personnel for major program planning through the Quality Planning Checklist,
preparation and delivery of documents, purchase of goods and services,
scheduling, training, calibration and maintenance of equipment, inspection and
testing of products, and assessments. CTC s Demonstration Test PI an/Work Order
Request System, ISO Procedure 0902, requires the use of test protocols (Generic
Verification Protocols and Test/QA Plans) and Demonstration Test Plans
(equipment and process operation SOPs), which are specified in this QMP.

Work Instructions, Tier 3, provide operating, process, and laboratory personnel
with step-by-step instructions on performing a job function.

Forms, Tier 4, are used to capture the information required to document the quality
system and test results. When forms are completed, they become some of the
quality records used to establish documentation that the system is being operated
correctly.

Tier 4 documents are equivalent to SOPs specified in EPA's ETV QMP. Data
required by Test/QA Plans will use existing forms when possible or new ones
developed specifically for the ETV-MF Program.

2.2.2 Quality Controls

Internal audits are conducted approximately three times a year in Johnstown,
Pennsylvania, to evaluate the CTC ISO Quality Management System. Qualified
staff members who have lead auditor status conduct these audits. Additional staff
members who have received internal training assist in the internal audit process.
In addition, third party surveillance audits are conducted every six months, at CTC
in Johnstown, Pennsylvania. The surveillance audits are conducted by KEMA
Registered Quality, Inc.

This QMP, with reference to CTC's certified ISO 9001 and 14001 Management
System, offers an overall guide to the management system of the ETV-MF
Program. As the quality and environmental management system for CTC, the
Quality Manual, Environmental Manual, ISO Procedures, and Work Instructions
serve as the basis for project planning, work implementation, and assessment of
the ETV-MF Program.

11


-------
Revision 1 03/26/01

2.2.3 Management System Review

As part of CTC s Quality and Environmental Management System requirements,
the management system is reviewed by executive management for continued
effectiveness and suitability (ISO Procedure 0401). The management system also
includes project reviews at different tiers on a monthly and quarterly basis (ISO
Procedure 1001). These reviews are both technical and management in nature.
Details of these reviews are also discussed in section 2.9 of this QMP.

2.3 Personnel Qualifications and Training

CTC uses a matrix management system where personnel of various tiers of education,
experience, and capability are available to a project as a manager, consultant, or team
member. Certain management personnel possess MBA degrees or are licensed
professionals. Numerous personnel maintain a membership in a professional society.
This allows for access to experts in key areas. CTC also maintains a network of academic,
military, and private company contacts who are available as consultants.

Only qualified personnel will be used on the ETV-MF Program. Qualifications include
work experience in the Metal Finishing field, sample collection and analysis, and
equipment setup and operation.

The qualifications of CTC and subcontractor employees participating in the ETV-MF
Program are documented in resumes that are maintained on file at CTC for reference as
necessary.

Generic Verification Protocols and Test/QA Plans will identify training requirements to
ensure that CTC and subcontractor program participants are trained in appropriate courses
prior to verification testing.

Identification of specific training requirements for this ETV-MF Program is the
responsibility of the CTC ETV-MF Program Manager. The CTC ETV-MF Program
Manager, along with line management, monitors changes in program personnel and
ensures that staff members are qualified and trained. These training requirements range
from technical competency to awareness training cn the environmental, safety and health
aspects of an employee's or contractor's job.

As stated in CTC ISO Procedure 4420, training will be implemented and documented as
required. Training documentation will be maintained at the Largo, Florida, CTC office
and at CTC Corporate Human Resources in Johnstown, Pennsylvania, and will be
available for review by EPA.

Conferences, seminars, and vendor-provided training will be utilized when identified and
deemed necessary for program execution. Records are maintained for each employee who
receives training from internal and external sources.

12


-------
Revision 1 03/26/01

2.4 Procurement

Procurement is implemented through CTC s Contract Resources per ISO Procurement
Procedure 0301. Contract Resources uses the CTC Procurement Manual, which contains
all procedures necessary for the purchase of items and services for the program. Contract
Resources not only has responsibility for ensuring that all requirements of a contract are
met, but also initiates the purchasing activity subsequent to receiving input from program
personnel.

Procurement of equipment and services is initiated using a Procurement Requisition (PR).
The PR includes the following information:

•	Date PR is prepared

•	Date item is required

•	Account number to which the item will be charged

•	Contract number for the program

•	PR number for tracking

•	Item number

•	Description of item

•	Vendor's identification number

•	Quantity desired

•	Unit cost and total price

•	Vendor name, address, and phone number

•	Signature of person preparing the PR

•	Signature of Program Manager

•	Signature of Line Manager

•	Appropriate approvals

Project personnel are responsible for providing information concerning the specifications
of equipment and supplies to procurement personnel. Additional requirements or
specifications, such as quality or technical requirements, may be included as an
attachment to the PR. By completing the appropriate section on the PR, an inspection of
the item or service for conformance to quality and technical requirements will be
performed at the time it is received. This inspection is documented on a Material
Inspection Receiving Report or a similar document at the Metal Finishing shop where the
equipment or supplies will be used.

The PR is inspected by a procurement representative to ensure that all necessary
information has been provided. The representative will contact the person who initiated
the PR if additional information is required. Major purchases must be made through a
competitive bid unless a sole provider can be justified. The bid process is initiated by
checking Request for Quotation (RFQ) on the PR. Procurement personnel, along with
project personnel, will select vendors and suppliers based on their ability to deliver the
appropriate quality and quantity of material or equipment that meets program
requirements. Procurement personnel may make these decisions based on past history,
performance and reliability history, self-assessment, on-site visit, or product inspection.

13


-------
Revision 1 03/26/01

At a minimum, all material will be inspected at the time of receipt to ensure that the
correct item and quantity has been received. Shipping and Receiving personnel will reject
unacceptable or damaged material upon receipt. In the event that unacceptable goods or
services are received, Contract Resources will act as the agent for negotiating replacement
of parts or services, rework, or refund. Any equipment purchase may be assessed through
an acceptance test to ensure proper operation.

Subcontractors will be evaluated according to their ability to provide the appropriate
quality of service that meets program requirements. An evaluation may be made based
upon self-assessment, onsite visit, past performance and reliability history, or by
contacting references. For laboratories the review will also include evaluation of their
quality control plans and SOPs for all proposed non-standard analytical procedures. In
addition to this review, a determination will be made as to the laboratory's ability to meet
the requirements specified in this QMP, particularly sections 2.5.1, 3.3, and 3.4.

2.5 Documents and Records

CTC procedures are established, controlled, and maintained for identifying, preparing,
reviewing, approving, revising, collecting, indexing, filing, storing, maintaining,
retrieving, distributing, and disposing of pertinent quality documents and records. All
forms of documents and records, including both printed and electronic media, are subject
to these procedures. Original records, developed on behalf of this ETV-MF Program and
required to demonstrate the quality of information and data provided in ETV-MF
verification reports, are specifically included in these procedures. In addition, records
requiring document control are identified and properly controlled.

Contract Resources will issue document numbers and maintain a chronological log of all
memoranda, letters and reports issued to EPA. Contract Resources, along with the CTC
ETV-MF Program Manager, is responsible for ensuring that required documents are
prepared and delivered as scheduled. The following document types may be generated
and submitted to EPA as part of ETV-MF activities:

•	ETV-MF QMP (this document)

•	Generic Verification Protocols

•	Test/QA Plans

•	Standard Operating Procedures (SOPs)

•	Engineering Drawings

•	Resumes

•	Quarterly Status Reports (QSRs)

•	Raw Data

•	Stakehol der Memb er Li st

•	Stakeholder Meeting Summaries

•	Technology Provider Agreements

•	Verification Reports and Verification Statements

•	Management and Quality System Audit reports

•	Non-Conformance and Preventive/Corrective Action Reports

14


-------
Revision 1 03/26/01

• Work PI an Revi si ons

2.5.1	Laboratory Data and Reports

Copies of all quality records, process records, bench sheets and data printouts will
be maintained by the laboratories and be available for inspection by CTC upon
request. All data, calculations, and records are identified by verification trail and
the specific technology evaluated. Documents and records generated by analytical
laboratories shall be retained by the laboratory in accordance with the laboratory
QMP. Laboratory final reports are held for three years after final payment on the
Cooperative Agreement #CR826492-01-0, per 40 CFR30.53.

2.5.2	Protocols and Test/QA Plans

Generic Verification Protocols and technology-specific Test/QA Plans are
prepared, reviewed, approved, and distributed as shown in Table 1, Records
Management Scheme. These documents are held for three years after the final
payment on the Cooperative Agreement #CR826492-01-0, per 40 CFR30.53.

15


-------
Revision 1 03/26/01

Record Type

Preparation/Updating

Review

Approval

Finals Distributed To:

ETV-MF Quality Management
Plan

CTC QA Manager

CTC ETV-MF Program
Manager

CTC Leadership Team
EPA ETV Center Manager
EPA QA Manager

CTC QA Manager
CTC ETV-MF Program
Manager

EPA ETV Center Manager

CTC QA Manager
CTC ETV-MF Program
Manager

EPA ETV Center Manager
EPA QA Manager
CTC Contract Resources

Generic Verification Protocols

ETV-MF Project Manager

CTC ETV-MF Program
Manager

CTC QA Manager
CTC Leadership Team
EPA ETV Center Manager
EPA QA Manager
Stakeholders

CTC QA Manager
CTC ETV-MF Program
Manager

EPA ETV Center Manager

CTC QA Manager
CTC ETV-MF Program
Manager

EPA ETV Center Manager
EPA QA Manager
CTC Contract Resources
EPA ETV Webmaster

Test/QA Plans

ETV-MF Project Manager

CTC ETV-MF Program
Manager

CTC QA Manager
CTC Leadership Team
EPA ETV Center Manager
EPA QA Manager
Stakeholders

CTC QA Manager
CTC ETV-MF Program
Manager

EPA ETV Center Manager
Technology Providers
User industry
ETV-MF Project Manager

CTC QA Manager
CTC ETV-MF Program
Manager
ETV MF Team
EPA ETV Center Manager
EPA QA Manager
CTC Contract Resources
EPA ETV Webmaster
Technology Providers
User Industry
Laboratory Manager (LM)

Standard Operating Procedures

Technology Provider
User Industry

ETV-MF Project Manager

Technology Provider
User Industry
ETV-MF Project Manager

Technology Provider
User Industry
ETV-MF Project Manager

Engineering Drawings

Technology Provider
User Industry

ETV-MF Project Manager

Technology Provider
User Industry
ETV-MF Project Manager

Technology Provider
User Industry
ETV-MF Project Manager

Resumes

(Employee Qualifications)

ETV-MF Team

CTC ETV-MF Program
Manager

CTC Leadership Team

CTC ETV-MF Program
Manager

CTC Leadership Team

CTC ETV-MF Program
Manager

Quarterly Status Reports

CTC ETV-MF Program
Manager

CTC Leadership Team
CTC Line Manager

CTC Line Manager
CTC Leadership Team

CTC ETV-MF Program
Manager

EPA ETV Center Manager
CTC Contract Resources











16


-------
Revision 1 03/26/01

Record Type

Preparation/Updating

Review

Approval

Finals Distributed To:

Raw Data and Engineering
Calculations

ETV-MF Project Manager

CTC QA Manager
CTC ETV-MF Program
Manager
ETV MF Team

CTC ETV-MF Program
Manager

CTC QA Manager

CTC ETV-MF Program
Manager

EPA ETV Center Manager

Stakeholder Member List

CTC ETV-MF Program
Manager

EPA ETV Center Manager

EPA ETV Center Manager

CTC ETV-MF Program
Manager

EPA ETV Center Manager
EPA ETV Webmaster

CTC QA Reviews and Audits

See Section 2.9

Section Section 2.9

See Section 2.9

See Section 2.9

Stakeholder Meeting Summaries

CTC ETV-MF Program
Manager

EPA ETV Center Manager

ETV-MF Team

CTC Leadership Team

CTC Leadership Team
EPA ETV Center Manager

CTC ETV-MF Program
Manager

EPA ETV Center Manager
EPA ETV Webmaster

Technology Provider Agreements

CTC Contract Resources

CTC ETV-MF Program
Manager

CTC Leadership Team

Technology Provider
CTC Leadership Te am

Technology Provider
CTC ETV-MF Program
Manager

CTC Contract Resources

Verification Reports

ETV-MF Project Manager

Technology Provider
CTC ETV-MF Program
Manager

CTC Leadership Team
EPA QA Manager
EPA ETV Center Manager
CTC QA Manager

CTC ETV-MF Program
Manager

CTC Leadership Team
EPA ETV Center Manager
EPA ETV Program Manager

Technology Provider
CTC ETV-MF Program
Manager

EPA ETV Center Manager
EPA ETV Webmaster
CTC Contract Resources

Verification Statements

ETV-MF Project Manager

Technology Provider
CTC ETV-MF Program
Manager

CTC Leadership Team
EPA QA Manager
EPA ETV Center Manager
CTC QA Manager

CTC ETV-MF Program
Manager

CTC Leadership Team
EPA ETV Center Manager
EPA Laboratory Director
EPA ETV Program Manager

EPA ETV Webmaste
CTC ETV-MF Program
Manager

EPA ETV Center Manager
CTC Contract Resources

Work Plan Revisions

CTC ETV-MF Program
Manager

CTC Leadership Team
EPA ETV Center Manager

CTC ETV-MF Program
Manager

CTC Leadership Team
EPA ETV Center Manager

CTC ETV-MF Program
Manager

EPA ETV Center Manager
CTC Contract Resources

ETV Verification Strategy

ETV Coordinator

CTC ETV-MF Program

Manager

ETV Team

EPA Laboratory Directors

ORD Deputy Assistant
Administrator

EPA ETV Webmaster

17


-------
Revision 1 03/26/01

Record Type

Preparation/Updating

Review

Approval

Finals Distributed To:

ETV Quality and Management
Plan

ETV Directors of Quality
Assurance

CTC ETV-MF Program
Manager

EPA QA Manager
ETV Team

ETV Coordinator
EPA Laboratory Directors
EPA Division Directors

EPA ETV Webmaster

COAG/IAG Records

CTC ETV-MF Program
Manager

EPA ETV Center Manager

Not Applicable

Not Applicable

Not Applicable

Annual ETV Progress Report

Evaluation Contractor

CTC ETV-MF Program
Manager
Stakeholders
EPA Team

ETV Coordinator

EPA Laboratory Directors

EPA Reviews/Audit Reports

EPA QA Manager

ETV Directors of Quality

Assurance

EPA ETV Center Manager
ETV Coordinator

Not Applicable

EPA Laboratory Directors
CTC ETV-MF Program
Manager

CTC QA Manager

Table 1. Records Management Scheme

18


-------
Revision 1 03/26/01

2.5.3	Document Control and Distribution

Documents and records will be managed in accordance with Table \ to include
preparation/updating, review, approval, and final distribution of ETV-MF related
documents.

The CTC ETV-MF Program Manager assigns the appropriate personnel for
document preparation. Documents are issued and reviewed according to an
established distribution list. Reviewers return the document with comments,
initials, and date of revision. An approval page is affixed for signatures after
revisions are made. The latest versions of documents for this ETV-MF Program
will be maintained at CTC offices in Largo, Florida. Copies of both electronic and
printed documents will be saved. The CTC QA Manager is responsible for
ensuring that all project personnel are using the most recent and approved
document.

Contract Resources issues the final approved document to EPA. Printed copies of
reports are maintained in Contract Resources' offices in Johnstown, Pennsylvania.
Copies are filed according to contract number, letter number, and document
number. Copies are held for a period of three years after final payment on the
Cooperative Agreement #CR826492-01-0, per 40 CFR 30.53.

2.5.4	Verification Reports and Verification Statements

Results of verification testing will be documented in a verification report. The
report will include all data collected from both process and laboratory testing and
engineering calculations used in data reporting. The report will include a QA
section that documents data quality indicators, deviations from the approved
Test/QA Plan, and limits of the data. In addition, a Verification Statement will be
issued that summarizes the testing performed and the results. The persons
responsible for preparation, review, approval, and final distribution are shown in
Table 1, Records Management Scheme.

2.6 Computer Hardware and Software

CTC maintains adequate computer resources to provide computing capability to the
program. Computer hardware, software, and networking capabilities are geared toward
three major areas. These are corporate computing, process control, and data collection.

2.6.1 Corporate Computing

Corporate computing is based upon advanced computing, software, and
networking technologies that allow personnel to solve a broad range of industrial
problems. Readily available and proven standard technologies, reasonably priced
yet geared toward current trends and technological advances, are used. The

19


-------
Revision 1 03/26/01

company has established guidelines to ensure a uniform yet flexible system that is
able to meet the daily demands of the contracts it supports.

Computing is based on a workgroup approach with each workgroup or contract
having its own server. The system requires password login from desktop personal
computers (PCs) for security. Desktop computers are connected to the server to
provide applications such as word processing, spreadsheet, presentation graphics,
forms, printing, filing, network access, and electronic mail. Backup procedures are
performed at the end of each workday in order to preserve information stored in
network files.

2.6.2	Process Control

Computerized process control, if required for the ETV-MF Program, will be
performed by the technology provider or user industry. Proper operation of all
monitoring devices will be confirmed according to a regular maintenance and
calibration schedule specified by the manufacturer. Whenever possible, industry
recognized standards will be used for the calibration of computerized process
control devices.

2.6.3	Data Collection

Computers and electronic readouts are used to collect data where computers are
interfaced directly to instruments. Vendor-supplied software is employed for this
type of data collection. Software allows for multiple point calibrations and
automatic readout of sample value. Calibration data, raw data, and final results are
assessed by laboratory management to ensure proper equipment operation and data
accuracy.

2.6.4	ETV Website

The ETV Program website is established and maintained by EPA to post ETV
information and documents for availability to the public. The following
documents will be submitted by the CTC ETV-MF Program Manager to the EPA
ETV Webmaster:

•	Stakeholder lists and updates

•	Meeting announcements and summaries

•	Generic Verification Protocols

•	Test/QA Plans

•	ETV Verification Statements

•	ETV Verification Reports

•	News of upcoming meetings/speeches/announcements

•	Commerce Business Daily Announcements

20


-------
Revision 1 03/26/01

2.7 Planning

At the beginning of the ETV-MF Program and after significant changes in scope, budget,
or time constraints, the CTC ETV-MF Program Manager will ensure completion of the
Quality Planning Checklist (Form 3027 - Appendix B). This checklist is used to ensure
that the important aspects of the program have been considered, and it identifies and
references the applicable management system procedure. These include Design of
Equipment/Facilities, appropriate and sufficient personnel, use of proper methods,
processes, materials, and equipment, use of appropriate inspection/assessment/validation
criteria, health and safety, waste disposal, training, cost, and deliverable schedules. The
CTC ETV-MF Program Manager is also responsible for identifying any legal or regulatory
requirements that apply to execution of program activities (i.e., air permits or monitoring).

2.7.1	Planning Meetings

The CTC ETV-MF Program Manager will regularly schedule a project review
meeting to include all applicable project personnel. The EPA ETV Center
Manager will be invited to attend this meeting and will receive a project report.
The meetings will typically address the following:

•	Project schedule

•	Project performance

•	Product delivery

•	Personnel requirements

•	Identification and justification of the specific Metal Finishing technologies to
be evaluated and the performance criteria that will be examined

•	Identification of the technical and quality goals in order to appropriately
evaluate the technology

•	Translation of the technical and quality goals into a test plan that will
adequately validate the technology

•	Development of a test plan that fits within the cost and schedule constraints

•	Design of an experiment, with acceptance criteria included, that will
adequately verify the technology

CTC has established a Stakeholder Advisory Group composed of representatives
from buyers and users of Metal Finishing equipment, technology developers and
vendors, members of technical associations, and Federal, state and local
government. This group, assembled with the guidance and approval of EPA, will
assist in identifying and selecting technologies for verification.

2.7.2	Generic Verification Protocols

After all of the technologies within a focus area have been tested, a Generic
Verification Protocol will be developed for that focus area. This protocol will
contain a wide range of test parameters that apply to the focus area. Included in
the protocol will be all testing required to gather sufficient data for verification of

21


-------
Revision 1 03/26/01

technologies within a focus area. The issues that may be addressed in the Generic
Verification Protocol include:

•	General description of the ETV-MF Center

•	Responsibilities of all involved parties

•	Experimental design

•	Equipment capabilities and description

•	Description and use of test sites

•	Description and use of laboratory test sites

•	QA/QC

•	Data handling

•	Requirements for other documents

•	Environmental, health, and safety regulatory requirements

•	References

The QA/QC section of the Generic Verification Protocol will describe activities
that verify the quality and consistency of the work. Preparation and use of
appropriate QA procedures (such as field duplicates, blanks, spiked samples, and
performance evaluation samples) will be described. Frequency of calibrations, the
QC checks and the rationale for them will be described. Procedures for reporting
QC data and results will be identified. Who or what organization is responsible for
each QA activity, and who has the responsibility for identifying and taking
corrective action, will be identified. Variations between tests will be described in
the Test/QA Plan.

The protocol will cite documents that explain, extend and/or enhance the protocol,
such as related procedures, published literature, or methods manuals. The specific
location of any reference, not readily available from a full citation in the reference
section (as in a facility-specific standard operating procedure) will be given or
attached to the protocol.

2.7.3 Test/QA Plans

The requirements for verification testing are included in the Test/QA Plan prepared
for each Metal Finishing technology to be evaluated. As stated previously in this
QMP, Test/QA Plans will be prepared for each technology to be verified, and will
contain the following elements where applicable:

•	Title and approval sheet

•	Table of contents, distribution list

•	Test description, test objectives

•	Identification of the critical measurements, data quality objectives, and data
quality indicator goals

•	Test (including QA) organization and responsibilities

•	Documentation and records

22


-------
Revision 1 03/26/01

•	Experimental design

•	Sampling procedures

•	Sample handling and custody

•	Analytical procedures (including review of SOPs for non-standard methods)

•	Test-specific procedures for assessing data quality indicators

•	Instrument calibration and frequency

•	Data acquisition and data management procedures

•	Technical Systems Audits

•	Corrective action procedures (response actions to audit findings)

•	Data reduction, data review, data validation, and data reporting

•	ETV-MF Operation Planning Checklist (required by the Job Training Analysis
Plan [Ref. 2])

•	Environmental, health, and safety requirements

•	Test/QA Plan Modification Procedure

The draft Test/QA Plans are prepared by the ETV-MF Project Manager and CTC
engineers, and reviewed by the CTC QA Manager, CTC ETV-MF Program
Manager, and CTC Leadership team member for accuracy according to CTC's ISO
0905 Procedure, Deliverable Documents.

2.7.4	Data Quality Objectives (DQO)

CTC uses the DQO process to establish test objectives and test measurement
quality criteria. The DQO process will involve systematic and strategic planning,
based on scientific methods, to identify and define the type, quality, and quantity
of data needed to establish test objectives. The key elements of the DQO process
are:

•	Concisely defining the problem

•	Identifying the decision to be made

•	Identifying the key inputs to that decision

•	Defining the boundaries of the study

•	Developing the decision rule

•	Specifying tolerable limits on potential decision errors

•	Selecting the most resource-efficient data collection design

Based upon the qualitative and quantitative statements derived from this process,
test objectives and test measurement quality criteria will be established.

2.7.5	Standard Operating Procedures (SOPs)

If another level of detail is required for describing test activities (e.g., operation of
instrumentation, equipment or a system), an SOP will be written to support the
Test/QA Plan as Tier 4 documentation. The following topics, from EPA's
Guidance for Development of Standard Operating Procedures (SOPs) for Quality

23


-------
Revision 1 03/26/01

Related Documents, (QA/G-6^, may be included (or a reference provided) in the
SOP where applicable:

•	Scope and applicability

•	Summary of procedures

•	Definitions (acronyms, abbreviations, etc.)

•	Personnel qualifications

•	Health and safety warnings (warning of activities that could result in possible
personal injury)

•	Cautions (warning of activities that could damage equipment, degrade samples,
or invalidate results)

•	Apparatus and materials

•	Calibration

•	Sample collection, sample labeling, and sample tracking

•	Handling and preservation of samples

•	Interferences

•	Sample preparation and analysis

•	Data acquisition, calculations, and data reduction

•	Requirements for computer hardware and software used in data reduction and
reporting

•	Data management and records management

CTC's existing procedures and work instruction documents will be used to meet
the requirements of SOPs as they contain the information listed above.

2.8	Implementation of Work Processes

Work will be performed according to approved planning and technical documents. The
planning for the implementation of technology performance verifications is contained in
Part B of this QMP, sections 3.1 and 3.3. All technology verification work will occur
according to Test/QA Plans, and SOPs developed and agreed upon by EPA, CTC, user
industry, and the technology provider. The approved Test/QA Plans and SOPs will be
present on test site, and the work will be implemented in accordance with them. During
the work phase, modifications to plans and procedures will be documented, and the
modifications will be incorporated into the final protocols, Test/QA Plans, and SOPs.
CTC will implement all work processes in accordance with this QMP.

2.9	Assessment and Response

Regular assessments of both the management and technical aspects of the project are
planned and scheduled. Assessments are conducted both internally through self-
assessment and externally by KEMA Registered Quality, Inc. Results are documented in
audit reports and reviewed by appropriate management.

Project Management tracks program status through any of CTC's assessment tools such
as:

24


-------
Revision 1 03/26/01

•	Cost Tracking System

•	Matrix Management System

•	Software Tools

2.9.1 Numbers and Types of Assessments

Assessments will be planned, scheduled, and conducted to measure the
effectiveness of the implemented management and quality systems. The following
assessments will be performed in accordance with the frequency noted in Table 2.

Assessment

Assessor(s)

Basis for
Assessment

Reason for
Assessment

Frequency

Management

Systems

Review

CTC QA
Manager

ISO 9001, ISO
14001, & ETV-
MF QMP

Assess quality of

management

practices

Annual

Technical

Systems

Audits

Self:

CTC QA Mgr

Independent:
EPA Quality
Assurance Mgr

Test/QA Plans

Assess quality of
verification test

Self- the first test in
a Focus Area and
twice thereafter if
the technology is
significantly
different

Independent -
Twice during
project

Performance

Evaluation

Audits

Self:

CTC ETV
Program Mgr

Independent:
EPA Quality
Assurance Mgr

Contract
Requirements,
ETV-MF QMP,
Test/QA Plans,
& Verification
Reports

Assess and
measure
performance in
meeting project
goals and
objectives

Self - Monthly

Independent - As
applicable

Audits of
Data Quality

Self:

CTC QA Mgr

Independent:
EPA Quality
Assurance Mgr

Raw and
Summary Data

Assess data
calculations and
reporting

Self - Verily all
data and perform
statistical
assessments for at
least 10 percent of
data sets

Independent -
As applicable

Table 2. Assessments

Any corrective actions required, based on these assessment activities noted above,
will be handled through CTCs ISO Procedure 1401 as outlined in section 2.9.7.
The CTC ETV-MF Program Manager will be responsible for sending the requests
for corrective actions to program subcontractors.

25


-------
Revision 1 03/26/01

2.9.2 Management Assessments

Management system assessments will consist of Performance Evaluation Audits of
the ETV-MF Program, Management Systems Audits of the CTC system-wide ISO
management systems, and a specific Management System Review for this QMP
and its interaction with the overall ISO 9001 and 14001 management systems.

Performance Evaluation Audits - The CTC ETV-MF Program Manager will
regularly conduct quality assessments to ensure that the quality objectives of the
program are being met. These internal reviews will be documented and reported to
CTC management in a monthly Project Report that documents the cost, schedule,
and technical performance of the project.

Management Systems Audits - Internal audits of CTC s ISO 9001 and ISO 14001
management system (under which this ETV-MF QMP operates) are conducted
periodically to assess compliance with the written and approved Quality Manual
and Procedures. CTC has a highly qualified staff to guide and direct this effort.
The staff includes assessors certified by the RAB, quality engineers certified by the
ASQ, as well as quality professionals with expertise in Quality Improvement
Programs, ISO 9000 Systems Development and Auditing, and Industrial Problem
Solving.

Audits of the QMS and the Environmental Management System are conducted
according to CTC's established ISO Procedure 1701. Results of these internal
audits are reported to CTC's Quality Committee, chaired by the Management
Systems Representative, who reports to a Senior Vice President within the
organization. The Quality Committee is responsible for guidance, direction, and
oversight of the management system, as well as providing the necessary resources
for successful realignment of the QMS and EMS system to meet the ISO
requirements.

An audit of the management system is also conducted by an independent
organization (KEMA Registered Quality, Inc.) every six months as part of the
surveillance requirement for ISO 9001 and ISO 14001 certifications. These audits
are conducted to ensure that CTC s policies, practices, and procedures are
sufficient to provide results of the type and quality required. Results of these audit
activities (internal and third party) are reported to CTC s Executive Management.

Management System Reviews - CTC QA Manager, using information from the
Performance Evaluation Audits and the Management Systems Audits, will perform
an audit of the ETV-MF Program by auditing the project at the Largo, Florida,
office of CTC. Audit procedures will follow CTCs ISO Procedure 1701. All
elements that impact process and product quality and those elements that have the
potential for significant EHS impact will be internally audited. Results of the audit
will be reported to the CTC Management Systems Representative, the Quality

26


-------
Revision 1 03/26/01

Committee, the CTC ETV-MF Program Manager, and the EPA ETV Center
Manager.

2.9.3	Technical Assessments

A Test/QA Plan is prepared for each Metal Finishing technology evaluated. The
Test/QA Plan contains the specific requirements for each technology evaluated.
Project personnel are responsible for implementing the Test/QA Plans. CTC or
subcontractor personnel are present during testing and data collection to assess
project technical quality. These personnel document adherence to specified
procedures and have the authority to suspend work when unsafe or unacceptable
quality conditions arise.

Technical System Audits (TSAs) are planned at the frequency specified in Table
2. These audits will be conducted during system operation & testing by the CTC
QA Manager to ensure that testing and data collection are performed according to
the Test/QA Plan requirements. Results of these audits will be reported to the EPA
ETV Center Manager, CTC ETV-MF Program Manager, and ETV-MF Project
Manager. Independent TSAs will also be conducted by EPA.

In addition to these technical audits, all Verification Reports will be reviewed by
the CTC ETV-MF Program Manager, CTC QA Manager, technology providers,
and EPA to ensure that conclusions are technically sound.

2.9.4	Data Quality Assessments

Reviews will include specific data quality indicators and procedures specified in
individual Test/QA Plans prepared for each technology assessed. The CTC QA
Manager conducts validations for all of the testing performed to determine whether
the quality objectives specified in the Test/QA Plan are met. These validations
include an evaluation to determine whether the data collected fulfills the test
objectives and design of experiment established for the technology.

Data quality (statistical) assessments, which require review of a minimum of 10
percent of the data sets (all data for the same parameter analyzed by the same test
method), will be conducted according to the procedures specified in the Test/QA
Plan prepared for specific technology using EPA's Guidance for Data Quality
Assessment, EPA/600/R-96/084 (QA/G-9J as a guideline. These reviews will be
selected from data sets where complex methods are used, non-standard methods
are used, or where method deviations are required. These assessments are the
responsibility of the CTC QA Manager. Results of these reviews are recorded and
are reported to the EPA ETV Center Manager, CTC ETV-MF Program Manager,
and ETV-MF Project Manager.

27


-------
Revision 1 03/26/01

2.9.5	Assessment Procedures

Assessments will be performed according to written and approved procedures,
based on careful planning of the scope of the assessment and the information
needed. All assessments are based on interviews, on the physical examination of
objective evidence, and on the examination of the documentation of past
performance. Results are documented in audit reports and reviewed by the
appropriate parties shown in Table 1.

2.9.6	Assessment Personnel Qualifications, Responsibility, and Authority

Personnel conducting assessments will have the appropriate technical and/or
management skills necessary to perform the assigned assessment. CTC will
determine and document the tier of competence, experience, and training of audit
personnel during hiring and periodic performance reviews. Qualified audit
personnel will have access to the appropriate management personnel and
documents required in the performance of their audit duties. Audit personnel will
have the responsibility and authority to:

•	Identify and document problems that affect quality of verification results

•	Propose recommendations for resolving problems that affect quality of
verification processes or results

•	Independently confirm implementation and effectiveness of solutions

If assessment personnel identify a severe problem affecting verification quality or
endangering the health and safety of personnel, they have the responsibility to
bring it to the immediate attention of the ETV-MF Project Manager and the CTC
ETV-MF Program Manager.

2.9.7	Response

If the recommendations and conclusions from the findings of an assessment are
adverse, corrections will be implemented in accordance with CTC s Preventive and
Corrective Action Procedure 1401, ISO 9001 and ISO 14001 management system.
A Preventive/Corrective Action Request is submitted to a QA Representative. The
QA Representative assigns the responsibility for corrective action to the ETV-MF
Project Manager. Corrective actions will be identified within 10 days of receiving
the assessment or audit report. The QA Representative verifies effectiveness of the
corrective action, closes out the Preventive/Corrective Action Request, and
provides the status to the Quality Committee as input to management reviews as
needed.

2.10 Quality Improvement

A quality improvement process will be established and implemented to continuously
develop and improve the ETV-MF Program. As Test/QA Plans are developed, testing is

28


-------
Revision 1 03/26/01

performed, and reports are generated, CTC will provide feedback on the QMP's
competency based on input received and lessons learned.

2.10.1	Detecting and Correcting Quality System Problems

Procedures are established and implemented to address problems identified within
the quality system. CTC's ETV-MF Program Manager and QA Manager will
remain alert for problems in the following areas:

•	Adequacy of the ETV-MF Quality Management Plan

•	Consistency of the quality system

•	Implementation of the quality system

•	Correction of quality system procedures

•	Quality of data

•	Completeness of documented information

•	Quality of planning documents

•	Implementation of the work process

CTC Leadership Team meetings and Management Team meetings are held
regularly to evaluate and gauge quality management systems. Important project
requirements are covered and appropriate briefings are held to communicate
findings.

Any deficiencies are documented using a Preventive/Corrective Action Request.
The QA Representative is responsible for conducting an investigation of the root
cause of the deficiency and documenting the findings.

2.10.2	Cause and Effect Relationship

CTC ISO Procedure 1401 requires root cause analysis for preventive and
corrective actions. When a significant problem is discovered, the CTC QA
Manager determines and documents the relationship between cause and effect that
is evidenced by the problem and, when possible, determines and documents root
cause. The CTC QA Manager communicates this information to the CTC ETV-
MF Program Manager so corrective action can be authorized and implemented.

A significant problem is identified as any problem requiring:

•	A change in testing protocol, or

•	A management system change, or

•	A quality system change within the ETV-MF Program

CTC's QA Manager, in accordance with CTC s ISO 9001 and ISO 14001
management system, will continually review and assess the ETV-MF Program for
conformance with internal management system documents.

29


-------
Revision 1 03/26/01

2.10.3	Root Cause

The CTC ETV-MF Program Manager ensures that root causes are determined,
whenever possible, before permanent preventive measures are planned and/or
implemented.

2.10.4	Quality Improvement Action

The CTC ETV-MF Program Manager, the CTC QA Manager, and process
personnel devise and implement the appropriate corrective action. Effectiveness of
the corrective action is verified and documented by the CTC QA Manager.
Procedures and instructions are revised as necessary to reflect the implemented
changes.

30


-------
Revision 1 03/26/01

PART B: COLLECTION AND EVALUATION OF ENVIRONMENTAL DATA

Part B of this ETV-MF Quality Management Plan contains the specifications and guidelines
applicable to test-specific environmental activities involving the generation, collection, analysis,
evaluation, and reporting of test data.

31


-------
Revision 1 03/26/01

3.0 COLLECTION AND EVALUATION OF ENVIRONMENTAL DATA (Part B)

This section contains the specifications and guidelines applicable to test-specific environmental
activities involving the generation, collection, analysis, evaluation, and reporting of test data.
Evaluation and verification of environmental technologies for Metal Finishing will be carried out
at commercial Metal Finishing shops. CTC and subcontractor personnel will be responsible for
the performance or supervision of this testing. All activities will be carried out according to the
Test/QA Plan approved for the technology to be verified.

Important aspects of the collection and evaluation of environmental data are:

•	Planning and Scoping - sufficient data collection to ensure objectives of the testing are
adequately addressed

•	Design of the Technology Verification Tests - identification of the important experimental
parameters to be collected and development of a design of experiment

•	Implementation of Planned Operations - execution of the designed experiments and planned
operations

•	Assessment and Response - assessment of activities and correction of non-conformances

•	Assessment and Verification of Data Usability - assessment of data and documentation of
reporting limitations

3.1 Planning and Scoping

The work of the ETV-MF Program is to verify the performance of commercial-ready
technologies. CTC implements a systematic process to plan the individual tests. The
following actions will be performed:

•	Joint identification of test objectives, critical measurements, and non-critical
measurements by technology developers, industry consultants, and CTC

•	Review and/or development of analytical methods, sample collection techniques,
holding times, and QA objectives by the analytical laboratory, industry consultant, and

CTC

•	Preparation of Test/QA Plans for the acquisition of data to verify performance of the
vendors' technologies

•	Review of Test/QA Plans by EPA, technology provider, test site, CTC QA Manager,
and CTC ETV-MF Program Manager

•	Completion of an ETV-MF Operation Planning Checklist prior to testing

3.1.1 Planning Personnel

CTC will coordinate test planning with the participating organizations, including
EPA, Stakeholders, technology providers, and any testing organizations or
laboratories participating in the test. CTC will identify the planning roles of the
participants and conduct planning activities by shared communication via
teleconferencing, video conferencing, and/or face-to-face meetings, as appropriate.

32


-------
Revision 1 03/26/01

3.1.2	Purpose, Scope, and Objectives

The purpose of this testing is to verify the performance of commercial-ready Metal
Finishing technologies and to establish an effective method for future testing of
similar technologies. These objectives are met through the preparation of Test/QA
Plans and Generic Verification Protocols.

3.1.3	Collected Data and Design of Experiment

During the test planning phase, the method of collecting process data,
environmental data, laboratory data, and QA data are identified. Testing
organizations, test personnel, skill levels, methods, procedures, and equipment are
also identified.

3.1.4	Documentation and Reporting

Records generated during the tests are listed in section 2.5. Records consist of
both paper and electronic data.

3.1.5	Assessments

Assessments and frequencies are identified in section 2.9.

3.1.6	Constraints and Suspension of Work

CTC is working under the constraints of time and resources as identified by EPA.
When constraints are determined by CTC to affect quality, resolution of the
problem will proceed as described in EPA's ETV QMP, section 1.5.
Circumstances under which work can be suspended are discussed in section 2.1.2
of this QMP.

3.1.7	Waste Minimization and Disposal

CTC has implemented ISO 14001 into its EHS operations. As such, EHS is
responsible for compliance to Federal, state, and local regulations, including
Superfund Amendments and Reauthorization Act (SARA), Occupational Safety
and Health Administration (OSHA), Resource Conservation and Recovery Act
(RCRA), and emission permitting. EHS conducts employee training in Hazardous
Communication, Safe Work Practices, and Worker-Right-to-Know. EHS
personnel will work with the CTC ETV-MF Program Manager to ensure applicable
environmental, health and safety issues are identified and resolved prior to
verification testing. The QMP requires the CTC ETV-MF Program Manager to
identify any legal requirements and environmental aspects and impacts of project
operations during the project planning phase. Environmental issues are also
addressed on the Operation Planning Checklist during the job training analysis

33


-------
Revision 1 03/26/01

[Ref. 2] conducted prior to each test. Regulatory requirements will be specified in
each Test/QA Plan.

3.2 Design of Technology Verification Tests

Test/QA Plans are used to convey the objectives and requirements of the verification
testing to be conducted for each technology.

3.2.1	Test/QA Plans

Technology verification design and data collection requirements, for a specific
technology to be verified, are included in the Test/QA Plan. The Test/QA Plan is
prepared with input from tie CTC ETV-MF Program Manager, CTC QA Manager,
technical staff, and EHS, as well as user industry and technology providers. With
respect to design of technology verification tests, the Test/QA Plan will address the
requirements identified in section 2.7.3, where applicable.

The Test/QA Plan is reviewed by appropriate parties listed in Table 1 to ensure it
is technically sound and that the analytical parameters and collection of data are
sufficient to meet the objectives of the testing.

The Test/QA Plan is reviewed by all program participants to ensure a clear
understanding of the work to be performed. Comments and technical points are
incorporated into the final Test/QA Plan prior to the initiation of any work.

3.2.2	Intended Use of the Data

The data collected may be used not only to verify the performance of an
environmentally friendly technology, but also to verify pollution prevention
potential of the technology. The data may also be used to identify utility and
operating costs and compliance to regulatory standards. Therefore, the data
collected must be of sufficient quality to allow these types of analyses with
reasonable certainty.

3.2.3	Data Quality Indicators

The data quality indicators required for each verification test will be identified in
the technology-specific Test/QA Plan. The values for data quality indicators such
as sample bottle type and size, sample preservative, holding time, quantity,
identification number, collection location, date and time, and analysis method will
be identified on a Chain of Custody (COC) form. Information to be delivered to
CTC by the laboratory performing the work for each sample include: identification
number; results; reporting units; analysis date, time, and method detection limit
(MDL); method quantification limit (MQL); precision; accuracy; blanks;
duplicates; spikes; and percent recovery as applicable. Values greater than the
MDL but less than the MQL will be reported as estimated.

34


-------
Revision 1 03/26/01

3.2.4	Performance Characteristics

All critical and non-critical parameters will be listed in the Test/QA Plan, along
with the required method, performance characteristics (accuracy, precision, etc.),
and calibration requirements. Parameters will be analyzed using approved
methods such as American Society for Testing Material (ASTM), EPA, Military
Specifications (MIL-SPEC), or Society of Automotive Engineers (SAE) whenever
applicable. Field records and laboratory records for calibration will include
supplier, lot number, expiration date, and date of usage of standards. Program
personnel are responsible for assessing the equipment prior to operation or
measurement during verification testing. Laboratory personnel are responsible for
assessing the proper operation of the analytical instrumentation prior to testing.
This validation will be done using nationally recognized performance standards
whenever possible.

In some instances, no commonly accepted method may exist due to the lack of a
suitable calibration material or the proprietary or innovative nature of the
technology. The performance of these non-standard methods will be evaluated
based on the manufacturer's specifications or on in-house developed and
documented protocols. Deviation from approved standard methods or non-
standard method SOPs must be approved by the CTC QA Manager or CTC ETV-
MF Program Manager. Requirements and results of spikes, duplicates, blanks, and
reference standards will be fully documented.

3.2.5	Specification

Trained and qualified personnel will carry out all sampling and data collection
activities. Sampling methods and requirements will be listed in the Test/QA Plan.
These methods will be taken from the appropriate source such as EPA air and
wastewater regulations promulgated in the Code of Federal Regulations, or the
National Institute of Occupational Safety and Health (NIOSH). Testing occurs
according to the methods and procedures specified in the Test/QA Plan. Any
modification of or deviation from an approved method is documented on process
Data Collection Sheets and test plan modification form provided in each Test/QA
Plan.

3.3 Implementation of Planned Operations

Technology performance verifications will be implemented according to Test/QA Plans,
which are prepared during the planning process. Test personnel will have access to the
approved planning documents, approved changes to these documents, and all referenced
documents. All implementation activities are traceable to the planning documents and to
test personnel.

35


-------
Revision 1 03/26/01

Only qualified services and items will be used in the performance of verification
operations. Acceptance will be noted on the items themselves and/or in documents
traceable to the items.

If additional level of detail is required for describing a test activity (e.g., operation of
instrumentation, equipment, or a system), an SOP is generated. This SOP provides the
operating personnel with all necessary setup parameters such as pressure, temperature,
flows, etc., of process baths, ovens, equipment, etc. All testing and data collection
activities to be performed both during and after processing are listed in the SOP or similar
document. A checklist is used to document the settings, parts, and equipment actually
used. Process Data Collection Sheets are used to document conditions during processing.
These are prepared in response to the critical and non-critical parameters listed in the
Test/QA Plan for the specific verification conducted. Any deviations from the methods
and parameters specified in the Test/QA Plan and SOP are recorded and reported to the
CTC ETV-MF Program Manager.

3.3.1	Technology Provider Equipment

The technology provider is responsible for calibrating equipment, associated with
the technology, used to record process conditions. Gauges and monitoring devices
used for process monitoring and data collection will be maintained and calibrated
at the manufacturer's specified interval. Nationally recognized performance
standards will be used whenever possible. Records of the calibration data
generated during calibration will be collected as part of the verification test data.

3.3.2	Analytical Laboratories

Whenever possible, analytical laboratories used by the ETV-MF Program will be
accredited by the National Environmental Laboratory Accreditation Program
(NELAP) or the American Association for Laboratory Accreditation (A2LA).
NELAP accredits laboratories to standards developed by the National
Environmental Laboratory Accreditation Conference (NELAC), a cooperative
association of state and Federal agencies (including EPA) formed to establish and
promote mutually acceptable performance standards for the operation of
environmental laboratories. The basis of the NELAC system is ISO Guide 17025,
"General Requirements for the Competence of Calibration and Testing
Laboratories." The A2LA is a nonprofit, professional membership society that
administers a broad spectrum, nationwide laboratory accreditation system, which,
like NELAC, uses ISO Guide 17025.

Proper operation of all analytical equipment will be verified by the laboratory
according to the laboratory QMP prior to testing any samples. This will be done
using samples of known value or, when available, purchased standard reference
material. A standard is analyzed with each batch of samples to verify proper
equipment function. Duplicates and spikes are also analyzed and precision and
accuracy data generated. Re-calibration, maintenance, or repair is indicated when

36


-------
Revision 1 03/26/01

unacceptable recovery of data quality indicators occurs. For example, balances
will be verified using traceable weights, and ovens will be checked using
thermometers calibrated against a National Institute of Standards and Technology
(NIST)-traceable thermometer.

Proper operation of equipment using a computer software program to calibrate,
collect data, and calculate results is verified by the laboratory according to their
QMP prior to analysis of samples. A typical scenario uses a minimum of three
standards plus a blank to establish a calibration curve. This calibration is verified
using a "check" standard from a different supply source. If the check standard is
acceptable, the curve is verified by checking the highest standard that must meet
the method requirements (typically a 5 percent to 10 percent error). To completely
verify the performance of the method and sample preparation steps, a method
blank, blank spike, and, when available, a known sample/standard from a third
source are analyzed. These checks and spikes must meet method requirements for
recovery before analysis of samples can begin. The data quality indicators
(precision, accuracy, and recovery) must meet method requirements for the data to
be acceptable. Finally, the calibration blank, highest standard, and check standard
are again analyzed to verify that the calibration is still accurate and no drift has
occurred. Other instrument checks include spectral interference samples, internal
standards, or surrogates. Only after the above requirements are met is data
reported; otherwise, the instrument is re-calibrated and re-analysis is performed.

3.3.3	Sample Handling Procedures

Samples are collected at the location and frequency specified by the design of
experiment detailed in the Test/QA Plan. Samples are preserved at the time of, or
as soon as possible after, collection. Samples are clearly labeled with the date,
time, sample identification (ID), location, and process at the time of collection.
Samples transported by common carrier will use security tape to ensure sample
integrity upon delivery to the analytical laboratory. Upon arrival at the analytical
laboratory, samples are handled in accordance with the laboratory QMP.

Equipment usage and calibration logs are used to indicate the standards, samples,
and reagents used during analysis of samples. A log is used to document any
repair or maintenance performed on the equipment.

3.3.4	Laboratory Data Handling, Validation, Reporting, and Archiving

Data collected in the Metal Finishing shops will be placed on Process Data
Collection Sheets. This information is reviewed by the CTC QA Manager and
forwarded to the CTC ETV-MF Program Manager.

Data collected during laboratory testing is processed in accordance with the
laboratory QMP. This includes, at a minimum, the parameter and method used for
testing, date, analyst initials, identification number of samples and standards,

37


-------
Revision 1 03/26/01

equipment ID number, calibration number, dilution, instrument reading, and final
result. Results for the analysis of duplicate, standard, blank, and spike samples are
transferred onto a QA/QC Summary.

A member of laboratory management will review all of the data on these sheets to
ensure that the proper samples, method, standards, reagents, and equipment have
been used and that all program requirements are met. Calculations, dilution, and
data quality indicators are verified as well as percent recovery of standard and
spike samples. The results from the analysis of duplicate, standard, and spike
samples are compared to the method requirements and values stated in the
Test/QA Plan.

The reviewer will initial and date all of the sheets after review to show that all
information has been verified. The data is forwarded for entry onto the laboratory
report form.

The CTC ETV-MF Program Manager may request interim results for preliminary
evaluation of the data. In these instances, the data is clearly marked to show
results are preliminary.

Restrictions on the use of data are indicated in the comment section of the
laboratory report. This may occur when matrix interference results in poor
recovery of spike standards and the sample is diluted to remove the interference.

3.3.5	Control of SOPs, Test/QA Plans, and Methods

SOPs and Test/QA Plans are clearly marked "DRAFT" to indicate that they are
preliminary, in the review process, and are not to be used to conduct testing. After
the SOP or Test/QA Plan is approved, the document is given a revision number.
Revision 0 is used to indicate the first level of revision. Any change to an
approved document will require another review and approval step. The changed
document will become Revision 1, etc. The date is also imprinted on each page of
the document to indicate the date approved. Project personnel will be required to
use the current revision with the approval date. Laboratory management is
responsible for ensuring that the most recent document is available to laboratory
personnel.

3.3.6	Work Instructions

The operation of equipment is documented in Work Instructions or SOPs.
Engineering staff members, responsible for the equipment, write the Work
Instruction. The engineer prepares the instruction based on information provided
from equipment manuals, vendor training, EHS staff, and experience of the
personnel responsible for the operation of the equipment.

38


-------
Revision 1 03/26/01

The ETV-MF Program primarily specifies EPA methods and Standard Methods
for the Examination of Water and Wastewater as sources for environmental sample
analysis. ASTM, MIL-SPEC, SAE, or industry specific protocols are followed for
finish quality testing.

3.3.7	Verification of Equipment Operation Prior to Testing

Prior to verification testing, equipment and instruments will be checked to ensure
proper operation and function. The requirements will be specified in the Test/QA
Plan.

3.3.8	Procedures for Data Validation

Data reduction will be performed according to the requirements and calculations
listed in the technology-specific Test/QA Plans. An assessment and statistical
analysis of the data will be conducted to determine if the data meets the Design of
Experiment and objectives specified in the Test/QA Plan. Data assessment will be
conducted according to the EPA QA/G-9 guidance document.

A Quality Assurance Summary will be written to document all qualitative and
quantitative quality measurements such as precision, accuracy, and problems
encountered during processing and testing. The QA Summary will be included in
the Verification Report.

Instructions for data management (i.e., transmittal, storage, validation, assessment,
processing, and retrieval) are discussed in other sections of this document.

3.4 Assessment and Response

The project personnel are responsible for evaluating the performance of testing and the
collection of data during verification testing. The collected data is assessed by CTC after
the testing is completed. CTC engineers check data collected from both in-line monitoring
and laboratory testing to confirm adequate calibration, correct method usage, data
transcription, and that all data quality indicators are met. The evaluation is based upon the
requirements of the Test/QA Plan and the defined data quality objectives.

Sample analysis involves the use of blanks, duplicates, spiked samples, or standards of
known value. Transcription errors are confirmed with the analyst prior to any correction
of data. A request for re-analysis is completed and issued to the analyst when data quality
indicators are not met. Re-analysis is performed when an inadequate calibration or an
inappropriate method is used. Laboratory management reviews final data packages to
make sure all results conform to chemical principles, mass balances, or charge balances.
Statistical methods will be used to evaluate the collected data. This may involve using
control charts and/or identifying outliers in the data quality indicators. Deviations are
reported to the CTC ETV-MF Program Manager.

39


-------
Revision 1 03/26/01

3.4.1	Maintenance and Calibration Frequency

Preventive maintenance and calibration of CTC equipment and instruments are
performed on a regular basis. Maintenance and calibration schedules for
equipment are maintained using calibration-scheduling software. Each piece of
equipment requiring manufacturer calibration is labeled with a unique
identification number and logged into the software. The software provides a
complete history and traceability for calibration performed. Use, calibration, and
maintenance activities are traced for each piece of equipment. A monthly schedule
is issued for maintenance and calibration activities to be performed on equipment.

A Preventive/Corrective Action Request is completed and forwarded to the QA
Representative when a piece of equipment is found to be out of calibration, when
improper methods or procedures are used, or equipment, supplies, and instruments
do not meet quality requirements. The QA Representative is responsible for
determining the root cause and implementing corrective action. In the event that
equipment or instruments have been out of calibration during testing and results
have been reported, the Calibration Technician issues an Out-of-Calibration
Assessment. Supervisory personnel are responsible for determining the impact on
the quality of the reported data using the equipment. The Out-of-Calibration
Assessment is forwarded to the QA Representative, who is responsible for follow-
up and corrective action. The findings are reported to management. Technology
providers and the user industries will be required to have a similar maintenance
and calibration system for equipment used in the ETV-MF Program.

3.4.2	Analysis of Known Samples/Standards

Proper operation and calibration of equipment are verified prior to testing as stated
in the specific Test/QA Plan or SOP. Nationally recognized performance
standards will be used, when available, for this type of calibration. Calibration of
analytical laboratory equipment shall be in accordance with the laboratory QMP.

3.4.3	Internal and External Audits

As part of CTCs management system, the facilities in Johnstown, Pennsylvania,
undergo quarterly internal (self) audits as described in sections 2.2 and 2.9 of this
QMP. As part of CTC's ISO system registration process, the facilities in
Johnstown, Pennsylvania, are audited every six months by an RAB-accredited
registrar, KEMA Registered Quality, Inc.

CTC's Quality Committee, chaired by a Management Systems Representative,
reports to a Senior Vice President within the organization. This committee is
responsible for guidance, direction, and oversight of the management system, and
for providing the necessary resources for successful realignment of the QMS and
EMS system in meeting the ISO requirements. A report of the audit results is
posted electronically and provided to management. A Preventive/Corrective

40


-------
Revision 1 03/26/01

Action Request is used to document findings of these audits and to initiate the
corrective action process. The QA Representative is responsible for determining
the root cause, documenting the action to be taken, and ensuring implementation of
the corrective action.

As indicated in section 2.9.2, the CTC QA Manager will conduct an annual audit
of the management system at the Largo office. Data from the ISO audits in
Johnstown along with the project Performance Audits will be used in the audit
planning. Effectiveness of corrective actions related to any area of the program
will be verified as applicable.

Representatives from EPA may also conduct assessments of the program and
perform inspections prior to and during verification testing.

3.4.4 Assessment of Data Usability

After the execution of a SOP (test or equipment operating procedure), an
evaluation is conducted to identify any problems encountered during processing or
collection of data. The ETV-MF team assesses the usability of the data after
laboratory analysis is completed. This evaluation involves the ETV-MF Project
Manager, laboratory management, quality assurance, and CTC engineers.
Evaluation is based upon the requirements specified in the Test/QA Plan and will
include any limitations (such as ranges or confidence intervals) on data parameters.

A statistical analysis may be performed on the data using DataQuest, MiniTab 12,
or Excel software programs. Measurements such as mean, median, standard
deviation, t-test, and f-test will be generated to describe the results and the
confidence tier. EPA may also conduct an assessment of the data.

After the evaluation is conducted, the CTC QA Manager is responsible for
generating a report to document the findings. This report will identify any
problems encountered during testing, calibration, analysis, or data reduction and
provide the corrective action steps required for resolving the issues. The impact on
the test objective and verification results will be provided in the QA section of the
verification report.

3.5 Reporting

Upon completion of verification testing in accordance with the Test/QA Plan, a report will
be prepared for EPA that details the results of the verification testing. The report will
comprehensively and accurately document the results of the verification testing of the
product against the specific criteria detailed in the Test/QA Plan. The draft reports are
prepared by the ETV-MF Project Manager and CTC engineers, and reviewed by the CTC
QA Manager, CTC ETV-MF Program Manager, and Leadership team member for
accuracy according to CTC's ISO 0905 Procedure, Deliverable Documents. Technology
providers will have the opportunity to review the verification data after EPA approval.

41


-------
Revision 1 03/26/01

The report and statement will be submitted to EPA. To maximize the ETV-MF Program's
exposure to the Metal Finishing industry, the verification statement will be made available
over the Internet on the ETV website (http: www.epa.gov e/v) as well as through other
sources as approved by EPA. Should the technology provider not agree with the
verification results, the test report will still be submitted to EPA; however, no verification
statement will be issued. EPA and the CTC ETV-MF Program Manager will keep a copy
of all reports on file.

Verification Reports will be prepared for each verification conducted. Verification
Reports will include:

•	Verification Statement

•	ETV Overview

•	Technology and Application

•	Test Methods and Procedures

•	Results and Discussion

•	Quality Assurance Narrative

•	Summary and Conclusions

Reports and presentations of data will be reviewed independently, by others than those
who produced the data or report, to ensure that these items are technically sound and of
sufficient quality to meet the objectives of the task.

42


-------
Revision 1 03/26/01

PART C: DESIGN, CONSTRUCTION AND OPERATION OF
ENVIRONMENTAL TECHNOLOGY

Part C of this ETV-MF Quality Management Plan describes the quality system elements
pertaining to the design, construction, installation, and operation of the treatment system
technology to be verified.

43


-------
Revision 1 03/26/01

4.0 DESIGN, CONSTRUCTION AND OPERATION OF ENVIRONMENTAL
TECHNOLOGY (Part C)

Part C applies only if the technology is to be installed only for the purposes of verification testing
and will be removed after testing. Part C does not apply if installation is occurring under an
agreement (sale or evaluation) between the technology provider and user industry only, with
verification testing to occur after the technology is released for unrestricted use.

The elements must be used in conjunction with Parts A and B to provide an adequate quality
system for the ETV-MF Program. There are large numbers of purportedly low-emission MF
technologies that are commercially ready. These technologies will be reviewed to identify
candidates for this verification program. The program will result in the collection of
environmental, technical (operational), and cost data for dissemination to industry. This section
of the ETV-MF QMP contains the following elements:

•	Planning

•	Design of Systems

•	Construction and Fabrication of Systems and Components

•	Operation of Systems

•	Assessment and Response

•	Verification and Acceptance of Systems

4.1	Planning

Planning activities will be coordinated between CTC, EPA, the user industries in which
the treatment systems will be installed, and the technology providers using the CTC
Quality Planning Checklist (Form 3027). The checklist addresses the key items identified
in section 4.1.1 of the E4 Standard and includes such items as:

•	Delivery, handling, and installation requirements

•	Notification of organizations that have been identified previously to participate in the
project and their role in planning, design, construction/fabrication, operation, and
assessment activities

•	Personnel, equipment, and other resources required

•	Program technical reviews, peer reviews, surveillances, technical and QA audits,
readiness reviews, and other assessment processes

•	Project and QA records required

•	Technical, performance, regulatory and quality standards, criteria, and objectives

Documentation of project and activity planning will consist of meeting minutes, Test/QA
Plans, schedules, and installation drawings.

4.2	Design of System

The design of the technology to be verified is the responsibility of the technology
provider. Designing the process interface with the technology is the responsibility of the

44


-------
Revision 1 03/26/01

user industry. Prior to installation, the designs will be reviewed by the ETV-MF Project
Manager to ensure that, to the best of their knowledge, the technology is designed using
sound engineering/scientific principles and appropriate standards. The review will
address:

•	The ability of the components and system to perform under expected conditions of use

•	Delivery documentation requirements

•	Compliance with regulatory requirements

•	Safety and maintenance requirements

The results of the design review, including installation requirements, will be documented
in a letter to the technology provider and user industry. A readiness review will be
performed and documented by the user industry, technology provider, and ETV-MF
Project Manager prior to operation of the technology.

4.3	Construction/Fabrication of Systems and Components

The construction required during installation of the technology will be performed
according to drawings and specifications identified during the design phase. Periodic
preventive and corrective maintenance of equipment used during installation will be
performed according to the manufacturer's specifications. Measurement and test
equipment requiring calibration will be calibrated according to the frequency and
specifications stated by the manufacturer. Traceability to nationally recognized
performance standards will be maintained when they are used.

Documentation of construction equipment requirements will be specified and maintained
by the user industry and technology provider. Construction personnel qualification
requirements will be specified and maintained by the user industry and technology
provider.

4.4	Operation of Systems

Documented operation of the various technologies within established parameters is
essential to ensure valid performance verification. Key elements necessary to control
operation of systems are described in the following sections.

4.4.1	Management of Consumables

Any materials that could directly affect system operations such as auxiliary
materials, utilities, consumables (e.g., water, compressed air, electric power, and
chemical feedstocks) will be identified in the SOPs and monitored appropriately.

4.4.2	Operational Controls

As stated in section 3.3 of this QMP, Work Instructions or SOPs will be developed
for operation of process equipment. The development process will include the

45


-------
Revision 1 03/26/01

identification of status indicators with tolerance limits in order to ensure that the
system operates within regulatory and safety parameters. Parameters to be
monitored will also include those required to validate that the system is operating
within applicable environmental permit conditions. For high-risk operations, the
SOPs will address special requirements such as emergency shutdown procedures.

4.4.3 Maintenance and Calibration Frequency

Measurement and test equipment requiring calibration will be calibrated according
to the frequency and specifications stated by the manufacturer. Traceability to
nationally recognized performance standards will be maintained when they are
used. Documentation of system requirements will be maintained by the user
industry and technology provider.

4.5	Assessment and Response

CTCs ISO Procedures for project management, internal audits, and corrective actions,
previously identified in section 3.4 of this QMP for environmental data collection, will
also be used for monitoring of process parameters during operation.

4.6	Verification and Acceptance of Systems

The performance of environmental technology will be verified according to the applicable
Test/QA Plan as stated in Part B of this QMP. Start-up testing will be performed as
specified by the technology provider. When acceptance criteria are not met, deficiencies
will be resolved and reassessments conducted as necessary. If minimum operational
criteria are not met, the situation will be handled through section 4.5.

46


-------
Revision 1 03/26/01

5.0 DISTRIBUTION

Alva Edwards Daniels, EPA ETV Center Manager
Lauren Drees, EPA QA Manager
Clinton Twilley, CTC QA Manager

James E. Voytko, CTC Director, Engineering & Manufacturing

Jacqueline Carr, CTC Contract Resources

Donn W. Brown, CTC ETV-MF Program Manager (5)

Chris Start, MMTC

Peter Gallerani, ITI

George Cushnie, CAI Resources

Gus Eskamani, CAMP, Inc.

Analytical Laboratories

47


-------
Revision 1 03/26/01

6.0 REFERENCES

1.	Environmental Technology Verification Program Metal Finishing Pollution Prevention
Technologies Pilot Generic Verification Protocol, Revision 0, May 10, 1999.

2.	Environmental Technology Verification Program Metal Finishing Pollution Prevention
Technologies Pilot Job Training Analysis Plan, Revision 0, May 10, 1999.

48


-------
Revision 1 03/26/01

APPENDIX A

Concurrent Technologies Corporation
Quality Policy Statement,
Environmental, Health, and Safety Policy Statement


-------
Revision 1 03/26/01

CTC Quality Policy Statement

Revised: January 10, 2001

Concurrent Technologies Corporation (CTC) provides management and technology-
based solutions to a wide array of clients representing Federal and state government
organizations as well as the private sector, nationally and internationally. As such, we are
committed to:

•	Providing quality products and services that meet or exceed the requirements of
our internal and external clients, on-time and every time

•	Seeking and achieving continuous improvement in the quality and value added of
our products and services

•	Improving efficiency and reducing waste in all of our business processes

It is the responsibility of each individual in the Company to comply with the provisions
of this Policy.

Daniel R. DeVos

President & Chief Executive Officer

A-l


-------
Revision 1 03/26/01

CTC Environmental, Health, & Safety Policy Statement

Revised: January 10, 2001

Concurrent Technologies Corporation (CTC) is committed to protecting and improving
the environment, and safeguarding the health and safety of its employees, clients,
contractors, and the public. CTC integrates environmental, health, and safety (EHS)
consciousness throughout all facets of its activities. Through a company-wide
commitment to continuous improvement, CTC strives to achieve excellence in EHS
performance and environmental stewardship.

In addition, CTC exerts beneficial EHS influence far beyond its own operations. CTC
demonstrates world-class leadership in the development and deployment of advanced
EHS technologies and management practices throughout Federal and state government
organizations as well as the private sector, nationally and internationally.

As such, we are committed to:

•	Regulatory Compliance: Meeting or exceeding the requirements of applicable
EHS legislation and regulations.

•	Safe Facilities: Building, operating, and maintaining safe facilities in a manner
that uniformly protects the environment and the health and safety of our
employees and the public.

•	Workplace Health and Safety: Providing a hazard-free workplace to prevent
occupational injuries and illnesses.

•	Pollution Prevention: Preventing pollution in our operations and activities.

•	Resource Conservation: Minimizing our energy and natural resource
consumption in fulfilling our mission.

Daniel R. DeVos

President & Chief Executive Officer

A-2


-------
APPENDIX B
ETV-MF Quality Planning Checklist


-------
QUALITY PLANNING CHECKLIST

The purpose of this Quality Planning Checklist is to aid in project planning by providing a structured
and guided approach to the various aspects of the Quality and EHS Management System. The Checklist,
therefore, serves as a guideline and is an integral part of the overall quality management process. The checklist
has been designed to walk PMts through the necessary procedures and supporting documentation necessary to
comply with the QMS/EMS system requirements.

The completed Checklist shall be retained by the Project Manager and appropriately filed with project
documents.

B-l


-------
QUALITY PLANNING CHECKLIST

ETV-MF Program			Donn W. Brown	

(Project)	(PMt)

U.S. Environmental Protection Agency			1/5/99	 	11/20/00

(Client)	(Original Date)	(Revised Date)

QUESTION

YES

NO

N/A

PROCEDURE
REFERENCE

DOCUMENTATION

required during project
execution

COMMENTS

I. Contract Review

A. General













1.

Has proposal and/or contract review been
performed and documented? (i.e.. Is the
statement of work clear, task discrete, and
acceptance criteria clearly defined?)

X





See CTC 0301 or
0302

Notes/documentation that review
was conducted prior to acceptance
(NCEMT review documentation
controlled by Program Manager)

Contracts & Business
Analyst reviewed
supplemental budget
request

2.

Are the financial resources adequate to cover
applicable CTC costs for the level of quality
required?



X



See CTC 0301 or
0302

Notes/documentation that review
was conducted prior to acceptance
(NCEMT financial documentation
controlled by Program Manager)

Additional funding
requirements being
developed for review
by EPA

B. Personnel













3.

Have personnel issues been addressed:
• Does adequate staffing exist with
the appropriate skill set?

X





See CTC 0301 or
0302

Notes/documentation that review
was conducted prior to acceptance

One additional
Engineer hired and
will start 11/27/00

• Is additional training required for any
personnel associated with the project to
supplement or improve the skill set
currently available?

X









Training
requirements
identified in QMP,
test plans, and
performance
appraisals



• Will additional personnel (new hires,
consultants) need to be recruited to
support project requirements?



X





B-2


-------
QUALITY PLANNING CHECKLIST

QUESTION

YES

NO

N/A

PROCEDURE
REFERENCE

DOCUMENTATION

required during project
execution

COMMENTS

C. Equipment/Tooling Capability













4.

Have equipment capacity requirements been
appropriately identified?





X

See CTC 0301 or
0302

Notes/documentation that review
was conducted prior to acceptance



5.

Does the equipment have the capability to
meet applicable precision, repeatability, or
tooling requirements?





X

See CTC 0301 or
0302

Notes/documentation that review
was conducted prior to acceptance



6.

Does a new piece of equipment or tooling
have to be purchased or can existing
equipment or tooling be upgraded or rebuilt?





X

See CTC 0301 or
0302

Notes/documentation that review
was conducted prior to acceptance



7.

Have acceptance criteria been identified and
documented for new equipment or tooling?





X

See CTC 0301 or
0302

Notes/documentation that review
was conducted prior to acceptance



D. Test Capability













8.

Is testing capacity sufficient?





X

See CTC 0406

Notes/documentation that review
was conducted prior to acceptance



9.

Is new measuring and test equipment
required?

X





See CTC 0406

Notes/documentation that review
was conducted prior to acceptance

New equipment will
be reviewed,
purchased or loaned
from within CTC and
listed in test plan

II. Project Planning

A. General













10.

Has a project plan been developed for this
project?

X





See CTC 0406

Approved Project Plan

Approved project
plan in file

11.

Does the project involve software related
products or tasks developed under the
National Security Division?





X

See CTC 2002

Documentation in accordance
with CTC 2000 series procedures



12.

Will equipment, outside services

X





See CTC 0601

PRs (Form 201), Pos, MIRRs and

Materials & services

B-3


-------
QUALITY PLANNING CHECKLIST



(subcontractors), and/or materials be
purchased?

Note: Subcontractor services may include
design services, materials, equipment,
testing, or other processing services.









subcontractor agreements

will be procured
through contracts or
purchasing

13.

Are standardized test methods (i.e., NIST or
ASTM) required?

X





See CTC 0403
and 0406

List of test requirements on work
orders

Methods identified in
test plans

QUESTION

YES

NO

N/A

PROCEDURE
REFERENCE

DOCUMENTATION

required during project
execution

COMMENTS

14.

Are there cost, schedule or technical risks
associated with the perfomance of this work?

X





See CTC 0406

Risk assessment in project plan

Risk assessment in
Project Plan

15.

Have appropriate facility servicing or
installation requirements been identified,
such as equipment being vented outside the
building, etc.?





X

See CTC 0903
See CJC 4472

FSRS work order should be
maintained by Facility Services
Operational Planning Checklist
(form 308)



16.

Are the resources accounted for in the
schedule or project budget to implement new
equipment or tools?

X





See CTC 0406

Breakdown of cost elements in
Project Plan

See program budget
and schedule

17.

Have you prepared a WBS with resource
assignments?

X





See CTC 0406

WBS in Project Plan

See WBS in Project
Plan

18.

Has funding been allocated for project
support and management including:

X





See CTC 0406

Breakdown of cost elements in
Project Plan

See Program Budget

a. Travel

X









b. Briefings

X







c. Computer and network support

X







d. Technical Writers

X







e. Graphics

X







f. Report production

X







g. Long distance phone calls

X







h. Overnight mail services

X







i. Calibration of equipment

X







j. Project review

X







k. ISO compliance

X







1. Audits

X







B-4


-------
QUALITY PLANNING CHECKLIST

19.

Training:















a. Have training requirements
been identified? (EHS training)

X





See CTC 4420

Documentation of identification
of training needs

Will be identified in
each project JTA



b. Has training time been allocated
in the project schedule?

X







Schedule in Project Plan

Training done prior
to the first day of
each test



c. Have fimds been allocated in
training?

X







Breakdown of cost elements in
Project Plan

Training budgeted in
each test project cost

20.

Is configuration management required? (i.e.
version control)

X





See CTC 0401 &
0402

Documentation of version control,
etc.

Document versions
are controlled

21.

Have adequate lead times been accounted
for:

X





See CTC 0406

Schedule in Project Plan

See individual project
schedules

QUESTION

YES

NO

N/A

PROCEDURE
REFERENCE

DOCUMENTATION

required during project
execution

COMMENTS



a. Subcontracted services

X











b. Engineering/CAD





X







c. Purchases

X







d. Equipment installation

X







e. Software development/support

X







f. Support services





X



g. Client deliverables preparation and
review (including peer review)

X







h. Calibration of new equipment

X







22.

Does the project involve any client-supplied
products, equipment or samples?





X

See C7T0701
and 1003

Documentation indicating
verification and tracking



23.

Have product packaging and delivery
requirements been considered?





X

See CTC 1503 &
1504

DD Form 1149, Shipping &
Receiving, Property Records



24.

Have you planned to survey client
satisfaction at project completion?

X





See CJC 0104

Letter, fax or e-mail requesting
completion of Form 3030

Form 3030 sent to
EPA annually

25.

Will servicing after delivery of product be
required?





X

See CTC 1901

Documentation verifying service
activities



B-5


-------
QUALITY PLANNING CHECKLIST

B. Personnel













26.

Have the labor inputs been negotiated with
line management and made to MMS?

X





See CTC 0406

Documentation of MMS input

Personnel shall be
allocated per MMS



a. Have administrative support personnel
been allocated? (Library research,
deliverable prep, AP support, etc.)

X









Requirements
discussed with line
manager



b. Have technical support personnel been
allocated? (Quality engineering,
laboratory, machine shop, test
validation, receiving inspection,
software, pubs/graphics, computer and
network support, etc.)

X













c. Have engineering support personnel
been allocated? (CAD/CAM, drafting,
floor layout, tooling, etc.)





X







C. EHS Planning















QUESTION

YES

NO

N/A

PROCEDURE
REFERENCE

DOCUMENTATION

required during project
execution

COMMENTS

27.

Does the project involve any potential EHS
hazards, including special physical hazards
(confined spaces, robots, lasers)?

X





See CTC 4472

Form 308, documentation and/or
written instructions

Hazards shall be
identified in test plan.
Training will be
provided using JTA

28.

Is proper and sufficient quantity of personal
protective equipment available?

X





See CTC 0906 &
4472

Form 308, MSDSs and/or JSAs

Shall be identified in
test plans as required

29.

Have the environmental aspects and impacts
been identified for the project?

X





See CTC 4310

Completed Form 3013 (EHS
aspect/impact checklist)

Shall be identified in
JTA & test plans as
required

30.

Have waste disposal issues been addressed?

X





See CTC 0904

Waste container request form

Shall be identified in
JTA & test plan.

31.

Will chemical products be ordered/used?

X





See CTC 0906
See CTC 4472

MSDSs

Operation Planning Checklist
(form 308)

Shall be identified in
JTA & test plan

B-6


-------
QUALITY PLANNING CHECKLIST

32.

Is EHS monitoring required to determine
potential releases to the environment or
exposure to personnel?

X





See CTC 4510,
4320, & 1602

Form 3014 (legal & other
requirements checklist),
monitoring/measurement data,
records

Shall be identified in
JTA & test plan

D. Design













33.

Have customer requirements been addressed
in the design of your activity, product or
service?

X





See CTC 0403

Requirements document, design
document

Shall be identified in
test plan and reports

34.

Does the design require new materials?





X

See CTC 0406

Risk assessment in Project Plan



35.

Are acceptance criteria clearly defined and
understood?

X





See CTC 0403

Requirements document, design
document, design review meeting
minutes

EPA provides
acceptance criteria in
review comments

36.

Have all design inputs and outputs been
identified/documented?

X





See CTC 0403

Documentation of design inputs
and outputs

Shall identify in test
plan

37.

Are design reviews required, if so have the
reviews been scheduled at appropriate stages
of the design?

X





See CTC 0406

Design review requirement noted
in the Project Plan

Document reviews in
project file

38.

Have appropriate customer approvals been
obtained?

X





See CTC 0406

Documentation of client/
customer approval

Customer approval
maintained in project
file

III. Project Implementation

A. Process Control















QUESTION

YES

NO

N/A

PROCEDURE
REFERENCE

DOCUMENTATION

required during project
execution

COMMENTS

39.

Have all additional requirements (customer
specifications, industry standards, regulatory
requirements, Mil-Stds, etc.) been identified
and controlled?

X





See CTC 4320

Completed Fomi 3014
(legal/other reqts checklist),
master list of applicable
standards/revision status

Shall be identified in
test plans and reports

40.

Have Facility Support Services been
contacted for maintenance and operating





X

See CTC 0903

Facility Support Request System
(FSRS)



B-7


-------
QUALITY PLANNING CHECKLIST



support services for new equipment?













41.

Have material handling requirements been
identified?

X





See CTC 0902 &
1501

Demo test plan/work order
request reqts. and/or PO
instructions

Identified in
agreements, JTA, &
test plans

42.

Have you addressed the handling/controlling
of nonconforming products?





X

See CTC 1301

Form 3004 (non-conforming
material report)



43.

Are the use of Statistical Techniques such as
Design of Experiments needed?

X





See CTC 2001

Documentation of statistical
techniques used

Identified in Generic
Verification Protocol
and test plans

44.

Is product traceability and identification
required?





X

See CTC 0801 &
1201

Material tags, markings, etc.,
documentation of inspection/test
status



45.

Are project reviews scheduled at appropriate
stages of development?

X





See CTC 1004

Documentation of project review
(e.g., forms, reports, etc.)

Project reviews
conducted weekly
and monthly

46.

Have you budgeted additional time for senior
management review?

X





See CTC 1004

Documentation of project review
(e.g., forms, reports, etc.)

Budgeted in MMS
and documented on
Form 908

47.

Have you allowed time in your schedule for
internal quality and EMS audit staff?

X





See CTC 1701

Internal Audit Reports

Audits documented
in QMP & schedule

48.

Have you established a contingency budget
to address possible corrective and preventive
action requrests (PCARs)?

X





See CTC 1401

Preventive and Corrective Action
Request Form 3026

Resources allocated
in budget

49.

Has final inspection been addressed?

X





See CTC 0905 &
1001

Form 908 or contract-specific
approval form, or other
documentation of inspection

Shall be identified in
test plan & ETV-MF
QMP

50.

Have you planned to maintain appropriate
records (i.e., project plans, test plans, design
review documentation, etc.)

X





See CTC 1602

Controlled records as required

Requirements
identified in QMP

51.

Are any engineering models or drawings
(internal or customer supplied) associated
with this project?



X



See CTC 0505

Controlled drawings or models



B-8


-------
QUALITY PLANNING CHECKLIST

QUESTION

YES

NO

N/A

PROCEDURE
REFERENCE

DOCUMENTATION

required during project
execution

COMMENTS

52.

Have operating, maintenance, safety, and/or
calibration instructions been generated for
each piece of equipment? If new equipment
has been purchased, follow procedure 1101,
Control of Measuring and Test Equipment, to
have the initial calibration performed and
establish a future calibration schedule.

X





See CTC 0501 &
CJC1101

Controlled work instructions

Identified in test
plans. Equipment
calibration scheduled

by CTC

53.

Do existing procedures/work instructions
need to be updated to account for changes in
methodology?





X

See CTC 0504

Form 3002 (document change
request form)



54.

Have maintenance requirements been
considered to ensure on-going suitability of
process and /or demonstration factory
equipment?





X

See CTC 0907

PSRS requests, maintenance
records, etc.



55.

Are shipping/receiving acceptance
instructions necessary for the receipt of
purchased materials?





X

See CTC 1003

Acceptance instructions



B. Validation/Test













56.

Have appropriate validation/inspection tests
been identified and scheduled?

X





See CTC 0403

Verification plans, validation
plans, test plans, etc.

Identified in test plan

57.

Are there any design requirements specified
that cannot be evaluated using known
validation/inspection techniques?





X

See CTC 0403

Verification plans, validation
plans, test plans, etc.



58.

Have various stages of in-process
inspection/validation been identified?

X





See CTC 1201

Schedule of testing/inspection

Identified in test plan

59.

Have appropriate work orders or test plans
with applicable process parameters been
developed to conduct experiments?

X





See CTC 0902

Demonstration test plans or work
order requests

Test plan to be
developed by
subcontractor and
finalized by CTC

60.

Will calibration of measuring and test
equipment be required?

X





See CTC 1101

Calibration schedules, stickers,
certificates, etc.

Field calibration
identified in test plan.

B-9


-------
QUALITY PLANNING CHECKLIST















Mfg calibration
scheduled by CTC

61.

Do personnel in the review process
understand the client acceptance criteria?

X





See CTC 0404

Verification of qualified,
independent reviewers

EPA review
comments distributed
to subcontractors

62.

Are engineering performance testing
requirements identified?

X





See CTC 0403

Verification plans, validation
plans, test plans, etc.

Identified in test plan

63.

Does the client require any in-process
validations or tests conducted in their
presence?



X



See CTC 0403

Verification plans, validation
plans, test plans, etc.



B-10


-------

-------
APPENDIX C

CTC ISO/EPA ETV Documentation Nomenclature Matrix


-------
CTC ISO/EPA ETV DOCUMENTATION NOMENCLATURE MATRIX

Tier

CTC ISO Documentation

EPA ETV Documentation







1

ISO Quality Manual

ETV-MF QMP







2

Procedures

Generic Verification Protocol







3

Work Instructions

Test/QA Plan







4

Records and Forms

Standard Operating Procedures

C-l


-------