Batteiie
The Business of Innovation
Environmental Technology
Verification Program
Advanced Monitoring
Systems Center
Quality Management Plan (QMP)
for the
ETV Advanced Monitoring Systems
Center
Version 7.0
ET/ET/ET/
-------
QUALITY MANAGEMENT PLAN (QMP)
for the
ETV ADVANCED MONITORING SYSTEMS CENTER
Version 7.0
(SIGNATURE ON FILE)
John McKernan 12-01-08
EPA AMS CENTER PROJECT OFFICER Date
Lauren Drees 11-26-08
EPA AMS CENTER QA MANAGER Date
Amy Dindal 11-18-08
BATTELLE AMS CENTER MANAGER Date
Zachary Willenberg 11-25-08
BATTELLE AMS CENTER QUALITY Date
MANAGER
BATTELLE
505 King Avenue
Columbus, OH 43201
-------
Section TOC
Pagel of 2
Version 7.0
Date: 11/17/08
TABLE OF CONTENTS
Page
1.0 GENERAL PROVISIONS 1
1.1 Introduction 1
1.2 Purpose 1
1.3 Scope and Field of Application 1
1.4 Background 1
1.5 Definitions 4
2.0 MANAGEMENT SYSTEMS 1
2.1 Management and Organization 1
2.2 Quality System and Description 1
2.3 Personnel Responsibilities, Qualifications and Training 2
2.4 Procurement and Acceptance of Items and Services 7
2.5 Documents and Records 7
2.6 Computer Hardware and Software 11
2.7 Planning 12
2.8 Design of Technology Verification Operations 14
2.9 Implementation 18
3.0 ASSESSMENT AND RESPONSE 1
3.1 Scope 1
3.2 General Requirements 2
3.3 Planning and Procedures 3
3.4 Data Validation 5
3.5 Report Review 5
3.6 Quality Improvement 5
LIST OF TABLES
Table 1.0 Personnel Responsibilities for the AMS Center for Verification Testing Activities 4
Table 2.0 Records Management Responsibilities for the AMS Center 10
Table 3.0 Assessments for the AMS Center 2
LIST OF FIGURES
Figure 1.0 AMS Center Organization 3
Figure 2.0 Systematic Planning of Verification Tests 13
-------
APPENDICES
APPENDIX I. NAMES, ADDRESSES, AND PHONE NUMBERS OF BATTELLE
AMS CENTER KEY STAFF
APPENDIX II. ETV VERIFICATION STATEMENT
APPENDIX III. AMENDMENT AND DEVIATION FORMS
APPENDIX IV. ETV KICK-OFF MEETING CHECKLIST
APPENDIX V. ETV ASSESSMENT REPORTING FORMS
Section TOC
Page 2 of 2
Version 7.0
Date: 11/17/08
-------
Section 1
Pagel of 5
Version 7.0
Date: 11/17/08
1.0 GENERAL PROVISIONS
1.1 INTRODUCTION
1.1.1 This document, the Quality Management Plan (QMP) for the Advanced Monitoring
Systems (AMS) Center, describes the quality systems that will be employed by Battelle in
conducting the AMS Center. These quality systems are designed to be consistent with
ANSI/ASQ E4-1994, "Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs," the U. S.
Environmental Protection Agency (EPA) document "Environmental Technology
Verification Program Quality Management Plan", Version 3.0, dated January 2008, and
EPA Requirements for Quality Management Plans (QA/R-2, dated March 2001).
1.2 PURPOSE
1.2.1 The purpose of the AMS Center is to verify commercially-available systems for monitoring
natural species and contaminants in air, water, and soil. The AMS Center encompasses the
full range of environmental monitoring technologies, and as part of the larger
Environmental Technology Verification (ETV) program is designed to provide technology
users with objective, high quality performance data to support monitoring technology
selection decisions.
1.3 SCOPE AND FIELD OF APPLICATION
1.3.1 This document encompasses activities that Battelle, as an ETV verification organization
(VO), shall utilize to assure the quality of products and services provided by the AMS
Center. The AMS Center is one of five centers operating under the ETV program.
1.3.2 This QMP applies to personnel involved in, and activities conducted by those staff for the
AMS Center, and contains the minimum specifications and guidelines that are applicable to
AMS Center quality management functions and activities based upon ANS/ASQ E4-2004
(an update to the ANSI/ASQC E4-1994 document). These include, but are not limited to,
personnel qualification and training, procurement of items and services, documents and
records, computer hardware and software, planning, implementation for work processes,
assessment and response, and quality improvement provisions.
1.4 BACKGROUND
1.4.1 Battelle (Memorial Institute) was established in 1929 by Gordon Battelle and serves as a
memorial to his family. Governed by a self-perpetuating Board of Trustees, Battelle is a
nonprofit Ohio corporation. Battelle and the laboratories it manages and co-manages has a
staff of 20,000 scientists, engineers, and support specialists. Battelle conducts $4 billion in
annual research and development.
Battelle's headquarters are in Columbus, Ohio. In addition to headquarters in Columbus,
Ohio, Battelle has major technology centers in Richland, Washington, where we manage
the Department of Energy's (DOE's) Pacific Northwest National Laboratory (PNNL);
-------
Section 1
Page 2 of 5
Version 7.0
Date: 11/17/08
Upton, New York, where we partner with the Research Foundation of the State of New
York in managing Brookhaven National Laboratory (BNL); Golden, Colorado, where we
partner with MRI in managing the National Renewable Energy Laboratory (NREL); Oak
Ridge, Tennessee, where we partner with the University of Tennessee in managing the Oak
Ridge National Laboratory (ORNL); Idaho Falls, Idaho, where we have formed the Battelle
Energy Alliance to manage the Idaho National Laboratory (INL); Frederick, Maryland,
location of the National Biodefense Analysis and Countermeasures Center, managed by
Battelle National Biodefense Institute; Livermore, California, where we partner with the
University of California in managing Lawrence Livermore National Laboratory; and
Aberdeen, Maryland, where we manage the Battelle Eastern Science and Technology
Center. Specialized facilities, regional centers, and offices are located in 159 other cities in
the United States and worldwide.
Battelle's organization includes three global lines of business: National Security, Health
and Life Sciences, and Energy Technology. The AMS Center is managed within Battelle's
National Security Global Business (NSGB) which includes approximately 3,500 chemists,
engineers, statisticians, and support personnel. Staff and facilities will be drawn from
NSGB and other Battelle organizations as needed to support the AMS Center. Staff
involved in the AMS Center include those with expertise in environmental monitoring,
stakeholder involvement, and outreach and communication. Key Battelle facilities that are
available for use on the AMS Center include comprehensive laboratory analysis equipment;
field sampling and analysis equipment; source simulators such as pilot plants;
environmental chambers; and real-world test sites.
The organization chart for the AMS Center is provided in Figure 1.0 and shows key AMS
Center staff and their reporting lines. The key AMS Center staff are:
Battelle AMS Center Manager: Ms. Amy Dindal is Battelle's AMS Center Manager with
responsibility for meeting all technical, budget, and schedule goals for the Center. Ms.
Dindal reports directly to Ms. Karen Riggs, a Section Manager in the Environmental
Technologies Product Line within NSGB, who will provide Ms. Dindal and the other key
AMS Center staff with direct support in securing and deploying Battelle resources for the
AMS Center. Ms. Tracy Stenner, the Environmental Technologies Product Line Manager,
has ultimate responsibility for ensuring that necessary Battelle facility and staff resources
are available to support the AMS Center. Ms. Dindal serves as the primary point of contact
for the EPA AMS Center Project Officer, Mr. John McKernan. She also directs the
activities of the leaders in three organizational areas within the Center: Quality Assurance;
Verification Testing; and Stakeholders.
Quality Assurance: Mr. Zachary Willenberg is the Battelle AMS Center Quality Manager.
He is a Quality Assurance Officer in NSGB and reports directly to the Environmental
Technologies Product Line Manager, Ms. Stenner, and for the AMS Center, to Ms. Dindal.
These relationships are illustrated in Figure 1.0. Mr. Willenberg serves as the primary point
of contact for the Acting EPA AMS Center Quality Manager, Ms. Lauren Drees.
Verification Testing: Dr. Thomas Kelly is the AMS Center Verification Testing Leader
and has responsibility planning and leading verification tests, including management of
-------
Section 1
Page 3 of 5
Version 7.0
Date: 11/17/08
Acting EPA
AMS Center
Quality Manager
L. Drees
Stakeholder
Coordinator
R. Sell
Stake
ikeholder
Committees
EPA
AMS Center
Project Officer
J. McKeman
AMS
Center Manager
A. Dindal
Verification
Testing Leader
T. Kelly
Battelle
National
Security Global
Business
President
S. Kelly
\
Battelle
Environmental
Technologies
Product Line
Manager
T. Stenner
Battelle
1 Quality Manager
Z. Willenberg
Verification Test Coordinators
Figure 1.0 AMS Center Organization
(dotted lines indicate indirect reports)
-------
Section 1
Page 4 of 5
Version 7.0
Date: 11/17/08
verification test coordinators. Dr. Kelly is an Associate Manager in Battelle's Environmental
Technologies Product Line. He reports directly to Ms. Dindal on AMS Center activities and
can serve in the role as AMS Center Manager for tasks delegated by Ms. Dindal.
Stakeholders: Ms. Rachel Sell is the AMS Center Stakeholder Coordinator with primary
responsibility for stakeholder recruitment, communications, and stakeholder meeting
facilitation. Ms. Sell is a Research Scientist at Battelle and reports directly to Ms. Dindal on
the AMS Center.
Names, mailing/email addresses, and phone/facsimile numbers of these Battelle AMS
Center key staff are included in Appendix I.
1.5 DEFINITIONS
1.5.1 Verbs for clarity:
Shall, must: when the element is required and deviation from the specification will
constitute nonconformance with this QMP
Should, will: when the element is recommended
May: when the element is optional.
1.5.2 Center Quality Management Plan (QMP) - Procedures for quality-related activities
developed and implemented by Battelle to assure quality in the work processes and services
developed for the AMS Center.
Generic Verification Protocol - A generisized description of a verification test for a
technology category.
Stakeholders - Representatives of verification customer groups including buyers and users
of technology, consulting engineers, finance and export communities, and government
(local, state, federal) permitters and regulators. Stakeholders are selected based upon their
expertise and interest in environmental monitoring and their availability and willingness to
participate in the AMS Center. A list of AMS Center Air and Water Stakeholder
Committee members are available on the ETV website (http://www.epa. gov/etv).
Test/Quality Assurance (QA) Plan - The plan developed by Battelle, with appropriate
input for each individual test of a technology or technology class. The test/QA plan
provides the experimental approach with clearly stated test objectives and associated
quality objectives for the related measurements and may incorporate or reference an AMS
Center Generic Verification Protocol and/or standard operating procedures (SOPs).
Vendor - An individual, company, or organization which submits a commercially-available
environmental monitoring technology for verification testing.
-------
Section 1
Page 5 of 5
Version 7.0
Date: 11/17/08
Verification Organization - A public or private sector organization selected by EPA to
cooperate with the implementation of the ETV program by conducting verification testing
and to provide unbiased and objective test performance data on environmental
technologies.
Verification Organization Center Manager - The person designated by the verification
partner with the responsibility to manage the Center and serve as the chief point of contact
with the EPA. For the AMS Center, this is the Battelle AMS Center Manager.
Verification Organization Quality Manager - The person designated by the verification
partner with the responsibility to manage quality assurance for the AMS Center on behalf
of the verification partner Center manager. For the AMS Center, this is the Battelle AMS
Center Quality Manager.
Verification Report - A complete detailed summary of procedures and results for a
verification test of a single technology.
Verification Statement - A summary statement developed by Battelle, and signed by EPA
and Battelle, which reports quantitatively but without endorsement, the performance of a
tested technology in a verification test. An example of an ETV verification statement is
included in Appendix II.
-------
Section 2
Pagel of 20
Version 7.0
Date: 11/17/08
2.0 MANAGEMENT SYSTEMS
Battelle's quality policy is to provide services, products, and data of the highest quality that meet or
exceed our client's requirements and expectations. To this end, quality programs such as this AMS
Center QMP, and quality achievement, shall be fully supported by Battelle management and staff.
2.1 MANAGEMENT AND ORGANIZATION
2.1.1 Battelle management is responsible for committing to a quality policy and for creating
work environments in which all personnel strive for the highest quality of services and
products. Management shall also provide the Battelle AMS Center Manager the authority
to ensure the following:
• That all applicable elements of the quality system as described in this QMP are
understood and are implemented in the AMS Center.
• That adequate personnel and resources are available to plan, implement, assess, and
improve services and products relevant to the AMS Center.
• That staff is (are) clearly designated to stop unsafe work and work of inadequate
quality as affects the AMS Center.
2.2 QUALITY SYSTEM AND DESCRIPTION
2.2.1 The Battelle quality system to be implemented for the AMS Center according to this QMP
(and, historically, previous versions of this QMP) is intended to conform with the
specifications listed in:
• ANSI/ASQ E4-2004, "Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs".
• EPA document "Environmental Technology Verification Program Quality
Management Plan", Version 3.0, January 2008.
• EPA document "EPA QA/R-2, EPA Requirements for Quality Management Plans,
March 2001.
It should be noted the E4 standard is comparable to the International Standards
Organization (ISO) 9000 series.
2.2.2 The principal quality system document governing general and specific responsibilities for
AMS Center management and staff, responsibility and authority for all technical activities,
and reporting lines is this document, the "Quality Management Plan for the ETV Advanced
Monitoring Systems Center".
Individual verification tests will conform both to this QMP and to the applicable test/QA
plan document(s) and applicable SOPs.
The AMS Center QMP and any revisions will be controlled documents identified by a
unique Battelle document number (see Section 2.5.1) and will be distributed according to a
published list maintained by the Battelle AMS Center Quality Manager.
-------
Section 2
Page 2 of 20
Version 7.0
Date: 11/17/08
The QMP review will be documented by the Battelle AMS Center Quality Manager and
Battelle AMS Center Manager by signing and dating the copy of the QMP routed for review.
Any revisions to the QMP will be compiled by the Battelle AMS Center Quality Manager for
review, approval, and distribution. The approved QMP has a scheduled review interval of
one (1) year, although this may be adjusted by the EPA AMS Center Quality Manager,
depending upon factors such as the revision cycle of the ETV QMP.
The initial approved QMP will serve as Version 1.0, which will be designated, with its
effective date, in the upper right corner of each document page. Revisions will be so
designated beginning with "2.0" and will subsequently be numbered and dated as
applicable. Battelle staff to whom controlled copies are issued will be responsible for
disposal of outdated QMP versions.
2.2.3 The scope of the AMS Center quality system applies to all Battelle personnel providing
products and services for the AMS Center. All AMS Center key staff shall be
knowledgeable regarding the QMP requirements.
2.2.4 Quality procedures documentation includes maintenance of all inspection and
review/assessment records, listing of all controlled documents (see Section 2.5.1), and
retention of records pertaining to personnel training and qualification, instrument
maintenance and calibration, and test methods/operating procedures.
2.2.5 Center-specific quality controls are initiated upon approval of Battelle's QMP prior to
implementing any verification testing activities. Planning actions documented through
approved test/QA plans shall also serve as quality control mechanisms for verification
testing.
In-process quality controls, through conduct of inspections followed by assessment reports
and verification of corrective actions when required, shall also be performed and recorded.
Implementation of a complete and consistent assessment of technical operations provides
overall control of Center activities. This will be accomplished by the Battelle AMS Center
Quality Manager according to Section 3.0 in this QMP.
2.2.6 An external quality system audit (QSA) of the Battelle quality system will be performed at
least once by the EPA AMS Center Quality Manager. In addition, an independent technical
systems audit will be performed by the EPA AMS Center Quality Manager or designee, at
least one time per year for the AMS Center.
2.3 PERSONNEL RESPONSIBILITIES, QUALIFICATIONS, AND TRAINING
2.3.1 Responsibilities
2.3.1.1 Verification Partner Responsibilities. In accordance with EPA's ETV QMP dated
January 2008, Battelle's responsibilities for the AMS Center include the
following:
-------
Section 2
Page 3 of 20
Version 7.0
Date: 11/17/08
• Establish, attend, and/or conduct meetings of stakeholder committees with
representation from major customer groups
• Maintain communication with EPA to assure mutual understanding and
conformance with EPA quality procedures and expectations and ETV policies
and procedures
• Develop, review, and revise test/QA plans in cooperation with technology
vendors and stakeholders
• Solicit technology vendors and verification test collaborators
• Manage participation of and conduct verification activities
• Assure that quality procedures are incorporated into all aspects of the AMS
Center
• Perform ETV activities within the documented quality system
• Prepare ETV verification reports and statements at the completion of each
technology verification
• Appoint a quality manager, responsible for ensuring the AMS Center quality
systems are in compliance with the E4 standard and the EPA ETV QMP dated
January 2008, and AMS Center staff complies with this QMP.
• Submits a written request to the EPA AMS Center Project Officer and EPA
AMS Center Quality Manager if desired in specific instances, the
responsibility for reviewing and approving test/QA plans be delegated to the
Battelle AMS Center Quality Manager.
2.3.1.2 Key Staff Responsibilities. Battelle is committed to operate an effective quality
system that ensures compliance with all program requirements. The
responsibilities of Battelle key staff who will be performing verification testing
activities addressed by this AMS Center QMP are listed in Table 1.0.
2.3.1.3 Stakeholder Responsibilities. The responsibilities of stakeholders for the AMS
Center include the following:
• Assist in development of the generic verification protocol.
• Assist in prioritizing the types of technologies to be verified, focusing on
these technologies with greatest environmental and sustainability impacts.
• Review Center-specific procedures and AMS Center documents including
test/QA plans, verification reports, and verification statements, as requested.
• Participate in verification testing as collaborators, provide funding and/or in-
kind support, or recommend collaborators to the AMS Center.
• Assist in the definition and conduct of outreach activities appropriate to the
technology area and customer groups.
• Serve as information conduits to the particular constituencies that each
member represents.
2.3.2 Qualification and Training
Battelle personnel qualifications and training shall target technical work performed directly
in support of verification testing activities. These qualifications and training may include:
• Formal education in physical sciences (e.g., chemistry, physics, engineering).
-------
Section 2
Page 4 of 20
Version 7.0
Date: 11/17/08
Table 1.0. Personnel Responsibilities for the AMS Center for Verification Testing
Activities
AMS Center Team
Member
Battelle AMS Center
Manager
Amy Dindal
Responsibilities
Ultimate responsibility for all aspects of the AMS Center
Conduct and oversee activities to establish and maintain active stakeholder committees
Maintain adequate communication with EPA
Manage oversight and conduct of verification activities
Assure that quality procedures are incorporated and implemented
Review/approve test/QA plans
Solicit technology vendors
Assign Verification Test Coordinators to technology categories
Operate ETV activities within the documented quality system
Issue stop work orders
Review and approve verification reports
Review and approve verification statements
Battelle Verification
Testing Leader
Thomas Kelly
Coordinate planning, performance, and data reviews of technology verification testing consistent
with the AMS Center QMP requirements
Coordinate review of applications from technology vendors wanting to have their technology verified
Work with stakeholders and EPA to identify and prioritize technologies for verification
Schedule verification tests
Provide recommendations to the AMS Center Manager for verification teams to perform specific
technology verification test/data reviews.
Review and approve test/QA plans and amendments and deviations to test/QA plans
Prepare, review, and/or approve verification reports
Prepare, review, and/or approve verification statements
Oversee/assist in problem resolution involving verification tests
Battelle AMS Center
Quality Manager
Zachary Willenberg
Ensure that the quality system is compliant with EPA-specified standards
Advise the Battelle AMS Center Manager of any QA/QC problems and oversee corrective actions
Ensure that the AMS Center QMP includes sufficient and appropriate specifications for QA/QC as
required for the AMS Center
Interact with AMS Center management and technical personnel to ensure that QA/QC procedures
are understood
Ensure that Battelle AMS Center QMP and the EPA/ETV QMP are followed for performing system
inspections and audits
Perform a TSA and ADQ for every verification test or ensure that a TSA and ADQ are performed by
a designee
Participate in pre-test kick-off meetings to review QA requirements with verification testing staff
Review training records of verification testing staff
Notify the Battelle AMS Center Manager to issue a stop work order if assessments indicate health,
safety, or quality concerns
Review QA documentation of reference laboratories for each verification test, as appropriate
Review QC data (including reference laboratories and vendor technologies) generated during
verification tests
Ensure that inspection reports are prepared and distributed that detail appropriate corrective action
and that implementation will be responded to by personnel. Problems that are not addressed will be
brought to the attention of the Battelle AMS Center Manager
Review test/QA plans, SOPs, verification reports, and verification statements
Review and approve amendments and deviations to test/QA plans
-------
Section 2
Page 5 of 20
Version 7.0
Date: 11/17/08
Table 1.0 (continued)
AMS Center Team
Member
Responsibilities
Review all quality system documentation, including this document, at intervals necessary to ensure
their integrity. Such reviews will be recorded and documents will be revised if necessary. All
previous original (i.e., signed) revisions will be retired and archived.
Act as a QA resource to respond to quality needs and problems. Answer questions and train
laboratory staff in QA/QC requirements and procedures as needed.
Battelle Verification
Test Coordinators
Provide technical support to verification testing as needed, and interact with the Battelle AMS
Center Quality Manager during inspections and implementation of corrective actions when needed
Perform QA/QC activities specified in this document, applicable test/QA plans, and in pertinent
SOPs
Conduct QC measures and activities required for sample analyses
Verify 100% of data and evaluate results of QC analyses to determine if quality goals and objectives
have been met
Inform the Verification Testing Leader of potential problems
Perform corrective action at the direction of the Battelle AMS Center Manager and Battelle AMS
Center Quality Manager in response to TSA and ADQ audit reports
Document results of QC analyses and include them with sample results and historical data files
Maintain instrumentation (vendor and/or reference instrumentation) in accordance with the QMP,
test/QA plan, SOPs, and the manufacturer's instructions
Prepare test/QA plans and amendments and deviations to these plans, as appropriate
Perform pre-test kick-off meetings to review technical, project management, and QA aspects of
verification testing with verification testing staff
Perform performance evaluation (PE) audits of reference laboratories and calibrated equipment for
each verification test, as appropriate
Maintain verification test records (in bound Laboratory Record Book and/or test binders) that
adequately capture the quality of data collected
Develop and implement test/QA plans
Prepare verification reports
Prepare verification statements
• Experience in sampling and analysis of air, water, and soil.
• Training on standard analytical instrumentation such as gas chromatographs, mass
spectrometers, ion chromatographs, etc.
• Experience in designing experiments to verify the performance of monitoring
technologies.
• Experience with specific monitoring techniques such as immunoassay, ion
chromatography, polymerase chain reaction techniques, X-ray fluorescence, etc.
Battelle personnel working on the AMS Center shall have, at a minimum, documentation
maintained by Battelle permanently for each of the following, as applicable:
• Education history which can include formal qualification or certification relevant to
technical, quality assurance, or management disciplines.
• Work experience as academic or on-the-job performance in technical and/or
management areas.
• Experience in the application of quality assurance/quality control requirements in
technical performance or data verification.
-------
Section 2
Page 6 of 20
Version 7.0
Date: 11/17/08
2.3.2.1 Formal qualifications and certifications in the form of actual or verified-copy
documentation for specific disciplines shall be maintained in the staff member's
qualification/training file. Training documents will be reviewed by the Battelle
AMS Center Quality Manager, as appropriate.
2.3.2.2 Technical management and training received in-house or offsite shall be recorded
and forms, memos, or certificates retained. Performance on either task, project, or
program assignments is to be considered as part of training.
2.3.2.3 Retraining needs based on job requirements shall be determined by the staff
member and respective management. To maintain staff proficiency, opportunities
provided by Battelle or other sources shall be made available, preferably on an
annual basis.
2.3.2.4 Personnel job proficiency based on witnessed performance on-the-job by a
qualified trainer/staff member designee shall be documented. Specific method
requirements for instrument inspection, performance, and maintenance are
objective measures that could be considered. Specific performance based on
national certification requirements can be recorded with certificates or other
documentation. Basic areas of proficiency for verification testing may include, at
a minimum:
• Sample management practices, such as chain of custody records
• Sample handling and storage and use of standards and reagents
• Instrument inspection, use, and maintenance
• Data acquisition, analysis, and verification.
2.3.2.5 Training resources should be offered on-site by Battelle for facility requirements,
such as general computer software use (E-mail, spreadsheets) or project
management. Off-site training, project/program meetings, and technical society
membership should be available for specific disciplines contributing to the staff
member's overall job proficiency.
2.3.2.6 Verification test collaborators working on behalf of Battelle in support of the
AMS Center and/or individual test operations are expected to provide the
Verification Test Coordinator, or designee, with:
• Educational background and/or degree(s) relevant to technical areas
represented in the AMS Center
• Work experience related to the technology category undergoing verification.
This information may be reviewed by the Battelle AMS Center Quality Manager.
2.3.2.7 Battelle personnel will receive update memorandum with the QMP when the
document is revised specifying changes made. In addition, the Battelle AMS
Center Quality Manager will provide testing staff with an annual refresher on
QMP requirements.
-------
Section 2
Page 7 of 20
Version 7.0
Date: 11/17/08
2.4 PROCUREMENT AND ACCEPTANCE OF ITEMS AND SERVICES
2.4.1 Policy
Procurement technical and quality requirements are generally based upon value (cost,
durability, maintainability), performance (specification compliance, operating conditions,
calibration capacity), delivery (timeliness, ease of ordering), customer support
(responsiveness, technical ability), past experience with a particular vendor, and
completeness and coherence of instructions (clarity, accuracy).
2.4.2 Procurement
Technical and quality requirements for items and services procured for a specific
verification test may be included in the test/QA plan. These requirements will typically be
specified under materials and/or measurement system equipment (test/QA plan Section B8,
Inspection/Acceptance Requirements for Supplies and Consumables). The request for
items or services will initiate from the Verification Test Coordinator or designee with
approval for purchase from the Verification Testing Leader, Battelle AMS Center Manager,
line manager, or designee.
2.4.3 Acceptance
2.4.3.1 Testing equipment procured for activities affecting quality shall be calibrated to
ensure accuracy with required specifications listed in the test/QA plan and may be
verified prior to use in the verification test (e.g. PE audits), as appropriate. Any
discrepancies shall result in a recalibration of the equipment, or if the equipment is
unusable, then a return of the item to the supplier for repair/replacement as
necessary. Verification, storage, and maintenance records will be included in
individual verification test records.
2.4.3.2 Testing materials procured for activities affecting quality (e.g. reference standards
or gases) shall be accompanied with a Certificate of Analysis (COA) where
appropriate. The COA will be examined to ensure that the listed specifications are
within the required limits. The COA will be retained and included in the
verification test records.
2.4.3.3 Methods to accept procurement of services (i.e. subcontractors; installation, repair,
or maintenance work; etc.) includes technical verification of the data produced,
surveillance and/or audit of the activity being performed, or review of objective
evidence for conformance to procurement document requirements.
2.5 DOCUMENTS AND RECORDS
2.5.1 Controlled Documents
Document control is the system which ensures that only the latest revision of the defined
documents are used by Battelle staff participating in the AMS Center. The system includes
-------
Section 2
Page 8 of 20
Version 7.0
Date: 11/17/08
retention of the document with original signed page(s) in a limited access storage area, a
unique numbering system for all documents (typically identified by revision number and/or
date), and a distribution list for each document. Such documents are defined as "controlled
documents" and can be revised only by the personnel listed within each document or this
QMP. The following is a list of the controlled documents within the AMS Center:
• Quality Management Plan for the ETV Advanced Monitoring Systems Center (this
document)
• Standard Operating Procedures
• Test/QA plans, including amendments and deviations
• Generic Verification Protocols
Controlled document identification will consist of a number (if applicable), date, and
version, if applicable, assigned to the document by the Battelle AMS Center Quality
Manager or designee. A current Master List of Controlled Documents and Distribution
shall be maintained by the Battelle AMS Center Quality Manager.
As a controlled document, approved copies of the QMP will be maintained and issued to
AMS Center staff by the Battelle AMS Center Quality Manager or designee.
Obsolete or superseded documents shall be removed from operations when new documents
are provided. Notification will accompany new document versions that the previous
version is to be removed from use and destroyed. Staff members are responsible for
destroying outdated versions of documents assigned to their person. The Battelle AMS
Center Quality Manager is authorized to remove outdated documents observed during
inspections and reviews. All controlled documents, including historical revisions, will be
retained at least 10 years or six years and three months after final payment of the
cooperative agreement, with the exception of the Standard Operating Procedures which will
be permanently archived.
2.5.2 Verification Test Records
2.5.2.1 Active Verification Test Records. All verification test records shall carry
minimum identification pertaining to title, responsible person or author, and date.
All manual entries shall be entered using ink and initial and dated by the
individual recording the entry. No changes to entries, manual or electronic, shall
obscure the original record during the correction process, and corrections shall be
initialed and dated by the individual recording the correction. A short explanation
will be added to non-obvious corrections.
2.5.2.2 Storage of Verification Test Records. Verification test records specific to the
AMS Center shall be retained for at least 10 years or six years and three months
after final payment of the cooperative agreement. All AMS Center records needed
to reconstruct test activities and verify that reported data were collected in a
consistent manner with this QMP and AMS Center requirements will be
maintained in an appropriate area of limited access, until either transferred to EPA
Office of Research and Development (ORD) Records Management or properly
destroyed with EPA permission. The Battelle AMS Center Quality Manager will
-------
Section 2
Page 9 of 20
Version 7.0
Date: 11/17/08
retain, as a permanent record, documentation of the transfer or destruction of
Battelle's AMS Center records.
2.5.3 AMS Center Program Records
The following program records will be retained, as per ETV directives, for at least 10 years
or six years and three months after final payment of the cooperative agreement.
• Minutes of stakeholder meetings
• Cooperative agreement records
• Test/QA plans
• Verification reports
• Verification statements
• Battelle quality assessment reports.
2.5.4 Record Preparation, Review, Approval, and Distribution
Responsibilities for these activities are summarized in Table 2.0 and are detailed below.
2.5.4.1 Preparation. Individual case requirements and this QMP shall guide document
and record content and/or format. For the AMS Center, guidance for content
and/or format are derived by EPA/ETV directive and the following documents:
• ANSI/ASQ E4-2004, "Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs".
• EPA document "Environmental Technology Verification Program Quality
Management Plan", Version 3.0, January, 2008.
• EPA document "EPA QA/R-2, EPA Requirements for Quality Management
Plans, March 2001.
2.5.4.2 Review/Approval. Record review/approval shall be performed by qualified
technical and/or management personnel as deemed appropriate. The individual
reviewer shall have access to all needed references.
All Battelle prepared documents in QMP Sections 2.5.1 through 2.5.3 shall require
at least one review by a Battelle staff member prior to external distribution by
Battelle. Document and record reviews are performed at the request of the
Battelle AMS Center Manager, Quality Manager, Verification Testing Leader, or
other staff personnel.
In addition, ETV record review assigned to Battelle extends to the following
documents, at a minimum:
• EPA/ETV strategy
• EPA/ETV QMP
• Annual Center progress reports.
-------
Section 2
Page 10 of 20
Version 7.0
Date: 11/17/08
Table 2.0 Records Management Responsibilities for the AMS Center
Preparation/
Record Type Updating Review Approval Finals Distributed to:
ETV Verification
Strategy
ETV Quality
Management Plan
CA Records
AMS Center Quality
Management Plan
Minutes of
Stakeholder
Meetings
Test/QA Plan
(including SOPs,
amendments and
deviations)
Generic Verification
Protocol
Raw data
ETV Verification
Report
ETV Verification
Statement
Annual ETV
Progress Report
Quarterly ETV
Progress Report
EPA Reviews/Audit
Reports
Battelle
Reviews/Audit
Reports
N/A
N/A
Battelle AMS
Center
Manager
Battelle AMS
Center
Quality
Manager
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
N/A
N/A
N/A
Battelle AMS
Center
Quality
Manager
Battelle AMS Center Manager
Battelle AMS Center Manager
EPA AMS Center Project Officer
EPA AMS Center Quality Manager
EPA AMS Center Project Officer
Stakeholders
Battelle AMS Center Manager
Battelle AMS Center Qualify
Manager
EPA AMS Center Quality Manager
EPA AMS Center Project Officer
Assigned Stakeholders
Peer Reviewers
EPA AMS Center Project Officer
Battelle AMS Center Quality
Manager
Assigned Stakeholders
Internal Technical Peer Review
Battelle AMS Center Manager
Battelle AMS Center Quality
Manager
Battelfe Verification Testing Leader
EPA AMS Center Quality Manager
Vendor
Peer Reviewers
Battelle AMS Center Manager
Battelle AMS Center Quality
Manager
Battelfe Verification Testing Leader
EPA AMS Center Project Officer
EPA AMS Center Quality Manager
Vendor
ETV Program Director
Peer Reviewers
Battelle AMS Center Manager
Battelle AMS Center Manager
N/A
Battelle AMS Center Manager
Battelle Verification Testing Leader
N/A
N/A
N/A
EPA AMS Center Project Officer
EPA AMS Center Quality
Manager
Battelfe AMS Center Manager
Battelle AMS Center Qualify
Manager
N/A
Vendors
EPA AMS Center Project Officer
EPA AMS Center Quality
Manager
EPA AMS Center Project Officer
EPA AMS Center Quality
Manager
N/A
EPA AMS Center Project Officer
EPA AMS Center Quality
Manager
EPA Laboratory Director
Battelle Management
EPA AMS Center Project Officer
EPA AMS Center Quality
Manager
N/A
Battelle AMS Center Manager
N/A
N/A
N/A
N/A
N/A
Testing Staff
ETV webmaster
EPA AMS Center Quality Manager
EPA AMS Center Project Officer
Stakeholders
ETV Webmaster
EPA AMS Center Project Officer
Testing Staff
Vendors
EPA AMS Center Project Officer
EPA AMS Center Quality Manager
ETV Webmaster
EPA AMS Center Project Officer
EPA can request copies
ETV Program Director
EPA AMS Center Project Officer
ETV Webmaster
Vendors
ETV Program Director
EPA AMS Center Project Officer
ETV Webmaster
Vendors
N/A
EPA AMS Center Project Officer
EPA AMS Center Quality Manager
ETV Program Director
EPA Laboratory Directors
Battelle AMS Center Manager
Battelle AMS Center Qualify
Manager
EPA AMS Center Project Officer
EPA AMS Center Quality Manager
NA = Indicates Battelle does not have responsibility for preparing/updating record; conducting or obtaining review; providing or obtaining approval; or distributing
and/or receiving final record.
-------
Section 2
Page 11 of 20
Version 7.0
Date: 11/17/08
2.5.4.3 Distribution. Once records have been reviewed and approved as required,
distribution will be made as listed in Table 2.0. AMS Center documents
specifically requiring EPA approval before release include:
• AMS Center QMP
• Verification Protocols (both Center and technology-specific)
• Test/QA plans
• ETV verification reports
• ETV verification statements
2.6 COMPUTER HARDWARE AND SOFTWARE
This QMP requires that Battelle staff understand the necessity for all computer hardware and
software specifications. Staff shall attempt to utilize computer hardware and software within the
acceptance criteria specified, and ensure that hardware and software are installed, maintained, and
used according to specifications. Any time a change in hardware components or configuration or
a software modification is needed, retesting and recalibration must be performed and
documentation included with facility records.
2.6.1 Hardware
All computer hardware at Battelle contains Intel based Pentium processors running a
Microsoft operating system. Each personal computer (PC) primarily consists of a standard
complement of Microsoft software (e.g., Word, Excel, Access, PowerPoint, and Outlook)
with capabilities of running other commercial software (e.g., WordPerfect, Quattro, Lotus,
SAS) and delivery of data in various standard formats. These computers are replaced
approximately once every three years to ensure staff have access to the most updated, state-
of-the-art equipment, especially those staff with the heaviest computational needs.
Computer hardware is upgraded to improve performance and provide complete
compatibility with current standards. The decision to upgrade computer hardware is made
when a project that requires specific computer capabilities is received. Next, an assessment
of impact is completed. This assessment includes a review of current computer programs
and the impact or upgrading hardware on data accessibility.
2.6.2 Software
Specific software required for a verification test will be identified in the test/QA plan. Most
software used at Battelle is acquired commercially, loaded, and tested as specified by the
publisher. Independently-developed software is not used within the ETV AMS Center;
only commercial products are used. Software used for data management activities may
include Microsoft Excel or Access. Standard word processing software (e.g. Word) is used
to create reports. Currently, Battelle does not use nor are the systems currently compatible
with Windows Vista.
2.6.3 Validation Policy
Since all hardware and software used on the ETV AMS Center is commercially available,
and wide public use and continued market viability is considered proof of software
-------
Section 2
Page 12 of 20
Version 7.0
Date: 11/17/08
dependability, validation is not considered necessary. However, verification of data
analysis techniques within each program (e.g. the use of formulas and macros) is required.
For each defined spreadsheet a performance test document will be prepared which will
contain the following:
• An overview of the application. The overview will describe what the application is
required to do and specify the methods used to meet the predetermined requirements.
• References to the productivity software used (e.g., Excel 2003, SigmaPlot V8.0, etc),
and the operating system (e.g., Windows 2000, Windows XP, etc.).
• A description of important equations used to derive data.
• A description of what test(s) were conducted to confirm the accuracy of the application.
2.7 PLANNING
This QMP addresses the purpose and scope of systematic, timely, and effective planning
necessary to assure services and products of the highest quality.
2.7.1 Stakeholder committee(s) containing representatives of appropriate technology interest
groups shall be jointly established by the EPA AMS Center Project Officer and Battelle.
Individual stakeholders shall be selected for these committee(s) based on their expertise and
interest in environmental monitoring and their availability and willingness to participate.
A joint meeting of the EPA AMS Center Project Officer, Battelle, and each stakeholder
committee will be held at least once annually, with minutes of such meetings recorded,
reviewed, and circulated to the stakeholders, the EPA AMS Center Project Officer,
Battelle, and the ETV Webmaster. The meeting can be conducted in person or by
teleconference.
The planned quality-related purposes of this meeting are to:
• Identify, revise, and/or clarify the technical and quality goals of the work to be
accomplished
• Determine testing priorities and evaluate customer satisfaction
• Define and review verification plans, and identify verification test collaborators.
2.7.2 Systematic Planning of Verification Tests
An overall view of the EPA ETV verification process is shown in Figure 2.0. Battelle, in
cooperation with the EPA AMS Center Project Officer, begins a systematic process to plan
the individual verification tests. Systematic planning may be accomplished through any
demonstrated technique such as the data quality objectives process (EPA QA/G-4,
Guidance on Systematic Planning Using the Data Quality Objectives Process, February
2006). The planners perform the following actions:
• Convene stakeholder committees of representatives of verification customer groups
who advise during the planning process
• Mediate and facilitate the selection of prioritized technologies
• Refine the scope of respective technology areas
-------
Section 2
Page 13 of 20
Version 7.0
Date: 11/17/08
Determine interest in verification from the vendors of commercial-ready technologies
within the defined scope of these areas and other collaborators that could contribute
funding or in-kind support
Identification of Technology
Categories for Testing
(Stakeholder/EPA/Battelle)
Recruitment of Vendors/Collaborators
(Battelle)
r
Draft Test/QA Plan
(Battelle/Vendors/Stakeholders/Collaborators)
,
'endor Review of Test/QA Plan
(Battelle)
I
Peer and QA Review of Revised
Test/QA Plan
(Battelle)
r/
Coordination with Host-Facility/
Subcontractors/Collaborators
Final
Test QA Plan
(Battelle)
lx
V
i \
r/
Finalize Host/Subcontractor/
Subcontractor Roles
(Battelle)
To Testing
V
Figure 2 Systematic Planning of Verification Tests
Solicit vendor and prepare vendor agreements to participate in verification of their
products based on the test/QA plan
Prepare test/QA plan(s) which are developed to promote uniform testing for a given
type of technology
Involve host facilities, collaborator organizations, and any subcontracted laboratories in
the planning process
Coordinate the review and revision of the test/QA plan(s) (by vendors, EPA, and peer
reviewers) keeping in mind both the customer and EPA objectives for verification as
defined in the ETV Strategy
Prepare final test/QA plans after testing a given type of technology
-------
Section 2
Page 14 of 20
Version 7.0
Date: 11/17/08
• Prepare amendments and deviations to the test/QA plan, as necessary, to include
revisions based on actual test experience.
Systematic planning process-control documents for the AMS Center include:
• The ETV Program Policy Compendium
• The ETV Program QMP
• This QMP which defines the operational quality system necessary to provide
acceptable products and services.
• Written quality procedures specific to the technology and verification test including
test/QA plans and Standard Operating Procedures.
• Outputs from stakeholder committee meetings in the form of reviewed and distributed
minutes.
• Quarterly financial and progress reports to the EPA ETV program.
2.7.3 Planning Personnel
Verification test planning shall be coordinated by Battelle among the participating
organizations including EPA, the stakeholders, the vendors, and any testing organizations
and laboratories participating in the test. Battelle, with the concurrence and oversight of
the EPA AMS Center Project Officer, shall identify the planning roles of the participants,
and shall conduct planning activities by shared communication via e-mail, teleconference,
video conference, and in-person meetings, as appropriate, and within the constraints of
budget.
2.7.4 Existing Data
Existing data may be used according to the procedures described in the test/QA plan for
each verification test and in accordance with the ETV QMP.
2.8 DESIGN OF TECHNOLOGY VERIFICATION OPERATIONS
2.8.1 Design Process
The design process produces a test/QA plan based upon the data quality objectives for the
verification.
2.8.1.1 Design Technique. In designing verification tests, Battelle staff use consensus-
accepted test design including statistical methods, as appropriate. The design
takes into account constraints of time, scheduling, and resources. All relevant
activities pertaining to environmental data operations shall be identified, as well as
performance specifications and the appropriate controls.
2.8.1.2 Field and Laboratory Equipment and Methods. During the design process, the
appropriate field and laboratory equipment which were identified during planning
for the testing of the technology verification performance, are incorporated.
Appropriate test methods and operating parameters are specified.
-------
Section 2
Page 15 of 20
Version 7.0
Date: 11/17/08
2.8.1.3 Sampling and Analysis. If samples for analysis are taken in the field, they are
handled according to procedures specified in the test/QA plan. The oversight
responsibility of Battelle is to determine that the approved systems and plans
contain adequate procedures for handling, storage, cleaning, packaging, shipping,
and preservation of field and laboratory samples to prevent damage, loss,
deterioration, artifacts, or interferences. Battelle will provide adequate chain of
custody procedures, if they are required. The following sampling and analysis
design parameters should be addressed in the test/QA plan.
• Experiments to be conducted, the baseline parameters, the number of replicate
tests, and the controls.
• Sampling methods, sample types, numbers, quantities, handling, packaging,
shipping, and custody (if sampling is performed).
• Sample locations, storage conditions, and holding times.
• Analysis methods, quantitative measures of performance, calibration
standards, calibration check standards, and performance evaluation samples,
as appropriate, and as identified in the planning process.
• Methods and procedures to ensure the test produces traceable data of known
and acceptable quality.
• Field and/or laboratory QA/QC activities.
• Requirements for qualifications of technical staff responsible for obtaining,
analyzing, and evaluating the data.
• Protection of the health and safety of testing personnel and the public.
• Procedures for the minimization and disposal of waste generated.
2.8.1.4 Assessments. Assessments incorporated into the design include self-assessments
(internal audits) by Battelle and independent assessments by EPA. The
assessments identified in the planning process are incorporated into the design.
The type and minimum number of assessments are identified in Section 3.0.
2.8.2 Verification Protocol, Test/QA Plans, and Standard Operating Procedures
Three types of planning documents have been identified for operation of an ETV Center:
the verification protocol, the test/QA plan, and Standard Operating Procedures (SOPs). The
verification protocol is meant to promote uniform testing for a technology category, and
therefore, is a more general document. The test/QA plan gives the specific information
needed to conduct a verification test. If another level of detail is required for describing
test activities, for example operation of an instrument, an SOP will be written and attached
to the test/QA plan.
2.8.2.1 Verification Protocol. The Battelle AMS Center Manager will be responsible for
assuring that the verification protocols are prepared and transferred to the EPA
AMS Center Quality Manager and stakeholders for review. The issues that may
be addressed in the verification protocol are the following:
• General description of the Center
• Responsibilities of all involved organizations
• Experimental design
• Equipment capabilities and description
-------
Section 2
Page 16 of 20
Version 7.0
Date: 11/17/08
• Description and use of field test sites
• Description and use of laboratory test sites
• QA/QC
• Data handling
• Requirements for other documents
• Health and safety
• References.
2.8.2.2 Test/QA Plans. Test/QA plans are the responsibility of the Battelle AMS Center
Manager and are reviewed by the Battelle AMS Center Quality Manager and EPA
AMS Center Quality Manager. The appropriate verification protocol is
incorporated by reference. Appropriate guidance for writing test/QA plans is
available in EPA/QA G-5, Guidance for Quality Assurance Project Plans,
December 2002. Planned changes to the test/QA plan are made by amendment.
Deviations from the plan must be fully documented including, date and
description of deviation, and impact on the verification test. Amendment and
deviation forms are in Appendix III. Elements of the test/QA plan may include
the following. Elements listed that are not appropriate for the test will be listed as
such:
• Group A: Verification Management - This group of elements covers the
general areas of verification management, verification history and objectives,
and roles and responsibilities of the participants. The following nine elements
ensure that the verification's goals are clearly stated, that all participants
understand the goals and the approach to be used, and that verification planning
is documented:
Al Title and Approval Sheet
A2 Table of Contents and Document Control Format
A3 Distribution List
A4 Verification Organization and Schedule
A5 Problem Definition/Background
A6 Verification Description
A7 Quality Objectives and Criteria for Measurement Data
- A8 Special Training Requirements/Certification
- A9 Documentation and Records
• Group B: Measurement/Data Acquisition - This group of elements covers all of
the aspects of measurement system design and implementation, ensuring that
appropriate methods for sampling, analysis, data handling, and QC are
employed and will be thoroughly documented:
B1 Sampling Process Design (Experimental Design)
- B2 Sampling Methods Requirements
B3 Sample Handling and Custody Requirements
B4 Analytical Methods Requirements
- B5 Quality Control Requirements
B6 Instrument/Equipment Testing, Inspection, and Maintenance
Requirements
-------
Section 2
Page 17 of 20
Version 7.0
Date: 11/17/08
- B7 Instrument Calibration and Frequency
- B8 Inspection/Acceptance Requirements for Supplies and Consumables
B9 Data Acquisition Requirements (Non-Direct Measurements)
- BIO Data Management
• Group C: Assessment/Oversight - The purpose of assessment is to ensure that
the test/QA plan is implemented as prescribed. This group of elements
addresses the activities for assessing the effectiveness of the implementation of
the verification and the associated QA/QC activities:
Cl Assessments and Response Actions
C2 Reports to Management
• Group D: Data Validation and Usability - Implementation of Group D elements
ensures that the individual data elements conform to the specified criteria, thus
enabling reconciliation with the verification's objectives. This group of
elements covers the QA activities that occur after the data collection phase of
the verification has been completed:
D1 Data Review, Validation, and Verification Requirements
- D2 Validation and Verification Methods
- D3 Reconciliation with Data Quality Objectives
2.8.2.3 Standard Operating Procedures. The follow topics, from EPA QA/G-6, Guidance
for Development of Standard Operating Procedures (SOPs), April 2007, may be
included (or a reference provided) in standard operating procedure that are
prepared for AMS Center verification tests:
• Title Page
• Table of Contents
• Procedures - The following are topics that may be appropriate for inclusion in
technical SOPs. Not all will apply to every procedure or work process
detailed.
Scope & Applicability
Summary of Method
Definitions
Health & Safety Warnings (indicating operations that could result in
personal injury or loss of life)
Cautions (indicating activities that could result in equipment damage,
degradation of sample, or possible invalidation of results)
Interferences (describing any component of the process that may
interfere with the accuracy of the final product)
Personnel Qualifications
Equipment and Supplies
Procedure (identifying all pertinent steps, in order, and materials needed
to accomplish the procedure such as:
• Instrument or method calibration and standardization
• Sample Collection
• Sample Handling and Preservation
-------
Section 2
Page 18 of 20
Version 7.0
Date: 11/17/08
• Sample Preparation and Analysis
• Troubleshooting
• Data Acquisition, Calculations & Reduction
• Requirements for Computer Hardware & Software used in Data
Reduction and reporting
Data and Records Management
Quality Control and Quality Assurance Section
References
2.9 IMPLEMENTATION
2.9.1 General
Technology performance verifications are implemented according to the test/QA plans and
technical documents (e.g., Standard Operating Procedures) prepared during planning. A
kick-off meeting will be held prior to the start of each verification test to review procedures
for the test with all verification testing staff. The kick-off meeting checklist is provided in
Appendix IV. Test personnel have access to the approved planning documents, approved
changes to planning documents, and all referenced documents. When a prescribed
sequence for the work is defined during the planning stages, work performed shall follow
that sequence. Changes to that sequence need to be documented by either amendment
(planned changes) or deviation (unplanned changes). All implementation activities are
documented. Suitable documents are bound notebooks (e.g. laboratory record books, or
LRBs), field and laboratory data sheets, spreadsheets, computer records, and output from
instruments (both electronic and hardcopy). All documentation is implemented as
described in the planning documents. All implementation activities are traceable to the
planning documents and traceable to test personnel.
2.9.1.1 Conformance of implementation to planning is accomplished by following
approved documents for the Battelle quality system implementation, verification
testing, and for any field and laboratory technical operations.
Generation of verification test data will not be initiated until the approved test/QA
plan is in place.
When work cannot be implemented according to the approved planning and test
document, Battelle shall be responsible for providing a written amendment to the
test/QA plan or a deviation report for the test records. Amendments are produced
for changes that are made to the test/QA plan before the proposed change will be
made. Amendments must be approved internally by the Battelle Verification
Testing Leader, Battelle AMS Center Quality Manager, and AMS Center
Manager. Following approval, the amendment will be distributed to all internal
personnel holding a copy of the parent test/QA plan and the EPA AMS Center
Quality Manager. A deviation report is produced for any changes to the test/QA
plan that occurred during the test. Deviation reports must be retained in the
verification test records and summarized in the verification test report. Frequent
deviations from established procedures should result in a retrospective review of
the written document and possible revision. Amendments and deviations will
include all the information displayed on the forms shown in Appendix III.
-------
Section 2
Page 19 of 20
Version 7.0
Date: 11/17/08
All persons responsible for performing verification testing and those participating
vendors shall receive copies of the current revision of the test/QA plan and
associated documentation provided by Battelle.
Current versions of test/QA plans and any applicable methods and SOPs are
required to be physically in place at each technology verification testing site.
2.9.1.2 Battelle oversight and inspection of a verification test shall be provided by the
Battelle AMS Center Quality Manager or designee at intervals prescribed in each
test/QA plan. This frequency, at a minimum, will be once for each verification
test of a technology category. To verify full implementation of the test/QA plan,
the inspection will include the testing process and any documentation associated
with the process, such as sample tracking records; instrument maintenance and
calibration; sample preparation and actual analysis; and data records. The Battelle
AMS Center Quality Manager will provide a written report, verify the completion
of any corrective actions needed, and retain a copy of the report with permanent
Battelle AMS Center Quality Manager records. The EPA AMS Center Project
Officer will be included in the routing of the inspection results and a written copy
provided to both the EPA AMS Center Project Officer and EPA AMS Center
Quality Manager.
2.9.2 Implementation Procedures
2.9.2.1 Testing procedures shall be documented in approved test/QA plans and SOPs.
Testing personnel, by virtue of training requirements described in this QMP, shall
demonstrate proficiency of performance and knowledge of QA and AMS Center
requirements for the verification test operations.
2.9.2.2 Content requirements for testing procedures may include those of existing Battelle
SOPs or other referenced documents.
2.9.2.3 Before the initiation of testing, a test kickoff meeting will be held by the
Verification Test Coordinator. The Battelle AMS Center Manager, Verification
Testing Leader, Battelle AMS Center Quality Manager, and all Battelle technical
staff who will be utilized for the verification test will attend the kickoff meeting.
Subjects to be discussed at the meeting will include, but not be limited to, a
general overview of the test/QA plan, staff assignments, schedules, and
assessments (see Section 3.0). A separate meeting may be held with external staff
(e.g. collaborators, external personnel, vendors, etc...) prior to the test start to
discuss schedules,
2.9.2.4 Review of technical Center-specific procedures shall be done by personnel
technically competent with respect to the procedure. Time must be allowed for the
composition, review, and approval of technical procedures to be completed in
advance of the actual performance.
-------
Section 2
Page 20 of 20
Version 7.0
Date: 11/17/08
2.9.3 Implementation Monitoring
2.9.3.1 Quality assessments during implementation of individual verification tests will be
prescribed at a minimum frequency/interval in the test/QA plan. Specifically, the
test/QA plan will address:
• A routine monitoring schedule and,
• The required specifications of performance, or particular aspects of the
process, that are determined to be critical for monitoring
2.9.3.2 Monitoring of the work process is conducted by the Battelle AMS Center Quality
Manager or designee and is done to:
• Ensure satisfactory performance based on requirements,
• Ensure required actions (as specified in implementation documents) are
performed so that routine measurements meet specifications,
• Ensure preventive maintenance is performed and documented as specified in
facility and study records,
• Ensure calibrations are performed as planned and prescribed,
• Ensure corrective actions are implemented and documented as planned in
response to items of nonconformance.
-------
Section 3
Page 1 of 7
Version 7.0
Date: 11/17/08
3.0 ASSESSMENT AND RESPONSE
3.1 SCOPE
3.1.1 Assessments shall be planned, scheduled, conducted, and reported in order to measure the
efficacy of the Battelle quality system.
3.1.2 Assessment and response elements shall include assigning appropriate, qualified persons to
conduct assessments at planned, scheduled intervals; having provisions for timely
responses and implementation of corrective actions if needed; and completing the
evaluation process with written reports to technical and management staff.
3.1.3 Assessment types, responsibility, and schedule for the AMS Center as shown in Table 3.0,
and are defined as follows:
Quality Systems Audit, an on-site review of the implementation of the AMS Center quality
system as documented in the AMS Center QMP. This review is used to verify the existence
of, and evaluate the adequacy of, the internal quality system. A Quality Systems Audit
(QSA) may be a self-assessment or an independent assessment by EPA.
Technical Systems Audit, a qualitative on-site evaluation of sampling and/or measurement
systems associated with a particular verification test. The objective of the Technical
Systems Audit (TSA) is to assess and document the acceptability of all facilities,
maintenance, calibration procedures, reporting requirements, sampling, and analytical
activities, and quality control procedures in the test. Conformance with the test/QA plan
and associated methods and/or Standard Operating Procedures is the basis for this
assessment. The Battelle AMS Center Quality Manager, or designee, conducts a TSA at
least once during each verification test. The EPA AMS Center Quality Manager conducts
an independent TSA once per year, as applicable, for the AMS Center.
Performance Evaluation Audits, a quantitative evaluation of a measurement system. The
type and frequency of performance evaluation self-audits to be performed by the Battelle
Verification Testing Leader or designee are specified in the test/QA plan for each
verification test. The value or composition of reference materials must be certified or
verified prior to use, and the certification or verification must be adequately documented.
The Battelle AMS Center Quality Manager, or designee, will review results of PE audits.
The need for independent performance evaluation audits will be determined by the EPA
AMS Center Quality Manager.
Audits of Data Qualityi an examination of the verification data after they have been
collected and 100% verified by project personnel. The Battelle AMS Center Quality
Manager, or designee, will audit at least 10% of all verification data, including equations
and calculations. The need for independent audits of data quality will be determined by the
EPA AMS Center Quality Manager.
-------
Section 3
Page 2 of 7
Version 7.0
Date: 11/17/08
Table 3.0 Assessments for the AMS Center
Assessment Respond Subject of Minimum Reason for
Level Tool Assessors ers Assessment Frequency Assessment Report Reviewed by
Center
Center
Center
Center
Quality
Systems
Audit
Technical
Systems
Audits
Performance
Evaluation
Audits
Audits of
Data Quality
M
BattelleAMS
Center Quality
Manager
Independent
EPA AMS Center
Quality Manager
M
BattelleAMS
Center Quality
Manager
Independent
EPA AMS Center
Quality Manager
Self
BattelleAMS
Center Quality
Manager
Independent
EPA AMS Center
Quality Manager
M
BattelleAMS
Center Quality
Manager
Independent
EPA AMS Center
Quality Manager
Battelle
Battelle
Battelle
Battelle
center QMP
test/QA plans
test/QA plans
raw data and
summary data
once;
thereafter, as
requested
M
Once per
verification test
Independent
once per year,
as applicable
M
each test, as
applicable
Independent
for each center,
as applicable
M
At least 10% of
the verification
data
Independent
for each center,
as applicable
assess quality
management
practices of
verification
partner
assess
technical
quality of
verification
tests
assess
measurements
performance
assess data
calculations
and reporting
EPA directors of
quality assurance
EPA AMS Center
Project Officer
Battelle AMS Center
Manager
ETV Program Director
EPA AMS Center
Project Officer
EPA AMS Center
Quality Manager
Battelle AMS Center
Manager
Self
Battelle AMS Center
Manager
Independent
EPA AMS Center
Project Officer
EPA AMS Center
Quality Manager
M
Battelle AMS Center
Manager
Independent
EPA AMS Center
Project Officer
EPA AMS Center
Quality Manager
3.2 GENERAL REQUIREMENTS
3.2.1 Each assessment must be fully documented. The Battelle AMS Center Quality Manager
will archive all internal assessment reports generated on the AMS Center.
Each assessment must be responded to by the appropriate level of management. The
Battelle quality assessment reports shall require a written response by the person
performing the inspected activity, and acknowledgment of the assessment by the Battelle
Verification Testing Leader and the Battelle AMS Center Manager. The assessment
reporting forms are provided in Appendix V.
3.2.2 Corrective action must be documented and approved on the original assessment report, with
detailed narrative in response to the assessor's finding. Initials and date are required for
-------
Section 3
Page 3 of 7
Version 7.0
Date: 11/17/08
each corrective action response. Acknowledgment of the response will be provided by the
Battelle Verification Testing Leader and Battelle AMS Center Manager.
3.2.3 Implementation of corrective actions must be verified by the Battelle AMS Center Quality
Manager or designee to ensure that corrective actions are adequate and have been
completed. This will be done in real-time if corrective actions can be immediately
performed and signed off on the assessment report. Alternatively, should the corrective
action require additional approvals not immediately available on-site, the Battelle AMS
Center Quality Manager or designee may need to repeat the inspection in order to
corroborate the implementation and effectiveness of the corrective action.
3.3 PLANNING AND PROCEDURES
3.3.1 Assessment Planning
Assessment planning is performed by Battelle's Quality and AMS Center Managers prior
to the actual performance of any assessments. Planning the assessment scope helps provide
the type of evaluation information needed to determine whether procedural compliance and
technical requirements are being met during verification testing.
Assessment planning by Battelle shall include a kickoff meeting with the verification
testing team where at least the following information may be discussed:
• Schedule of assessment(s),
• Proper completion of data records
• Notification to affected parties,
• Specific assessment requirements (personnel lists, equipment lists, and availability of
test/QA plans),
• Follow-up procedures for corrective action, including debriefing and discussion of
possible resolutions,
• Corrective action guidelines to facilitate completion of the reported assessment,
• Appropriate management signature approval of the reviewed assessment report.
The kick-off meeting checklist is included in Appendix IV.
3.3.2 Personnel Qualifications for Assessment
The principal Battelle inspector shall be the Battelle AMS Center Quality Manager, who
will have an extensive quality assurance laboratory and field inspection background, and
technical and management experience, and who will be directly familiar with the AMS
Center assessment requirements. Should the need arise, the Battelle AMS Center Quality
Manager will designate an individual to perform scheduled assessments, based upon that
person's technical skill and knowledge of QMP compliance requirements and test/QA plan
specifications. Battelle personnel conducting assessments shall have the responsibility and
authority to:
• identify and document problems affecting the quality of verification results,
-------
Section 3
Page 4 of 7
Version 7.0
Date: 11/17/08
• propose recommendations for resolving these problems,
• independently confirm implementation and effectiveness of solutions.
3.3.3 Stop Work
Assessor responsibility and authority to stop work during a verification test for safety and
quality considerations is delegated to Battelle, who must ensure compliance with all
applicable federal, state, and local safety policies during the performance of verification
testing.
Should it be determined during an assessment that adverse health effects could result, or
that test objectives of acceptable quality cannot be achieved during performance of
verification testing, the Battelle AMS Center Quality Manager is responsible for
immediately notifying the Battelle Center Manager of the need to consider a stop work
order. The Battelle AMS Center Manager shall then direct the AMS Center staff
accordingly.
Should any AMS Center staff suspect compromise to personal health or test objectives
during the conduct of verification testing, that staff member shall immediately contact the
Battelle Verification Testing Leader, who shall through vested authority from the Battelle
AMS Center Manager, issue the stop work order and subsequently notify the Battelle AMS
Center Manager.
The EPA AMS Center Quality Manager is delegated to notify the EPA AMS Center Project
Officer who will notify the Battelle AMS Center Manager to facilitate a stop work order if
work of inadequate quality is discovered.
Documentation is required of any stop work order and the corrective action implemented
and shall be maintained as part of the Battelle quality records, with a copy provided to the
EPA AMS Center Project Officer and EPA AMS Center Quality Manager.
3.3.4 Internal Assessment Reporting
Authority to effectively report internal technical system audits, performance evaluation
audits, and audits of data quality is assigned to the Battelle AMS Center Quality Manager
or designee. These reports should:
• Identify and document problems that affect quality and the achievement of objectives
required by the QMP, test/QA plan, and any associated Standard Operating Procedures,
• Identify and cite noteworthy practices that may be shared with others to improve the
quality of their operations and products,
• Propose recommendations (if requested) for resolving problems that affect quality,
• Independently confirm implementation and effectiveness of solutions,
• Provide documented assurance (if requested) to line management that, when problems
are identified, further work performed is monitored carefully until the problems are
suitably resolved.
-------
Section 3
Page 5 of 7
Version 7.0
Date: 11/17/08
3.3.5 Response
Responses to TSA adverse findings should be addressed within 10 working days after the
TSA report is completed. However, it is expected that findings that have a direct impact on
the conduct of a verification test will be corrected immediately following notification of the
finding.
Responses to each adverse finding shall be documented in the assessment report (QMP
Section 3.3.4). Ideally, assessment reports will provide space after each adverse finding for
a response to be recorded. The response will indicate the corrective action taken or planned
to address the adverse finding. The response should be signed and dated by the staff
responsible for implementing the corrective action.
Any corrective action that cannot be immediately implemented should be verified
following completion by the Battelle AMS Center Quality Manager or designee. Once all
corrective action associated with an assessment report has been taken, the Battelle AMS
Center Quality Manager or designee will initial the corrective action in the assessment
report thus documenting verification of the corrective action. Any impact that an adverse
finding had on the quality of verification test data should be addressed in the verification
test report.
The TSA report, with responses to adverse findings recorded within, will be sent to EPA
within 10 working days after the Battelle AMS Center Quality Manager has verified all
corrective actions.
3.4 DATA VALIDATION
Validation is based on the performance measures for the test specified during the design process.
The usability of a verification report and statement is determined relative to how well it
determines the performance of the tested technology under the conditions of testing. Any
limitations on the data and recommendations for limitations on data usability are documented in
the data audit report and the ETV verification report.
3.5 REPORT REVIEW
Review and approval procedures for verification reports and statements are given in Table 2.0.
Verification reports are peer-reviewed by external reviewers and verification statements are
signed by an EPA laboratory director and Battelle management.
3.6 QUALITY IMPROVEMENT
3.6.1 Policy
A continuous quality improvement process is considered essential for Battelle staff to develop
a more responsive quality system in all aspects of technical and management activities.
-------
Section 3
Page 6 of 7
Version 7.0
Date: 11/17/08
3.6.2 Annual QMP Review
An annual review of the QMP for the AMS Center shall be conducted by the Battelle AMS
Center Quality Manager and technical and management staff in order to incorporate
improvements to the quality system process. AMS Center QMP revisions may be delayed
beyond one year if an update to the ETV QMP is pending. Approval for the delay will be
obtained from the EPA AMS Center Quality Manager.
Any revisions to the QMP will be compiled by the Battelle AMS Center Quality Manager for
review, approval, and distribution. The QMP review will be documented by the Battelle AMS
Center Quality Manager and Battelle AMS Center Manager by signing and dating the revised
QMP routed for review and approval.
3.6.3 Problem Identification and Resolution
Detecting and correcting quality system problems is a result of qualified AMS Center technical
and management staff implementing not only this QMP, but also the test/QA plan and other
procedures. All staff are encouraged to identify problems and offer solutions to problems in
the following quality areas:
• Adequacy of the quality system, as defined in the QMP,
• Consistency of the quality system,
• Implementation of the quality system to specific verification tests,
• Correction of quality system procedures,
• Completeness of documented information,
• Quality of data,
• Quality of planning documents, such as the test/QA plans,
• Implementation of the work process.
Cause and effect relationships of significant problems shall be documented by the Battelle
AMS Center Quality Manager. When problems are reported to the Battelle AMS Center
Quality Manager, attempts to determine the root cause based on cause and effect during
performance of planned and documented procedures will be made, through intensified
observations of testing activities and audits of test data. When problems are identified for
the quality system, the Battelle AMS Center Manager will contact the EPA AMS Center
Project Officer of the problem(s) and corrective action(s).
Collaboration with trained technical/management staff associated with or performing the
activity can provide insight and determine whether any of the following is required:
• A test/QA plan change,
• A management system change, or
• A quality system change within the AMS Center.
Assessment reports can also serve as tools to determine cause and effect relations of
significant problems that might require testing protocol, management system, or quality
system changes. Continual monitoring and evaluation by the EPA AMS Center Quality
-------
Section 3
Page 7 of 7
Version 7.0
Date: 11/17/08
Manager, for example, may indicate trends or common and recurring problems for an entire
technology evaluation. In this case, the situation is immediately communicated to the ETV
Program Director, who then provides information and any corrective actions to the EPA
AMS Center Project Officer.
Root cause determination is immediately reported by Battelle to the EPA AMS Center
Project Officer prior to any planned implementation of preventative measure. Once the
root cause determination is verified, appropriate actions can be planned, documented, and
implemented by the AMS Center staff.
3.6.4 Ongoing Quality Improvement
Quality improvement action is ongoing in the Battelle quality system, where quality issue
action items can be reviewed by all levels of line management at periodic continuous
improvement meetings. Quality processes are continually monitored and both short-term
and long-term quality issues are identified through customer feedback and client
involvement, peer review and internal lessons learned, and program reviews.
-------
APPENDIX I
NAMES, ADDRESSES, AND PHONE NUMBERS OF
BATTELLE AMS CENTER KEY STAFF
-------
KEY BATTELLE AMS CENTER STAFF
Battelle Center Manager:
Ms. Amy Dindal
1801 Waldorf Dr.
Royal Palm Beach, FL 33411
Phone:561-422-0113
Fax: 614-458-6697
e-mail: dindala@battelle.org
Battelle Quality Manager:
Mr. Zachary Willenberg
505 King Avenue
Columbus, OH 43201
Phone: 614-424-5795
Fax: 614-458-5795
e-mail: willenbergz@battelle.org
Verification Testing Leader:
Dr. Thomas Kelly
505 King Avenue
Columbus, OH 43201
Phone: 614-424-3495
Fax: 614-458-3495
e-mail: kellvt@battelle.org
Stakeholder Coordinator:
Ms. Rachel Sell
505 King Avenue
Columbus, OH 43201
Phone: 614-424-3579
Fax: 614-458-3579
e-mail: sellr@battelle.org
-------
APPENDIX II
ETV VERIFICATION STATEMENT
-------
THE ENVIRONMENTAL TECHNOLOGY VERIFICATION
PROGRAM
&EPA PT%r
U.S. Environmental Protection Agency ".I I/ JL
Baireiie
BuSmCSS O/ Innovation
ETV Joint Verification Statement
TECHNOLOGY TYPE: CHEMILUMINESCENT OZONE ANALYZER
APPLICATION: MEASURING OZONE IN AMBIENT AIR
TECHNOLOGY NAME: 3.02 P-A
COMPANY: JSC OPTEC Ltd.
ADDRESS: 1985 West Henderson Road PHONE: (614) 477-1020
Columbus, Ohio 43220 FAX: (614) 486-2693
WEB SITE: www.optec-corp.com
E-MAIL: mkozliner@gmail.com
The U.S. Environmental Protection Agency (EPA) has established the Environmental Technology
Verification (ETV) Program to facilitate the deployment of innovative or improved environmental
technologies through performance verification and dissemination of information. The goal of the ETV
Program is to further environmental protection by accelerating the acceptance and use of improved and
cost-effective technologies. ETV seeks to achieve this goal by providing high-quality, peer-reviewed
data on technology performance to those involved in the design, distribution, financing, permitting,
purchase, and use of environmental technologies. Information and ETV documents are available at
www.epa.gov/etv.
ETV works in partnership with recognized standards and testing organizations, with stakeholder groups
(consisting of buyers, vendor organizations, and permitters), and with individual technology developers.
The program evaluates the performance of innovative technologies by developing test plans that are
responsive to the needs of stakeholders, conducting field or laboratory tests (as appropriate), collecting
and analyzing data, and preparing peer-reviewed reports. All evaluations are conducted in accordance
with rigorous quality assurance (QA) protocols to ensure that data of known and adequate quality are
generated and that the results are defensible.
The Advanced Monitoring Systems (AMS) Center, one of five verification centers under ETV, is
operated by Battelle in cooperation with EPA's National Exposure Research Laboratory. The AMS
Center evaluated the performance of a chemiluminescent ozone analyzer, a continuous monitor for
determining ozone in air. This verification statement provides a summary of the test results for the JSC
Optec Ltd. 3.02 P-A ozone analyzer.
-------
VERIFICATION TEST DESCRIPTION
The objective of this verification test was to evaluate the 3.02 P-A performance for determining ozone
in air in part by comparing it to the response of the ultraviolet (UV)-absorption Federal Equivalent
Method (FEM) for ozone. The specific commercial FEM monitor used in this test was the Thermo
Environmental Model 49 C (FEM EQOA-880-047). The verification test was conducted between
June 13 and June 28, 2007 at Battelle laboratories located in Columbus, Ohio.
The response of the 3.02 P-A to ozone in ambient air and at low (< 5%) and high (~ 80%) relative
humidity (RH) in clean air was used to evaluate for accuracy, linearity, interference effects,
comparability to the FEM, data completeness, and operational factors. Two 3.02 P-A instruments were
operated, one with no additional equipment and per the manufacturer's recommendations, and the other
with a Nafion humidity equilibration tube added to the monitor's inlet. Results from the first unit are
the primary results of this verification. The Nafion tube was added to the second unit to assess the
impact of humidity and humidity control on the 3.02 P-A response. Both 3.02 P-A units relied entirely
on their internal automated calibration systems, as specified by the vendor.
Accuracy was calculated from the response of the 3.02 P-A with respect to different levels of ozone
challenges, established either by a known ozone source or by FEM response. Linearity was assessed by
a linear regression analysis using the FEM reading as the independent variable and the response of the
3.02 P-A as the dependent variable. Interferences tested were naphthalene, o-nitrophenol, and p-
tolualdehyde, each at approximately 6 to 15 parts per billion by volume (ppbv) in both dry and
humidified air; mercury vapor at 630 nanograms per cubic meter (ng/m3) in dry air and 54 ng/m3 in
humidified air; nitrogen dioxide (NO2) at up to 200 ppbv in humidified air; and a 17-component mix of
volatile organic compounds at up to 593 ppbv total concentration in humidified air. Interference effects
were calculated in terms of the ratio of the 3.02 P-A response to the actual concentration of the
interferent. Comparability was assessed by comparing the 3.02 P-A response to that of the FEM during
generation of ozone in photochemical chamber tests at 80% RH, and in ambient ozone monitoring.
Data completeness was assessed as the percentage of maximum data return achieved by the 3.02 P-A
over the test period. Operational factors were evaluated by means of observations during testing and
records of needed maintenance, vendor activities, and expendables use.
QA oversight of verification testing was provided by Battelle and EPA. Battelle QA staff conducted a
technical systems audit, a performance evaluation audit, and a data quality audit of 10% of the test data.
EPA QA staff also conducted an on-site technical systems audit. This verification statement, the full
report on which it is based, and the test/QA plan for this verification test are all available at
www. epa.gov/etv/centers/centerl. html.
TECHNOLOGY DESCRIPTION
The following description of the 3.02 P-A was provided by the vendor and does not represent verified
information.
The 3.02 P-A chemiluminescent ozone analyzer combines a solid phase chemiluminescence approach
with menu-driven software with diagnostic functions. It is designed to measure ozone concentrations in
ambient air.
The Model 3.02 P-A is designed to have the following features:
- Automatic continuous measurements
- Automatic internal calibration
-------
- High sensitivity
- Fast response time
- Linearity
- Menu driven software
- Digital display
- Electronic data output
The 3.02 P-A detects ambient ozone by means of its chemical reaction with a solid-phase reactant of
proprietary composition, resulting in the emission of light with peak intensity near 560 nm wavelength.
The emitted light is detected by a photomultiplier tube, and converted to a digital signal that is linearly
proportional to the gaseous ozone concentration. An internal pump draws sample air through two
alternate flow paths: in the measurement path sample air passes directly into contact with the solid-
phase reactant, whereas in the zeroing path ozone in the sample air is removed by a selective scrubber
element before the air contacts the reactant. The 3.02 P-A thus measures ozone by comparison of the
signals from these two paths. An internal ozone generator (UV lamp) provides a calibration mixture to
the 3.02 P-A at 10-minute intervals, and the internal software automatically adjusts instrument response
with each calibration. The measured ozone concentration is displayed on the front panel of the 3.02 P-A
and can be transmitted via analog outputs. The estimated price of the base model analyzer is $5,000.
VERIFICATION RESULTS
The addition of the Nafion humidity equilibration tube to the inlet of one unit of the 3.02 P-A reduced
the performance of that unit in nearly all performance measures. The performance of the 3.02 P-A unit
operated normally is summarized below.
Accuracy: The accuracy of the Optec 3.02 P-A ozone analyzer was assessed in terms of percent
recovery (%R). The 3.02 P-A averaged a %R of 85.4 to 107.7% at concentrations of 98 to 289 parts
ppbv when ozone was delivered to the analyzer in dry air directly from a calibration source. When
ozone was added to clean air in the test chamber in stepwise concentrations of 51 to 257 ppbv at high
humidity (~ 80% RH), the average %R was 93.2 to 110%. In the corresponding chamber test at
concentrations of 69 to 260 ppbv in dry test conditions (< 5% RH), the average %R of the 3.02 P-A was
83.4 to 88.3%.
Linearity: Linearity was evaluated in terms of slope, intercept, and coefficient of determination (r2).
The linearity of the 3.02 P-A under ~ 80% RH test conditions showed a slope of 0.914, an intercept of
6.2 ppbv, and an r2 value of 0.998 over a concentration range of 51 to 257 ppbv. The linearity of the
3.02 P-A in < 5% RH conditions showed a slope of 0.838, an intercept of 1.0 ppbv, and an r2 value of
0.999 over a concentration range of 69 to 260 ppbv.
Interference Effects: None of the interferents tested caused any response on the 3.02 P-A. As a result,
all interferent response ratios were zero.
Comparability: Comparability was evaluated in terms of the slope, intercept, and r2 of a linear
regression of 3.02 P-A readings against FEM readings. The comparison between the 3.02 P-A and FEM
during the photochemical ozone test with high precursor concentrations resulted in a slope of 0.815, an
intercept of 4.1 ppbv, and an r2 value of 0.999 over an ozone concentration range of 20 to 130 ppbv.
The same comparison in the photochemical test with lower precursor concentrations resulted in a slope
of 0.941, an intercept of-1.2 ppbv, and an r2 value of 0.997 over an ozone concentration range of 20 to
80 ppbv. The comparison of the 3.02 P-A and FEM readings over the four day ambient monitoring
-------
period resulted in a slope of 0.998, an intercept of 0.19 ppbv, and an r2 value of 0.995 over an ozone
concentration range of 3 to 80 ppbv. The average relative percent difference of the 3.02 P-A with
respect to the FEM in these three comparisons was -14.2%, -8.0%, and 0.3%, respectively.
Data Completeness: Data completeness for the 3.02 P-A was 100%, based on its operation over a total
of 6.07 test days during a 16 day operational period. Considering only those 6.07 days on which the
3.02 P-A was tested, there were 4.6 days of ambient monitoring, 0.27 days spent in calibration/
zeroing/other instrument checks, and 1.2 days total spent conducting measurements in the
environmental chamber. Both 3.02 P-A units also operated without problems throughout the 16-day
period in which those 6 test days occurred.
Operational Factors: The Optec 3.02 P-A was operated on a 220 V to 120 V converter during testing.
When the 3.02 P-A was turned on, it took approximately 1 hour for the 3.02 P-A to stabilize and it then
remained functional throughout the entire 16-day test period. No repair was needed during the test and
the need for vendor assistance was minimal. The analyzer calibrated itself internally every ten minutes.
The ozone measurements were displayed on the front panel in parts per million. An operating manual
was provided and although translated from Russian to English, the manual was somewhat difficult to
understand. The monitor includes an audible alarm which sounds when ozone readings exceed the
maximum full scale value (i.e., above about 250 ppbv).
Signed by Martin E Toomajian 3/06/08 Signed by Sally Gutierrez 4/15/08
Martin E Toomajian Date Sally Gutierrez Date
Manager Director
Chemical, Environmental, and National Risk Management Research Laboratory
Material Operations Office of Research and Development
Battelle U.S. Environmental Protection Agency
NOTICE: ETV verifications are based on an evaluation of technology performance under specific,
predetermined criteria and the appropriate quality assurance procedures. EPA and Battelle make no expressed or
implied warranties as to the performance of the technology and do not certify that a technology will always
operate as verified. The end user is solely responsible for complying with any and all applicable federal, state,
and local requirements. Mention of commercial product names does not imply endorsement.
-------
APPENDIX III
AMENDMENT AND DEVIATION FORMS
-------
TEST/OA PLAN AMENDMENT
TEST/QA PLAN TITLE AND DATE:
AMENDMENT NUMBER:
EFFECTIVE DATE:
PART TO BE CHANGED/REVISED:
CHANGE/REVISION:
REASON FOR CHANGE:
ORIGINATED BY:
Battelle Verification Test Coordinator
DATE
APPROVED BY:
Battelle Verification Testing Leader
Battelle AMS Center Quality Manager
DATE
DATE
Required Distribution - All individuals/organizations listed on distribution for the
applicable Test/QA Plan, including but not limited to:
Battelle AMS Center Management
Battelle AMS Center Testing Staff
Battelle AMS Center Quality Manager
Subcontractors (if any)
Verification Test Collaborators (if any)
EPA/ETV AMS Center Project Officer
EPA/ETV AMS Center Quality Manager
Vendors
Distribution must be documented
-------
TEST/OA PLAN DEVIATION REPORT
TEST/QA PLAN TITLE AND DATE:
DEVIATION NUMBER:
DATE OF DEVIATION:
DESCRIPTION OF DEVIATION:
CAUSE OF DEVIATION:
IMPACT OF DEVIATION ON THE TEST:
CORRECTIVE ACTION:
ORIGINATED BY:
Battelle Verification Test Coordinator
DATE
ACKNOWLEDGED BY:
Battelle Verification Testing Leader
Battelle AMS Center Quality Manager
DATE
DATE
Required Distribution - All individuals/organizations listed below:
Battelle AMS Center Management
Battelle AMS Center Quality Manager
Distribution must be documented
-------
APPENDIX IV
ETV KICK-OFF MEETING CHECKLIST
-------
ETV VERIFICATION TEST KICK-OFF MEETING
PURPOSE
To prepare verification testing staff for an upcoming test and review critical logistical,
technical, and administrative aspects of the test.
STAFF TO ATTEND
• Verification Test Coordinator
• Verification Testing Leader
• ETV Center Manager
• QA Manager(s)
• All testing staff involved in all. phases of test (include staff from partnering
organizations/subcontractor or conduct separate kickoff meeting with external
staff, if applicable)
TIMING AND LENGTH
• The kick-off meeting should be scheduled prior to the start of testing. It should be
near the start of the test but allow time for the test coordinator to address any
lingering issues.
• A tentative date/time should be set for the kick-off meeting a month in advance of
the start of the test, if possible, to allow the meeting to be scheduled when the
critical staff mentioned above are all available.
• It may be necessary to schedule a second kick-off meeting for tests that are
conducted at multiple locations if all staff cannot attend the original meeting.
PROJECT MANAGEMENT
• Review roles/responsibilities of all staff attending meeting
• Work authorization distributed to all staff? (Internal only)
• All staff have project number and subaccount number(s)? (Internal only)
• All testing staff have budget for their time on each subaccount involved?
(Internal only)
• Stakeholders, EPA/ETV program manager, and EPA/QA staff pre-notified of
testing schedule and start date? (Internal only)
• Review test schedule
• Formal distribution of final, signed hard-copy test/QA plan made to all Battelle
staff, subcontractor (if any), vendors, and EPA?
• Documentation: All pertinent forms should be signed, copied for Amy, and
brought to the meeting.
o Peer review forms on test/QA plan. Must include one EPA reviewer/two
non-EPA peer reviewers.
o Final test/QA plan signed by all vendors? Where are original signature
pages stored? All staff attending meeting should be told to bring their copy
-------
of test/QA plan to kick-off meeting prior to meeting or final test/QA plan
should be distributed at meeting.
o All vendor agreements signed/checks received/copies sent to Amy for
project files?
• If Battelle, collaborator, or subcontractor staff is operating technology, has vendor
trained those staff in operating the technology and signed formal training form
certifying training and vendor's acceptance of Battelle generated results? Where
is original signed form located/filed?
• Subcontractors (if used)
Subcontract fully signed? Provide copy to Amy for project files.
Due dates well defined in subcontract?
- Get test data from subcontractor/partner real time or at close of test
• Has a 508-compliant pdf file been generated of the final test/QA plan and has it
been sent to the ETV web master?
QUALITY ASSURANCE
• Are annual training records of all testing staff on file and up-to-date? Training
memo in place for temporary staff? Do the training records adequately reflect the
abilities of the staff involved for the tasks involved with this verification testing
(e.g., reference sampling or reference sample analysis)?
• Copies of all standard methods cited in test/QA plan available to testing staff and
in laboratory where test will be performed?
• EPA QA staff pre-notified of test start date and external technical systems audit
planned/coordinated? When is external TSA scheduled? Who will perform?
Who has been coordinating with EPA/QA staff? Remind testing staff to not offer
information during external TSA.
• When is internal TSA scheduled? Who will perform?
• Remind testing staff to sign and date everything
• If samples are to be transported between labs, or between
Battelle/partner/subcontractor, bring chain-of-custody form to meeting, review
how to complete, and where to obtain form
• Review deviation/amendment procedures at meeting - what to do in the middle of
a test if test/QA plan cannot be followed - who to notify/what forms to file
• Review test/QA plan at meeting - identify key testing procedures and critical
steps to ensure no ambiguity or questions
• Are or will there be copies of the certificates of analysis in the verification test
binder?
• When will PE audit be performed? Who will perform? Has materials/equipment
been purchased or obtained for the PE audit? What are QC limits? What to do if
QC limits are not met? Who to contact?
• If testing is being performed off-site, will regular communication with the staff at
the test site be maintained? If so, how?
-------
TECHNICAL
• Emphasize to testing staff to document anything and everything that is observed
about the technologies, particularly if there are unusual sample results (e.g.,
sample color).
• Will the samples be blind and randomly distributed to the operators?
• Are provisions made to handle daily preparation of solutions/standards, if
necessary?
• Take digital photos of all test activities.
DATA/REPORTING
• Review data recording forms or sheets at meeting or discuss how/where will data
be recorded for each testing activity
• How are data going to be converted electronically? Are data saved in technology
undergoing verification and then exported to Excel? Or will data be recorded
manually by the operators? If so, how will transcription errors be avoided?
• Data review - who will be doing two week review for each data set collected? If
Battelle staff not on-site, how will data be transmitted to Battelle for two-week
review?
• Who is Battelle verification report author? Distribute and review report
schedule. Reporting should begin at the same time as testing. (Internal only)
• Has the test/QA plan been sent to PMP for report template generation? (Internal
only)
-------
APPENDIX V
ETV ASSESSMENT REPORTING FORMS
-------
Quality Assurance Routing Sheet
ETVAMS Center
Verification Test:
Audit Type:
Test Coordinator:
Vendor:
Auditor: Date:
Test Coordinator, please complete the attached form indicating CORRECTIVE ACTION
TAKEN (IF NEEDED), sign and date this Routing Sheet in the space provided beside
your name, and return the entire set when completed to the Battelle AMS Center Quality
Manager no later than .
Route To Signature Date
Test Coordinator
Battelle AMS Center
Quality Manager
Approval
Testing Leader
AMS Center Manager
Battelle AMS Center
Quality Manager
-------
Verification Test Designation:
QA Auditor:
Audit Type:
Page
of
Audit Date:
Audit Comment Sheet
Instructions: The Battelle AMS Center Quality Manager will fill out the first column for the audit indicated above. The Verification Testing Leader (or assigned
responder such as the Verification Testing Coordinator) will respond to the comments and initial and date the response in column three. The Battelle AMS Center
Quality Manager will verify and document that the response/corrective action has been completed by initialing and dating the final column.
QA Comment
Testing Leader/Coordinator
Response/Corrective Actions
Responder
Initials/ Date
QA Initials/
Date
------- |