Batrene
The Business of Innovation
Environmental Technology
Verification Program
Quality Management Plan (QMP)
for the
ETV Materials Management and
Remediation Center
Version 1.0
-------
QUALITY MANAGEMENT PLAN (QMP)
for the
ETV MATERIALS MANAGEMENT AND REMEDIATION
CENTER
Version 1.0
(SIGNATURE ON FILE)
Teri Richardson 3-13-09
EPA MMR CENTER PROJECT OFFICER Date
Scott Jacobs 3-12-09
EPA MMR CENTER QUALITY MANAGER Date
Amy Dindal 2-23-09
BATTELLE MMR CENTER MANAGER Date
Zachary Willenberg 2-25-09
BATTELLE MMR CENTER QUALITY Date
MANAGER
BATTELLE
505 King Avenue
Columbus, OH 43201
-------
Section TOC
Pagel of 2
Version 1.0
Date: 2/23/09
TABLE OF CONTENTS
Page
1.0 GENERAL PROVISIONS 1
1.1 Introduction 1
1.2 Purpose 1
1.3 Scope and Field of Application 1
1.4 Background 1
1.5 Definitions 4
2.0 MANAGEMENT SYSTEMS 1
2.1 Management and Organization 1
2.2 Quality System and Description 1
2.3 Personnel Responsibilities, Qualifications and Training 2
2.4 Procurement and Acceptance of Items and Services 7
2.5 Documents and Records 7
2.6 Computer Hardware and Software 11
2.7 Planning 12
2.8 Design of Technology Verification Operations 14
2.9 Implementation 18
3.0 ASSESSMENT AND RESPONSE 1
3.1 Scope 1
3.2 General Requirements 2
3.3 Planning and Procedures 3
3.4 Data Validation 5
3.5 Report Review 5
3.6 Quality Improvement 5
LIST OF TABLES
Table 1.0 Personnel Responsibilities for the MMR Center for Verification Testing Activities 4
Table 2.0 Records Management Responsibilities for the MMR Center 10
Table 3.0 Assessments for the MMR Center 2
LIST OF FIGURES
Figure 1.0 MMR Center Organization 3
Figure 2.0 Systematic Planning of Verification Tests 13
-------
APPENDIX I.
Section TOC
Page 2 of 2
Version 1.0
Date: 2/23/09
APPENDICES
NAMES, ADDRESSES, AND PHONE NUMBERS OF BATTELLE
MMR CENTER KEY STAFF
APPENDIX II. AMENDMENT AND DEVIATION FORMS
APPENDIX III. ETV ASSESSMENT REPORTING FORMS
LIST OF ACRONYMS AND ABREVIATIONS
ADQ Audit of Data Quality
CA Cooperative Agreement
COA Certificate of Analysis
EPA U.S. Environmental Protection Agency
ETV Environmental Technology Verification Program
GVP Generic Verification Protocol
LRB Laboratory Record Book
MMR Materials Management and Remediation Center
NSGB National Security Global Business (Battelle)
PE Performance Evaluation
QA Quality Assurance
QC Quality Control
QMP Quality Management Plan
QSA Quality Systems Audit
SOP Standard Operating Procedure
TSA Technical Systems Audit
-------
Section 1
Pagel of 5
Version 1.0
Date: 2/23/09
1.0 GENERAL PROVISIONS
1.1 INTRODUCTION
1.1.1 This document, the Quality Management Plan (QMP) for the Materials Management and
Remediation (MMR) Center, describes the quality systems that will be employed by
Battelle in conducting the MMR Center. These quality systems are designed to be
consistent with ANSI/ASQ E4-1994, "Specifications and Guidelines for Quality Systems
for Environmental Data Collection and Environmental Technology Programs," the U. S.
Environmental Protection Agency (EPA) document "Environmental Technology
Verification Program Quality Management Plan", Version 3.0, dated January 2008, and
EPA Requirements for Quality Management Plans (QA/R-2, dated March 2001).
1.2 PURPOSE
1.2.1 The purpose of the MMR Center is to verify materials management technologies, including
recycling, beneficial use of waste materials, recovery of useful components of wastes, and
treatments to minimize disposal requirements (e.g., containment, volume, cost). The center
will also verify technologies to remediate contaminated land and ground water, such as is
found at Superfund sites and other properties where industrial or commercial activities
resulted in a legacy of hazardous constituents that limits future use of the property. The
MMR Center encompasses the full range of materials management and remediation
technologies, and as part of the larger Environmental Technology Verification (ETV)
program is designed to provide technology users with objective, high quality performance
data to support materials management or remediation technology selection decisions.
1.3 SCOPE AND FIELD OF APPLICATION
1.3.1 This document encompasses activities that Battelle, as an ETV verification organization,
shall utilize to assure the quality of products and services provided by the MMR Center.
The MMR Center is one of six centers operating under the ETV program.
1.3.2 This QMP applies to personnel involved in, and activities conducted by those staff for the
MMR Center, and contains the minimum specifications and guidelines that are applicable
to MMR Center quality management functions and activities based upon ANSI/ASQ E4-
2004 (an update to the ANSI/ASQC E4-1994 document). These include, but are not
limited to, personnel qualification and training, procurement of items and services,
documents and records, computer hardware and software, planning, implementation for
work processes, assessment and response, and quality improvement provisions.
1.4 BACKGROUND
1.4.1 Battelle (Memorial Institute) was established in 1929 by Gordon Battelle and serves as a
memorial to his family. Governed by a self-perpetuating Board of Trustees, Battelle is a
nonprofit Ohio corporation. Battelle and the laboratories it manages and co-manages has a
staff of 20,000 scientists, engineers, and support specialists. Battelle conducts $4 billion in
annual research and development.
-------
Section 1
Page 2 of 5
Version 1.0
Date: 2/23/09
Battelle's headquarters are in Columbus, Ohio. In addition to headquarters in Columbus,
Ohio, Battelle has major technology centers in Richland, Washington, where we manage
the Department of Energy's Pacific Northwest National Laboratory; Upton, New York,
where we partner with the Research Foundation of the State of New York in managing
Brookhaven National Laboratory; Golden, Colorado, where we partner with MRI in
managing the National Renewable Energy Laboratory; Oak Ridge, Tennessee, where we
partner with the University of Tennessee in managing the Oak Ridge National Laboratory;
Idaho Falls, Idaho, where we have formed the Battelle Energy Alliance to manage the
Idaho National Laboratory; Frederick, Maryland, location of the National Biodefense
Analysis and Countermeasures Center, managed by Battelle National Biodefense Institute;
Livermore, California, where we partner with the University of California in managing
Lawrence Livermore National Laboratory; and Aberdeen, Maryland, where we manage the
Battelle Eastern Science and Technology Center. Specialized facilities, regional centers,
and offices are located in 159 other cities in the United States and worldwide.
Battelle's organization includes three global lines of business: National Security, Health
and Life Sciences, and Energy Technology. The MMR Center is managed within Battelle's
National Security Global Business (NSGB) which includes approximately 3,500 chemists,
engineers, statisticians, and support personnel. Staff and facilities will be drawn from
NSGB and other Battelle organizations as needed to support the MMR Center. Staff
involved in the MMR Center includes those with expertise in materials management and
remediation, stakeholder involvement, and outreach and communication. Key Battelle
facilities that are available for use on the MMR Center include comprehensive laboratory
analysis equipment; field sampling and analysis equipment; source simulators such as pilot
plants; environmental chambers; and real-world test sites.
The organization chart for the MMR Center is provided in Figure 1.0 and shows key MMR
Center staff and their reporting lines. The key Battelle MMR Center staff are:
Center Manager: Ms. Amy Dindal is Battelle's MMR Center Manager with responsibility
for meeting all technical, budget, and schedule goals for the Center. Ms. Dindal is
organizationally in the Environmental Technologies Product Line within NSGB. Ms.
Tracy Stenner, the Environmental Technologies Product Line Manager, has responsibility
for ensuring that necessary Battelle facility and staff resources are available to support the
MMR Center. Ms. Dindal serves as the primary point of contact for the EPA MMR Center
Project Officer, Ms. Teri Richardson. She also directs the activities of the leaders in three
organizational areas within the Center: Quality Assurance; Verification Testing; and
Stakeholders.
Quality Manager: Mr. Zachary Willenberg is the Battelle MMR Center Quality Manager.
He is a Quality Assurance Officer in NSGB and reports directly to the Environmental
Technologies Product Line Manager, Ms. Stenner, and for the MMR Center, to Ms. Dindal.
These relationships are illustrated in Figure 1.0. Mr. Willenberg serves as the primary point
of contact for the EPA MMR Center Quality Manager, Mr. Scott Jacobs.
-------
Section 1
Page 3 of 5
Version 1.0
Date: 2/23/09
EPA
MMR Center
Quality Manager
S.Jacobs
Battelle
Stakeholder
Coordinator
M. Gordon
Stakeholder
Committees
EPA
MMR Center
Project Officer
T. Richardson
Battelle National
Security Global
Business Division
President
S. Kelly
Battelle
Environmental
Technologies Product
Line Manager
T. Stenner
Verification Test Coordinators
Battelle MMR
Center Manager
A. Dindal
Battelle
MMR Center
Quality Manager
Z. Willenberg
Figure 1.0 MMR Center Organization
(dotted lines indicate indirect reports)
-------
Section 1
Page 4 of 5
Version 1.0
Date: 2/23/09
Stakeholder Coordinator: Ms. Maria Gordon is the MMR Center Stakeholder Coordinator
with primary responsibility for stakeholder recruitment, communications, and stakeholder
meeting facilitation. Ms. Gordon reports directly to Ms. Dindal on the MMR Center.
Names, mailing/email addresses, and phone/facsimile numbers of these Battelle MMR
Center key staff are included in Appendix I.
1.5 DEFINITIONS
1.5.1 Verbs for clarity:
Shall, will, must: when the element is required and deviation from the specification will
constitute nonconformance with this QMP
Should: when the element is recommended
May: when the element is optional.
1.5.2 Center Quality Management Plan (QMP) - Procedures for quality-related activities
developed and implemented by Battelle to assure quality in the work processes and services
developed for the MMR Center.
Generic Verification Protocol (GVP) - A document that outlines the generic steps to be
taken in conducting a verification test for a technology category.
Stakeholders - The representatives of verification customer groups including buyers and
users of technology, consulting engineers, finance and export communities, and
government (local, state, federal) permitters and regulators. Stakeholders are selected
based upon their expertise and interest in materials management and remediation and their
availability and willingness to participate in the MMR Center. A list of MMR Center
Stakeholder Committee members will be available on the ETV website
(http://www.epa.gov/etv).
Test/Quality Assurance (QA) Plan - The plan developed by Battelle, with appropriate
input for each individual test of a technology or technology class. The Test/QA Plan
provides the experimental approach with clearly stated test objectives and associated
quality objectives for the related measurements and may incorporate or reference an MMR
Center GVP and/or standard operating procedures (SOPs).
Vendor - An individual, company, or organization which submits a commercially-available
remediation technology for verification testing.
Verification Organization - A public or private sector organization selected by EPA to
cooperate with the implementation of the ETV program by conducting verification testing
and to provide unbiased and objective test performance data on environmental
technologies.
-------
Section 1
Page 5 of 5
Version 1.0
Date: 2/23/09
Verification Organization Center Manager - The person designated by the verification
partner with the responsibility to manage the Center and serve as the chief point of contact
with the EPA. For the MMR Center, this is the Battelle MMR Center Manager.
Verification Organization Quality Manager - The person designated by the verification
partner with the responsibility to manage quality assurance for the MMR Center on behalf
of the verification partner Center manager. For the MMR Center, this is the Battelle MMR
Center Quality Manager.
Verification Report - A complete detailed summary of procedures and results for a
verification test of a single technology.
Verification Statement - A summary statement developed by Battelle, and signed by EPA
and Battelle, which reports quantitatively but without endorsement, the performance of a
tested technology in a verification test.
2.0 MANAGEMENT SYSTEMS
Battelle's quality policy is to provide services, products, and data of the highest quality that meet or
exceed our client's requirements and expectations. To this end, quality programs such as this MMR
Center QMP, and quality achievement, shall be fully supported by Battelle management and staff.
2.1 MANAGEMENT AND ORGANIZATION
2.1.1 Battelle management is responsible for committing to a quality policy and for creating
work environments in which all personnel strive for the highest quality of services and
products. Management shall also provide the Battelle MMR Center Manager the authority
to ensure the following:
• That all applicable elements of the quality system as described in this QMP are
understood and are implemented in the MMR Center.
• That adequate personnel and resources are available to plan, implement, assess, and
improve services and products relevant to the MMR Center.
• That staff is (are) clearly designated to stop unsafe work and work of inadequate
quality as affects the MMR Center.
2.2 QUALITY SYSTEM AND DESCRIPTION
2.2.1 The Battelle quality system to be implemented for the MMR Center according to this QMP
is intended to conform with the specifications listed in:
• ANSI/ASQ E4-2004, "Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs".
• EPA document "Environmental Technology Verification Program Quality
Management Plan", Version 3.0, January 2008.
• EPA document "EPA QA/R-2, EPA Requirements for Quality Management Plans,
March 2001.
-------
Section 1
Page 6 of 5
Version 1.0
Date: 2/23/09
It should be noted the E4 standard is comparable to the International Standards
Organization 9000 series.
2.2.2 The principal quality system document governing general and specific responsibilities for
MMR Center management and staff, responsibility and authority for all technical activities,
and reporting lines is this document, the "Quality Management Plan for the ETV Materials
Management and Remediation Center".
Individual verification tests will conform both to this QMP and to the applicable Test/QA
Plan document(s) and applicable SOPs.
The MMR Center QMP and any revisions will be controlled documents identified by a
unique Battelle document number (see Section 2.5.1) and will be distributed according to a
published list maintained by the Battelle MMR Center Quality Manager.
The QMP review will be documented by the Battelle MMR Center Quality Manager and
Battelle MMR Center Manager by signing and dating the copy of the QMP routed for
review. Any revisions to the QMP will be compiled by the Battelle MMR Center Quality
Manager for review, approval, and distribution. The approved QMP has a scheduled review
interval of one (1) year, although this may be adjusted by the EPA MMR Center Quality
Manager, depending upon factors such as the revision cycle of the ETV QMP.
The initial approved QMP will serve as Version 1.0, which will be designated, with its
effective date, in the upper right corner of each document page. Revisions will be so
designated beginning with "2.0" and will subsequently be numbered and dated as
applicable. Battelle staff to which controlled copies are issued will be responsible for
disposal of outdated QMP versions.
2.2.3 The scope of the MMR Center quality system applies to all Battelle personnel providing
products and services for the MMR Center. All MMR Center key staff shall be
knowledgeable regarding the QMP requirements.
2.2.4 Quality procedures documentation includes maintenance of all inspection and
review/assessment records, listing of all controlled documents (see Section 2.5.1), and
retention of records pertaining to personnel training and qualification, instrument
maintenance and calibration, and test methods/operating procedures.
2.2.5 Center-specific quality controls are initiated upon approval of Battelle's QMP prior to
implementing any verification testing activities. Planning actions documented through
approved Test/QA Plans shall also serve as quality control mechanisms for verification
testing.
In-process quality controls, through conduct of inspections followed by assessment reports
and verification of corrective actions when required, shall also be performed and recorded.
-------
Section 1
Page 7 of 5
Version 1.0
Date: 2/23/09
Implementation of a complete and consistent assessment of technical operations provides
overall control of Center activities. This will be accomplished by the Battelle MMR Center
Quality Manager according to Section 3.0 in this QMP.
2.2.6 An external quality system audit (QSA) of the Battelle quality system will be performed at
least once by the EPA MMR Center Quality Manager. In addition, an independent
technical systems audit (TSA) will be performed by the EPA MMR Center Quality
Manager or designee, at least one time per year for the MMR Center.
2.3 PERSONNEL RESPONSIBILITIES, QUALIFICATIONS, AND TRAINING
2.3.1 Responsibilities
2.3.1.1 Verification Partner Responsibilities. In accordance with EPA's ETV QMP dated
January 2008, Battelle's responsibilities for the MMR Center include the
following:
• Establish, attend, and/or conduct meetings of stakeholder committees with
representation from major customer groups
• Maintain communication with EPA to assure mutual understanding and
conformance with EPA quality procedures and expectations and ETV policies
and procedures
• Develop, review, and revise Test/QA Plans in cooperation with technology
vendors and stakeholders
• Solicit technology vendors and verification test collaborators
• Manage participation of and conduct verification activities
• Assure that quality procedures are incorporated into all aspects of the MMR
Center
• Perform ETV activities within the documented quality system
• Prepare ETV verification reports and statements at the completion of each
technology verification
• Appoint a quality manager, responsible for ensuring the MMR Center quality
systems are in compliance with the E4 standard and the EPA ETV QMP dated
January 2008, and MMR Center staff complies with this QMP.
• Submit a written request to the EPA MMR Center Project Officer and EPA
MMR Center Quality Manager if desired in specific instances, the
responsibility for reviewing and approving Test/QA Plans be delegated to the
Battelle MMR Center Quality Manager.
2.3.1.2 Key Staff Responsibilities. Battelle is committed to operate an effective quality
system that ensures compliance with all program requirements. The
responsibilities of Battelle key staff who will be performing verification testing
activities addressed by this MMR Center QMP are listed in Table 1.0.
2.3.1.3 Stakeholder Responsibilities. The responsibilities of stakeholders for the MMR
Center include the following:
-------
Section 1
Page 8 of 5
Version 1.0
Date: 2/23/09
• Assist in prioritizing the types of technologies to be verified, focusing on
these technologies with greatest environmental and sustainability impacts.
• Review Center-specific procedures and MMR Center documents including
Test/QA Plans, verification reports, and verification statements, as requested.
• Participate in verification testing as collaborators, provide funding and/or in-
kind support, or recommend collaborators to the MMR Center.
• Assist in the definition and conduct of outreach activities appropriate to the
technology area and customer groups.
• Serve as information conduits to the particular constituencies that each
member represents.
2.3.2 Qualification and Training
Battelle personnel qualifications and training shall target technical work performed directly
in support of verification testing activities. These qualifications and training may include:
• Formal education in physical sciences (e.g., chemistry, physics, engineering).
-------
Section 1
Page 9 of 5
Version 1.0
Date: 2/23/09
Table 1.0. Personnel Responsibilities for the MMR Center for Verification Testing
Activities
MMR Center Team
Member
Battelle MMR
Center Manager
Amy Dindal
Responsibilities
Ultimate responsibility for all aspects of the MMR Center
Conduct and oversee activities to establish and maintain active stakeholder committees
Maintain adequate communication with EPA
Manage oversight and conduct of verification activities
Assure that quality procedures are incorporated and implemented
Review/approve Test/QA Plans
Solicit technology vendors
Assign Verification Test Coordinators to technology categories
Operate ETV activities within the documented quality system
Issue stop work orders
Review and approve verification reports
Review and approve verification statements
Battelle MMR
Center Quality
Manager
Zachary Willenberg
Ensure that the quality system is compliant with EPA-specified standards
Advise the Battelle MMR Center Manager of any QA/quality control(QC) problems and oversee
corrective actions
Ensure that the MMR Center QMP includes sufficient and appropriate specifications for QA/QC as
required for the MMR Center
Interact with MMR Center management and technical personnel to ensure that QA/QC procedures
are understood
Ensure that Battelle MMR Center QMP and the EPA/ETV QMP are followed for performing system
inspections and audits
Perform a ISA and ADQ for every verification test or ensure that a ISA and Audit of Data Quality
(ADQ) are performed by a designee
Participate in pre-test kick-off meetings to review QA requirements with verification testing staff
Review training records of verification testing staff
Notify the Battelle MMR Center Manager to issue a stop work order if assessments indicate health,
safety, or quality concerns
Review QA documentation of reference laboratories for each verification test, as appropriate
Review QC data (including reference laboratories and vendor technologies) generated during
verification tests
Ensure that inspection reports are prepared and distributed that detail appropriate corrective action
and that implementation will be responded to by personnel. Problems that are not addressed will be
brought to the attention of the Battelle MMR Center Manager
Review Test/QA Plans, SOPs, verification reports, and verification statements
Review and approve amendments and deviations to Test/QA Plans
Review all quality system documentation, including this document, at intervals necessary to ensure
their integrity. Such reviews will be recorded and documents will be revised if necessary. All
previous original (i.e., signed) revisions will be retired and archived.
Act as a QA resource to respond to quality needs and problems. Answer questions and train
laboratory staff in QA/QC requirements and procedures as needed.
-------
Section 1
Page 10 of 5
Version 1.0
Date: 2/23/09
MMR Center Team
Member
Battelle Verification
Test Coordinators
Responsibilities
Provide technical support to verification testing as needed, and interact with the Battelle MMR
Center Quality Manager during inspections and implementation of corrective actions when needed
Perform QA/QC activities specified in this document, applicable Test/QA Plans, and in pertinent
SOPs
Conduct QC measures and activities required for sample analyses
Verify 100% of data and evaluate results of QC analyses to determine if quality goals and objectives
have been met
Inform Battelle MMR Center Manager of potential problems
Perform corrective action at the direction of the Battelle MMR Center Manager and Battelle MMR
Center Quality Manager in response to ISA and ADQ audit reports
Document results of QC analyses and include them with sample results and historical data files
Maintain instrumentation (vendor and/or reference instrumentation) in accordance with the QMP,
Test/QA Plan, SOPs, and the manufacturer's instructions
Prepare Test/QA Plans and amendments and deviations to these plans, as appropriate
Perform pre-test kick-off meetings to review technical, project management, and QA aspects of
verification testing with verification testing staff
Perform performance evaluation (PE) audits of reference laboratories and calibrated equipment for
each verification test, as appropriate
Maintain verification test records (in bound Laboratory Record Book and/or test binders) that
adequately capture the quality of data collected
Develop and implement Test/QA Plans
Prepare verification reports
Prepare verification statements
• Experience materials management or remediation techniques such as recycling,
beneficial use of waste materials, recovery of useful components of waste, and
treatment to minimize disposal requirements
• Training on remediation technologies
• Experience in designing experiments to verify the performance of materials
management and/or remediation technologies.
Battelle personnel working on the MMR Center shall have, at a minimum, documentation
maintained by Battelle permanently for each of the following, as applicable:
• Education history which can include formal qualification or certification relevant to
technical, quality assurance, or management disciplines.
• Work experience as academic or on-the-job performance in technical and/or
management areas.
• Experience in the application of quality assurance/quality control requirements in
technical performance or data verification.
2.3.2.1 Formal qualifications and certifications in the form of actual or verified-copy
documentation for specific disciplines shall be maintained in the staff member's
qualification/training file. Training documents will be reviewed by the Battelle
MMR Center Quality Manager, as appropriate.
-------
Section 1
Page 11 of 5
Version 1.0
Date: 2/23/09
2.3.2.2 Technical management and training received in-house or offsite shall be recorded
and forms, memos, or certificates retained. Performance on either task, project, or
program assignments is to be considered as part of training.
2.3.2.3 Retraining needs based on job requirements shall be determined by the staff
member and respective management. To maintain staff proficiency, opportunities
provided by Battelle or other sources shall be made available, preferably on an
annual basis.
2.3.2.4 Personnel job proficiency based on witnessed performance on-the-job by a
qualified trainer/staff member designee shall be documented. Specific method
requirements for instrument inspection, performance, and maintenance are
objective measures that could be considered. Specific performance based on
national certification requirements can be recorded with certificates or other
documentation. Basic areas of proficiency for verification testing may include, at
a minimum:
• Sample management practices, such as chain of custody records
• Sample handling and storage and use of standards and reagents
• Instrument inspection, use, and maintenance
• Data acquisition, analysis, and verification.
2.3.2.5 Training resources should be offered on-site by Battelle for facility requirements,
such as general computer software use (E-mail, spreadsheets) or project
management. Off-site training, project/program meetings, and technical society
membership should be available for specific disciplines contributing to the staff
member's overall job proficiency.
2.3.2.6 Verification test collaborators working on behalf of Battelle in support of the
MMR Center and/or individual test operations are expected to provide the
Verification Test Coordinator, or designee, with:
• Educational background and/or degree(s) relevant to technical areas
represented in the MMR Center
• Work experience related to the technology category undergoing verification.
This information may be reviewed by the Battelle MMR Center Quality Manager.
2.3.2.7 Battelle personnel will receive update memorandum with the QMP when the
document is revised specifying changes made. In addition, the Battelle MMR
Center Quality Manager will provide testing staff with a refresher on QMP
requirements with each update.
2.4 PROCUREMENT AND ACCEPTANCE OF ITEMS AND SERVICES
-------
Section 1
Page 12 of 5
Version 1.0
Date: 2/23/09
2.4.1 Policy
Procurement technical and quality requirements are generally based upon value (cost,
durability, maintainability), performance (specification compliance, operating conditions,
calibration capacity), delivery (timeliness, ease of ordering), customer support
(responsiveness, technical ability), past experience with a particular vendor, and
completeness and coherence of instructions (clarity, accuracy).
2.4.2 Procurement
Technical and quality requirements for items and services procured for a specific
verification test may be included in the Test/QA Plan. These requirements will typically be
specified under materials and/or measurement system equipment (Test/QA Plan Section
B8, Inspection/Acceptance Requirements for Supplies and Consumables). The request for
items or services will initiate from the Verification Test Coordinator or designee with
approval for purchase from the Battelle MMR Center Manager, line manager, or designee.
2.4.3 Acceptance
2.4.3.1 Testing equipment procured for activities affecting quality shall be calibrated to
ensure accuracy with required specifications listed in the Test/QA Plan and may
be verified prior to use in the verification test (e.g. PE audits), as appropriate. Any
discrepancies shall result in a recalibration of the equipment, or if the equipment is
unusable, then a return of the item to the supplier for repair/replacement as
necessary. Verification, storage, and maintenance records will be included in
individual verification test records.
2.4.3.2 Testing materials procured for activities affecting quality (e.g. reference standards
or gases) shall be accompanied with a Certificate of Analysis (COA) where
appropriate. The COA will be examined to ensure that the listed specifications are
within the required limits. The COA will be retained and included in the
verification test records.
2.4.3.3 Methods to accept procurement of services (i.e. subcontractors; installation, repair,
or maintenance work; etc.) includes technical verification of the data produced,
surveillance and/or audit of the activity being performed, or review of objective
evidence for conformance to procurement document requirements.
2.5 DOCUMENTS AND RECORDS
2.5.1 Controlled Documents
Document control is the system which ensures that only the latest revision(s) of the defined
documents are used by Battelle staff participating in the MMR Center. The system
includes retention of the document with original signed page(s) in a limited access storage
area, a unique numbering system for all documents (typically identified by revision number
-------
Section 1
Page 13 of 5
Version 1.0
Date: 2/23/09
and/or date), and a distribution list for each document. Such documents are defined as
"controlled documents" and can be revised only by the personnel listed within each
document or this QMP. The following is a list of the controlled documents within the
MMR Center:
• Quality Management Plan for the ETV Materials Management and Remediation Center
(this document)
• Standard Operating Procedures
• Test/QA Plans, including amendments and deviations
• Generic Verification Protocols
Controlled document identification will consist of a number (if applicable), date, and
version, if applicable, assigned to the document by the Battelle MMR Center Quality
Manager or designee. A current Master List of Controlled Documents and Distribution
shall be maintained by the Battelle MMR Center Quality Manager.
As a controlled document, approved copies of the QMP will be maintained and issued to
MMR Center staff by the Battelle MMR Center Quality Manager or designee.
Obsolete or superseded documents shall be removed from operations when new documents
are provided. Notification will accompany new document versions that the previous
version is to be removed from use and destroyed. Staff members are responsible for
destroying outdated versions of documents assigned to their person. The Battelle MMR
Center Quality Manager is authorized to remove outdated documents observed during
inspections and reviews. All controlled documents, including historical revisions, will be
retained at least 10 years or six years and three months after final payment of the
cooperative agreement, with the exception of the Standard Operating Procedures which will
be permanently archived.
2.5.2 Verification Test Records
2.5.2.1 Active Verification Test Records. All verification test records shall carry
minimum identification pertaining to title, responsible person or author, and date.
All manual entries shall be entered using ink and initial and dated by the
individual recording the entry. No changes to entries, manual or electronic, shall
obscure the original record during the correction process, and corrections shall be
initialed and dated by the individual recording the correction. A short explanation
will be added to non-obvious corrections.
2.5.2.2 Storage of Verification Test Records. Per the ETV QMP, verification test records
specific to the MMR Center shall be retained for at least 10 years or six years and
three months after final payment of the cooperative agreement. All MMR Center
records needed to reconstruct test activities and verify that reported data were
collected in a consistent manner with this QMP and MMR Center requirements
will be maintained in an appropriate area of limited access, until either transferred
to EPA Office of Research and Development Records Management or properly
destroyed with EPA permission. The Battelle MMR Center Quality Manager will
-------
Section 1
Page 14 of 5
Version 1.0
Date: 2/23/09
retain, as a permanent record, documentation of the transfer or destruction of
Battelle's MMR Center records.
2.5.3 MMR Center Program Records
The following program records will be retained, as per the ETV QMP, for at least 10 years
or six years and three months after final payment of the cooperative agreement.
• Minutes of stakeholder meetings
• Cooperative agreement (CA) records
• Test/QA Plans
• Verification reports
• Verification statements
• Battelle quality assessment reports.
2.5.4 Record Preparation, Review, Approval, and Distribution
Responsibilities for these activities are summarized in Table 2.0 and are detailed below.
2.5.4.1 Preparation. Individual case requirements and this QMP shall guide document
and record content and/or format. For the MMR Center, guidance for content
and/or format are derived by EPA/ETV directive and the following documents:
• ANSI/ASQ E4-2004, "Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs".
• EPA document "Environmental Technology Verification Program Quality
Management Plan", Version 3.0, January, 2008.
• EPA document "EPA QA/R-2, EPA Requirements for Quality Management
Plans, March 2001.
2.5.4.2 Review/Approval. Record review/approval shall be performed by qualified
technical and/or management personnel as deemed appropriate. The individual
reviewer shall have access to all needed references.
All Battelle prepared documents in QMP Sections 2.5.1 through 2.5.3 shall require
at least one review by a Battelle staff member prior to external distribution by
Battelle. Document and record reviews are performed at the request of the
Battelle MMR Center Manager, Quality Manager, or other staff personnel.
In addition, ETV record review assigned to Battelle extends to the following
documents, at a minimum:
• EPA/ETV strategy
• EPA/ETV QMP
• Annual Center progress reports.
-------
Table 2.0 Records Management Responsibilities for the MMR Center
Section 1
Page 15 of 5
Version 1.0
Date: 2/23/09
Preparation/
Record Type Updating Review Approval Finals Distributed to:
ETV Verification
Strategy
ETV Quality
Management Plan
CA Records
MMR Center Quality
Management Plan
Minutes of
Stakeholder
Meetings
Test/QA Plan
(including SOPs,
amendments and
deviations*)
Generic Verification
Protocol
Raw data
ETV Verification
Report
ETV Verification
Statement
Annual ETV
Progress Report
Quarterly ETV
Progress Report
EPA Reviews/Audit
Reports
Battelle
Reviews/Audit
Reports
N/A
N/A
Battelle MMR
Center
Manager
Battelle MMR
Center
Quality
Manager
Battelle
Battelle
Battelle
Battelle
Battelle
Battelle
N/A
N/A
N/A
Battelle MMR
Center
Quality
Manager
Battelle MMR Center Manager
Battelle MMR Center Manager
EPA MMR Center Project Officer
EPA MMR Center Quality Manager
EPA MMR Center Project Officer
Stakeholders
Battelle MMR Center Manager
Battelle MMR Center Qualify
Manager
EPA MMR Center Quality Manager
EPA MMR Center Project Officer
Assigned Stakeholders
Peer Reviewers
EPA MMR Center Project Officer
Battelle MMR Center Quality
Manager
Assigned Stakeholders
Internal Technical Peer Review
Battelle MMR Center Manager
Battelle MMR Center Quality
Manager
EPA MMR Center Quality Manager
Vendor
Peer Reviewers
Battelle MMR Center Manager
Battelle MMR Center Qualify
Manager
EPA MMR Center Project Officer
EPA MMR Center Quality Manager
Vendor
ETV Program Director
Peer Reviewers
Battelle MMR Center Manager
Battelle MMR Center Manager
N/A
Battelle MMR Center Manager
N/A
N/A
N/A
EPA MMR Center Project Officer
EPA MMR Center Quality
Manager
Battelfe MMR Center Manager
Battelle MMR Center Qualify
Manager
N/A
Vendors
EPA MMR Center Project Officer
EPA MMR Center Quality
Manager
EPA MMR Center Project Officer
EPA MMR Center Quality
Manager
N/A
EPA MMR Center Project Officer
EPA MMR Center Quality
Manager
EPA Laboratory Director
Battelle Management
EPA MMR Center Project Officer
EPA MMR Center Quality
Manager
N/A
Battelle MMR Center Manager
N/A
N/A
N/A
N/A
N/A
Testing Staff
ETV webmaster
EPA MMR Center Quality Manager
EPA MMR Center Project Officer
Stakeholders
ETV Webmaster
EPA MMR Center Project Officer
Testing Staff
Vendors
EPA MMR Center Project Officer
EPA MMR Center Quality Manager
ETV Webmaster
EPA MMR Center Project Officer
EPA can request copies
ETV Program Director
EPA MMR Center Project Officer
ETV Webmaster
Vendors
ETV Program Director
EPA MMR Center Project Officer
ETV Webmaster
Vendors
N/A
EPA MMR Center Project Officer
EPA MMR Center Quality Manager
ETV Program Director
EPA Laboratory Directors
Battelle MMR Center Manager
Battelle MMR Center Qualify
Manager
EPA MMR Center Project Officer
EPA MMR Center Quality Manager
N/A = Indicates Battelle does not have responsibility for preparing/updating record; conducting or obtaining review; providing or obtaining approval; or distributing
and/or receiving final record.
* = Amendments and deviations are not reviewed or approved by EPA. Only amendments are distributed.
-------
Section 1
Page 16 of 5
Version 1.0
Date: 2/23/09
2.5.4.3 Distribution. Once records have been reviewed and approved as required,
distribution will be made as listed in Table 2.0. MMR Center documents
specifically requiring EPA approval before release include:
• MMR Center QMP
• Generic Verification Protocols
• Test/QA Plans
• ETV verification reports
• ETV verification statements
2.6 COMPUTER HARDWARE AND SOFTWARE
This QMP requires that Battelle staff understand the necessity for all computer hardware and
software specifications. Staff shall attempt to utilize computer hardware and software within the
acceptance criteria specified, and ensures that hardware and software are installed, maintained,
and used according to specifications. Any time a change in hardware components or
configuration or a software modification is needed, retesting and recalibration must be performed
and documentation included with facility records.
2.6.1 Hardware
All computer hardware at Battelle contains Intel based Pentium processors running a
Microsoft operating system. Each personal computer primarily consists of a standard
complement of Microsoft software (e.g., Word, Excel, Access, PowerPoint, and Outlook)
with capabilities of running other commercial software (e.g., WordPerfect, Quattro, Lotus,
SAS) and delivery of data in various standard formats. These computers are replaced
approximately once every three years to ensure staff have access to the most updated, state-
of-the-art equipment, especially those staff with the heaviest computational needs.
Computer hardware is upgraded to improve performance and provide complete
compatibility with current standards. The decision to upgrade computer hardware is made
when a project that requires specific computer capabilities is received. Next, an assessment
of impact is completed. This assessment includes a review of current computer programs
and the impact or upgrading hardware on data accessibility.
2.6.2 Software
Specific software required for a verification test will be identified in the Test/QA Plan.
Most software used at Battelle is acquired commercially, loaded, and tested as specified by
the publisher. Independently-developed software is not used within the ETV MMR Center;
only commercial products are used. Software used for data management activities may
include Microsoft Excel or Access. Standard word processing software (e.g. Word) is used
to create reports. Currently, Battelle does not use nor are the systems currently compatible
with Windows Vista.
-------
Section 1
Page 17 of 5
Version 1.0
Date: 2/23/09
2.6.3Validation Policy
Since all hardware and software used on the ETV MMR Center is commercially available
and wide public use and continued market viability is considered proof of software
dependability, validation is not considered necessary. However, verification of data
analysis techniques within each program (e.g. the use of formulas and macros) is required.
For each defined spreadsheet a performance test document will be prepared which will
contain the following:
• An overview of the application. The overview will describe what the application is
required to do and specify the methods used to meet the predetermined requirements.
• References to the productivity software used (e.g., Excel 2003, SigmaPlot V8.0, etc),
and the operating system (e.g., Windows 2000, Windows XP, etc.).
• A description of important equations used to derive data.
• A description of what test(s) were conducted to confirm the accuracy of the application.
2.7 PLANNING
This QMP addresses the purpose and scope of systematic, timely, and effective planning
necessary to assure services and products of the highest quality.
2.7.1 Stakeholder committee(s) containing representatives of appropriate technology interest
groups shall be jointly established by Battelle with input from the EPA MMR Center Project
Officer. Individual stakeholders shall be selected for these committee(s) based on their
expertise and interest in materials management and remediation and their availability and
willingness to participate.
A joint meeting of the EPA MMR Center Project Officer, Battelle, and each stakeholder
committee will be held at least once annually, with minutes of such meetings recorded,
reviewed, and circulated to the stakeholders, the EPA MMR Center Project Officer,
Battelle, and subsequently posted on the ETV Website. The meeting can be conducted in
person or by teleconference.
The planned quality-related purposes of this meeting are to:
• Identify, revise, and/or clarify the technical and quality goals of the work to be
accomplished
• Determine testing priorities and evaluate customer satisfaction
• Define and review verification plans, and identify verification test collaborators.
2.7.2 Systematic Planning of Verification Tests
An overall view of the EPA ETV verification process is shown in Figure 2.0. Battelle, in
cooperation with the EPA MMR Center Project Officer, begins a systematic process to plan
the individual verification tests. Systematic planning may be accomplished through any
demonstrated technique such as the data quality objectives process (EPA QA/G-4,
-------
Section 1
Page 18 of 5
Version 1.0
Date: 2/23/09
Guidance on Systematic Planning Using the Data Quality Objectives Process, February
2006). The planners perform the following actions:
• Convene stakeholder committees of representatives of verification customer groups
who advise during the planning process
• Mediate and facilitate the selection of prioritized technologies
• Refine the scope of respective technology areas
• Determine interest in verification from the vendors of commercial-ready technologies
within the defined scope of these areas and other collaborators that could contribute
funding or in-kind support
Identification of Technology
Categories for Testing
(Stakeholder/EPA/Battelle)
Recruitment of Vendors/Collaborators
(Battelle)
Draft Test/QA Plan
(Battelle/Vendors/Stakeholders/Collaborators)
Vendor Review of Test/QA Plan
(Battelle)
Peer and QA Review of Revised
Test/QA Plan
(Battelle)
Coordination with Host-Facility/
Subcontractors/Collaborators
Final Test QA Plan
(Battelle)
Finalize Host/Subcontractor/
Subcontractor Roles
(Battelle)
To Testing
Figure 2 Systematic Planning of Verification Tests
-------
Section 1
Page 19 of 5
Version 1.0
Date: 2/23/09
• Solicit vendor and prepare vendor agreements to participate in verification of their
products based on the Test/QA Plan
• Prepare Test/QA Plan(s) which are developed to promote uniform testing for a given
type of technology
• Involve host facilities, collaborator organizations, and any subcontracted laboratories in
the planning process
• Coordinate the review and revision of the Test/QA Plan(s) (by vendors, EPA, and peer
reviewers) keeping in mind both the customer and EPA objectives for verification as
defined in the ETV Strategy
• Prepare final Test/QA Plans after testing a given type of technology
• Prepare amendments and deviations to the Test/QA Plan, as necessary, to include
revisions based on actual test experience.
Systematic planning process-control documents for the MMR Center include:
• The ETV Program Policy Compendium
• The ETV Program QMP
• This QMP which defines the operational quality system necessary to provide
acceptable products and services.
• Written quality procedures specific to the technology and verification test including
Test/QA Plans and Standard Operating Procedures.
• Outputs from stakeholder committee meetings in the form of reviewed and distributed
minutes.
• Quarterly financial and progress reports to the EPA ETV program.
2.7.3 Planning Personnel
Verification test planning shall be coordinated by Battelle among the participating
organizations including EPA, the stakeholders, the vendors, and any testing organizations
and laboratories participating in the test. Battelle, with the concurrence and oversight of
the EPA MMR Center Project Officer, shall identify the planning roles of the participants,
and shall conduct planning activities by shared communication via e-mail, teleconference,
web meetings, and in-person meetings, as appropriate, and within the constraints of budget.
2.7.4 Existing Data
Existing data may be used according to the procedures described in the Test/QA Plan for
each verification test and in accordance with the ETV QMP.
2.8 DESIGN OF TECHNOLOGY VERIFICATION OPERATIONS
2.8.1 Design Process
The design process produces a Test/QA Plan based upon the data quality objectives for the
verification.
-------
Section 1
Page 20 of 5
Version 1.0
Date: 2/23/09
2.8.1.1 Design Technique. In designing verification tests, Battelle staff use consensus-
accepted test design including statistical methods, as appropriate. The design
takes into account constraints of time, scheduling, and resources. All relevant
activities pertaining to environmental data operations shall be identified, as well as
performance specifications and the appropriate controls.
2.8.1.2 Field and Laboratory Equipment and Methods. During the design process, the
appropriate field and laboratory equipment which were identified during planning
for the testing of the technology verification performance are incorporated.
Appropriate test methods and operating parameters are specified.
2.8.1.3 Sampling and Analysis. If samples for analysis are taken in the field, they are
handled according to procedures specified in the Test/QA Plan. The oversight
responsibility of Battelle is to determine that the approved systems and plans
contain adequate procedures for handling, storage, cleaning, packaging, shipping,
and preservation of field and laboratory samples to prevent damage, loss,
deterioration, artifacts, or interferences. Battelle will provide adequate chain of
custody procedures, if they are required. The following sampling and analysis
design parameters will be addressed in the Test/QA Plan.
• Experiments to be conducted, the baseline parameters, the number of replicate
tests, and the controls.
• Sampling methods, sample types, numbers, quantities, handling, packaging,
shipping, and custody (if sampling is performed).
• Sample locations, storage conditions, and holding times.
• Analysis methods, quantitative measures of performance, calibration
standards, calibration check standards, and performance evaluation samples,
as appropriate, and as identified in the planning process.
• Methods and procedures to ensure the test produces traceable data of known
and acceptable quality.
• Field and/or laboratory QA/QC activities.
• Requirements for qualifications of technical staff responsible for obtaining,
analyzing, and evaluating the data.
• Protection of the health and safety of testing personnel and the public.
• Procedures for the minimization and disposal of waste generated.
2.8.1.4 Assessments. Assessments incorporated into the design include self-assessments
(internal audits) by Battelle and independent assessments by EPA. The
assessments identified in the planning process are incorporated into the design.
The type and minimum number of assessments are identified in Section 3.0.
2.8.2 Generic Verification Protocols, Test/QA Plans, and Standard Operating Procedures
Three types of planning documents have been identified for operation of an ETV Center:
the GVP, the Test/QA Plan, and Standard Operating Procedures (SOPs). The GVP is meant
to promote uniform testing for a technology category, and therefore, is a more general
document. The Test/QA Plan gives the specific information needed to conduct a
-------
Section 1
Page 21 of 5
Version 1.0
Date: 2/23/09
verification test. If another level of detail is required for describing test activities, for
example operation of an instrument, an SOP will be written and attached to the Test/QA
Plan.
2.8.2.1 Generic Verification Protocol. The Battelle MMR Center Manager will be
responsible for assuring that the GVPs are prepared and transferred to the EPA
MMR Center Quality Manager and stakeholders for review. The issues that may
be addressed in the GVP are the following:
• General description of the Center
• Responsibilities of all involved organizations
• Experimental design
• Equipment capabilities and description
• Description and use of field test sites
• Description and use of laboratory test sites
• QA/QC
• Data handling
• Requirements for other documents
• Health and safety
• References.
2.8.2.2 Test/QA Plans. Test/QA Plans are the responsibility of the Battelle MMR Center
Manager and are reviewed by the Battelle MMR Center Quality Manager and
EPA MMR Center Quality Manager. The appropriate GVP is incorporated by
reference. Appropriate guidance for writing Test/QA Plans is available in
EPA/QA G-5, Guidance for Quality Assurance Project Plans, December 2002.
Planned changes to the Test/QA Plan are made by amendment. Deviations from
the plan must be fully documented including, date and description of deviation,
and impact on the verification test. Amendment and deviation forms are in
Appendix II. Elements of the Test/QA Plan will include the following. Elements
listed that are not appropriate for the test will be listed as such:
• Group A: Verification Management - This group of elements covers the
general areas of verification management, verification history and objectives,
and roles and responsibilities of the participants. The following nine elements
ensure that the verification's goals are clearly stated, that all participants
understand the goals and the approach to be used, and that verification planning
is documented:
- Al Title and Approval Sheet
- A2 Table of Contents and Document Control Format
A3 Distribution List
- A4 Verification Organization and Schedule
A5 Problem Definition/Background
A6 Verification Description
- A7 Quality Objectives and Criteria for Measurement Data
A8 Special Training Requirements/Certification
-------
Section 1
Page 22 of 5
Version 1.0
Date: 2/23/09
- A9 Documentation and Records
• Group B: Measurement/Data Acquisition - This group of elements covers all of
the aspects of measurement system design and implementation, ensuring that
appropriate methods for sampling, analysis, data handling, and QC are
employed and will be thoroughly documented:
B1 Sampling Process Design (Experimental Design)
- B2 Sampling Methods Requirements
B3 Sample Handling and Custody Requirements
B4 Analytical Methods Requirements
- B5 Quality Control Requirements
B6 Instrument/Equipment Testing, Inspection, and Maintenance
Requirements
- B7 Instrument Calibration and Frequency
B8 Inspection/Acceptance Requirements for Supplies and Consumables
- B9 Data Acquisition Requirements (Non-Direct Measurements)
- BIO Data Management
• Group C: Assessment/Oversight - The purpose of assessment is to ensure that
the Test/QA Plan is implemented as prescribed. This group of elements
addresses the activities for assessing the effectiveness of the implementation of
the verification and the associated QA/QC activities:
Cl Assessments and Response Actions
- C2 Reports to Management
• Group D: Data Validation and Usability - Implementation of Group D elements
ensures that the individual data elements conform to the specified criteria, thus
enabling reconciliation with the verification's objectives. This group of
elements covers the QA activities that occur after the data collection phase of
the verification has been completed:
- Dl Data Review, Validation, and Verification Requirements
- D2 Validation and Verification Methods
D3 Reconciliation with Data Quality Objectives
2.8.2.3 Standard Operating Procedures. The following topics, from EPA QA/G-6,
Guidance for Development of Standard Operating Procedures (SOPs), April
2007, will be included (or a reference provided), as appropriate, in standard
operating procedure that are prepared for MMR Center verification tests:
• Title Page
• Table of Contents
• Procedures - The following are topics that may be appropriate for inclusion in
technical SOPs. Not all will apply to every procedure or work process
detailed.
-------
Section 1
Page 23 of 5
Version 1.0
Date: 2/23/09
Scope & Applicability
Summary of Method
Definitions
Health & Safety Warnings (indicating operations that could result in
personal injury or loss of life)
• Cautions (indicating activities that could result in equipment damage,
degradation of sample, or possible invalidation of results)
• Interferences (describing any component of the process that may
interfere with the accuracy of the final product)
Personnel Qualifications
Equipment and Supplies
Procedure (identifying all pertinent steps, in order, and materials needed
to accomplish the procedure such as:
• Instrument or method calibration and standardization
• Sample Collection
• Sample Handling and Preservation
• Sample Preparation and Analysis
• Troubleshooting
• Data Acquisition, Calculations & Reduction
• Requirements for Computer Hardware & Software used in Data
Reduction and reporting
Data and Records Management
Quality Control and Quality Assurance Section
References
2.9 IMPLEMENTATION
2.9.1 General
Technology performance verifications are implemented according to the Test/QA Plans and
technical documents (e.g., Standard Operating Procedures) prepared during planning. A
kick-off meeting will be held prior to the start of each verification test to review procedures
for the test with all verification testing staff. Test personnel have access to the approved
planning documents, approved changes to planning documents, and all referenced
documents. When a prescribed sequence for the work is defined during the planning
stages, work performed shall follow that sequence. Changes to that sequence need to be
documented by either amendment (planned changes) or deviation (unplanned changes). All
implementation activities are documented. Suitable documents are bound notebooks (e.g.
laboratory record books, or LRBs), field and laboratory data sheets, spreadsheets, computer
records, and output from instruments (both electronic and hardcopy). All documentation is
implemented as described in the planning documents. All implementation activities are
traceable to the planning documents and traceable to test personnel.
2.9.1.1 Conformance of implementation to planning is accomplished by following
approved documents for the Battelle quality system implementation, verification
testing, and for any field and laboratory technical operations.
-------
Section 1
Page 24 of 5
Version 1.0
Date: 2/23/09
Generation of verification test data will not be initiated until the approved
Test/QA Plan is in place.
When work cannot be implemented according to the approved planning and test
document, Battelle shall be responsible for providing a written amendment to the
Test/QA Plan or a deviation report for the test records. Amendments are produced
for changes that are made to the approved Test/QA Plan before the proposed
change will be made. Amendments must be approved internally by the Battelle
MMR Center Quality Manager and the Battelle MMR Center Quality Manager.
Following approval, the amendment will be distributed to all internal personnel
holding a copy of the approved Test/QA Plan and the EPA MMR Center Quality
Manager. A deviation report is produced for any changes to the approved
Test/QA Plan that occurred during the test. Deviation reports must be retained in
the verification test records and summarized in the verification test report.
Frequent deviations from established procedures should result in a retrospective
review of the written document and possible revision. Amendments and
deviations will include all the information displayed on the forms shown in
Appendix II.
All persons responsible for performing verification testing and those participating
vendors shall receive copies of the current revision of the Test/QA Plan and
associated documentation provided by Battelle.
Current versions of Test/QA Plans and any applicable methods and SOPs are
required to be physically in place at each technology verification testing site.
2.9.1.2 Battelle oversight and inspection of a verification test shall be provided by the
Battelle MMR Center Quality Manager or designee at intervals prescribed in each
Test/QA Plan. This frequency, at a minimum, will be once for each verification
test of a technology category. To verify full implementation of the Test/QA Plan,
the inspection will include the testing process and any documentation associated
with the process, such as sample tracking records; instrument maintenance and
calibration; sample preparation and actual analysis; and data records. The Battelle
MMR Center Quality Manager will provide a written report, verify the completion
of any corrective actions needed, and retain a copy of the report with permanent
Battelle MMR Center Quality Manager records. The EPA MMR Center Project
Officer will be included in the routing of the inspection results and a written copy
provided to both the EPA MMR Center Project Officer and EPA MMR Center
Quality Manager.
2.9.2 Implementation Procedures
2.9.2.1 Testing procedures shall be documented in approved Test/QA Plans and SOPs.
Testing personnel, by virtue of training requirements described in this QMP, shall
demonstrate proficiency of performance and knowledge of QA and MMR Center
requirements for the verification test operations.
-------
Section 1
Page 25 of 5
Version 1.0
Date: 2/23/09
2.9.2.2 Content requirements for testing procedures may include those of existing Battelle
SOPs or other referenced documents.
2.9.2.3 Before the initiation of testing, a test kickoff meeting will be held by the
Verification Test Coordinator. The Battelle MMR Center Manager, Battelle
MMR Center Quality Manager, and all Battelle technical staff who will be utilized
for the verification test will attend the kickoff meeting. Subjects to be discussed at
the meeting will include, but not be limited to, a general overview of the Test/QA
Plan, staff assignments, schedules, and assessments (see Section 3.0). A separate
meeting may also be held with external staff (e.g. collaborators, external
personnel, vendors, etc...) prior to the test.
2.9.2.4 Review of technical Center-specific procedures shall be done by personnel
technically competent with respect to the procedure. Time must be allowed for the
composition, review, and approval of technical procedures to be completed in
advance of the actual performance.
2.9.3 Implementation Monitoring
2.9.3.1 Quality assessments during implementation of individual verification tests will be
prescribed at a minimum frequency/interval in the Test/QA Plan. Specifically, the
Test/QA Plan will address:
• A routine monitoring schedule and,
• The required specifications of performance, or particular aspects of the
process, that are determined to be critical for monitoring
2.9.3.2 Monitoring of the work process is conducted by the Battelle MMR Center Quality
Manager or designee and is done to:
• Ensure satisfactory performance based on requirements,
• Ensure required actions (as specified in implementation documents) are
performed so that routine measurements meet specifications,
• Ensure preventive maintenance is performed and documented as specified in
facility and study records,
• Ensure calibrations are performed as planned and prescribed,
• Ensure corrective actions are implemented and documented as planned in
response to items of nonconformance.
3.0 ASSESSMENT AND RESPONSE
3.1 SCOPE
3.1.1 Assessments shall be planned, scheduled, conducted, and reported in order to measure the
efficacy of the Battelle quality system.
-------
Section 1
Page 26 of 5
Version 1.0
Date: 2/23/09
3.1.2 Assessment and response elements shall include assigning appropriate, qualified persons to
conduct assessments at planned, scheduled intervals; having provisions for timely
responses and implementation of corrective actions if needed; and completing the
evaluation process with written reports to technical and management staff.
3.1.3 Assessment types, responsibility, and schedule for the MMR Center as shown in Table 3.0,
and are defined as follows:
Quality Systems Audit, an on-site review of the implementation of the MMR Center
quality system as documented in the MMR Center QMP. This review is used to verify the
existence of, and evaluate the adequacy of, the internal quality system. A QSA may be a
self-assessment or an independent assessment by EPA.
Technical Systems Audit, a qualitative on-site evaluation of sampling and/or measurement
systems associated with a particular verification test. The objective of the Technical
Systems Audit (TSA) is to assess and document the acceptability of all facilities,
maintenance, calibration procedures, reporting requirements, sampling, and analytical
activities, and quality control procedures in the test. Conformance with the Test/QA Plan
and associated methods and/or Standard Operating Procedures is the basis for this
assessment. The Battelle MMR Center Quality Manager, or designee, conducts a TSA at
least once during each verification test. The EPA MMR Center Quality Manager conducts
an independent TSA once per year, as applicable, for the MMR Center.
Performance Evaluation Audits, a quantitative evaluation of a measurement system. The
type and frequency of performance evaluation (PE) self-audits to be performed will be
specified in the Test/QA Plan for each verification test. The value or composition of
reference materials must be certified or verified prior to use, and the certification or
verification must be adequately documented. The Battelle MMR Center Quality Manager,
or designee, will review results of PE audits. The need for independent PE audits will be
determined by the EPA MMR Center Quality Manager.
Audits of Data Qualityi an examination of the verification data after they have been
collected and 100% verified by project personnel. The Battelle MMR Center Quality
Manager, or designee, will audit at least 10% of all verification data, including equations
and calculations. The need for independent audits of data quality (ADQ) will be
determined by the EPA MMR Center Quality Manager.
-------
Table 3.0 Assessments for the MMR Center
Section 1
Page 27 of 5
Version 1.0
Date: 2/23/09
Assessment Respond Subject of Minimum Reason for
Level Tool Assessors ers Assessment Frequency Assessment Report Reviewed by
Center
Center
Center
Center
Quality
Systems
Audit
Technical
Systems
Audits
Performance
Evaluation
Audits
Audits of
Data Quality
M
Battelle MMR
Center Quality
Manager
Independent
EPA MMR Center
Quality Manager
Self
Battelle MMR
Center Quality
Manager
Independent
EPA MMR Center
Quality Manager
Self
Battelle MMR
Center Quality
Manager
Independent
EPA MMR Center
Quality Manager
Self
Battelle MMR
Center Quality
Manager
Independent
EPA MMR Center
Quality Manager
Battelle
Battelle
Battelle
Battelle
center QMP
Test/QA Plans
Test/QA Plans
raw data and
summary data
once;
thereafter, as
requested
M
Once per
verification test
Independent
once per year,
as applicable
M
each test, as
applicable
Independent
for each center,
as applicable
Self
At least 10% of
the verification
data
Independent
for each center,
as applicable
assess quality
management
practices of
verification
partner
assess
technical
quality of
verification
tests
assess
measurements
performance
assess data
calculations
and reporting
EPA directors of
quality assurance
EPA MMR Center
Project Officer
Battelle MMR Center
Manager
ETV Program Director
EPA MMR Center
Project Officer
EPA MMR Center
Quality Manager
Battelle MMR Center
Manager
M
Battelle MMR Center
Manager
Independent
EPA MMR Center
Project Officer
EPA MMR Center
Quality Manager
Self
Battelle MMR Center
Manager
Independent
EPA MMR Center
Project Officer
EPA MMR Center
Quality Manager
3.2 GENERAL REQUIREMENTS
3.2.1 Each assessment must be fully documented. The Battelle MMR Center Quality Manager
will archive all internal assessment reports generated on the MMR Center.
Each assessment must be responded to by the appropriate level of management. The
Battelle quality assessment reports shall require a written response by the person
performing the inspected activity, and acknowledgment of the assessment by the Battelle
MMR Center Manager and the Battelle MMR Center Manager. The assessment reporting
forms are provided in Appendix III.
3.2.2 Corrective action must be documented and approved on the original assessment report, with
detailed narrative in response to the assessor's finding. Initials and date are required for
-------
Section 1
Page 28 of 5
Version 1.0
Date: 2/23/09
each corrective action response. Acknowledgment of the response will be provided by the
Battelle MMR Center Manager.
3.2.3 Implementation of corrective actions must be verified by the Battelle MMR Center Quality
Manager or designee to ensure that corrective actions are adequate and have been
completed. This will be done in real-time if corrective actions can be immediately
performed and signed off on the assessment report. Alternatively, should the corrective
action require additional approvals not immediately available on-site, the Battelle MMR
Center Quality Manager or designee may need to repeat the inspection in order to
corroborate the implementation and effectiveness of the corrective action.
3.3 PLANNING AND PROCEDURES
3.3.1 Assessment Planning
Assessment planning is performed by Battelle's Quality and MMR Center Managers prior
to the actual performance of any assessments. Planning the assessment scope helps provide
the type of evaluation information needed to determine whether procedural compliance and
technical requirements are being met during verification testing.
Assessment planning by Battelle shall include a kickoff meeting with the verification
testing team where at least the following information may be discussed:
• Schedule of assessment(s),
• Proper completion of data records
• Notification to affected parties,
• Specific assessment requirements (personnel lists, equipment lists, and availability of
Test/QA Plans),
• Follow-up procedures for corrective action, including debriefing and discussion of
possible resolutions,
• Corrective action guidelines to facilitate completion of the reported assessment,
• Appropriate management signature approval of the reviewed assessment report.
3.3.2 Personnel Qualifications for Assessment
The principal Battelle inspector shall be the Battelle MMR Center Quality Manager, who
will have an extensive quality assurance laboratory and field inspection background, and
technical and management experience, and who will be directly familiar with the MMR
Center assessment requirements. Should the need arise, the Battelle MMR Center Quality
Manager will designate an individual to perform scheduled assessments, based upon that
person's technical skill and knowledge of QMP compliance requirements and Test/QA Plan
specifications. Battelle personnel conducting assessments shall have the responsibility and
authority to:
• identify and document problems affecting the quality of verification results,
• propose recommendations for resolving these problems,
• independently confirm implementation and effectiveness of solutions.
-------
Section 1
Page 29 of 5
Version 1.0
Date: 2/23/09
3.3.3 Stop Work
Assessor responsibility and authority to stop work during a verification test for safety and
quality considerations is delegated to Battelle, who must ensure compliance with all
applicable federal, state, and local safety policies during the performance of verification
testing.
Should it be determined during an assessment that adverse health effects could result, or
that test objectives of acceptable quality cannot be achieved during performance of
verification testing, the Battelle MMR Center Quality Manager is responsible for
immediately notifying the Battelle Center Manager of the need to consider a stop work
order. The Battelle MMR Center Manager shall then direct the MMR Center staff
accordingly.
Should any MMR Center staff suspect compromise to personal health or test objectives
during the conduct of verification testing, that staff member shall immediately contact the
Battelle MMR Center Manager who will issue the stop work order.
The EPA MMR Center Quality Manager is delegated to notify the EPA MMR Center
Project Officer who will notify the Battelle MMR Center Manager to facilitate a stop work
order if work of inadequate quality is discovered.
Documentation is required of any stop work order and the corrective action implemented
and shall be maintained as part of the Battelle quality records, with a copy provided to the
EPA MMR Center Project Officer and EPA MMR Center Quality Manager.
3.3.4 Internal Assessment Reporting
Authority to effectively report internal technical system audits, performance evaluation
audits, and audits of data quality is assigned to the Battelle MMR Center Quality Manager
or designee. These reports should:
• Identify and document problems that affect quality and the achievement of objectives
required by the QMP, Test/QA Plan, and any associated Standard Operating
Procedures,
• Identify and cite noteworthy practices that may be shared with others to improve the
quality of their operations and products,
• Propose recommendations (if requested) for resolving problems that affect quality,
• Independently confirm implementation and effectiveness of solutions,
• Provide documented assurance (if requested) to line management that, when problems
are identified, further work performed is monitored carefully until the problems are
suitably resolved.
3.3.5 Response
Responses to TSA adverse findings should be addressed within 10 working days after the
TSA report is completed. However, it is expected that findings that have a direct impact on
-------
Section 1
Page 30 of 5
Version 1.0
Date: 2/23/09
the conduct of a verification test will be corrected immediately following notification of the
finding.
Responses to each adverse finding shall be documented in the assessment report (QMP
Section 3.3.4). Ideally, assessment reports will provide space after each adverse finding for
a response to be recorded. The response will indicate the corrective action taken or planned
to address the adverse finding. The response should be signed and dated by the staff
responsible for implementing the corrective action.
Any corrective action that cannot be immediately implemented should be verified
following completion by the Battelle MMR Center Quality Manager or designee. Once all
corrective action associated with an assessment report has been taken, the Battelle MMR
Center Quality Manager or designee will initial the corrective action in the assessment
report thus documenting verification of the corrective action. Any impact that an adverse
finding had on the quality of verification test data should be addressed in the verification
test report.
The TSA report, with responses to adverse findings recorded within, will be sent to EPA
within 10 working days after the Battelle MMR Center Quality Manager has verified all
corrective actions.
3.4 DATA VALIDATION
Validation is based on the performance measures for the test specified during the design process.
The usability of a verification report and statement is determined relative to how well it
determines the performance of the tested technology under the conditions of testing. Any
limitations on the data and recommendations for limitations on data usability are documented in
the data audit report and the ETV verification report. All validation procedures are performed by
the Verification Test Coordinator and then reviewed by the Battelle MMR Center Quality
Manager.
3.5 REPORT REVIEW
Review and approval procedures for verification reports and statements are given in Table 2.0.
Verification reports are peer-reviewed by external reviewers and verification statements are
signed by an EPA laboratory director and Battelle management.
3.6 QUALITY IMPROVEMENT
3.6.1 Policy
A continuous quality improvement process is considered essential for Battelle staff to develop
a more responsive quality system in all aspects of technical and management activities.
-------
Section 1
Page 31 of 5
Version 1.0
Date: 2/23/09
3.6.2 Annual QMP Review
An annual review of the QMP for the MMR Center shall be conducted by the Battelle MMR
Center Quality Manager and technical and management staff in order to incorporate
improvements to the quality system process. MMR Center QMP revisions may be delayed
beyond one year such as when an update to the ETV QMP is pending. Approval for any delay
to the annual review will be obtained from the EPA MMR Center Quality Manager.
Any revisions to the QMP will be compiled by the Battelle MMR Center Quality Manager for
review, approval, and distribution. The QMP review will be documented by the Battelle
MMR Center Quality Manager and Battelle MMR Center Manager by signing and dating the
revised QMP routed for review and approval.
3.6.3 Problem Identification and Resolution
Detecting and correcting quality system problems is a result of qualified MMR Center
technical and management staff implementing not only this QMP, but also the Test/QA Plan
and other procedures. All staff are encouraged to identify problems and offer solutions to
problems in the following quality areas:
• Adequacy of the quality system, as defined in the QMP,
• Consistency of the quality system,
• Implementation of the quality system to specific verification tests,
• Correction of quality system procedures,
• Completeness of documented information,
• Quality of data,
• Quality of planning documents, such as the Test/QA Plans,
• Implementation of the work process.
Cause and effect relationships of significant problems shall be documented by the Battelle
MMR Center Quality Manager. When problems are reported to the Battelle MMR Center
Quality Manager, attempts to determine the root cause based on cause and effect during
performance of planned and documented procedures will be made, through intensified
observations of testing activities and audits of test data. When problems are identified for
the quality system, the Battelle MMR Center Manager will contact the EPA MMR Center
Project Officer of the problem(s) and corrective action(s).
Collaboration with trained technical/management staff associated with or performing the
activity can provide insight and determine whether any of the following is required:
• A Test/QA Plan change,
• A management system change, or
• A quality system change within the MMR Center.
Assessment reports can also serve as tools to determine cause and effect relations of
significant problems that might require testing protocol, management system, or quality
system changes. Continual monitoring and evaluation by the EPA MMR Center Quality
-------
Section 1
Page 32 of 5
Version 1.0
Date: 2/23/09
Manager, for example, may indicate trends or common and recurring problems for an entire
technology evaluation. If recurring problems are identified, Battelle will immediately
communicate this situation to the EPA MMR Center Project Officer.
Root cause determination is immediately reported by Battelle to the EPA MMR Center
Project Officer prior to any planned implementation of preventative measure. Once the
root cause determination is verified, appropriate actions can be planned, documented, and
implemented by the MMR Center staff.
3.6.4 Ongoing Quality Improvement
Quality improvement action is ongoing in the Battelle quality system, where quality issue
action items can be reviewed by all levels of line management at periodic continuous
improvement meetings. Quality processes are continually monitored and both short-term
and long-term quality issues are identified through customer feedback and client
involvement, peer review and internal lessons learned, and program reviews.
-------
Section 1
Page 33 of 5
Version 1.0
Date: 2/23/09
APPENDIX I
NAMES, ADDRESSES, AND PHONE NUMBERS OF
BATTELLE MMR CENTER KEY STAFF
-------
Section 1
Page 34 of 5
Version 1.0
Date: 2/23/09
KEY BATTELLE MMR CENTER STAFF
Center Manager:
Ms. Amy Dindal
1801 Waldorf Dr.
Royal Palm Beach, FL 33411
Phone:561-422-0113
Fax: 614-458-6697
e-mail: dindala@battelle.org
Quality Manager:
Mr. Zachary Willenberg
505 King Avenue
Columbus, OH 43201
Phone: 614-424-5795
Fax: 614-458-5795
e-mail: willenbergz@battelle.org
Stakeholder Coordinator:
Ms. Maria Gordon
505 King Avenue
Columbus, OH 43201
Phone: 614-424-5278
Fax: 614-458-5278
e-mail: gordonmigibattelle.org
-------
Section 1
Page 35 of 5
Version 1.0
Date: 2/23/09
APPENDIX II
AMENDMENT AND DEVIATION FORMS
-------
Section 1
Page 36 of 5
Version 1.0
Date: 2/23/09
TEST/QA PLAN AMENDMENT
TEST/QA PLAN TITLE AND DATE:
AMENDMENT NUMBER:
EFFECTIVE DATE:
PART TO BE CHANGED/REVISED:
CHANGE/REVISION:
REASON FOR CHANGE:
ORIGINATED BY:
Battelle Verification Test Coordinator
DATE
APPROVED BY:
Battelle MMR Center Manager Battelle MMR Center Quality Manager
DATE DATE
Required Distribution - All individuals/organizations listed on distribution for the
applicable Test/QA Plan, including but not limited to:
Battelle MMR Center Manager
Battelle MMR Center Testing Staff
Battelle MMR Center Quality Manager
Subcontractors (if any)
Verification Test Collaborators (if any)
EPA/ETV MMR Center Project Officer
EPA/ETV MMR Center Quality Manager
Vendors
Distribution must be documented
-------
Section 1
Page 37 of 5
Version 1.0
Date: 2/23/09
TEST/QA PLAN DEVIATION REPORT
TEST/QA PLAN TITLE AND DATE:
DEVIATION NUMBER:
DATE OF DEVIATION:
DESCRIPTION OF DEVIATION:
CAUSE OF DEVIATION:
IMPACT OF DEVIATION ON THE TEST:
CORRECTIVE ACTION:
ORIGINATED BY:
Battelle Verification Test Coordinator
DATE
ACKNOWLEDGED BY:
Battelle MMR Center Manager Battelle MMR Center Quality Manager
DATE DATE
Required Distribution - All individuals/organizations listed below:
Battelle MMR Center Manager
Battelle MMR Center Quality Manager
Distribution must be documented
-------
Section 1
Page 38 of 5
Version 1.0
Date: 2/23/09
APPENDIX III
ETV ASSESSMENT REPORTING FORMS
-------
Section 1
Page 39 of 5
Version 1.0
Date: 2/23/09
Quality Assurance Routing Sheet
ETVMMR Center
Verification Test:
Audit Type:
Test Coordinator:
Vendor:
Auditor: Date:
Test Coordinator, please complete the attached form indicating CORRECTIVE ACTION
TAKEN (IF NEEDED), sign and date this Routing Sheet in the space provided beside
your name, and return the entire set when completed to the BattelleMMR Center Quality
Manager no later than .
Route To Signature Date
Test Coordinator
Battelle MMR Center
Quality Manager
Approval
Battelle MMR Center
Manager
Battelle MMR Center
Quality Manager
-------
Section 1
Page 40 of 5
Version 1.0
Date: 2/23/09
Audit Comment Sheet
Instructions: The Battelle MMR Center Quality Manager will fill out the first column for the audit indicated above. The Verification Test Coordinator will respond to
the comments and initial and date the response in column three. The Battelle MMR Center Quality Manager will verify and document that the response/corrective action
has been completed by initialing and dating the final column.
QA Comment
Test Coordinator
Response/Corrective Actions
Responder
Initials/Date
QA Initials/
Date
------- |