Environmental Technology Verification
Water Quality Protection Center
Quality Management Plan
Revision 3
November 2004
Developed by:
NSF International
789 Dixboro Road
Ann Arbor, MI 48105
Authors and Contributors:
Thomas Stevens
Maren Roush
WQPC QMP Page 1 of 32
Revision 3 - November 2004
-------
Environmental Technology Verification (ETV)
Water Quality Protection Center (WQP)
Quality Management Plan
Revision 3
November 2004
Thomas Stevens Raymond Frederick
NSF International U.S. Environmental Protection Agency
WQP Center Project Manager WQP Center Project Officer
Bruce DeMaine Carolyn Esposito
NSF International U.S. Environmental Protection Agency
WQP Center Quality Manager WQP Center Quality Manager
WQPCQMP Page 2 of 32
Revision 3 - November 2004
-------
Contents
Contents 3
Acronyms and Abbreviations 5
Introduction 6
Background 6
Environmental Technology Verification Program 6
Program Description 7
Operation of the Center 7
Program and Quality Management Documents 8
The ETV Quality Management Plan 8
WQP Center 8
WQP Center - Quality Management Plan 9
1. Management Systems 10
1.1. Management and Organization 10
1.1. WQP Center Quality Policy 10
1.2. Organization Structure 10
1.2.1. NSF Corporate Organization Structure 10
1.2.2. Center Structure - NSF Personnel 10
1.2.2.1. Vice President, Research 10
1.2.2.2. Project Manager, WQP Center 10
1.2.2.3. Project Coordinators, WQP Center 11
1.2.2.4. Secretary, Federal Programs 11
1.2.2.5. Director, Quality Assurance 11
1.2.3. Center Structure - EPA Personnel 12
1.2.4. WQP Center Stakeholder Advisory Groups 12
1.2.5. WQP Center Technology Panels 12
2. Quality System and Description 13
2.1. Corporate 13
2.2. Federal Programs 14
2.3. WQP Center 14
2.4. Laboratories 14
3. Assessments 15
3.1. Quality Management Systems Review 15
3.1.1. Assessment of Corporate Quality Assurance 15
3.1.2. Center Quality Systems Audits 15
3.1.2.1. Independent Assessment -EPA 15
3.1.2.2. Self Assessment-NSF 15
3.1.3. Other Assessments 15
4. Conformance to E4 16
4.1. Water Quality Protection Center QMP 16
4.2. Quality Management System (Part A of E4) 16
4.3. Collection and Evaluation of Data (Part B of E4) 16
5. Collection and Evaluation of Data 17
5.1. Planning and Design 17
5.2. Implementation 17
5.3. Reporting 17
5.4. Assessments 18
5.4.1. Technical Systems Audits 18
5.4.1.1. Independent - EPA 18
5.4.1.2. Self-NSF 18
5.4.2. Performance Evaluation Audits 18
5.4.2.1. Independent - EPA 18
5.4.2.2. Self-NSF 18
5.4.3. Audits of Data Quality 19
5.4.3.1. Independent - EPA 19
5.4.3.2. Self-NSF 19
6. Use of Existing Data 20
WQPCQMP Page 3 of 32
Revision 3 - November 2004
-------
Attachment A: Location of E4 Requirements in NSF Reference Documents 22
Appendix A: NSF Standard Operating Procedures Referenced in WQP QMP 33
WQPCQMP Page 4 of 32
Revision 3 - November 2004
-------
Acronyms and Abbreviations
ANSI American National Standards Institute
ASQC American Society for Quality Control
CQAM Corporate Quality Assurance Manual
E4 ANSI/ASQC E4-1994, Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs
EPA Environmental Protection Agency
ETV Environmental Technology Verification Program
GSA General Services Administration
LQAM NSF Laboratory Quality Assurance Manual
NERL National Exposure Research Laboratory
NRMRL National Risk Management Research Laboratory
NSF NSF International
ORD EPA's Office of Research and Development
QMP Quality Management Plan
SOP Standard Operating Procedure
WQPCQMP Page 5 of 32
Revision 3 - November 2004
-------
Introduction
Background
Environmental Technology Verification Program
The following background information on the ETV Program is excerpted from the Environmental Protection Agency's
Environmental Technology Verification Program Quality Management Plan (EPA Report No. EPA/600/R-03/021).
The Environmental Technology Verification Program (ETV) was established by the Environmental Protection Agency (EPA) to
evaluate the performance characteristics of innovative environmental technologies across all media and to report objective
performance information to the permitters, buyers, and user of environmental technology. ETV evolved in response to the
following mandates:
• A 1995 Presidential directive to EPA in Bridge to a Sustainable Future, to "work with the private sector to establish
a market-based verification process...which will be available nationally for all environmental technologies within
three years."
• Goals articulated in the Administration's Reinventing Government: A Performance Review, which directed EPA to
begin a comprehensive environmental technology verification program no later than October 1995.
• Congressional appropriation language contained in the FY96 and FY97 budgets, that the Agency fund technology
verification activities at the $10 million level in each year.
To comply with these directives, EPA's Office of Research and Development (ORD) established a five-year pilot program to
evaluate alternative operating parameters and determine the overall feasibility of a technology verification program. ETV began
the five-year pilot period in October 1995. At the conclusion of the pilot period, the Agency prepared a Report to Congress
containing an evaluation of the results of the pilot program and recommendations for its future operation.
Credible, high-quality performance information is one of the tenets of ETV. Therefore, the highest appropriate level of quality
assurance is used throughout the program. EPA's Office of Research and Development, under which ETV operates, has
implemented an Agency-wide quality system to assure that activities conducted in EPA research laboratories and other facilities
or at facilities being operated on behalf of or in cooperation with the EPA are supported by data of known and acceptable quality
for their intended use. Each of the ORD laboratories involved in ETV, the National Risk Management Research Laboratory
(NRMRL) and the National Exposure Research Laboratory (NERL), operate under laboratory-specific quality management plans
(QMPs). The ETV QMP is consistent with the policies expressed in the individual laboratory QMPs and is intended to provide an
overarching uniform quality system for all aspects of the ETV program.
WQPCQMP Page 6 of 32
Revision 3 - November 2004
-------
Program Description
Developers of innovative environmental technology report numerous impediments to commercialization. Among those most
frequently mentioned is the lack of acceptance of vendor performance claims. The success of the pilot program shows that
objective, independently acquired, high-quality performance data and operational information on new technologies significantly
facilitates the use, permitting, financing, export, purchase, and general marketplace acceptance of such technologies. ETV
provides this data and information to the customer groups that require them to accelerate the real world implementation of
improved technology. Improved technology more thoroughly, rapidly, and efficiently protects human health and the
environment. It is important to stress that the product of ETV is high-quality data and information, not technology approval or
endorsement. Although there is substantial EPA involvement in guiding and administering this program, ETV does not provide
EPA endorsement or certification of commercial products.
At the conclusion of the pilot period the Agency internally reviewed the performance and operation of the program to assess its
future direction and scope. The ETV Director recommended consolidation of the program into six technology centers:
• Advanced Monitoring Systems Center (AMS)
• Air Pollution Control Technology Center (APCT)
• Greenhouse Gas Technology Center (GHG)
• Drinking Water Protection Center (DWS)
• Water Quality Protection Center (WQP)
• Pollution Prevention, Recycling and Waste Treatment Center (P2,R,WT)
During 2000 and 2001 the first five centers above were established. The sixth center was not put in place due to a lack of
adequate funding to support it. In addition, subsequent to the terrorist attacks of September 11th, 2001, the role of ETV in
verifying homeland security type environmental technologies was brought to the forefront. As vulnerabilities in the Nation's
critical infrastructure were identified, two areas of particular concern - drinking water supply systems, and the health and safety of
the Nation's workforce in their places of employment - became a focus for EPA. ETV was called upon to support the Nation's
homeland security efforts by adapting its testing and evaluation process for technologies for protecting and cleaning up drinking
water systems and buildings. ORD realized the tremendous testing and evaluation capability that resided in the various
technology verification organizations that have been and continue to operate each center, therefore, it was determined to be in the
best interest of the Nation to utilize the existing technology verification organizations, where possible and practical, to execute
this mission. Three of the existing verification organizations - Battelle, Research Triangle Institute, and NSF International - were
enlisted to support the homeland security-related technology testing and evaluation needs. A new center operated by Battelle - the
Building Decontamination Technology Center (BDT) - was established, as the existing verification organization funding
agreements did not have this type technology verification within their scopes of work.
The homeland security efforts fall into two categories: water security and safe buildings. The water security efforts are being
accomplished through the existing cooperative agreements with the AMS, DWS, and WQP centers. The safe buildings efforts are
being accomplished through General Services Administration (GSA) contract to two existing verification organizations, Battelle
and Research Triangle Institute. Contracts through GSA were chosen in lieu of cooperative agreements because it was believed
that for this area the Agency and federal government would have to be directive in many of the elements of the effort: the
stakeholders, the test methods, deciding on the technologies to be tested, and possibly other aspects. This was in contrast to the
water security effort in which the ETV management process consistent with cooperative agreements was to be followed. None of
the contract funding will be commingled with cooperative agreement funding by the technology verification organizations. The
ETV project officers have been trained in the distinctions between managing cooperative agreements and contracts.
Operation of the Center
The technology verification organizations are all not-for-profit entities that work with or for EPA through an extramural
agreement (a cooperative agreement, an interagency agreement, or an existing General Services Administration (GSA) contract).
EPA and the technology verification organization roles are identical to those established during the pilot period. Each agreement
has oversight by an EPA project officer who may also be the EPA Center Manager. EPA provides substantial oversight through
an active quality assurance program. Each technology verification organization is contractually required to fully implement EPA
QA requirements for planning, auditing, and documenting the testing and reporting activities. Qualified peer reviewers are also
utilized to review the technical aspects of the test plans and of the final reports.
WQPCQMP Page 7 of 32
Revision 3 - November 2004
-------
Program and Quality Management Documents
Several documents define the overall operation of the Program. The first to be published (February, 1997) was the Environmental
Technology Verification Strategy. This document describes the goals, customer and key word definitions, basic operating
principles, project selection criteria, and the programmatic and budgetary vision of the program. The Strategy is evaluated
periodically for the need for modification and amplification. The second major program management document being used by
ETV to guide its operation is the ETV Quality Management Plan (QMP).
The ETV Quality Management Plan
The ETV QMP uses the structure, policies, and standards established in the American National Standard Specifications and
Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs (E4)1. This
document "...describes a basic set of mandatory specifications and non-mandatory guidelines by which a quality system for
programs involving environmental data collection and environmental technology can be planned, implemented, and assessed."
Based on the structure and standards of E4, the ETV QMP contains the definitions, procedures, processes, inter-organizational
relationships, and outputs that assure the quality of both the data and the programmatic elements of ETV. Part A of the ETV
QMP contains the specifications and guidelines that are applicable to common or routine quality management functions and
activities necessary to support the ETV program. Part B contains the specifications and guidelines that apply to test-specific
environmental technology testing activities involving the generation, collection, analysis, evaluation, and reporting of test data.
The ETV QMP was also developed in accordance with EPA Requirements for Quality Management Systems2, a document that
provides the development and content requirements for Quality Management Plans for organizations that conduct environmental
data operations for EPA through contracts, assistance agreements, and interagency agreements; however, it may be used by EPA
as well.
The ETV QMP is designed to play a major role in clearly delineating the roles and responsibilities of all the diverse and important
participants. The ETV Program is organizationally complex. Within EPA, the Program is coordinated though ORD's ETV
Team, consisting of staff from eight branches located in three divisions in two laboratories, NRMRL and NERL, and one ORD
center, the Homeland Security Research Center, including the quality assurance staff assigned to each organizational element.
There are also numerous outside organizations involved through the stakeholder process, the technology verification
organizations who bear most of the quality assurance responsibilities, and testing and consulting companies hired by technology
verification organizations to conduct field and laboratory work. Finally, EPA program offices and regions are increasingly
involved in outreach activities, as are other Federal agencies and states.
WQP Center
The U.S. Environmental Protection Agency's partner in the ETV Water Quality Protection (WQP) Center is NSF International
(NSF), a not-for-profit, non-governmental organization that provides public health and safety-based risk management solutions
and is dedicated to protection of the environment. The goal of the WQP Center is to verify commercial-ready environmental
technologies that protect ground- and surface waters from contamination. Under the WQP Center, technologies are evaluated by
a third party organization, following technically sound test procedures, appropriate QA/QC, and a managed process, to provide
purchasers, specifiers and permitters with credible and relevant data.
Verification protocols are developed for specific technology areas following an open process with broad-based stakeholder input.
The protocols then serve as templates for developing test plans for the evaluation of individual technologies at specific locations.
Verification reports detailing the results of the technology evaluations are made publicly available to assist in marketing, purchase
and permitting of the technologies. Verification statements, executive summaries of each verification test, are also provided.
1 American Society for Quality (ASQ); American National Standard and Guidelines for Quality Systems for Environmental Data
Collection and Environmental Technology Programs: ASQ, Milwaukee, Wisconsin, 1994. (E4)
2 U.S. Environmental Protection Agency; EPA Requirements for Quality Management Systems. EPA/240/B-01/002; 2001
(available on the Internet at www.epa.gov/quality/qs-docs/r2-final.pdf).
WQPC QMP Page 8 of 32
Revision 3 - November 2004
-------
WQP Center - Quality Management Plan
As the Verification Organization for the WQP Center, NSF is required to maintain a center-specific QMP that adheres to the
guidelines set forth in the American National Standard, Specifications and Guidelines for Quality Systems for Environmental
Data Collection and Environmental Technology Programs (E4) and the ETV QMP. NSF has prepared a Water Quality
Protection Center Quality Management Plan (WQP QMP) that outlines NSF's quality systems and demonstrates conformance
with the relevant provisions of E4, EPA QMP Requirements, and the ETV QMP.
Many of the provisions of E4 and the ETV QMP were already addressed in NSF's existing management and quality systems, as
documented in the NSF Corporate Quality Assurance Manual, the NSF Laboratories Quality Assurance Manual, NSF Standard
Operating Procedures (SOPs) and other NSF documents. The NSF documents that are relevant to the operation of the WQP
Center are referenced herein. Other provisions of E4 and the ETV QMP (particularly Part B, Collection and Evaluation of
Environmental Data) are addressed in the SOPs, Protocols, and Test Plans developed specifically for the WQP Center
The purpose of the WQP QMP is to provide the definitions, procedures, processes, inter-organizational relationships, and outputs
that assure the quality of both the data and programmatic elements of the WQP Center. Sections 1 through 4 of the WQP QMP
contain the specifications and guidelines that are applicable to common or routine quality management functions and
administrative activities necessary to support the WQP Center. These sections also detail the organizational structure of both NSF
and the WQP Center
Sections 5 through 6 of the WQP QMP contain the specifications and guidelines that apply to test-specific environmental
activities involving the generation, collection, analysis, evaluation, and reporting of data.
WQPCQMP Page 9 of 32
Revision 3 - November 2004
-------
1. Management Systems
1.1. Management and Organization
1.1. WQP Center Quality Policy
The quality system for the WQP Center seeks to be consistent with that of the ETV Program, EPA QMP Requirements, and
industry consensus standards. To that purpose, the quality system for the WQP Center conforms to the relevant provisions of the
ETV QMP, EPA QMP Requirements, and E4. Attachment A is a table that lists the requirements of E4 and the NSF Reference
Documents (SOPs) that fulfill them. In addition, all SOPs referenced herein are included in Appendix A. Each verification test
shall be performed according to planned, documented, and approved test/QA plans, which are developed by NSF International or
a subcontractor in accordance with NSF SOP AA-394-0002. All technical statements in ETV verification reports shall be
supported by the appropriate data.
The quality management system for NSF International is described in the NSF Corporate Quality Assurance Manual (CQAM),
NSF Document Control No. AF-700-0002, which is included in Appendix A. The CQAM documents NSF's Corporate Mission,
Corporate Values, Quality Policy and Objectives, Quality System Document Structure, and General and Administrative Structure.
In addition, the CQAM describes the quality systems in place for all of NSF's operations.
1.2. Organization Structure
1.2.1. NSF Corporate Organization Structure
The NSF corporate organization structure is described in Section 4 of the CQAM.
1.2.2. Center Structure - NSF Personnel
The WQP Center is one of two ETV Centers operated under separate cooperative agreements between NSF and EPA. The NSF
ETV Centers are overseen by the Vice President, Research. Training records of NSF staff are available from NSF's Human
Resources Department for review by EPA, upon request. Listed below are the general functional responsibilities of the NSF
personnel associated with the WQP Center.
1.2.2.1. Vice President, Research
Gordon Bellen, 734-827-6840
Responsibilities include:
• providing leadership on Center initiatives and long-term goals;
• overseeing outreach activities, including those that impact multiple ETV Centers or other NSF Programs;
and
• communicating with NSF Chief Executive Officer, Corporate Officers, and Senior Management regarding
ETV activities.
1.2.2.2. Project Manager, WQP Center
Thomas Stevens, 734-769-5347
Responsibilities include:
• reporting to Vice President, Research;
• establishing, attending, and conducting meetings of stakeholders;
• maintaining communication with EPA, including the ETV director, ETV team, EPA center manager and
the EPA center quality manager, to assure mutual understanding and conformance with EPA quality
procedures and expectations and ETV policies and procedures;
• managing the oversight and conduct of verification activities;
• assuring that quality procedures are incorporated into all aspects of the each ETV project;
• overseeing the development of verification protocols and test plans;
WQPC QMP Page 10 of 32
Revision 3 - November 2004
-------
• overseeing the development and implementation of the WQP Center Quality Management Plan;
• overseeing the development of ETV verification reports for technology evaluations;
• overseeing the development of an ETV verification statement at the completion of each technology
verification;
• approving contractual agreements with external parties such as protocol developers, peer-reviewers, and
testing organizations; and
• identifying, conducting, and managing Center outreach activities including paper presentations, trade
shows, agency visits, etc.
1.2.2.3. Project Coordinators, WQP Center
Patrick Davison, 734-913-5719
MarenRoush, 734-827-6821
Responsibilities include:
• reporting to Project Manager, WQP Center;
• establishing contact with interested stakeholders and soliciting their participation on Stakeholder Advisory
Groups and Technology Panels and with peer-review activities;
• preparing agendas, meeting materials, presentations, and summaries for meetings of Stakeholder Advisory
Groups and Technology Panels;
• soliciting vendor applications for verification testing;
• coordinating verification protocol/test plan development;
• soliciting potential field testing organizations/sites for verification activities;
• reviewing qualifications of potential field testing organizations/sites and making recommendations to the
Project Manager as to their suitability for conducting testing and preparing reports;
• conducting audits and inspections of verification tests;
• reviewing verification reports and supporting data for completeness and conformance with QA provisions
in the verification protocol and test plan;
• preparing and updating the WQP Center Quality Management Plan;
• preparing and maintaining the WQP Center web pages on the NSF web sites and forwarding content to
EPA for inclusion on its web sites; and
• ensuring that Center documents are distributed to the proper ETV Program personnel.
12.2A. Secretary, Federal Programs
Pat Wilkie, 734-913-5740
Responsibilities include:
maintaining mailing lists for the Stakeholder Advisory Groups, Technology Panels, Contract organizations,
and other stakeholders;
providing administrative support for Center activities such as mailings and faxes; and
making necessary arrangements for on-site and off-site meetings and conference calls.
1.2.2.5. Director, Quality Assurance
Bruce DeMaine, 734-769-5143
Responsibilities include:
ensuring that quality procedures are incorporated into all aspects of the Center;
ensuring that quality assurance reviews have been performed on all verification protocols, product- and
site-specific test plans and verification reports, and that feedback for improvement has been provided and
addressed;
reviewing and approving the WQP QMP; and
conducting annual audits of the WQP Center for conformance with the WQP QMP.
WQPC QMP Page 11 of 32
Revision 3 - November 2004
-------
1.2.3. Center Structure - EPA Personnel
The WQP Center is one of seven centers operating under the EPA's ETV Program. Section 1.2 of the ETV QMP, "Organization
Structure", lists the general functional responsibilities of the EPA personnel associated with the ETV program, including the EPA
center manager.
1.2.4. WQP Center Stakeholder Advisory Groups
The WQP Center Stakeholder Advisory Groups are comprised of individuals representing organizations concerned with water
quality protection. Five Stakeholder Advisory Groups were formed by the WQP Center to address the following technology
areas: Ballast Water Treatment, Decentralized Wastewater Treatment, Infrastructure Rehabilitation, Watershed Protection, and
Wet Weather Flow. Stakeholder Advisory Group members are selected by NSF and EPA and include users of technologies,
technology vendors, consultants, financial institutions, state and local permitters and regulators, EPA regional personnel, and
federal agency officials. Many members represent professional and trade organizations in addition to their specific employer.
Members of a Stakeholder Advisory Group:
• assist in prioritizing the types of technologies to be verified;
• assist in the development and review of verification protocols, as requested;
• review verification reports and statements, as requested;
• assist in definition and conduct of outreach activities appropriate to the technology area and customer groups;
• exchange information with the organizations represented; and
• provide general guidance to Center, especially with respect to Center policies and procedures.
For more information on the roles and responsibilities of WQP Center Stakeholder Advisory Groups, refer to NSF SOP No. AA-
394-0001, which is included in Appendix A.
7.2.5. WQP Center Technology Panels
Technology Panels consist of individuals who have expertise in a particular technology area. Whereas Stakeholder Advisory
Group members provide policy-related input to the Center and assist with prioritization of technology areas to be addressed by the
Center, Technology Panels are formed, as needed, to assist with the development and review/revision of Verification Protocols
and Test Plans for specific technologies or technology areas.
For more information on the roles and responsibilities of WQP Center Technology Panels, refer to NSF SOP No. AA-394-0001,
which is included in Appendix A.
All stakeholders in the WQP Center, in addition to those serving on Stakeholder Advisory Groups and Technology Panels, are
encouraged to participate in Center activities, including meetings and review of documents that have been released for public
comment.
WQPC QMP Page 12 of 32
Revision 3 - November 2004
-------
2. Quality System and Description
2.1. Corporate
NSF has a company-wide quality system to assure that the NSF quality policy is upheld and the NSF quality objectives are
achieved. The quality system is described in the CQAM and related standard operating procedures (SOPs). Corporate quality
systems are reviewed and their description updated annually to reflect changes in the organization and in policy.
Table 2-1 highlights the key elements of the NSF quality system and identifies the NSF Document, by Document Control Number
and Title, in which the element is addressed. The procedures and policies contained in the referenced NSF document apply to the
Federal Programs area of the Engineering and Research Services Department and the WQP Center.
Table 2-1. NSF Quality System Summary and References to Control Documents
Key Elements of the NSF Quality System
Established quality policy and objectives focused on
meeting customer and stakeholder needs applicable to
all of NSF' s operations
Training of NSF personnel
Documentation of procedures, policies and processes
used in NSF's operations
Process to control the distribution and dissemination of
documents such as processes, procedures and contracts
for all operations
Addressing customer feed back and comments
Internal and external audits and inspections of
operations
Record retention
Control Document Reference
AF-700-0002: SECTIONS 2.0 AND 2. 1 IN CQAM
AA-700-0188: CONTINUOUS IMPROVEMENT REPORT
AC-700-0007: CONTINUOUS IMPROVEMENT REPORT
FORM
AF -700-0003: CORPORATE TRAINING MANUAL
AF -700-0001: HUMAN RESOURSES HANDBOOK
AA-700-0001: PERFORMANCE APPRAISALS
SEE THE LIST OF CONTROL DOCUMENTS WHICH BEGIN
WITH THE LETTERS "AA". THESE LETTERS SIGNIFY THAT
THE DOCUMENT IS A PROCEDURE. OTHER LETTER
DESIGNATIONS INCLUDE FORMS (AC), ETC.
AA-759-0006: SUBMITTAL AND ISSUANCE OF
CONTROLLED DOCUMENTS
AA-700-0008: PROCEDURAL MODIFICATION
DOCUMENTATION
AA-700-0 191: CLIENT FEEDBACK MANAGEMENT
SYSTEM
AF-700-0002: SECTION 13 IN CQAM
AF-700-0002: SECTION 14 IN CQAM
AA-700-0 187: CORPORATE QUALITY ASSURANCE
AUDITS
AF-700-0002: SECTION 15 IN CQAM
WQPC QMP
Revision 3 - November 2004
Page 13 of 32
-------
2.2. Federal Programs
The Engineering and Research Services (ERS) Department consists of two main groups of staff: Federal Programs employees,
who are ETV personnel (NSF staff who work for the Drinking Water Systems Center and the Water Quality Protection Center)
and other Engineering and Research Services personnel. The ERS staff typically respond to and manage research projects, one-
time evaluations and special studies.
The highly qualified and credentialed members of the ERS Department have years of experience in developing and performing
test protocols for "non-routine" commercial, municipal, and government projects of all sizes. In addition to operating under the
company-wide quality system outlined in Table 2-1, the ERS Department develops, documents and implements procedures as
needed for its specific operations. SOPs for ERS operations, including those specific to the Federal Programs personnel, are
maintained in the NSF Document Control System.
2.3. WQP Center
In order to achieve the technical and quality goals of the WQP Center and the ETV Program, the WQP Center operates in
accordance with written procedures and guidelines. The procedures and guidelines specific to the Center include the following
documents:
AA-394-0001 Procedures for the Formation, Maintenance, and Role of Stakeholder Advisory Groups and Technology
Panels
AA-394-0002 Verification Protocol, Test Plan, and Verification Report Development Procedures
AK-394-0001 Guidelines for the Conduct of Meetings of the Stakeholder Advisory Groups and Technology Panels
2.4. Laboratories
NSF operates several laboratories for the purpose of conducting chemical, engineering, and microbiological assessments. NSF's
laboratories are accredited by more than 20 organizations at the state, national, and international levels. The laboratories operate
in accordance with the corporate quality system as described in the CQAM, and in accordance with the laboratories' quality
system, as described in the NSF Laboratories Quality Assurance Manual (LQAM), Document Control #AF-840-0001. Analyses
performed on behalf of the ETV Program are done in accordance with the QA/QC procedures established in the LQAM and the
relevant Test Plan(s).
WQPCQMP Page 14 of 32
Revision 3 - November 2004
-------
3. Assessments
3.1. Quality Management Systems Review
Periodic reviews of NSF Corporate and WQP Center quality systems are necessary to establish that the Center's quality
management structure, policies, practices and procedures are adequate for ensuring the quality of results demanded by its
customers.
3.1.1. Assessment of Corporate Quality Assurance
An independent audit of the NSF Corporate Quality Assurance Systems is conducted annually as prescribed in Section 14 of the
CQAM.
Independent accreditation bodies periodically audit NSF's Quality Assurance Systems. This is done as necessary to obtain and
maintain NSF's multiple accreditations.
3.1.2. Center Quality Systems Audits
3.1.2.1. Independent Assessment - EPA
The EPA Center Quality Manager will perform a quality systems audit of the WQP Center, using the WQP QMP as a basis, to
assess the quality management practices of the WQP Center. The quality systems audit will take place once in the first year after
approval of the WQP QMP, and thereafter as requested. The quality systems audit follows the guidelines presented in Part A:
Section 2.7 and Table 9.1 of the ETV QMP. The audit report will be forwarded for review to the EPA Director of Quality
Assurance, EPA Center Manager, the appropriate Project Manager(s) (NSF), and the ETV Director.
3.1.2.2. Self Assessment-NSF
The NSF Director, Quality Assurance, will perform a quality systems audit of the WQP Center, using the WQP QMP as a basis,
as specified in Part A: Section 2.7 and Table 9.1 of the ETV QMP. The quality systems audit will take place once in the first year
after approval of the WQP QMP, and thereafter as requested. The audit report will be forwarded to the Project Manager of the
WQP Center for review and final copies of the report will be distributed to the EPA Center Manager.
3.1.3. Other Assessments
NSF is also responsible for conducting quality assessments related to the collection and evaluation of data under the WQP Center.
Additional required assessments include Technical Systems Audits, Performance Evaluation Audits, and Audits of Data Quality,
the purpose and frequency of which are described in 5.4.
WQPC QMP Page 15 of 32
Revision 3 - November 2004
-------
4. Conformance to E4
The ETV Water Quality Protection Center is committed to conforming to the relevant provisions of E4 and EPA QMP
Requirements.
4.1. Water Quality Protection Center QMP
The WQP QMP is organized so that it can be compared to E4 and EPA QMP Requirements. Attachment A addresses NSF's
quality management system (Part A of E4) and the Center's processes for collecting and evaluating environmental data (Part B of
E4).
4.2. Quality Management System (Part A of E4)
The requirements in Part A of E4 for a quality management system are met by two NSF documents:
• AF-700-0002: Corporate Quality Assurance Manual
• AF-840-0001: Laboratory Quality Assurance Manual
The locations of the specific E4 requirements in these NSF documents are shown in Attachment A. The two referenced
documents are attached to this QMP.
4.3. Collection and Evaluation of Data (Part B of E4)
The requirements in Part B of E4 for the collection and evaluation of test data are met by the following NSF documents:
• AA-394-0002: Verification Protocol, Test Plan, and Verification Report Development Procedures
• Individual protocols and test plans developed for technology evaluations under the ETV Water Quality Protection
Center
WQPC QMP Page 16 of 32
Revision 3 - November 2004
-------
5. Collection and Evaluation of Data
5.1. Planning and Design
Verification Protocols have been and will continue to be developed for various categories of water quality protection technologies
identified as priorities by NSF, EPA, and the relevant Stakeholder Advisory Groups. Verification Protocols outline the scope,
data quality objectives, and the generic experimental design for verification. These protocols are developed in accordance with
NSF Document AA-3 94-0002, Verification Protocol, Test Plan, and Verification Report Development Procedures, which
conforms to the guidelines established in the ETV QMP Part B: Section 2.0 Design of Technology Verification Tests. The
Verification Protocols provide the framework for the development of Test Plans, which are also addressed in NSF Document AA-
394-0002. Test Plans describe the procedures to be followed when evaluating vendor-specific technologies at site-specific
locations.
Each Verification Protocol contains a QA/QC section that provides guidelines for the quality assurance elements to be included in
the technology- and site-specific Test Plans. The QA/QC section of the Verification Protocol addresses QA responsibilities, data
quality indicators, QC checks, data handling, corrective action plans, and other items necessary to verify the quality and
consistency of the work.
Each Test Plan contains a Quality Assurance Project Plan that describes the specific QA/QC methods and procedures to be
followed during all phases of equipment operation, sample collection, handling and analysis, instrument calibration, and data
handling. The Quality Assurance Project Plans contain written procedures for planning and conducting audits.
5.2. Implementation
Most verification tests are conducted at off-site facilities by organizations under contract with NSF. Procedures for NSF
subcontracting are explained in NSF SOP AA-840-0001. The subcontract testing organization, if any, is responsible for
conducting the testing in accordance with the approved Verification Protocol and Test Plan(s). The subcontract organization must
demonstrate its ability to implement the Verification Protocol and Test Plan(s) as written and operate in accordance with the WQP
QMP. If the subcontract organization is to draft the test plan and/or verification report, it must also demonstrate its ability to meet
the requirements outlined in AA-394-0002 Verification Protocol, Test Plan, and Verification Report Development Procedures.
Due to unforeseen occurrences it is sometimes necessary for a subcontract testing organization to propose deletions, additions, or
modifications to an approved Test Plan in order to complete the verification testing. Any changes or additions to the Test Plan
deemed necessary by the contract testing organization shall be documented and approved in advance by the Verification
Organization
5.3. Reporting
NSF and EPA are responsible for the issuance of all ETV Verification Reports and ETV Verification Statements developed under
the WQP Center.
NSF oversees the development of ETV Verification Reports in accordance with AA-394-0002 Verification Protocol, Test Plan,
and Verification Report Development Procedures prior to their submission to EPA. The Project Manager (NSF) or designated
Project Coordinator is responsible for overseeing the preparation of each verification report. Verification reports are reviewed by
the EPA Center Quality Manager, the NSF Director, Quality Assurance, peer-reviewers, and the vendor. The Center Quality
Manager and NSF Director, Quality Assurance, are ultimately responsible for the QA review of each verification report, but may
choose to designate other staff to perform the review, as needed to expedite the process. The reviewers' comments are considered
and the verification report is updated as appropriate. Following the required reviews of the verification report, the Project
Manager (NSF) forwards the reviewed reports to the EPA Center Manager for approval, as specified in the ETV QMP. Final
verification reports are distributed to the ETV Director and are provided to the ETV Webmaster to post on the Internet.
NSF oversees the development of ETV verification statements in accordance with AA-394-0002 Verification Protocol, Test Plan,
and Verification Report Development Procedures. A verification statement is included as an "executive summary" of the
verification test at the beginning of each verification report, and may also be used as a standalone document. The Project
Manager (NSF) is responsible for the preparation of each verification statement. Each verification statement is reviewed by the
EPA Center Manager, the EPA Center Quality Manager, the NSF Director, Quality Assurance, peer-reviewers, the vendor and the
WQPC QMP Page 17 of 32
Revision 3 - November 2004
-------
ETV Director. The Center Quality Manager and the NSF Director, Quality Assurance, are ultimately responsible for the QA
review of each verification statement, but may choose to designate other staff to perform the review, as needed to expedite the
process. The reviewers' comments are considered and the verification statement is updated as appropriate. Following the
required reviews of the verification statement, the revised verification statements are forwarded to the EPA laboratory directors
for approval, as specified in the ETV QMP. Final verification statements are distributed to the ETV Webmaster.
5.4. Assessments
5.4.1. Technical Systems Audits
5.4.1.1. Independent - EPA
The EPA Center Quality Manager will perform a technical systems audit at least once per year, as applicable, in accordance with
Part B: Section 4.1.2 of the ETV QMP. The purpose of the technical systems audit is to assess the technical systems of
verification tests, including all facilities, maintenance, calibration procedures, reporting requirements, sampling and analytical
activities, and quality control procedures, and to ensure their conformance to the approved test plans. Audit reports will be
forwarded to the appropriate EPA Center Manager for review.
5.4.1.2. Self-NSF
The Project Manager (NSF), Project Coordinator, Director, Quality Assurance, or other qualified NSF designee will conduct a
technical systems audit at least once during the verification testing period for each technology evaluated through the WQP Center.
The purpose of the technical systems audit is to assess the technical systems of verification tests, including all facilities,
maintenance, calibration procedures, reporting requirements, sampling and analytical activities, and quality control procedures,
and to ensure their conformance to the approved test plans. Technical systems audits are most useful when conducted early in the
life cycle of a project, when corrective actions (if necessary) can be performed that will minimize the loss of data. Audit reports
will be forwarded to the appropriate Project Manager (NSF), the EPA Center Quality Manager, and the Testing Organization for
review. Subcontract testing organizations will conduct serf-assessments of technical systems at a frequency specified in the
relevant Test Plan.
5.4.2. Performance Evaluation Audits
5.4.2.1. Independent - EPA
The EPA Center Quality Manager will conduct a performance evaluation audit of the measurement systems used in testing, as
applicable, according to Part A: Table 9.1 and Part B: Section 4.1.4 of the ETV QMP. Although each measurement in a test
program could be subjected to a performance evaluation, the critical measurements (designated in the test/QA plan) are more
commonly evaluated. An evaluation of a measurement system usually involves the measurement or analysis of a reference
material of known value or composition. The value or composition of reference material must be certified or verified prior to use,
and the certification or verification must be adequately documented. Ideally, the identity of the reference material is disguised so
that the operator or analyst will treat the material no differently than a test program sample. The audit report will be forwarded to
the EPA Center Manager for review.
5.4.2.2. Self-NSF
As applicable, NSF's Director, Quality Assurance, or another qualified NSF designee, will conduct a performance evaluation
audit of the measurement systems used in testing at least once during the verification testing period for each technology evaluated
under the WQP Center, as specified in Part A: Table 9.1 and Part B: Section 4.1.4 of the ETV QMP. Although each measurement
in a test program could be subjected to a performance evaluation, the critical measurements (designated in the test/QA plan) are
more commonly evaluated. An evaluation of a measurement system usually involves the measurement or analysis of a reference
material of known value or composition. The value or composition of reference material must be certified or verified prior to use,
and the certification or verification must be adequately documented. Ideally, the identity of the reference material is disguised so
that the operator or analyst will treat the material no differently than a test program sample. Audit reports will be forwarded to
the Project Manager (NSF) and the EPA Center Quality Manager for review. Contract testing organizations will conduct serf-
assessments of measurements performance at a frequency specified in the relevant Test Plan.
Laboratories conducting sample analyses for the WQP Center shall be accredited by:
WQPC QMP Page 18 of 32
Revision 3 - November 2004
-------
• The state in which the laboratory operates, if an accreditation program is offered;
• The American Association for Laboratory Accreditation;
• NSF International (according to the LQAM); or
• An equivalent program deemed acceptable by NSF.
Subcontract laboratories shall have documented quality control procedures that include participation in approved, independent
performance audits.
5.4.3. Audits of Data Quality
5.4.3.1. Independent - EPA
To determine the accuracy and usability of data calculations and reporting, the EPA Center Quality Manager will conduct an audit
of data quality for the WQP Center, as applicable. The audit of data quality will be performed on raw data and summary data.
Findings will be documented in a data quality audit report, which will be provided to the EPA Center Manager for review.
5.4.3.2. Self-NSF
To determine the accuracy and usability of data calculations and reporting, NSF's Director, Quality Assurance, or another
qualified NSF designee, will conduct an audit of data quality for each verification test conducted under the WQP Center using
raw data and summary data. Audits of Data Quality are performed on at least 10% of the test data ("10% of the test data" means a
random selection of 10% of the data from all of the measured parameters") that have already been 100% verified by project
personnel, as possible (i.e. it may not be feasible to verify every data point generated by a monitor that measures flow every five
seconds over the course of a verification test). Findings are documented in a data quality audit report, which is signed by NSF's
Director, Quality Assurance, and is provided to the appropriate NSF Project Manager and to the EPA Center Quality Manager for
review. Data quality audit reports will document that NSF's Quality Assurance Department reviewed the data associated with the
verification test for the data quality indicators and that they were found to be acceptable. The data quality audit report will outline
any deficiencies in 1) the data set and 2) the way in which the data were presented in the draft verification report. The data
quality report will note any data quality deficiencies, along with corrective actions taken.
WQPC QMP Page 19 of 32
Revision 3 - November 2004
-------
6. Use of Existing Data
The use of existing data (i.e., data collected outside of the ETV Program) toward the ETV Verification of a water quality
protection technology is considered in accordance with the guidelines established in the ETV QMP Appendix C - Environmental
Technology Verification Program Existing Data: Policy and Process.
WQPC QMP Page 20 of 32
Revision 3 - November 2004
-------
Attachment A: Location of E4 Requirements in NSF Reference Documents
E4: PART A
ITEM
001
002
003
004
005
006
007
008
009
010
Oil
012
013
014
015
016
017
018
019
020
021
022
023
024
E4
REFEREN
CE
NO.
2.01.1.001
2.01.1.002
2.01.1.003
2.01.1.004
2.01.1.005
2.01.1.006
2.01.1.007
2.01.1.008
2.01.1.009
2.01.1.010
2.01.1.011
2.02.1.001
2.02.1.002
2.02.1.003
2.02.1.004
2.02.1.005
2.02.1.006
2.02.1.007
2.02.1.008
2.02.1.009.0
0
2.02.1.009.0
1
2.02.1.009.0
2
2.02.1.009.0
o
6
2.02.1.010
E4
SECTIO
N
mgt
mgt
mgt
mgt
mgt
mgt
mgt
mgt
mgt
mgt
mgt
Qsyst
Qsys
Qsys
Qsys
Qsys
Qsys
Qsys
Qsys
Qsys
Qsys
Qsys
Qsys
Qsys
SPECIFICATION
quality policy
quality system resp. authority, mgt appr
ID of customers and suppliers
ID their needs/expectations, est. work
objectives
negotiation for quality when
problems/constraints
ensure E4 understood and implemented
provide adequate resources
stop unsafe work or delegate authority
assess and document adequacy of Q system
define assessment objectives, determine Q
system, impl. meas.
determine response actions, implement in a
timely manner
Qsys plan, established, documented,
implemented, and assessed
include as org, policy, requi., guidance
necessary
shall ensure product/results of type needed
no initiation of work until Qsys approved
Qsys in QMP approved by mgt for
implementation
controls for projects and how projects are
plan/impl/assess
address all applicable part of E4
general descr. items/prog/activities Qsys
applies to
ID/document activities affecting Q
mgt. resp.
technic. act. resp.
required interfaces
Qsys update, at least annual
NSF REFERENCE
DOCUMENT AND
SECTION
2.0-2.1 in CQ AM
4.0-4.3 in CQAM
2.0-2.1 in CQAM,
Protocols
3.0-3.4 in CQAM,
Protocols
3.0-3.4 in CQAM, AA
700-0191
5. 1.1 in CQAM
1.2 in CQAM
Technology-Specific Test
Plans
3.0-3.4 and 14.0-14.4 in
CQAM
3.0-3.4 and 14.0-14.4 in
CQAM
3.0-3.4 and 14.0-14.4 in
CQAM
3. 0-3. 5 in CQAM
3. 0-3. 5 and 4 in CQAM
3. 0-3. 5 in CQAM
3.0-3.5 and 5.1.1 in
CQAM
3. 5 and 14.4 in CQAM
3. 0-3. 5 in CQAM
3.0-3.5 and 5.11 in
CQAM
3. 0-3. 5 in CQAM
3. 0-3. 5 in CQAM
4.0-4.3 in CQAM
4.0-4.3 in CQAM
3. 2 and 4.0-4.3 in CQAM
3. 5 and 14.0 in CQAM
WQPC QMP
Revision 3 - November 2004
Page 21 of 32
-------
ITEM
025
026
027
028
029
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
E4
REFEREN
CE
NO.
2.03.1.001
2.03.1.002
2.03.1.003
2.03.1.004
2.03.1.005
2.03.1.006
2.04.1.001
2.04.1.002
2.04.1.003
2.04.1.004
2.04.1.005
2.04.1.006
2.04.1.007
2.05.1.001
2.05.1.002
2.05.1.003
2.05.1.004
2.05.1.005
2.05.1.006
2.05.1.007
2.05.1.008
2.05.1.009
2.05.1.010
2.05.1.011
2.05.1.012
E4
SECTIO
N
Per/Tr
Per/Tr
Per/Tr
Per/Tr
Per/Tr
Per/Tr
Procur
Procur
Procur
Procur
Procur
Procur
Procur
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
Doc/Rec
SPECIFICATION
trained for project before starting the work
need for training evaluated and impl.
documentation of training
ensure training when job reqs change
doc/maintain evidence of job proficiency
training resources
planned/controlled/documented to meet client
needs
docs describe item/service needed &
associated technical reqs/crit.
specify parts of E4 supplier resp. for
docs reviewed for accuracy before release
changes to docs subject to same review
procedures to ensure procured items meet
requirements
if stated, demonstrated capability needed
from suppliers
records mgt. procedures required
must be applicable to all, including electronic
media
ID need for control of any specified
documents
must be reviewed for conformance before
release by authoriz.
if used for work, must be kept current
ensure all users understand docs
ID obsolete/superseded docs/ remove from
workplace
maintain docs to show quality of work or
conformance to regs.
maintenance to include
retention/prot/pres/trac/retr
if evid. records, COC for records and
confidentiality proc.
retention times based on contract/stat
requirements
storage, protection from damage, loss,
deterioration
NSF REFERENCE
DOCUMENT AND
SECTION
8.0-8.3 in CQAM, AF-
700-0003, AA-759-0006
8.0-8.3 in CQAM, AF-
700-0003
8.0-8.3 in CQAM, AF-
700-0003, AA-700-0001
8.0-8.3 in CQAM, AA-
759-0006
8.0-8.3 in CQAM, AA-
700-0001
8.0-8.3 in CQAM, AF-
700-0003
CQAM 9.0-9.3, LQAM
5.11-5.13
CQAM 9.0-9.3, LQAM
5.11-5.13;AA-840-0001
NA
CQAM 9.0-9.3, LQAM
5.11-5.13
CQAM 9.0-9.3, LQAM
5.11-5.13
CQAM 9.0-9.3, LQAM
5.11-5.13;AA-840-0001
CQAM 9.0-9.3, LQAM
5.11-5.13; AA-840-0001
10.0-10.5 in CQAM, AF-
700-0002
NA
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM, AF-
700-0002, AA-759-0006
10.0-10.5 in CQAM
WQPC QMP
Revision 3 - November 2004
Page 22 of 32
-------
ITEM
050
051
052
053
054
056
057
058
059
060
061
062
063
064
065
066
067
068
069
070
071
072
073
074
075
E4
REFEREN
CE
NO.
2.06.1.001
2.06.1.002
2.06.1.003.0
0
2.06.1.003.0
1
2.06.1.003.0
2
2.06.1.004
2.06.1.005
2.06.1.006
2.06.1.007.0
0
2.06.1.007.0
1
2.06.1.007.0
2
2.06.1.007.0
3
2.06.1.007.0
4
2.06.1.007.0
5
2.07.1.001.0
0
2.07.1.001.0
1
2.07.1.001.0
2
2.07.1.001.0
3
2.07.1.001.0
4
2.07.1.001.0
5
2.07.1.002
2.07.1.003
2.08.1.001
2.08.1.002
2.08.1.003
E4
SECTIO
N
Comput
Comput
Comput
Comput
Comput
Comput
Comput
Comput
Comput
Comput
Comput
Comput
Comput
Comput
Planning
Planning
Planning
Planning
Planning
Planning
Planning
Planning
Impleme
nt
Impleme
nt
Impleme
nt
SPECIFICATION
all aspects to meets user reqs & conform to
appl. cone. stds.
test hardware/software before use
further testing of configs not req. unless:
software usage scope changes
mods to hard/software configs.
assess changes to configs re: impact on
technical & Qobj.
change components (& config), must retest
program changes, check (& test) config as
needed
configs covered include:
experimental design
design analysis
environmental modeling
process control systems (LIMS, auto
data acquisition, etc.)
environmental data bases
establ./imple/control/ and docu planning
process to:
ID customers, needs, work results
expected
ID tech and Q goals to meet needs
translate goals to tech. specs.
address cost/schedule constraints
ID acceptance criteria for results
plan, docs approved by authorized person.
Before work starts
plan doc include work plans, schedules, and
QAPPs
work accord, to plans and other docs, in the
correct order
mgt. oversight commensurate to work
importance
SOPs for standard, critical operations
NSF REFERENCE
DOCUMENT AND
SECTION
CQAM 7.0-7.3, LQAM
5.8
CQAM 7.0-7.3, LQAM
5.8
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
5. 0-5. 6 in CQAM
5.0-5.6 in CQAM,
Protocols
5.0-5.6 in CQAM,
Protocols
5.0-5.6 in CQAM, Test
Plans
5.0-5.6 in CQAM,
Protocols and Test Plans
5.0-5.6 in CQAM,
Protocols and Test Plans
5.0-5.6 in CQAM,
Protocols and Test Plans
5.0-5.6 in CQAM, Test
Plans
3.0-3.4 in CQAM,
Protocols
3.0-3.4 in CQAM,
Protocols
3. 0-3. 4 in CQAM
WQPC QMP
Revision 3 - November 2004
Page 23 of 32
-------
ITEM
078
079
080
081
082
083
084
085
086
087
088
089
090
091
092
093
094
095
096
097
098
099
100
101
102
E4
REFEREN
CE
NO.
2.08.1.004
2.08.1.005.0
0
2.08.1.005.0
1
2.08.1.005.0
2
2.08.1.005.0
o
5
2.08.1.006
2.08.1.007
2.08.1.008
2.08.1.009
2.09.1.001
2.09.1.002.0
0
2.09.1.002.0
1
2.09.1.002.0
2
2.09.1.002.0
3
2.09.1.002.0
4
2.09.1.003
2.09.1.004
2.09.1.005
2.09.1.006
2.09.1.007
2.09.1.008
2.09.1.009.0
0
2.09.1.009.0
0
2.09.1.009.0
0
2.09.1.009.0
0
E4
SECTIO
N
Impleme
nt
Impleme
nt
Impleme
nt
Impleme
nt
Impleme
nt
Impleme
nt
Impleme
nt
Impleme
nt
Impleme
nt
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
SPECIFICATION
SOPS in good format, sufficient detail
following addressed at minimum:
ID of operations needing SOPs
prep process for SOPs- form, content,
applicability
review/approval for SOPs
tech SOPs reviewed/approved by qualified
personnel
Impl. to include measurement of performance
to tech. Q specs.
monitor work proc.
independence of monitor. personnel
commensurate with work
assess, to be plan/sche/periodically cond.
four types of assessment:
mgt. self-assessment
mgt. ind.-assessment
tech. self-assessment
tech. ind.-assessment
ass. type selected in planning stage of project
must address tech requirement (not just
procedural)
ass. performed according to a described
process
ass. documented, reported to mgt, reviewed
by mgt.
assessors must be qualified
respo. and authority for ass. must be
documented/stop work
authority must include:
ID and document problems
ID lessons learned and share them
propose recommendations
NSF REFERENCE
DOCUMENT AND
SECTION
3.0-3.4 in CQAM, AA-
759-0006
3. 4 in CQAM
3. 4 in CQAM
3. 4 in CQAM
3. 4 in CQAM
3. 4 in CQAM
3. 0-3. 4 in CQAM
3. 0-3. 4 in CQAM
3. 0-3. 4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM, AA-
700-0187
14.0-14.4 in CQAM
14.0-14.4 in CQAM;
WQPQMP
14.0-14.4 in CQAM;
WQPQMP
14.0-14.4 in CQAM,
WQPQMP, Test Plans
14.0-14.4 in CQAM,
WQPQMP
5.0 and 14.0-14.4 in
CQAM, WQP QMP
14.0-14.4 in CQAM,
WQPQMP
14.0-14.4 in CQAM,
WQPQMP
14.0-14.4 in CQAM,
WQPQMP
14.0-14.4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM
WQPC QMP
Revision 3 - November 2004
Page 24 of 32
-------
ITEM
103
104
105
106
107
108
109
110
111
112
E4
REFEREN
CE
NO.
2.09.1.009.0
0
2.09.1.009.0
0
2.09.1.010
2.09.1.011
2.09.1.012
2.10.1.001
2.10.1.002
2.10.1.003
2.10.1.004
2.10.1.005
E4
SECTIO
N
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Resp
Improv.
Improv.
Improv.
Improv.
Improv.
SPECIFICATION
independently confirm corrective action
document to mgt, monitoring of work
where problems
resp. to adverse conclusions must be timely
condition, needing corrective action ID and
responded timely
follow-up required to confirm response
process to be est. / implemented
process to detect and correct problems in all
phases
if big problems, ID cause & effect and root
causes
ID root causes before permanent prev.
measures
plan/doc/imple. appro responses, timely
NSF REFERENCE
DOCUMENT AND
SECTION
14.0-14.4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM
14.0-14.4 in CQAM, AA-
700-0188
14.0-14.4 in CQAM, AA-
700-0188
14.0-14.4 in CQAM, AA-
700-0188
14.0-14.4 in CQAM, AA-
700-0188
WQPC QMP
Revision 3 - November 2004
Page 25 of 32
-------
Referenced NSF Documents:
CQAM: Corporate Quality Assurance Manual, Issue No. 2 (AF-700-0002)
LQAM: NSF International Laboratories Quality Assurance Manual (AF-840-0001)
WQP QMP: Water Quality Protection Center Quality Management Plan (AA-394-003)
AA-700-0001: Performance Appraisals
AA-700-0008: Procedural Modification Documentation
AA-700-0187: Corporate Quality Assurance Audits
AA-700-0188: Continuous Improvement Report
AA-759-0006: Submittal and Issuance of Controlled Documents
AA-840-0001: Qualifying a Subcontract Laboratory
AF-700-0001: Human Resources Handbook
AF-700-0003: Corporate Training Manual
Water Quality Protection Center Verification Protocols and Test Plans (available at www.epa.gov/etv and www.nsf.org/etv
WQPC QMP Page 26 of 32
Revision 3 - November 2004
-------
E4 - PART B
ITEM
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
E4
REFERENCE
NO.
3.01.1.001
3.01.1.002
3.01.1.003
3.01.1.004
3.01.1.005.00
3.01.1.005.01
3.01.1.005.02
3.01.1.005.03
3.01.1.005.04.0
0
3.01.1.005.04.0
1
3.01.1.005.04.0
2
3.01.1.005.04.0
o
5
3.01.1.005.05
3.01.1.005.06
3.01.1.005.07
3.01.1.005.08
3.01.1.005.09
E4
SECTION
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
Plan/Scope
SPECIFICATION
all work (involving generation, acquisition, & use
of environ, data) planned and documented
type & qual. of environ, data, ID and documented
using a systematic planning process
proj. spec, planning involves key users, customers
of data,& tech. staff
results of planning activities subject to review for
conformance to tech. and qual. expectations
proj. planning coordinated among participating
orgs. & include:
definition of proj. /task scope & objs. &
desired action or
result from the work
ID of orgs. (e.g., sampling groups & anal.
labs) that need to
participate in proj. & their role in
planning/imp/ass .
activities
ID of environ, data required to achieve
desired action or
result
ID of QA/QC requirements to establish qual. of
data collected or produced including:
data qual. indicator (e.g., precision, bias)
goals
acceptable level of confidence (or statistical
uncertainty)
level of data valid. & verif . needed
ID of documentation to describe qual. of results
ID of personnel, their skills & required equipment.
ID of applicable regulatory requirements & other
constraints (e.g., time & budget)
ID of conditions under which suspension of work
is necessary
determination of assessment tools (e.g., prog. tech.
reviews, peer reviews, surveillances, readiness
reviews, & tech. audits)
NSF
REFERENCE
DOCUMENT
AND SECTION
AA-3 94-0002 /
Verification
Protocols / Test
Plans
ibid.
AA-3 94-0001 /
AA-3 94-0002 /
Verification
Protocols / Test
Plans
ibid.
AA-3 94-0001 /
AA-3 94-0002 /
AK-3 94-0001
AA-3 94-0002
AA-3 94-0001
AA-3 94-0002
AA-3 94-0002 /
Verification
Protocols / Test
Plans
ibid.
ibid.
ibid.
ibid.
WQPQMP
AA-3 94-0002
AA-3 94-0002
AA-3 94-0002 /
Verification
Protocols / Test
Plans
WQPC QMP
Revision 3 - November 2004
Page 27 of 32
-------
ITEM
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
E4
REFERENCE
NO.
3.01.1.005.10
3.01.1.005.11
3.02.1.001
3.02.1.002
3.02.1.003.00
3.02.1.003.01
3.02.1.003.02
3.02.1.003.03
3.02.1.003.04
3.02.1.003.05
3.02.1.003.06
3.02.1.003.07
3.02.1.003.08
3.02.1.003.09
3.02.1.003.10
3.02.1.003.11
3.02.1.003.12
3.02.1.003.13
E4
SECTION
Plan/Scope
Plan/Scope
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
SPECIFICATION
ID of methods/procedures for storing, retrieving,
analyzing, & reporting data produced
ID of possible methods/procedures (including
waste minimization objs) for char. & disp. of cont.
sample material accumulated during the proj.
Define, control, verify, & document the design of
data collection operation
ID all relevant activities pertaining to environ, data
operations, establish performance specs, ID
appropriate ctrls
design process includes detailed specs for:
assessments during proj. (e.g., surveillance, audits,
performance evaluations).
data reporting requirements.
data validation & verification methods
integrating cost or schedule constraints into design
protection of health & safety of workers & the
public
readiness reviews prior to data collection
reqs. for calibration & perf. eval. samples for anal.
methods
requirements for data (and data base) security,
archival, & retention
requirements for field & lab QA/QC activities
requirements & qualifications for sampling &
analysis personnel
sample handling, packaging, shipping & custody
requirements
selection of analytical methods & their quality
performance expectations
selection of analytical facility or lab
NSF
REFERENCE
DOCUMENT
AND SECTION
AA-3 94-0002 /
Verification
Protocols / Test
Plans
Verification
Protocols / Test
Plans
AA-3 94-0002
AA-3 94-0002
4inWQPQMP/
AA-3 94-0002 /
Verification
Protocols
AA-3 94-0002 /
Verification
Protocols
Verification
Protocols
AA-3 94-0002 /
Verification
Protocols
Verification
Protocols
AA-3 94-0002 /
Test Plans
Verification
Protocols / Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
AA-394-0003
/Test Plans
WQPC QMP
Revision 3 - November 2004
Page 28 of 32
-------
ITEM
148
149
150
151
152
153
154
155
156
157
158
159
160
E4
REFERENCE
NO.
3.02.1.003.14
3.02.1.003.15
3.02.1.003.16
3.02.1.004
3.02.1.005
3.02.1.006
3.02.1.007
3.02.1.008
3.02.1.009
3.02.1.010
3.02.1.011
3.02.1.012
3.02.1.013
E4
SECTION
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
Design Ops
SPECIFICATION
selection of field sampling or testing methodology
(specific sampling or field analytical
instrumentation & other analytical testing
requirements)
techniques for assessing limitations on data use
disp. or min. proc. for wastes prod, during
sampling & analy. operations
ID & control (accord, specs determined during
design) key variables that det. or directly affect
qual. of results
ensure that data are traceable to the proc. (incl.
revisios) used to produce data & to the persons
generating or coll. data
deter. & document data transfer, red., verif. , and
valid, reqs
deter. & spec, in the design data interpretation and
analysis needs, such as the use of specific statist.
methods
ID & doc. reports to manag. re: work status,
interim work results, & ass. results
ID & state restrictions on using interim results (and
data) - defines restriction & the specific data to
which it applies
(if the data are stored in magnetic media) Encode
restrictions with the data and report in
accompanying documentation
document results of the environ, data collection
design process in QAPP & other plan docs
according to reqs. of QA system & line mgmt
review/approve QAPP &/or other plann documents
by designated persons - tech. capable of eval. the
proj.
organizations qual. system ID who must review &
approve proj. -specific QAPP & explain process for
conducting the review
NSF
REFERENCE
DOCUMENT
AND SECTION
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols
AA-3 94-0002 /
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
WQPC QMP
Revision 3 - November 2004
Page 29 of 32
-------
ITEM
161
162
163
164
165
166
167
168
169
170
171
172
173
E4
REFERENCE
NO.
3.02.1.014
3.03.1.001
3.03.1.002
3.03.1.003
3.03.1.004
3.03.1.005
3.03.1.006
3.03.1.007
3.03.1.008
3.03.1.009
3.03.1.010
3.03.1.011
3.03.1.012
E4
SECTION
Design Ops
Implement
Implement
Implement
Implement
Implement
Implement
Implement
Implement
Implement
Implement
Implement
Implement
SPECIFICATION
changes to data collection designs or procedures
(including field changes) subject to same
review/approval protocols as the original document
environ, data operations implemented according to
approved planning docs - by qual. persons Doc.
and report deviations to mgmt.
determine impact & significance of the deviation
on planned operations & make adjustments to such
operations
make approved changes to planning documents,
operating guides & manuals and distribute to proj.
personnel
data collected during impl. shall be traceable to
plans used & persons collecting data
use qual. and accepted services/items in environ.
data operations. Accept, shall be ID on the items
&/or in docs traceable to the items.
perform inspections/accept, test, of samp, meas.,
anal, instr. (or other meas. systems) and their
components to confirm intended use of items
when accept, criteria are not met, defic. resolved &
re-inspection performed
Ctrl tools, gauges, instruments, & other sampling,
measuring, & testing equipment for activites
affecting qual. as required & at specified intervals,
calibrated to maintain accuracy within specified
limits. ID unsuitable equipment
eval. the valid, of measure./ tests performed with
out-of-calibr. equipment - measurements and tests
will be repeated
document basis for the calib. Maintain docs of
calibr. - shall be traceable to the equipment
perform periodic preventive and corrective maint.
of meas./test equipment - ensure avail, and
satisfactory perf. of the systems
re-calibrate all equipment subj. to maintenance or
repair, before the equipment is used
NSF
REFERENCE
DOCUMENT
AND SECTION
AA-3 94-0002 /
Verification
Protocols /Test
Plans
4.3.2inWQP
QMP/
Verification
Protocols /Test
Plans
4.3.2inWQP
QMP/
Verification
Protocols /Test
Plans
4.3.2inWQP
QMP/
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols / Test
Plans
Verification
Protocols / Test
Plans
Verification
Protocols / Test
Plans
Verification
Protocols / Test
Plans
AA-3 94-0002 /
Verification
Protocols / Test
Plans
Verification
Protocols / Test
Plans
WQPC QMP
Revision 3 - November 2004
Page 30 of 32
-------
ITEM
174
175
176
177
178
179
180
181
182
183
E4
REFERENCE
NO.
3.03.1.013
3.03.1.014
3.04.1.001
3.04.1.002
3.04.1.003
3.05.1.001
3.05.1.002
3.05.1.003
3.05.1.004
3.05.1.005
E4
SECTION
Implement
Implement
Ass/Resp
Ass/Resp
Ass/Resp
Ass/Verif
Ass/Verif
Ass/Verif
Ass/Verif
Ass/Verif
SPECIFICATION
perform hand., stor, cleaning, pack., ship., and
pres. of field and lab samples accord, to req. specs,
protocols, or proc. to prevent damage, loss, deter.,
artifacts, or interfer. Track and document sample
chain of custody
perform data or info, mgmt, incl. trans., storage,
valid., assess., process., & retrieval, accord, to
approved instr., methods, procs.
assess activities perf. during environ, data ops. that
affect the qual. of the data & the find, reported to
mgmt. to ensure that the reqs stated in planni docss
(e.g., QAPPs, work & sampling plans) are being
implemented
take appropriate corr. actions and verify/ document
the ass. findings
eval. data obtained prev. from a method or
instrument found to be nonconforming to specs to
determine impact of a nonconform. on qual. of
data. Doc. impact and the approp. action taken
assess, verify, and qualify data obtained from
environ, data operations according to their
intended use
express limitations on this intended data use
(quantitatively) and doc. reporting of the data in
print or electronically
assess data obtained from sources that did not use
a qual. system equiv. to this Standard according to
approved/documented proc.
independ. review of proj. reports w/ data, OR
report results of environ, data ops to confirm that
data/r results are presented correctly.
reports shall be approved by mgmt. prior to
release, publ., or dist.
NSF
REFERENCE
DOCUMENT
AND SECTION
Verification
Protocols / Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
AA-3 94-0002 /
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
Verification
Protocols /Test
Plans
AA-3 94-0002
3.5inWQPQMP
AA-3 94-0002
AA-3 94-0002
Referenced NSF Documents:
WQP QMP: Water Quality Protection Center Quality Management Plan (AA-394-003)
AA-394-0001: Procedures for the Formation, Maintenance, and Roles of Stakeholder Advisory Groups and Technology Panels
AA-394-0002: Verification Protocol, Test Plan, and Verification Report Developmental Procedures
AK-394-0001: Guidelines for the Conduct of Meetings of the Stakeholder Advisory Groups and Technology Panels
WQPC QMP
Revision 3 - November 2004
Page 31 of 32
-------
Appendix A: NSF International SOPs
NSF considers all Controlled Documents, including SOPs, forms or laboratory manuals as
Proprietary Business Information. Therefore, NSF will provide EPA any Controlled Documents
referenced in this QMP upon request. NSF requests that EPA refrain from sharing these documents
with any other party.
WQPCQMP Page 32 of 32
Revision 3 - November 2004
------- |