oEPA
United States
Environmental Protection
Agency
Quality Management
Plan (QMP)
Office of Research and Development (ORD)
National Homeland Security Research Center (NHSRC), 26 W. Martin Luther King Drive, Cincinnati, OH 45268
-------
Quality Management
Plan (QMP)
Office of Research and Development (ORD)
National Homeland Security Research Center (NHSRC), 26 W. Martin Luther King Drive, Cincinnati, OH 45268
-------
Quality Management Plan Contact Information
Title: Quality Management Plan for the National Homeland Security Research Center
Organization: Office of Research and Development
National Homeland Security Research Center
Immediate Office
Address: U.S. Environmental Protection Agency
National Homeland Security Research Center (NHSRC)
26 W. Martin Luther King Drive (NG16)
Cincinnati. OH 45268
FAX: 513 487-2555
Phone: 513 569-7662
-------
APPROVALS
Reggie Cheatham, Director of Quality Staff
Signature
Date
Office of Environmental Information (OEI)
Jonathan G. Herrmann, Director
Signature
Date
National Homeland Security Research Center
Angela L. Hadley, Acting Deputy Director
Signature
Date
Immediate Office of the Director
Gregory Sayles Associate to Director
Signature
Date
Immediate Office of the Director
Nancy H. Adams. Division Director
Signature
Date
Decontamination Consequence Management
Kinr R. Fox. Division Director
Signature
Date
Water Infrastructure Protection
Deborah McKean. Acting Director
Signature
Date
Threat and Consequence Assessment Division
Oba Vincent. Team Leader
Signature Date Response Capability Enhancement
Eletha Brady-Roberts, Director of Quality Assurance
Signature Date Immediate Office of the Director
-------
This page left intentionally blank.
-------
Contents
Approvals iii
Abbreviations and Acronyms ix
List of Tables xi
List of Figures xi
List of Text Boxes xi
1.0 Background 1
1.1 Purpose of NHSRC's QMP 1
2.0 Organizational Structure and QA Management 3
2.1 Organizational Structure 3
2.1.1 Mission 3
2.1.1.1 Directives 3
2.1.2 Quality Policy Statement 3
2.1.3 Program Areas 4
2.1.3.1 Water Infrastructure Protection Research 4
2.1.3.3 Technology Testing and Evaluation Program 5
2.2 NHSRC's Management of Quality Assurance 5
2.2.1 QA Roles and Responsibilities 5
2.2.1.1 Director 5
2.2.1.2 Deputy Director for Management (DDM) 5
2.2.1.3 Deputy Director for Science and Policy (DDSP) 6
2.2.1.4 Director of Quality Assurance (DQA) 6
2.2.1.5 QA Coordinator (QAC) 7
2.2.1.6 QA Manager (QAM) 8
2.2.1.7 Associate Director ( AD) 9
2.2.1.8 Division Director (DD) 9
2.2.1.9 Programmatic Team Leader (PTL) and Program Manager (PM) 10
2.2.1.10 Cross Center Team Leaders (CCTLs) 10
2.2.1.11 Principal Investigator (PI) 10
3.0 Quality System and Description 13
3.1 Quality-Affecting Activities 13
-------
3.2 Principle Components of the Quality System 13
3.2.1 NHSRC Quality Management Plan (QMP) 13
3.2.2 Quality Assurance Annual Report and Work Plan (QAARWP) 13
3.2.3 Extramural Quality Management Plans (QMPs) 13
3.2.4 Systematic Planning 13
3.2.5 Quality Assurance Review Form (QARF) 14
3.2.6 Quality Assurance Project Plans (QAPPs) 15
3.2.7 Graded Approach to Research Projects 15
3.2.8 Facility Manuals 16
3.2.9 Standard Operating Procedures (SOPs) and Methods 17
3.2.10 QA Review and Response Procedures 17
3.2.11 Peer Review 17
3.2.12 Information Quality Guidelines 18
3.2.12.1 Request for Correction 18
3.2.12.2 Request for Reconsideration 18
3.13 A Stop Work Order 18
3.3 Assessments 18
3.3.1 Quality Systems Audit (QSA) 19
3.3.2 Technical Systems Audits (TSAs) 19
3.3.3 Audit of Data Quality (ADQ) 20
3.3.4 Performance Evaluations (PE) 20
3.3.5 Laboratory Surveillances 20
3.3.6 Data Quality Assessments 20
3.3.7 QA Review of Research Products 20
3.3.8 Computer Tracking System 20
3.3.9 Training 20
4.0 Personnel Qualifications and Training 21
4.1 Technical Training 23
4.2 Project Management Training 23
4.3 Quality System Training 23
4.4 Safety Training 23
4.5 Records Management Training 24
4.6 Information Security Training 24
4.6.1 Information Security (INFOSEC) and Physical Security Programs 24
4.6.2 Personnel Security (PERSEC) 24
vi
-------
4.6.3 Automated Information Systems (AIS) and Communications Security (COMSEC) 24
5.0 Procurement of Items and Services 25
5.1 Procurement of Services 26
5.1.1 Mechanisms for Procuring Services 26
5.1.2 Preparing the Extramural Package for Procuring Services 26
5.1.2.1 Contracts 27
5.1.2.2 Assistance Agreements 27
5.1.2.3 Interagency Agreements 27
5.2 Competitive Extramural Actions 27
5.2.1 Single-Project Effort 27
5.2.2 Multiple-Project Effort 28
5.2.3 Alternative Documentation 28
5.2.4 Additional Requirements for Competitive Extramural Actions Exceeding $500,000 29
5.3 Non-Competitive Extramural Actions 29
5.4 QA Approval Process for Extramural Actions 30
5.5 Evaluation of Quality Deliverables 30
5.6 Evaluation of NHSRC's Compliance with QMP 30
6.0 Documentation and Records 31
6.1 EPA Records Management Policy 31
6.2 Project Documentation and Records 31
6.3 Research Laboratory-Specific Documentation and Records 31
6.4 Software Documentation and Records 32
6.5 NHSRC Quality Documentation and Records 32
7.0 Computer Hardware and Software 33
8.0 Implementation of Work Processes 35
8.1 Q APP Development, Review, and Approval 35
8.2 SOP Development 35
8.3 QAPP/SOP Implementation 35
8.4 QAPP/SOP Revisions 35
9.0 Assessment and Response 37
9.1 Assessment Techniques and Practices 37
9.2 Types of Assessments 37
9.2.1 Self-Assessment of a Quality System 37
9.2.2 Independent Assessment of a Quality System 37
-------
9.2.3 Technical Self-Assessment
9.2.4 Independent Technical Assessment
9.3 Roles and Responsibility of the Assessors
9.3.1 Capabilities and Authority
9.3.2 Conflict of Interest
9.3.3 NHSRC-Wide Assessment and Response Procedures
9.4 Assessment Reports and Corrective Action
9.6 Assessment Response Procedures
9.7 Audit Close-out
10.0 Quality Improvement
10.1 Quality System Opportunities for Improvement
10.2 Quality Assurance Tracking System
10.3 Center-wide Opportunities for Improvement
11.0 Supplemental References
Appendix A: NHSRC Minimum Requirements For Center Research Activities
Appendix B: QAPP Requirements And Guidance
Requirements And Guidance
QAPP Requirements For Sampling And Analysis Projects
QAPP Requirements For Basic Research Projects
QAPP Requirements For Applied Research Projects
QAPP Requirements For Research Model Development And Application Projects
QAPP Requirements For Projects Using Secondary Data
QAPP Requirements For Method Development Projects
QAPP Requirements For Software And Data Management Projects
Appendix C: NHSRC Quality Assurance Review (Qarf) Forms
Appendix D: NHSRC QA Requirements/Definitions List
Appendix E: NHSRC Product QA/QC Verification Report Form
Appendix F: Requirements For Quality Management Plans (Qmps)
Appendix G: Routing Slip For Extramural Packages
-------
Abbreviations and Acronyms
ADP
Automated Data Processing
ADQ
Audit of Data Quality
ANSI
American National Standards Institute
ASQ
American Society for Quality
BOSC
Board of Scientific Counselors
CFR
Code of Federal Regulation
CIO
Chief Information Officer
CMD
Contracts Management Division
COMSEC
Communication Security
CRADA
Cooperative Research and Development Agreement
DCMD
Decontamination Consequences and Management Division
DD
Division Director
DQA
Director of Quality Assurance
DQI
Data Quality Indicator
DQO
Data Quality Objective
EPA
US Environmental Protection Agency
PTE
Full-Time Equivalent
FY
Fiscal Year
GALP
Good Automated Laboratory Practices
GPRA
Government Performance Results Act
GIAMD
Grants and Interagency Agreement Management Division
HSDN
Homeland Security Digital Network
H&S
Health and Safety
IA
Interagency Agreement
10
Immediate Office
IQG
Information Quality Guidelines
LAN
Local Area Network
L/C/0
Laboratory/Center/Office
NAS
National Academy of Science
NDPD
National Data Processing Division
NHSRC
National Homeland Security Research Center
NRMP
National Records Management Plan
OEI
Office of Environmental Information
OGD
Office of Grants and Debarment
IRM
Information Resources Management
ORD
Office of Research & Development
PCSC
Personal Computer Site Coordinator
-------
Abbreviations and Acronyms (Cont.)
PE
Performance Evaluation
PI
Principal Investigator
PTL
Programmatic Team Leaders
PM
Program Manager
PO
Project Officer
QA
Quality Assurance
QAARWP
Quality Assurance Annual Report and Work Plan
QAC
Quality Assurance Coordinator
QAM
Quality Assurance Manager
QAPP
Quality Assurance Project Plan
QATRAK
Quality Assurance Tracking System
QARF
Quality Assurance Review/Requirements Form
QC
Quality Control
QMP
Quality Management Plan
QS
Quality Staff
QSA
Quality System Audit
RCE
Response Capability Enhancement Team
RMO
Records Management Office
RTP
Research Triangle Park
SAB
Science Advisory Board
SOP
Standard Operating Procedure
STEs
Secure Terminal Equipment
STPs
Scientific Technical Products
TEP
Technical Evaluation Panel
TL
Team Leader
TCAD
Threat and Consequence Assessment Division
TSCA
Toxic Substances Control Act
TSA
Technical System Audit
TTEP
Technology Testing and Evaluation Program
WS
Water Security
WIPD
Water Infrastructure Protection Division
-------
List of Tables
Tabic 1 Types of Product Reviews 17
Table 2 Selected Scientific Schedules for ORD Research Activities 32
Table 3 Distribution and archival of QAPPs/SOPs 36
Table 4 Types and Frequency of QA Assessment 38
Table B-l Q APP Elements according to the Q A Category 65
Table B- 2 Recommended Documentation for Archiving by Q A Category 66
List of Figures
Figure 1 NHSRC Organization Chart 4
List of Text Boxes
Text Box 1 Quality Assurance related language for Statement of Work 25
-------
This page left intentionally blank.
-------
1.0
Background
The Environmental Protection Agency (EPA) Order
CIO 2105.0 (formerly Order 5360.1 A2) Policy and
Program Requirements for the Mandatory Agency-
Wide Quality System1, May 2000, establishes policy
and program requirements for the preparation
and implementation of quality management
systems. The intent of the Order is to develop a
consistent approach to environmental decisions
that ensures the collection of supporting data that
are scientifically sound, legally defensible, and
of known and documented quality. The Office of
Environmental Information (OEI) Quality Staff
is responsible for developing quality assurance
(QA) and quality control (QC) requirements and
for overseeing implementation of the Agency wide
Quality System.
The Office of Research and Development's National
Homeland Security Research Center (NHSRC) has
adopted the policy described in the EPA Quality
Manual for Environmental Programs, CIO 2105-P-
Ol-O-(formerly order 5360 A1), which articulates
that EPA organizations shall provide for the
follow ing: (1) a Quality Assurance Manager, or
equivalent position, who functions independently
of direct environmental data generation and reports
1 Policy and Program Requirements for the Mandatory Agency-
Wide Quality System, CIO 2105.0, 2000, OEI, USEPA, Wash-
ington, D.C. http://www.epa.gov/irmpoli8/ciopolicy/2105-0.pdf
2 EPA Quality Manual for Environmental Programs, CIO 2105-
P-01-0, 2000, OEI, USEPA, Washington, D.C. http://www.epa.
gov/irmpoli8/ciopolicy/2105-P-01 -0.pdf
on quality issues to the senior manager having
executive leadership authority for the organization,
and (2) a Quality Management Plan (QMP). This
QMP document defines the quality system that
is established and implemented at the National
Homeland Security Research Center. Moreover, it
provides the authority and guidance for how quality
assurance activities are planned, documented,
implemented, and assessed.
Quality System functions pertaining to the NHSRC
are covered by this QMP regardless of where the
work is performed - Cincinnati. Ohio; Research
Triangle Park. North Carolina; Washington D C.;
Las Vegas. Nevada; and contractor work sites.
1.1 Purpose of NHSRC's QMP
The purpose of this QMP is to describe the
NHSRC quality system. The NHSRC quality
system is designed to be useful to project
personnel, managers, and QA staff. This QMP is a
comprehensive document that incorporates quality
policies, and lines of authority and responsibility for
all Center personnel. NHSRC's management and
QA staff work to ensure that (1) standard policies
and procedures are in place for the quality system
elements described in the NHSRC quality system;
(2) all environmental data collection, evaluation,
and use are performed in accordance with an
approved planning document; and (3) all NHSRC
projects (both intramural and extramural) produce
defensible data of defined quality.
1
-------
This page left intentionally blank.
2
-------
2.0
Structure And QA Management
The National Homeland Security Research Center
(NHSRC) is a principal entity with EPA's Office
of Research and Development responsible for
homeland security related research. The Center
develops the scientific foundations that provide
decision makers with the tools they need to prevent
and manage a range of potential threats.
This section discusses NH SRC's organizational
structure, and the Center-wide management and
implementation of the quality system.
2.1 Organizational Structure
The National Homeland Security Research Center's
research focuses on the mitigation of environmental
contamination associated with terrorist threat
scenarios that involve chemical, biological or
radiological contaminants, including measuring
and modeling environmental exposures, studying
human health, quantifying human health risks, and
developing risk assessment methodologies.
The program areas for which NHSRC is responsible
involve numerous scientific and engineering
disciplines; therefore, the Center may engage
experts from other EPA organizations to serve as a
Principal Investigator (PI). The Quality Assurance
Manager (QAM) from the respective organization or
Laboratory/Center/Office (L/C/O) is responsible for
the QA of the research project. Moreover, the lead
PI who is responsible for a project being funded by
NHSRC will be supported by their assigned L/C/O
QAM. In the case where the lead PI is not a member
of NHSRC. the NHSRC quality staff will concur
and sign their initial next to the signature of the
partner QAM.
2.1.1 Mission
NHSRC develops and delivers reliable, responsive
expertise and products based on scientific research
and evaluations of technology. Our expertise and
products are widely used to prevent, prepare for.
and recover from public health and environmental
emergencies arising from terrorist threats and
incidents.
2.1.1.1 Directives
The Public Health Security and Bioterrorism
Preparedness and Response Act (Bioterrorism Act.
2002), together with Homeland Security Presidential
Directives 7, 9, 10 and 22, charge EPA with
protecting our nation's critical water infrastructure;
monitoring for chemical, biological, and
radiological terrorism threats to public health and
the environment; and supporting decontamination
approaches to be used during a terrorist attack.
2.1.2 Quality Policy Statement
The quality policy of NHSRC is that environmental
data generated, processed or used will be of
adequate and sufficient quality for their intended
use. The purpose of the NHSRC Quality Assurance
policy is to:
• Ensure that NHSRC meets EPA QA
requirements as defined in EPA CIO
21053 (http://www.epa.gov/irmpoli8/
ciopolicy/2105-0.pdf) and the guidance
provided in the EPA's Overview for EPA's
Quality System for Environmental Data and
Technology4;
• Implement a quality system that guides QA
planning and quality control activities for
individual projects or programs;
• Ensure that QA reviews are performed on
various products and activities in NHSRC
• Ensure that continuous improvement is
practiced in the implementation of the
NHSRC quality system; and
• Ensure that adequate resources including
3 Policy and Program Requirements for the Mandatory Agency-
Wide Quality System, CIO 2105.0, 2000, OEI, USEPA, Wash-
ington, D.C. http://www.epa.gov/irmpoli8/ciopolicy/2105-0.pdf
4 Overview of EPA Quality System for Environmental Data and
Technology, EPA/240/R-02/003, 2002, OEI, USEPA, http://
www.epa.gov/quality/qs-docs/overview-final.pdf
3
-------
full-time equivalents (FTEs), contractor
support, and travel funds are provided to
implement the NHSRC QMP across the
Center.
2.1.3 Program Areas
As shown in Figure 1, NHSRC has three divisions:
Water Infrastructure Protection and the Threat and
Consequence Assessment are based in Cincinnati,
Ohio and the Decontamination and Consequence
Management is located in Research Triangle
Park, North Carolina. In addition, the Center's
Response Capability and Enhancement team is
located in Cincinnati and the Technology Testing
and Evaluation Program is managed in Las Vegas,
Nevada.
The Center's main program areas are Water
Infrastructure Protection and Indoor/Outdoor
Decontamination. The risk assessment activities
and analytical methods development efforts are
embodied in those two programs areas.
2.1.3.1 Water Infrastructure Protection Research
The focus of water infrastructure protection
research is on improving the nation's ability to
protect from and respond to terrorist attacks on our
water and wastewater infrastructure. The thematic
research areas include:
• Protection and prevention research, which
involves developing tools and methods to
address the vulnerabilities of drinking water and
wastewater systems;
• Detection research, which involves developing
tools and methodologies to detect, confirm,
and measure accidental and intentional
contamination events, and support the
development of a laboratory network;
• Containment and mitigation research, which
involves supporting the development of
planning tools for contamination events, and
tools and methodologies for responding to and
mitigating such events; and
• Decontamination and water treatment
research, which involves developing a
better understanding of the treatment and
decontamination of water infrastructure and
contaminated water.
2.1.3.2 Indoor and Outdoor
Decontamination Research
The main focus of indoor and outdoor
Shawn Ryan (Associate DD) !_ Hiba Ernst (Associate DD) —-Tonya Nichols (Acting Associate DD)
Eletha Brady-Roberts -Dir. QA
Raniona Sherman - QA
Coordinator
Response Capability
Enhancement Team
Oba Vincent - Leader
Testing and Evaluation
Technology Program
Eric Koglin - Program
Manager
Peter Jutro
Deputy Director for
Science and
Policy
Angela Hadley
Acting Deputy Director for
Management
Andre Reinhart - Acting
Associate to Deputy Director
for Management
Decon & Consequence
Management Division
Nancy H. Adams, DD
Water Infrastructure
Protection Division
Kim R. Fox, DD
Threat & Consequence
Assessment Division
Deborah McKean,
Acting DD
Jonathan Herrmann, Director
Mary Sullivan
Administrative Officer and Records Liaison
Gregory Sayles - Associate to Dir.
Cynthia Sonich-Mullin - Assistant to Dir.
Figure 1 NHSRC Organization Chart
-------
decontamination research is on improving the
nation's ability to respond to terrorist attacks
affecting indoor and outdoor environments. The
thematic research areas include:
• Detection research, which results in detection
techniques (e.g., laboratory methods, technology
evaluations) that will enable the rapid
characterization of threats and identification of
contaminants;
• Containment and mitigation research, which
results in reports, databases, and computer
models that provide information on the
movement and health effects of contaminants;
and
• Remediation research, which delivers reports,
techniques, and tools to support the remediation
(decontamination and associated material
disposal) of buildings and outdoor environments
follow ing contamination event associated with
chemical, biological or radiological agents of
concern.
2.1.3.3 Technology Testing and Evaluation
Program
The Technology Testing and Evaluation Program
(TTEP) provides reliable information regarding the
performance of commercially available technologies
that might be used during a homeland security
related incident. Technologies are evaluated to
specifically determine their abilities to:
• Detect contaminants in environmental matrices
to determine which chemical or biological agents
may be present and to characterize areas that
may need to be restored after a terrorist attack;
and
• Decontaminate buildings and water distribution
systems in order to return them to a usable state
after a terrorist attack.
2.2 NHSRC's Management of Quality
Assurance
NHSRC's Director of Quality Assurance (DQA)
is assigned to the NHSRC Immediate Office of the
Director; therefore, the required independence of the
QA function is maintained.
2.2.1 QA Roles and Responsibilities
2.2.1.1 Director
The Director of NHSRC is responsible for:
• Ensuring that EPA quality policy is implemented
at NHSRC by including quality responsibilities
as described in this QMP in the performance
standards of the NHSRC Deputy Director.
Division Directors (DD). QA staff, and Pis;
• Promoting (by words, actions, and involvement)
the establishment of a quality culture at NHSRC;
• Promoting (by words, actions, and involvement)
continuous quality improvement within NHSRC;
• Providing facilities that are conducive to the
production of data of appropriate quality to meet
the needs of the research being conducted;
• Ensuring that information disseminated by
the NHSRC meets the Information Quality
Guidelines;
• Providing adequate resources (including FTEs,
contractor support, and travel) for the operation
of the NHSRC quality system;
• Reviewing and approving the NHSRC QMP;
• Serving as the ultimate decision maker on
unresolved quality issues; and
• Ensuring that any Request for Corrections
(RFC) that affect NHSRC. the ORD IQG Officer
will contact the NHSRC Center Director (or
designee) to participate in the development of a
draft response.
2.2.1.2 Deputy Director for Management (DDM)
The Deputy Director for Management has been
designated by the NHSRC Director as the senior
management designee for ensuring the NHSRC
quality system is implemented NHSRC. The DD
serves as the first-line supervisor for personnel
responsible for quality assurance, extramural, budget
execution, communication, information security,
travel and personnel assigned to the RCE team and
the TTEP effort. In the area of quality assurance, the
Deputy Director for Management is responsible for:
• Informing the Center Director of Q A program
developments;
• Review ing and recommending approval of the
NHSRC QMP;
• Selecting the NHSRC DQA;
• Serving as the NHSRC arbitrator for quality
issues that cannot be resolved at lower levels;
• Supervising and overseeing the activities of the
DQA;
-------
Ensuring that the NHSRC QA system is
implemented across the Center by including
quality responsibilities as described in this
QMP in the performance standards of all
managers QA staff, and Pis in the Center;
Promoting (by words, actions, and
involvement) the establishment of a quality
culture at NHSRC;
Promoting (by words, actions, and
involvement) the NHSRC quality system;
Promoting (by words, actions, and
involvement) continuous quality
improvement within the Center;
Providing adequate resources (including
PTEs, contractor support, and travel funds)
to implement the NHSRC QMP across the
Center;
Reviewing, approving, and subsequently
ensuring that the NHSRC QMP is
implemented and providing written review
comments regarding the QMP to the DQA;
Ensuring that the Center DQA attends
planning meetings when proposed,
new, and/or current research activities
(any activity that generates or collects
environmental data or uses secondary data
for situations related to the environment)
are discussed;
Ensuring that the DQA is aware of all
research activities in the center;
Resolving disagreements within the Center
(e.g.. appropriate corrective action to an
audit. Quality Assurance Project Plan
(QAPP). or final product review finding);
Ensuring that the responses to findings
of Quality System Audits (QS As) are
appropriately addressed by the DQA (while
this responsibility may be delegated, it
is the Deputy Director's responsibility to
ensure that this requirement is met); and
• Review ing and approving the Quality
Assurance Annual Report and Work Plan
(QAARWP).
2.2.1.3 Deputy Director for Science and Policy
(I) DSP)
The Deputy Director for Science and Policy serves
as the senior science adviser to the Director on
homeland security policy and science issues that
may impact EPA, ORD and/or NHSRC. The DDSP
also serves as NHSRC liaison to other Federal
agencies and program offices and:
• Coordinates closely with the Associate
Director, especially on matters relating to
science products, intra agency activities
with other Centers and Labs;
• Promotes (by words, actions, and
involvement) the NHSRC quality system
and;
• Promotes (by words, actions, and
involvement) continuous quality
improvement within the Center.
2.2.1.4 Director of Quality Assurance (DQA)
The DQA is a member of the Center's Immediate
Office and reports to the Deputy Director for
Management. This arrangement alleviates any
potential conflict of interest with program or data
activities and provides an independent review of
materials requiring quality assurance. Moreover,
the DQA provides QA support to the Center and
does not perform any tasks related to the research
conducted by the RCE. TTEP, WIPD. TCAD or
DCMD.
The DQA will attempt to resolve any issues
pertaining to general QA requirements or audit
close-out. However, if a dispute cannot be resolved,
the DQA will present the issue to their immediate
supervisor (DDM) and the Associate to Center
Director.
The DQA is responsible for:
• Reporting all unresolved NHSRC-specific
QA issues and information to the NHSRC
Deputy Director;
• Coordinating the NHSRC Quality Team
(including all Quality Assurance Managers
supporting NHSRC research) and serving
-------
as the facilitator for resolving quality
issues;
Leading Quality System Audits of the
implementation of the NHSRC quality
system;
Approving the schedules for regular
assessment of the quality system;
Preparing for and conducting Technical
Systems Audits (TSA) of NHSRC projects;
Coordinating the preparation and updating,
as necessary, of the NHSRC QMP;
2.2.1.5 OA Coordinator (QAC)
The QAC provides QA support for projects either
funded by or conducted by NHSRC Pis. The QAC
is a member of the Director's Immediate Office and
reports to the Deputy Director for Management.
This arrangement alleviates any potential conflict of
interest with program or data activities and provides
an independent review of quality assurance matters.
Moreover, the QAC provides QA support to the
Center and does not perform any tasks related to
the research conducted by the RCE. TTEP, WIPD.
TTEP, TCAD. or DCMD. The responsibilities of
Center's QAC are as follows:
• Reviewing, approving and signing the
QARF for extramural funding packages;
Serving as the NHSRC liaison to the
Agency's Office of Environmental
Information's (OEI) Quality Staff (QS);
Entering information into the NHSRC
Quality Assurance Tracking system
(QATRAk) used to track QAPPs;
Assisting QS in the review of policy and
concept documents;
Coordinating the preparation of the
NHSRC QA Annual Report and Work
Plan (QAARWP) and submitting it to the
NHSRC Deputy Director;
Initiating discussions w ith the Center
Director or Deputy Director to correct
quality problems as needed;
Identifying NHSRC quality training needs
and assisting in the development and
presentation of QA training courses;
Leading continuous quality improvement
efforts within NHSRC;
Reviewing, signing, and dating Quality
Assurance Review Form (QARF) for
funding packages;
Entering quality document tracking
information into the QA tracking system;
and
Managing QA support contracts.
Attending Center research project planning
meetings (e.g.. Center implementation plan
meetings) when proposed, new, and/or
current research activities are discussed;
Reporting issues regarding NHSRC QA
system noncompliance to the DQA for
further action when resolution cannot be
made;
Assisting in the preparation of audit
checklists and assisting the DQA in
conducting audits;
Participating in the NHSRC DQA's QSAs
and/or internal audits and preparing audit
reports for submission to the DQA;
Tracking corrective action as required at
project level;
Assisting the DQA and/or QACs in support
of the overall NHSRC quality system;
Assisting in the review of the NHSRC QA
program as needed;
Assisting Pis with in-house research
projects when requested. This may include
assisting the PI in the development of the
-------
QAPP; establishing appropriate project
objectives, experimental design, analytical
methods, sampling points; performing
audits to ensure implementation of QAPP;
and
• Performing some level of data validation
and reviewing the final report.
2.2.1.6 QA Manager (QAM)
As mentioned in section 2.1.3, the program areas
that NHSRC is responsible for are broad in scope
and involve numerous scientific and engineering
disciplines; therefore, the Center must involve
experts from outside of the Center to serve as
Pis. The QAM from their respective L/C/O is
responsible for the QA of the research project.
Moreover, the Pis will be supported by their
assigned L/C/O QAM. The responsibilities of the
Center's QAM are listed below. The responsibilities
of the QAMs from other L/C/Os should be listed
in their QMP and should be comparable to the
responsibilities listed below. Disagreement in
responsibilities should be communicated to the
Center DQA by the L/C/O QAM.
The NHSRC QAM is a member of the Center's
Immediate Office and reports directly to the Deputy
Director for Management. This arrangement
alleviates any potential conflict of interest with
program or data activities and provides an
independent review of quality assurance matters.
Moreover, the QAM will provide QA support to
the Center and does not perform any tasks related
to the research conducted by the RCE. WIPD.
TTEP, TCAD or DCMD. The typical QAM
responsibilities are responsible are as follows:
• Reviewing and signing the QARF form for
funding packages;
• Review ing and approving QAPPs.
SOPs. published reports (both paper and
electronic), journal articles, symposium/
conference papers, and extended abstracts;
• Entering quality document tracking
information into the QA tracking system;
• Managing QA support contracts;
• Attending Center research project planning
meetings (e.g.. Center implementation plan
meetings) when proposed, new, and/or
current research activities are discussed;
• Compiling Center information for the
QAARWP and submitting it to the NHSRC
DQA;
• Negotiating whenever there is disagreement
on the proposed resolution to an audit or
review (QAPP. final report, journal article,
symposium paper, extended abstract)
finding (if agreement cannot be reached,
negotiations will be elevated by the QAM
to the DQA);
• Reporting issues regarding NHSRC QA
system noncompliance to the DQA for
further action when resolution cannot be
made;
• Performing audits, as listed in Section 3.3,
of support laboratories and projects and
preparing audit reports for submission to
the Pis and TLs;
• Participating in the NHSRC DQA QSAs
and/or internal audits;
• Assisting in the development of Q A
guidance;
• Serving as a QA consultant for Center
projects;
• Assisting in the preparation of the NHSRC
QMP or any revisions to the NHSRC QMP
by providing information to NHSRC DQA;
• Providing QA training courses;
• Developing and revising quality system
forms as needed;
• Assisting in the development of internal
QA system SOPs; and
• Tracking corrective action as required at
project level.
Currently, the Center's DQA also serves as the
QAM.
-------
2.2.1.7 Associate Director (AD)
The Associate Director is a senior science advisor
to the NHSRC Director conducting science
planning, coordinating NHSRC's research program
with other Federal research programs including
those within EPA, promoting the quality of science
underway, coordinating program-wide, external
peer review, and reviewing the Center's science
products. The Associate Director is responsible for:
• Serving as an NHSRC arbitrator for quality
issues that cannot be resolved at lower
levels. Either the DQA or Division Director
can bring the unresolved issue to the AD
for resolution;
• Promoting (by words, actions, and
involvement) the NHSRC quality system;
• Promoting (by words, actions, and
involvement) continuous quality
improvement within the Center;
• As delegated by the Director, conducting
the final review of products including
examination of the QA review and
resolution of this review;
• Reviewing and subsequently promoting
the implementation of the NHSRC QMP
and providing written review comments
regarding the QMP to the DQA;
• Coordinating external, prog ram-wide peer
review such as that conducted by the Board
of Scientific Counselors;
• Ensuring that the DQ A is aw are of all
research activities in the center; and
• Resolving disagreements within the Center
(e.g.. appropriate corrective action to an
audit. Quality Assurance Project Plan
(QAPP), or final product review finding).
2.2.1.8 Division Director (I)I))
The Division Director serves as first-line supervisor
for individuals who are assigned to their division.
The DDs are responsible for:
• Ensuring that the NHSRC QA system is
implemented by their researchers and the
Center's DQA is aw are of all research
activities; they are accountable for ensuring
that the QA requirements identified in this
QMP are implemented;
• Promoting (by words, actions, and
involvement) the NHSRC quality system;
• Ensuring that quality planning documents
(i.e., QAPPs. Standard Operating
Procedures (SOPs)) are developed and
implemented for all research activities
occurring within their division (i.e., for all
intramural and extramural projects). (For
example, project-specific QAPPs need
to be prepared and approved for research
activities as soon as a PI is prepared to
collect data/information to support Agency
decisions or for inclusion in research
products e.g.. reports |both paper and
electronic], journal articles, symposium/
conference papers, extended abstracts,
computer products/software/models/
databases, or scientific data);
• Keeping the DQA informed of QA related
issues;
• Participating in and ensuring team
personnel participate in QA training
sessions as appropriate;
• Ensuring that research products (i.e.,
published reports |both paper and
electronic], journal articles, symposium/
conference papers) are reviewed by the
DQA or QAC and approved prior to
publication (research products shall be
subjected to QA review prior to being
distributed outside of NHSRC);
• Ensuring implementation of corrective
actions; and
• Ensuring that responses to any internal
audits are prepared and submitted to the
DQA OR QAC or DQA to ensure that all
unresolved QA issues are addressed.
-------
2.2.1.9 Programmatic Team Leader (PTL) and
Program Manager (PM)
The RCE Team Leader and TTEP Program
Manager are responsible for:
• Ensuring that the NHSRC QA system is
implemented by their team members;
• Keeping the Center's DQA abreast of all
research activities;
• Promoting (by words, actions, and
involvement) the NHSRC quality system;
• Resolving technical or administrative issues
relating to quality within the team (e.g..
appropriate corrective action to an audit.
QAPP. or final product review finding);
• Ensuring that quality planning documents
(QAPPs. Standard Operating Procedures)
are developed and implemented for all
research activities occurring within their
team and/or program area (i.e., for all
intramural and extramural projects by
assigning team personnel the responsibility
for developing project-specific QAPPs
and SOPs). A quality assurance project
plan needs to be prepared and approved
for research activities as soon as a PI is
prepared to collect data /information to
support Agency decisions or for inclusion
in research products e.g., reports |both
paper and electronic|. journal articles,
symposium/conference papers, extended
abstracts, computer products/software/
models/databases, or scientific data;
• Participating in and ensuring team
personnel participate in QA training
sessions as appropriate; and
• Ensuring that research products (i.e.,
published reports |both paper and
electronic], journal articles, symposium/
conference papers) are reviewed by the
DQA or QAC and approved prior to
publication (research products shall be
subjected to QA review prior to being
distributed outside of NHSRC).
2.2.1.10 Cross Center Team Leaders (CCTLs)
The team leaders are responsible for:
• Ensuring that the NHSRC QA system is
implemented by the team members;
• Promoting (by words, actions, and
involvement) the NHSRC quality system;
• Keeping the DQ A informed of Q A related
issues; and
• Resolving technical or administrative issues
relating to quality within the team.
2.2.1.11 Principal Investigator (PI)
The PI is the person who is responsible for the
project. For extramural contract work, the PI
is typically known as the contracting officer's
representative (COR) or the project officer (PO) on
Interagency Agreements (IA); for intramural work,
the PI is typically known as the lead researcher.
Pis may be full-time or non-Center staff. The PI is
responsible for:
• Preparing QARF form to be included in
procurement package (e.g.. full-time Pis
should prepare the NHSRC QARF form
and part-time Pis should prepare the QARF
form and submit to their assigned QAM for
approval);
• Ensuring that all extramural agreements
(i.e., contracts, cooperative agreements,
grants, interagency agreements) include
appropriate QA requirements and that the
requirements are met for their projects;
• Ensuring development of a planning
document consistent with NHSRC policy
for all intramural and extramural projects
that involve the collection or generation of
primary and/or secondary data (secondary
data are environmental data collected from
other sources, by, or for, EPA, that are used
for purposes other than those originally
intended);
• Ensuring that all project participants agree
to the project objectives and planned
experimental approach before the QAPP is
submitted for QA review;
• Ensuring that all environmental data
collection, evaluation, and use do not
proceed until there is an approved QAPP;
-------
Ensuring that the approved QAPP is
implemented and that significant changes
(i.e., those that may or do affect the quality
of data, the scope of the project, or the
successful completion of the project)
to the approved QAPP are documented
and approved before the change is
implemented;
Ensuring that research products (i.e.,
published reports | both paper and
electronic], journal articles, symposium/
conference papers, and extended abstracts)
are reviewed by the DQA or QAC and
approved prior to publication (research
products shall be subjected to QA review
prior to being distributed outside of
NHSRC);
• Ensuring that corrective action procedures
are initiated in a timely manner and issuing
documentation to DQA or QAC of all
corrective actions;
• Requesting or cooperating with any
project-specific audits as required in
Section 2.3.5; and
• Preparing responses to audit or review
(e.g.. QAPP, final report, or journal article)
findings.
-------
This page left intentionally blank.
12
-------
3.0
Quality System and Description
3.1 Quality-Affecting Activities
As indicated in the QA policy statement (Section
2.1.2), NHSRC's data generating research and
testing activities always require QA oversight.
However, NHSRC also performs other functions
that may not always be attributed to project-specific
efforts; certain aspects of NHSRC's operations,
such as the use of in-house data and software
systems, may be inherently part of projects and the
quality of these systems is often not addressed in a
QAPP. Establishing quality policy for these types of
activities and monitoring these activities is typically
performed through the development of SOPs. and
in other cases, facility manuals (Section 3.2.7).
In addition, there are some generally accepted
laboratory practices that need to be implemented
to ensure that at least a minimum level of quality is
established in each laboratory. The list of minimum
requirements for laboratory research activities is
presented in Appendix E.
3.2 Principle Components of the Quality
System
The follow ing sections describe the key
components of the NHSRC quality system.
3.2.1 NHSRC Quality Management Plan (QMP)
The purpose of this document is to provide details
about the Center's quality system and to discuss
management and technical activities necessary to
plan, implement, and assess the effectiveness of
QA and QC operations applied to all research and
environmental programs. The NHSRC Director
is responsible for the development of the NHSRC
QMP, although its preparation is delegated to
the NHSRC DQA. The QMP must be approved
and signed by the Director. Deputy Director for
Management, Division Directors. Associate to
the Director. Programmatic Team Leader, and
the DQA. The QMP is submitted to the Office
of Environmental Information's Quality Staff
(QS) for review. It must be approved and signed
by the QS Director. It is reviewed annually by
the NHSRC DQA and updated as needed. The
Quality Assurance Annual Report and Work Plan
(Section 3.2.2) will be used to identify changes in
the QMP prior to the next scheduled revision of
the document. The approved QMP is valid for five
years unless there is a significant change in the
organization or in the research areas emphasized.
After five years, or a significant change, the QMP
is reviewed and revised as needed and subjected to
the same approval processes described above.
3.2.2 Quality Assurance Annual Report and
Work Plan (QAARWP)
The Center assesses the effectiveness of its
quality system on an annual basis to ensure that
assessments and planned work activities for
the coming fiscal year are documented in the
QAARWP. The DQA prepares the QAARWP
and submits it to the OEI's Quality Staff Director
under the signature of the NHSRC Deputy Director
for Management. More information regarding the
QAARWP is available on the OEI Web site.5
3.2.3 Extramural Quality Management
Plans (QMPs)
The EPA contractors, assistance agreement
recipients, and some EPA program offices may be
required to submit a QMP. A graded approach is
used by the EPA PI in consultation with the DQA
or QAC to determine if a QMP is necessary (see
Section 5). If a QMP is required, it is reviewed
and approved by the EPA PI and DQA or QAC.
The NHSRC requirements for QMPs are given in
Appendix B.
3.2.4 Systematic Planning
The NHSRC PI is responsible for using a
systematic process when planning work. The
systematic planning process must involve defining
project objectives and then designing and refining
a plan that meets these objectives. The graded
approach is used to ensure that the level of detail
is in accord with the intended data use and the
resources available. Although not mandatory, the
data quality objectives process developed by EPA is
an example of a systematic planning process. This
5 http://www.epa.gov/quality/qaarwps.html
13
-------
process is described in Appendix B. Section 7.0 of
the QAPP requirement documents and in Guidance
on Systematic Planning Using the Data Quality
Objectives Process, EPA QA/G-4, EPA/240/B-
06/01, 2006, OEI, USEPA, Washington, D C.
(http://\Y\YVY.epa.gov/quality/qs-docs/g4-final.pdf).
Depending on the scope of research, as well as on
any contractual or Agency requirements, planning
documents (including QAPPs) appropriate to
the scope are developed. The NHSRC DQA and
QAC participate in NHSRC planning meetings as
directed by senior management.
The EPA PI for the project, in consultation with
project participants and the QA staff, determines
the appropriate QAPP requirements to be used. It
is the Pi s responsibility to identify and involve
any and all appropriate sponsoring organizations,
responsible official(s), project personnel, and
stakeholders, scientific experts (e.g.. all customers
and suppliers) in the planning of the project.
This is true for both intramural and extramural
projects. Once the planning is complete, project
documentation should include (at a minimum) a
complete description of the following:
• The project's goals, objectives, questions,
and issues to be addressed;
• The project's schedule, resources (including
budget), milestones, and any applicable
requirements (e.g.. regulatory requirements,
contractual requirements);
• The type of data needed and how the
data will be used to support the project's
objectives;
• The quantity of data needed and the
specification of performance criteria for
measuring data quality;
• How. when, and where the data will be
obtained (including existing data) and
identification of any constraints on data
collection;
• Specifications of needed QA/QC activities
to assess the quality performance criteria
(e.g., QC samples for the field and
laboratory, audits, technical assessments,
performance evaluations); and
• How the acquired data will be analyzed
(either in the field or the laboratory),
evaluated (i.e., QA review, validation,
verification), and assessed against the
quality performance criteria and for its
intended use.
Once all project planning is complete and
documented, it is the responsibility of the PI to
submit the QAPP to the designated QA staff.
The QA staff performs a review against QAPP
requirements and issues documentation to indicate
whether planning requirements have been met and
an assessment of whether project goals can be met.
The documentation from the Q A staff shall include
either an approval or a non approval. In the case
of non approval, detailed review comments shall
be provided to the PI. Resolution of all findings
shall be accomplished and documented before any
research is started. The document still needs to be
approved before research is started.
3.2.5 Quality Assurance Review Form (QARF)
The Quality Assurance Review Form must be
completed for extramural projects; preferably a
QARF is completed for intramural projects. For
intramural projects, the QARF is used by the
NHSRC quality staff to track in-house research
projects and Quality Assurance Project Plans. On
the other hand, if procured services are involved
in the environmental data collection or generation
activities, the QARF' helps to inform the vendor
regarding any QA requirements specific to the
project. Additional information is provided in
Sections 5.2.3 (Alternative Documentation) and
3.2.6 (Quality Assurance Project Plans) and. if
so, which quality documentation and activities
are required. If the project does not include data
collection or generation, then the QARF is required
only for levels above $50,000. The completed form
is signed by the Contract Officer Representative
(COR) or Project Officer (PO). and
DQA or QAC; then the QARF is attached to the
NHSRC extramural routing sheet (see Section 5.0).
6 L:\NHSRC\Quality AssuranceYNHSRC Quality System\
Forms and Templates or Appendix 46. ID in Contracts Manage-
ment Manual at http://intranet.epa.gov/oamintra/policy/cmm.
pdf
-------
The Contracts Management Manual (CMM),
Section 46.1.5.1 specifies that the QARF is used
to ensure that quality requirements of FAR 46.202
and 52.246 are communicated to the Contracting
Officer.
Therefore, the QARF must be completed for all
solicitations and contracts, work assignments,
delivery orders, and task orders; any modifications
that involve a significant change to the Statement
of Work; and simplified acquisitions except
those under a purchase order (See Appendix
C). The completed QARF is attached to the
extramural package and forwarded to the contracts
management division (CMD) or the Grants and
Interagency Agreements Management Division
(GIAMD). A QARF is not required if the
amendment to the original SOW does not impact
the contractor's work such as an incremental
funding actions or project time extensions.
For the convenience of the NHSRC PI. the QARF
is available in several locations: (1) Lotus Notes
database under applications; (2) on the NHSRC
share drive; and (3) listed on the EPA web forms
system.
3.2.6 Quality Assurance Project Plans (QAPPs)
The Quality Assurance Project Plan (QAPP)
documents the necessary quality and technical
activities that must be implemented to ensure
that the results of work performed will satisfy
the stated performance criteria. The information
in this section applies equally to both in-house
(intramural) and extramural projects. Quality
Assurance Project Plans (QAPPs) must be prepared
to document QA/QC requirements for all research
projects. A QAPP must be prepared before a
research project begins.
For QA planning purposes, a research project
begins as soon as a PI is prepared to collect data/
information to support Agency decisions or for
inclusion in research products (e.g.. paper and
electronic reports, journal articles, symposium/
conference papers, extended abstracts, computer
products/software/models/databases, or scientific
data). Before proceeding with the collection of
this data/information, a QAPP applicable to the
project's QA category must be prepared. As part
of the planning process for a research project, it
may be necessary to perform preliminary work
to gather information used to define its scope.
This preliminary work must comply with the
requirements specified in NH SRC's Minimum
Requirements for Center Research Activities
(Appendix A).
Guidance for QAPPs is available through the EPA
Web site7. For more information, the OEI's R and G
series documents can assist technical staff to further
understand EPA's quality specifications*.
For projects that require original environmental
data collection and/or sampling and analysis or use
of secondary data, a graded approach (see Section
3.2.7) is in place, and is used for both intramural
and extramural activities. Secondary use of data is
when data is used for a purpose other than which
it was collected. Prior to using data for a use other
than originally intended, the data must be qualified
for acceptability. The graded approach is used to
ensure that the level of detail is in accord with the
intended data use and the resources available.
The overall intent is to allow flexibility, while still
meeting NH SRC's interpretation of Agency policy
for QA. NHSRC has summarized the Agency
guidance for generic QAPPs (see Appendix B); the
NHSRC guidance and QAPP template are available
on the shared drive'.
3.2.7 Graded Approach to Research Projects
The graded approach is used to ensure that the level
of detail is in accord with the intended data use
and the resources available. The NHSRC quality
system utilizes a "graded approach" for establishing
appropriate requirements for QAPPs for various
types of research activities. Under the graded
approach, the intended use of the data dictates the
required level (or category) of quality. The graded
approach utilizes four QA categories for research
projects. The four QA categories are:
Category I: Research that directly and/or
immediately supports specific Agency rule-
making. enforcement, regulatory, or policy
decisions. This category may also include
research of significant national interest,
such as tasks that might be monitored by the
Administrator.
7 http://www.epa.gov/quality/qapps.html
8 http://www.epa.gov/quaIity/qa_docs.html).
9 L:\NHSRC/QA/QMP\QAPP Guidance
15
-------
Category I projects require QA planning
documents that comply with EPA R5/G5
requirements, including a statement of data
quality objectives; an audit plan that at a
minimum includes technical systems audits,
audits of data quality, data quality assessments,
and performance evaluation audits of
measurement systems (if possible); QA review
of products; and reports that have a readily
identifiable QA section.
Category II: Research of high programmatic
relevance that, in conjunction with other
ongoing or planned studies, is expected to
provide complementary support of Agency rule-
making. regulatory, or policy decisions.
Category 11 project QA requirements are the
same as for Category I. but the audit plan will
differ (the same minimum requirements apply
but the frequency/intensity will differ).
Category III: Projects involving applied
research or technology evaluation; method
validation studies.
Category III projects require QA planning
documents; QA review of products; and reports
that have a readily identifiable QA section.
Category IV: Basic, exploratory, conceptual
research to study basic phenomena or issues
that typically result in a peer-reviewed journal
article.
Category IV projects require QA planning
documents.
For both Category I and Category 11 projects,
NHSRC follows, in its entirety, the R-5 document1"
for QAPP preparation developed by EPA's Quality
Staff (QS). If a particular R-5 requirement does not
apply to an individual Category I/I I project, the PI
provides a brief explanation for why the specific
requirement docs not apply. For Category III and
IV projects, a subset of applicable R-5 requirements
is utilized.
The requirements used for Category III (applied
research) and Category IV projects (basic research)
are presented in Appendix B. Quality Assurance
10 http://www.epa.gov/quality/qs-docs/r5-final.pdf
Project Plan requirements are also presented in
Appendix B for projects involving the use of
secondary data, sampling and analysis, methods
development, modeling, and software development/
data management. (In some cases, these types
of projects may be designated Category I or
II; additional requirements in R-5 would then
be applicable). As guidance, the requirements
in Appendix B include short descriptions of
the applicable types of research. The required
quality level (or category) and associated QAPP
requirements are determined by the EPA PI. in
consultation with the QA staff, at the beginning of
a project.
3.2.8 Facility Manuals
National Homeland Security Research Center
facility managers, lead researchers, and QA staff
develop facility manuals to assist technical staff
(both EPA and contractor) in documenting the
performance of routine operations. A facility is
defined as a building (or portion of a building)
housing a set of equipment designed for a particular
area of research (such as a furnace to study the
impact of combustion parameters on the pollutants
emitted or a pilot plant to simulate a water
distribution system). Facility manuals may contain
the following information on a specific NHSRC
facility:
I.0 Introduction
2.0 Facility Charter (purpose)
3.0 Management
4.0 Description
5.0 Equipment
6.0 Documentation
7.0 Operation
8.0 Quality Assurance
9.0 Quality Control
10.0 Data Handling
II.0 Corrective Action
12.0 Health and Safety
13.0 QA/Test Plans
Appendices may contain:
A. Current Facility Personnel
B. Operating Procedures
C. Standard Methods
D. Technical Systems Audit Checklist
E. Performance Evaluation Audit Ranges
-------
Facility manuals are reviewed by the facility
manager or lead researcher, QA staff, and health
and safety (H&S) personnel if H&S requirements
are included in the manual; they are maintained
and updated by facility managers. Formal revisions
and reviews are initiated when substantial changes
are made to a facility manual. A facility manual
documents instrumentation and procedures, provides
a historical record of facility use, standardizes
procedures, provides a training aid for new
employees, and facilitates preparation of QAPPs
(which can reference standard sections in the facility
manual, with description of the specific tests and
schedules for testing).
3.2.9 Standard Operating Procedures (SOPs) and
Methods
The NHSRC encourages the development of SOPs
for those operations which have become, or will
become, routine, including analytical procedures,
sampling procedures, and instrument calibration
procedures which are used for more than one
project or performed by more than one person.
Before SOPs are implemented, they are reviewed
by a second person in the specific technical area
and the QA staff. Project-specific SOPs may also
be written and are particularly recommended to
facilitate updating the QAPP or when the procedure
is performed by contract personnel. The purpose of
the SOPs is to facilitate the uniform performance
of routine procedures. When appropriate. NHSRC
uses approved standard methods (e.g., SW-846,
MCAWW, EPA 500 and 600 series, Standard
Methods, CFR) to meet project-specific objectives.
3.2.19 QA Review and Response Procedures
As shown in Table 1, the DQA or QAC is responsible
for scheduling, and conducting the following type
of reviews. The DQ A or Q AC prepares written
comments for each review and submits them to the
PI. The Pis responsibility to address the comments
and provide a disposition of comments to the Q A
staff.
3.2.11 Peer Review
Internal and external peer reviews are a vital
component of the NHSRC quality system and will
be used to ensure quality research products. Internal
"technical" reviews are required for all scientific and
technical products such as written material or data,
technical reports, and published papers. National
Homeland Security Research Center's products
must meet the expectation of our clients outside
of NHSRC and our Government Performance
and Results Act (GPRA) goals. The policies and
procedures used to implement peer review are
contained in the Science Policy Council Handbook
on Peer Review". Since the final output of NHSRC
research is often in the form of journal articles,
another assessment of output quality is made
through the peer review systems of the technical
journals.
In addition to technical reviews of products, it is
ORD policy that major research programs must
undergo external peer review at least once every
three years. These program peer reviews examine
the underlying intent, scientific approach, direction,
progress and results of large-scale research program
areas, which are collections of individual research
projects, but do not focus on the details of how
the individual projects are conducted nor do they
duplicate the peer reviews. The NHSRC often relies
on the Board of Scientific Counselors (BOSC). the
11 Peer Review Handbook 3rd Edition, Science Policy Council,
USEPA. 2006. EPA/100/B-06/002
Products Reviews
Review of SOPs all NHSRC developed SOPs
Review of QAPPs all QAPPs
Final Research Products all published reports (both paper and electronic), journal articles,
symposium/conference papers, and extended abstracts
Table 1 Types of Product Reviews
17
-------
National Academy of Science (NAS) and the EPA
Science Advisory Board (SAB) for these types of
reviews.
3.2.12 Information Quality Guidelines
The EPA's Information Quality Guidelines12
contain EPA's policy and procedural guidance
for ensuring and maximizing the quality of
information we disseminate, and complements
EPA's Quality Management System for assuring
the quality of EPA's product and information.
The term "Information" generally includes any
communication or representation of knowledge
or position/policy such as facts or data, in any
medium or form. This includes "preliminary"
information that EPA has endorsed or adopted,
and also conclusions or facts drawn from or based
upon other existing information (secondary uses of
information).
Information that is either adopted endorsed or used
by EPA to support an Agency decision or position
is generally considered "information" for the
purposes of the IQG and should be subject to pre-
dissemination review. The pre-dissemination review
procedures are intended to provide assurance
that quality has been built into the information
we disseminate. To be in compliance with the
IQG concept. NHSRC reports must document the
quality of information and include a disclaimer
that clarifies the intended use and limitation of the
product, when appropriate. Usually, quality related
discussion can be found in a separate section of the
report titled. Quality Assurance.
3.2.12.1 Request for Correction
If a person believes information disseminated
by EPA does not meet the Information Quality
Guidelines, they have the opportunity to submit
a Request for Correction" (RFC). The Office of
Environmental Information manages the database
for RFCs and posts all RFCs to the IQG Web site"
For any RFCs that affect NHSRC, the OEI IQG
Officer will contact the NHSRC Center Director
12 Guidelines for Ensuring and Maximizing the Quality, Ob-
jectivity, Utility, and Integrity of Information Disseminated
by the Environmental Protection Agency (EPA/ 260R-02-008,
October 2002, available at http://www.epa.gov/quality/infor-
mationguidelines
13 http://www.epa.gov/quality/informationguidelines/iqg-faqs.
html
14 ibid
(or designee) to participate in the development of
a draft response. The draft response goes through
many reviews including ORD. Office of General
Council, (OGC). OEI, and Office of Management
and Budget (OMB). Once the response is approved,
OEI staff will post the final response on the IQG
Web site. A flow chart of the RFC process is found
on the IQG intranet1".
3.2.12.2 Request for Reconsideration
The Request for Reconsideration (RFR) process is
the appeal process for the results of an RFC. If a
person is not satisfied with the Agency's response
to an RFC, they can submit an RFR" The OEI
manages the database for RFRs and posts all RFRs
to the IQG Web site17 For any RFRs that affect
NHSRC, OEI IQG Officer will contact the NHSRC
Director to participate in the development of a draft
recommendation for review by a panel of senior
staff. The Review Panel makes their decision and
prepares a draft response. The draft response goes
through review at OMB. Once the response is
approved. OEI staff will post the final response on
the IQG Web site. A flowchart of the RFR process
is found on the IQG intranet18.
3.13 A Stop Work Order
Quality assurance staff members are often on-site
and able to interact directly with project personnel.
If a QA staff member observes work practices
which could have serious adverse impacts on data
quality, the QA staff member should promptly
notify the NHSRC PI or line manager for in-house
projects. For extramural projects, if the PI concurs
that work practices are unacceptable, the PI should
notify the Contracting Officer, who may issue a
Stop Work Order (SWO). When a SWO is issued
the work is halted until the issue (technical, quality
or safety) is resolved. The CO must cancel the
SWO before work can be resumed.
3.3 Assessments
The NHSRC conducts quality assessments to
determine if data collection operations and/or the
15 http://intranet.epa.gov/quality/informationguidelines/pdf/
rfc_process.pdf
16 http://www.epa.gov/quality/informationguidelines/iqg-faqs.
htmlMisagree
17 http://www.epa.gov/quality/informationguidelines/iqg-list.
html
18 http://intranet.epa.gov/quality/informationguidelines/pdf/
rfr_process.pdf
-------
organization are adhering to the prevailing quality
management structure, policies, and procedures;
and the existing structure, policies, and procedures
are adequate for ensuring that the necessary quality
of data is obtained. Assessments are used to
determine the effectiveness of the quality program
and the adequacy of resources and personnel
provided to ensure quality in all activities.
For measurement activities. NHSRC conducts
assessments to verify the integrity and accuracy of
the generated data, and to identify opportunities
for improvement. Quality assessments may include
internal/external audits, observations, internal
reviews, quality control checks/audits, performance
evaluations, and management reviews. For more
information regarding NHSRC assessments, please
refer to Section 10.
Internal assessments of the quality program will be
conducted annually. The internal assessments place
priority on projects that are high visibility projects
and have new test procedures or equipment, or as
determined by observation. Internal assessments
may focus on a selected division or program to
serve as a sample representative of NHSRC"s
quality system. A more frequent internal review
cycle may be required if serious deficiencies exist.
Results of the assessment will be documented and
archived and included in the appropriate Quality
Assurance Annual Report and Work Plan.
External audits are scheduled for specific systems,
programs, and projects based on required periodic
review, the importance of. or public interest, in the
program or project, problems in the measurement
system (noted by the PI or QA staff), requests by
the PI. managers, or selection by the QA staff.
The audits planned by the PI at the beginning of a
project or required by the program are specified in
the QAPP or QMP.
For all assessments, the lead auditor/assessor
(usually either the DQA or a QAC) must ensure
that auditor(s)/assessor(s) have no real or perceived
conflicts of interest with the project/system/
organization being assessed.
3.3.1 Quality Systems Audit (QSA)
The EPA quality system, in conformance with
national consensus standards, requires that each
organization assess the effectiveness of its quality
systems" implementation. It is expected NHSRC
will conduct a self assessment. EPA's Quality Staff
also performs assessments of NHSRC"s Quality
Systems on a 3-year cycle. A QSA is an on-site
review of the implementation of an organization's
quality system as documented in the organization's
approved QMP. This review is used to verify the
existence of. and evaluate the adequacy of. the
internal quality system.
The NHSRC DQA schedules QS As of the Center
annually to assess the implementation of the quality
system and may enlist the help of personnel from
other L/C/Os. In any event, the DQA is always
invited as a member of the evaluation team to
witness first hand the implementation of the
QMP observed during the QSA and to respond to
questions as needed.
3.3.2 Technical Systems Audits (TSAs)
A TSA is a qualitative on-site evaluation of
sampling and/or measurement systems. The
objective of the TSA is to assess and document
acceptability of all facilities, maintenance,
calibration procedures, reporting requirements,
sampling and analytical activities, and quality
control procedures. Normally, an approved QAPP
provides the basis for the TSA. Independent TSAs
are most often scheduled by the PI and conducted
by the DQA. Assistance for the TSA is available to
each DQA from other L/C/Os or from QA support
contractors. Technical Systems Audits are most
useful when conducted early in the life cycle of a
project when corrective actions (if necessary) can
be performed that will minimize any loss of data.
However, a TSA can be performed any time during
a project's life cycle.
The NHSRC DQA may schedule project-specific
TSAs to assess the quality system for individual
projects and may enlist the help of personnel in
other L/C/Os. In any event, the DQA is always
invited as a member of the evaluation team.
Technical System Audits are required for all
Category I & II projects (both intramural and
extramural). For Category III & IV projects, the
Center may audit two Category III /IV projects
each fiscal year. At least one Category 11 I/I V project
must be an intramural project.
19
-------
3.3.3 Audit of Data Quality (ADQ)
An ADQ is an examination of data after they have
been collected and verified by project personnel.
Assessing whether the Data Quality Indicator (DQI)
goals specified in the QAPP were met requires
a detailed review of the recording, transferring,
calculating, summarizing, and reporting of the
data. An ADQ is conducted as required by specific
programs or as requested by the PI or other EPA
stakeholder. ADQs may be conducted by EPA or
contractor personnel.
3.3.4 Performance Evaluations (PE)
A PE is a quantitative evaluation of a measurement
system. Although each measurement in a test
program could be subjected to a performance
evaluation, the critical measurements (designated
in the QAPP) are more commonly evaluated.
An evaluation of a measurement system usually
involves the measurement or analysis of a reference
material of known value or composition. The value
or composition of reference materials must be
certified or verified prior to use. and the certification
or verification must be adequately documented.
Ideally, the identity of the reference material is
disguised so that the operator or analyst will treat
the material no differently than a test program
sample. Performance evaluations are conducted as
required by specific programs or projects or at the
request of the PI or other EPA stakeholder.
3.3.5 Laboratory Surveillances
Laboratory surveillances are conducted using the
Minimum Requirements for Laboratory Research
Activities contained in Appendix A. The Center
may conduct surveillances of several in-house
and extramural laboratories per year. A laboratory
surveillance is less formal than a technical systems
assessment (TSA), which is based on the project
specific QAPP.
3.3.6 Data Quality Assessments
A Data Quality Assessment is a scientific and
statistical evaluation to determine if validated data
obtained from environmental data operations are of
the right type, quality, and quantity to support their
intended use. The Data Quality Assessment process
is described in Guidance for the Data Quality
Assessment: A Reviewer's Guide (QA/G-9R)19, and
the companion document Data Quality Assessment:
Statistical Tools for Practitioners (QA/G-9S)20-
EPA/240/B-06/002. February 2006 and EPA/240/B-
06/003 February 2006, respectively. For research
projects, these assessments are normally conducted
by the PI. and they should be performed routinely
on all projects.
3.3.7 QA Review of Research Products
Product reviews are conducted to determine
if the product (i.e., draft or final) of a research
activity is clear, complete, consistent, correct, and
coherent. These reviews evaluate the credibility
of data, realization of project goals, comparability
of data, validity of conclusions, and quality of
data. Research products should be subjected to
QA review prior to being distributed outside of
NHSRC. The NHSRC Product QA/QC Verification
Report Form21 must be completed on-line on
NHSRC « work. The product (e.g.. reports, journal
articles, symposium papers, conference papers,
extended abstracts, computer products/software/
models/databases) that contains environmental
data can be attached using the "attach" button
at the top of the form. Environmental data are
defined in EPA CIO 2105.0 (formerly Order
5360.1 A2) as any measurements or information
that describe environmental processes, location,
or conditions; health effects and consequences;
or the performance of environmental technology.
Moreover, environmental data include information
collected directly from measurements, produced
from models, and compiled from other sources such
as databases or the literature.
3.3.8 Computer Tracking System
Computer tracking systems for the various elements
of the quality system are in place in the Center.
The NHSRC intranet system- is available to the
19 Data Quality Assessment: A Reviewer's Guide, EPA QA/
G9-R, EPA/240/B-06/002, 2006, OEI, USEPA, Washington,
D.C. (http://www.epa.gov/QUALITY/qs-docs/g9r-final.pdf)
http://www.epa.gov/quality/qs-docs/g9r-finaI.pdf
20 Data Quality Assessment: Statistical Methods for Practitio-
ners, EPA QA/G-9S, EPA/240/B-06/003,2006, OEI, USEPA,
Washington, D.C. http://www.epa.gov/quality/qs-docs/g9s-
final.pdf)
21 http://nhsrc.intra.epa.gov/qa.aspx
22 http://nhsrc.intra.epa.gov/qa.aspx
20
-------
staff for entering their Q A product review forms.
The QARF is available on-line through the Lotus
Notes database (see Appendices C and D). The
Lotus Notes system is used to track QAPPs, QMPs,
SOPs. audits, and the NHSRC « work Web site is
used to track products.
3.3.9 Training
Mandatory quality system training for all NHSRC
management and technical personnel is described
in Section 4.0.
-------
This page left intentionally blank.
22
-------
4.0
Personnel Qualifications and Training
It is the policy of NHSRC that all persons
managing or directing NHSRC projects have
appropriate training for their assigned work.
Training needs may be identified by EPA regulation
(e.g.. requirements for project officer training),
by those persons planning a technical activity
(e.g.. PI), by those managing persons performing
a technical activity (e.g.. team leaders), or by
surveying NHSRC staff. Training programs are
generally designed by the persons providing
the training. The process for specifying and
documenting intramural training varies with the
type of training.
NHSRC staff is encouraged to participate in various
types of training courses and continuing education
programs which are identified in their Individual
Development Plan (IDP).
4.1 Technical Training
Documentation of technical training may be an
appropriate degree in an area of study. Other
documentation of technical training may include
professional certifications (e.g.. professional
engineer (PE)) or certification of specialized
training (e.g.. classes in computer applications or
instrument operation as provided by technology
vendors). Other specialized training requirements
established by the project officer/principal
investigator/PI for a given NHSRC project may be
described in the QAPP or facilities manual. The
QAPP is prepared by the project officer/principal
investigator/PI. reviewed by the TL, and DQA
(see Section 2.6.6). General technical training
needs for facility or team members may also be
identified by NHSRC managers. Each employee is
responsible for submitting his/her training records
to the Center's Human Resources Management
Representative who keeps a record of each NHSRC
employee's formal technical training.
4.2 Project Management Training
For extramural project management. NHSRC
adheres to the Agency's requirements regarding
initial and ongoing training, including requirements
for training as a Project Officer (PO). or Contract
Officer's Representative (COR). The Center's
Immediate Office of the Director keeps records
of project management training for NHSRC
personnel. A certification that the designated
EPA extramural project manager or COR has the
appropriate training is also required as part of each
funding package.
4.3 Quality System Training
Training in NHSRC's Quality System policies and
procedures is planned and performed by NHSRC's
QA staff. The DQA will retain copies of the training
certificates. Additional training needs are usually
identified by the QA staff through interactions
with NHSRC technical and managerial staff.
Surveys have also been used to identify quality
system training needs. Quality training is generally
designed to: (1) inform NHSRC staff of new
QA policies and procedures. (2) refresh NHSRC
staff of current QA policies and procedures. (3)
provide QA training to new staff member, or (4)
describe services provided by the NHSRC QA staff
(document review, and technical assistance).
At a minimum, the staff are expected to complete
the follow ing training modules: (1) Quality
System training module prepared by the Office of
Environmental Information; (2) NHSRC quality
management plan; and (3) How to prepare a quality
assurance review form. Additional training might
include: (1) How to prepare a quality assurance
project plan; and (2) How to define data quality
objectives. The researchers are strongly encouraged
to contact the Q A staff for assistance regarding any
QA related issue.
4.4 Safety Training
EPA specifies that personnel involved in laboratory
or field activities receive appropriate safety
training. The type and amount (hours) of training
depend on the specific assignment and the nature
of the potential hazards. Training requirements are
specified by line managers and compliance with
training requirements is monitored by Center/
locality safety officers and other health and safety
personnel.
23
-------
4.5 Records Management Training
Every EPA employee has responsibility for records
management. Each employee is responsible for:
• Creating records necessary to document
their activities and actions;
• Filing records for safe and efficient
retrieval; and
• Disposing of records only in accordance
with agency directives and Federal
regulations.
To assist employees and contractors with their
records management responsibilities, a Records
Management Officer (RMO) will assist personnel
with the implementation of the agency's National
Records Management Program (NRMP). The
NHSRC records file plan is available from the
NHSRC's records liaison officer or on the NHSRC
share drive. Further training and assistance is
available online at www.epa.gov/records/policy/
manual/index.htm.
4.6 Information Security Training
4.6.1 Information Security (INFOSEC) and
Physical Security Programs
The NHSRC possesses a mature INFOS EC
program for safeguarding, managing and using
classified information. Designated Review
Authorities (DRA) and the Security Manager assess
risk in all NHSRC SOWs. I As and work products
under the RASP program, ensuring that sensitive
and classified information is identified immediately
and properly managed. Guidance and training
is provided for classified materials safeguarding
and management. Guidance for identification
and classification/designation of information
is provided. These programs are run by the
Security Manager directly. References: National
Security Information Handbook, 2007, Identifying,
Safeguarding and Sharing Homeland Security
Risk Information, 2009 and NHSRC Policies and
Procedures Manual, 2008.
4.6.2 Personnel Security (PERSEC)
The NHSRC requires all hires to be capable
of acquiring a National Security clearance,
but does not require all staff to possess a
clearance. Justifications for clearances are
coordinated by the Assistant Security Manager,
who helps facilitate the process with Personnel
Security Branch. Training is provided for new
hires orientation, cleared personnel orientation
and annual refresher, foreign travel and foreign
contacts. Reference: Personnel Security Handbook
and NHSRC Policies and Procedures Manual,
2008.
4.6.3 Automated Information Systems (AIS) and
Communications Security (COMSEC)
The NHSRC controls and manages classified
computer assets in order to support classified
research. The NHSRC controls and
manages COMSEC assets, such as STEs and
HSDN. Training for classified AIS operation
and COMSEC is provided as needed. AIS and
COMSEC programs are run by the Assistant
Security Manager. References: National Security
Information Handbook, 2007, and NHSRC Policies
and Procedures Manual, 2008.
24
-------
5.0
Procurement of Items and Services
EPA policies require that parties to procurement
agreements or assistance agreements shall have
in place a quality system consistent with EPA
requirements. Procurement agreements include
contracts and simplified acquisitions, and parties
to these agreements include primary contractors,
subcontractors, and vendors. Assistance agreements
include grants, cooperative agreements, and
Text Box 1: Quality Assurance related language for
EPA Acquisition Regulations25 (EPAAR), were
developed to supplement the FAR.
Only Contracting Officers (COs). including
Simplified Acquisition Contracting Officers, are
authorized to procure items and services, unless
purchased via the U.S. Government Purchase Card
program26. The Federal government is not bound
by any commitments made by personnel other than
those authorized.
Statement of Work
In addition to completing the QARF, the COR must include the following statement,
preferably under Task 1 of the SOW: The awardee shall comply with all requirements as
delineated on the "Quality Assurance Planning Requirements Form (QARF)" included
with this extramural action, see attachment #1 and #2. The contractor shall prepare
a QAPP in accordance with for
Category or 11 projects. In most cases, the QAPP can be based on the type of research
that is being conducted. For guidance on preparing a category III or IV research-
specific QAPP, the preparer should refer to the project specific requirements provided
in NHSRC's QMP in Appendix B. The QAPP must be approved prior to the start of any
laboratory work. Additional information related to QA requirements can be found at
www.epa.gov/quality
interagency agreements (IAs). Other extramural
agreements covered by this policy include
Cooperative Research and Development
Agreements (CRADAs). Compliance with this
policy is achieved by including appropriate written
requirements into solicitation, award, and/or
agreement documents.
The Center utilizes the services of Cincinnati
Procurement Office Division (CPOD) and its
internal Simplified Acquisition Contracting Officers
(SACO) (procurement under $100,000.00) for
the procurement of items and services. Both
the CPOD and the SACOs follow the Federal
Acquisition Regulations23(FAR) which establishes
government-wide policies and procedures governing
the acquisition process. Two EPA publications,
the Contracts Management Manual24 and the
23 http://www.arnet.gov/far/
24 http://intranet.epa.gov/oamintra/policy/cmm.pd
Requests for purchases and identification of funds
begin at the planning stages of any project. Items
and services should be identified and specifications
to meet the government's minimum needs should
be detailed. These specifications will be required
during the procurement process and assure that the
requestor receives the proper item or service. It also
reduces the chances of purchase delays or incorrect
purchases being made because of inadequate
product specification.
All procurements are documented using EPA Form
1900-8 Procurement Request (PR). The Contract
Officers Representative (COR) or Project Officer
(PO) is responsible for originating the PR and
25 http://intranet.epa.gov/oamintra/policy/epaar.pdf
26 See Section 13.3 of Contracts Management Manual, http://
intranet.epa.gov/oamintra/policy/cmm.pdf. EPA-NHSRC Pur-
chase Card Standard Operating Procedures is found in shared
drive I:\share\PurchaseCardInformation\Pcd Guidance
25
-------
identification of funding. The PR is then sent for
approval to the divisional management. PRs are
reviewed for completeness and accuracy by the
appropriate Approving Officials ( AO). Funds
are certified as available by the Center's Funds
Certifying Official who assigns a document control
number (DCN).
5.1 Procurement of Services
It is NHSRC policy that any funding action that
falls into one of the following accounting and
object classification structures must include a
Quality Assurance Review Form (QARF) form.
With appropriate signatures:
25.32 Programmatic Research and
Development Contracts
25.05 Program Contracts
25.63 Programmatic Occupational Health
Monitoring
25.71 Programmatic - Research Interagency
Agreements.
41.00 Grants. Subsidies and Contributions
For contracts and IAs (Object Class 25.XX),
funding actions include the initial award funding
and any modifications that add additional funds.
For assistance agreements (Object Class 41.XX),
funding actions include the initial award funding,
incremental funding (partial funding, no new work),
and supplemental funding (additional funding of
new work).
All funding actions and agreement documents are
reviewed by NHSRC DQA or the QACs to ensure
the appropriate requirements are included in each
action.
5.1.1 Mechanisms for Procuring Services
Generally, two types of mechanisms are used
to procure services; contracts and assistance
agreements (e.g.. grants, cooperative agreements,
and interagency agreements). There are certain
activities of a policy-making and/or decision-
making nature that remain the sole authority of
EPA27 and can not be contracted.
27 See Chapter 7.3.5.5 Advisory and Assistance Services and
Vulnerable Services in the Contracts Management Manual;
http://intranet.epa.gov/oamintra/policy/cmm.pdf. See EPA
Order 1900.2 Contracting at EPA; http://intranet.epa.gov/rm-
policy/ads/orders/1900_2.pdf
It is required that an individual completes the
contract administration training-*prior to serving
as a COR. Similarly for Project Officers who
manage assistant agreements, completion of grants
administration training- ' is required prior to being
designated a grants project officer. Project officers
who manage interagency agreements are required to
take Interagency Grant Certification/Recertification
training.3"
5.1.2 Preparing the Extramural Package for
Procuring Services
The procurement package is initiated by the COR
or PO by completing the PR form and getting all
the required signatures. The COR is responsible for
preparing statements pertaining to the services to
be delivered, and the acceptance criteria related to
the quality of the service. Although, the COR has
overall responsibility to oversee the service that is
being provided, they must work through the CO's
authority. The CO is the only individual who can
enter into contractual agreements or amendments
with the supplier.
The statement of work (SOW)31 is a tool for
the COR to ensure that adequate services or
deliverables are provided. The COR must develop
a SOW that accurately defines the minimum
acceptable requirements for the service. The
SOW must succinctly state the expectations of the
product or service and the required format for the
deliverable. The follow ing statement should be
included in NHSRC SOWs: Products developed
under this SOW must conform to the requirements
of EPA's Handbook for Preparing Office of
Research and Development Reports (EPA/800/K-
95-002). Substantive portions of this handbook can
be found at www.epa.gov/nhsrc under the policy
and guidance tab.
Usually, the quality documentation required from
the contractor is a quality management plan32or
28 See Chapter 42 Contract Administration and Audit Services
in the Contracts Management Manual;
http://intranet.epa.gov/oamintra/policy/cmm.pdf
29 http://intranet.epa.gov/ogd/on_line_training/main/training.
htm
30 http://intranet.epa.gov/ogd/on_line_training/main/training-
infoiag.htm
31 See Chapter 11.1 Statements of Work in the Contracts
Management Manual; http://intranet.epa.gov/oamintra/policy/
cmm.pdf
32 http://www.epa.gov/quality/qmps.html
-------
equivalent and a quality assurance project plan33 or
equivalent. For extramural agreements, the QMP is
an organization or program-specific document; it
describes the general practices of an organization
or program. Project-specific details of individual
projects of the organization or program are
documented in a QAPP. In some cases where the
objective of the work and needs of NHSRC do not
require rigorous quality documentation, a combined
QMP and QAPP may be developed. This agreement
is detailed on the QA Review Form.
5.1.2.1 Contracts
Contracts are used when the government derives
sole benefit from a particular product or service.
The document which defines the authority
requiring quality assurance documentation is
48 CFR 4634. Through tailoring language to 48
CFR 46. EPA requires that applicants submit
a quality management plan or equivalent and
a quality assurance project plan or equivalent
when environmental data collection or generation
activities are present in the contract.
5.1.2.2 Assistance Agreements
Assistance agreements are used when both parties
(EPA and the group providing the service) derive
benefit out of the service. This usually occurs
with grants or cooperative agreements where
universities, states, or non-profit organizations
derive benefits. Grants are assistance agreements
where EPA has no substantial involvement in the
project. Cooperative agreements are assistance
agreements where EPA has substantial involvement
in the project.
Assistance agreement statements of work are
usually developed jointly. However, once the
SOW is complete, the parties must also agree on
the quality standards for assuring the product or
service. It is the responsibility of the PO to be
knowledgeable of EPA"s and NHSRC's quality
policies and to present these standards during the
development of the project's SOW.
Special conditions are usually included in
33 http://www.epa.gov/quality/qapps.html
34 http://ecfr.gpoaccess.gov/cgi/t/text/text- idx?c=ecfr&sid=7f
97a455bf850el0849c341f7fc9cb71&rgn=div5&view=text&no
de=48:1.0.1,7.45&idno=48
assistance agreements. Assistance agreements will
use the QA Review Form during the pre-award
planning process and include quality documentation
requirements (or similar requirements) in final
agreements.
5.1.2.3 Interagency Agreements
For interagency agreements (IA) funded by EPA,
NHSRC cannot require other Federal agencies to
comply with EPA quality system requirements. The
quality specifications must be negotiated between
EPA and the other agency. When agreement
is reached on the quality specifications, the
specifications must be included in the interagency
agreement.
When EPA receives funding from another agency
through an interagency agreement, the EPA
quality requirements shall apply in addition to any
specifications provided by the funding agency.
5.2 Competitive Extramural Actions
5.2.1 Single-Project Effort
For a single-project effort, the PI typically requires
that an offeror applicant address NHSRC's
Quality System Specifications for Extramural
Actions in their proposal (before award). The five
specifications are:
(1) A description of the organization's Quality
System (QS) and information regarding how
the QS is documented, communicated, and
implemented;
(2) An organizational chart show ing the position
of the Q A function;
(3) Delineation of the authority and
responsibilities of the QA function;
(4) The background and experience of the
QA personnel who will be assigned to the
project; and
(5) The organization's general approach for
accomplishing the QA specifications in the
SOW.
After-award, the PI typically requires the awardee
to prepare a QAPP that meets the project-specific
requirements specified on the QARF form. The
before-award documentation, in concert with the
after-award QAPP, constitutes a joint QMP/QAPP.
27
-------
5.2.2 Multiple-Project Effort
As part of the solicitation of a large contract
with multiple work assignments and large
assistance agreements with more than one level
of development, the PI typically requires that an
offeror or applicant submit a QMP (before award).
The QMP needs to meet the requirements of EPA
(which includes compliance with the American
National Standards Institute ANSI/ASQ E4-
2004. Quality Systems for Environmental Data
and Technology Programs - Requirements with
Guidance for Use - see EPA Procedure CIO 2105-
P-01-0) and is submitted by the offeror/applicant
as part of the proposal. NHSRC has developed a
set of requirements (see Appendix B) based on EPA
Requirements for Quality Management Plans, EPA
QA/R-2. March 2001. NHSRC's requirements are
to be addressed in QMPs submitted as part of a
proposal responding to a NHSRC solicitation. Each
QMP is evaluated by the technical evaluation panel
(TEP) to determine its acceptability. The specific
evaluation procedures are determined and agreed
to by the DQA or QAC, the PI. and the contracting
officer prior to issuing the solicitation.
After-award, the awardee must revise the QMP
according to comments received from the TEP.
For each subsequent funding action where
environmental data are used, generated, or
collected, the PI typically requires the awardee
to prepare a QAPP that meets the project-specific
requirements specified on the QARF form.
5.2.3 Alternative Documentation
The QA documentation (QMP, QAPP or both)
required is based on the type of extramural action.
For example, if the COR is conducting a "Request
for Proposals" (RFP) action then he/she must use
the "before Aw ard of Contract" identified in steps
1-6. On the other hand, if the COR is using an
existing vehicle, then the suitable option would be
steps 5 and 6.35
35 Quality Assurance Requirements in Contracts Management
Manual Chapter 46 http://epawww.epa.gov/oamintra/policy/
cmm.pdf
Before Award of Contract
STEP 1. Review the Statement of Work with the
QA Manager (or the appropriate QA personnel)
to determine if QA requirements apply. If
not. complete Sections I. II. and III of the QA
Review Form and sign the form and attach to
the extramural package.
STEP 2. If QA requirements apply, determine what
standards apply as allow ed by your organization's
Quality Management Plan (w ith the assistance of
the QA Manager).
STEP 3. Complete the QA Review Form and obtain
a concurrence signature of the QA staff as part of
the acquisition package.
For each type of documentation selected in Section 11
a and 11 b of the QA Review Form, identify (with
the assistance of the QA Manager) whether the
documentation should be prepared in accordance
with the standard EPA requirements [i.e., EPA
Requirements for Quality Management Plans (QA/
R-2) and EPA Requirements for Quality Assurance
Project Plans (QA/R-5)] or w hether other NHSRC-
approved requirements will be used. The standard
EPA requirements should be used unless the NHSRC
QA staff agrees to different requirements identified in
this Quality Management Plan.
STEP 4. If the potential value of the procurement
exceeds $500,000; or the estimate of the
percentage of costs or level-of-effort. allocated
to activities requiring quality requirements
exceeds 15%; or procedures defined in the
Agency-approved Quality Management
Plan of the organization sponsoring the work
apply; then the quality documentation (i.e.,
the Quality Management Plan or equivalent
documentation) shall be included as part of the
Technical Evaluation Criteria. The QA Manager.
QA Officer, or authorized QA designee as
defined in the organization's approved Quality
Management Plan, shall: (1) assist the Project
Officer with development of the Technical
Evaluation Criteria, and any associated technical
instructions, for the Request for Proposal, and (2)
serve as a member of the Technical Evaluation
Panel for the purpose of evaluating the QA aspects
28
-------
of the technical proposals when a Technical
Evaluation Panel is convened.
After Award of Contract - Perform these steps for
each Statement of Work under the contract.
STEP 5. Review the project and determine if it
requires quality documentation (for example, a
QA Project Plan). Incorporate the requirement
to develop this documentation and to implement
the EPA-approved documentation into the
project's Statement of Work. If the project
will be based on previously prepared and
current EPA-approved quality documentation,
incorporate the requirement to implement this
documentation into the project's Statement of
Work.
STEP 6. Complete a QA Review Form for each
project and attach it to the project's Statement
of Work (e.g., statement of work, delivery order,
task order). Obtain a concurrence signature of
the QA Manager.
Alternative documentation may be selected by the
PI in consultation with the DQA. For example,
acceptable alternative documentation includes:
Before-Award:
NHSRC's Quality System Specifications
After-Award:
QMP and QAPP for the entire effort
Before-Award:
NHSRC's Quality System Specifications
After-Award:
QMP for the entire effort and QAPP for each
applicable project
Before-Award:
QMP for the entire effort
After-Award: QAPP for the entire effort
5.2.4 Additional Requirements for Competitive
Extramural Actions Exceeding $500,000
In addition to the requirements in Section 5.1,
the following requirement exists for competitive
contract and assistance agreement solicitations
exceeding $500,000.
Part of the process for procurements where the
estimated value exceeds $500,000 or the percentage
of cost allocated to activities requiring quality
requirements exceeds 15% includes the use of a
Technical Evaluation Panel (TEP). The TEP must
include the Director of Quality Assurance or a
QA representative who is knowledgeable about
the service being procured. The TEP rates each
potential contractor against a standard set of criteria
including quality documentation such as the quality
management plan or equivalent. Portions of the
criteria can include various assessments such as
on-site audits and the analysis of performance
evaluation materials. Prior to solicitation for bid.
it must be determined what proportion of the TEP
rating will be allocated to quality activities; it is
suggested that a minimum of 10% of the overall
TEP rating be allocated to quality activities."
The role of the QA Manager, QA Coordinator,
or authorized QA designee as defined in this
Quality Management Plan, shall: (1) assist the
Project Officer with development of the Technical
Evaluation Criteria, and any associated technical
instructions, for the Request for Proposal, and (2)
serve as a member of the Technical Evaluation
Panel for the purpose of evaluating the Q A aspects
of the technical proposals.
5.3 Non-Competitive Extramural Actions
Similar requirements (as described in Section 5.2)
apply to non-competitive extramural actions. These
requirements shall be incorporated into the award/
agreement documentation. The PI must complete
the QARF Form (see Appendix C).
Additional documentation may be allowable after
consultation with the QAM and contracting officer.
36 Quality Assurance Requirements in Contracts Management
Manual Chapter 46 http://epawww.epa.gov/oamintra/policy/
cmm.pdf
29
-------
5.4 QA Approval Process for Extramural
Actions
For each extramural action, the COR must prepare
a routing sheet, which is tool that is used to track
and process all extramural packages, see Appendix
G. In addition to the standard extramural forms and
the QARF. it is required that NHSRC extramural
packages contain a RASP form. When NHSRC
project planning reaches final definition, a RASP
review is required to assess the project's potential
to generate classified or sensitive products. Because
R&D projects are ordinarily dynamic in scope, the
scope of information generated may also change
substantially.
5.5 Evaluation of Quality Deliverables
In order to ensure that extramural organizations
provide quality services that satisfy EPA QA/QC
requirements, the DQA (or their designees) and Pis
review QMPs, QAPPs, SOPs, and final products
and audit projects and programs. Review comments
generated by the QAM (or designee) are provided
to the PI. The PI and QAM (or their designee)
perform final review and issues approval. The PI
is ultimately responsible for the quality of services
provided.
5.6 Evaluation of NHSRC's Compliance
with QMP
The NHSRC DQA schedules and conducts reviews
of QA procedures pertaining to the procurement
of items and services in the Center as part of the
routine internal QSAs.
30
-------
6.0
Documentation and Records
Quality Management Plans, QAPPs, facility
manuals, and SOPs are prepared, reviewed,
approved, issued, used, and revised. As described in
Sections 3, 8, and 9, all these documents, including
revisions, must be reviewed for conformance with
the quality system requirements and approved
by authorized personnel before general use. The
follow ing is a discussion of NHSRC procedures for
documents and records, including computer-resident
records. Records shall be maintained by the PI or
PO so that all NHSRC research activities can be
reconstructed.
6.1 EPA Records Management Policy
EPA records management policy is established by
Office of Information and Resources Management
and documented in the Records Management
Manual EPA 2160.31 This procedure applies to all
records, as defined under the Federal Records Act
(44 U .S.C. 3101), regardless of media (including
paper, microform, electronic, audiovisual, and
record copies of Agency publications). All locations
of NHSRC comply with EPA records management
policy.
The management process that ensures that
records accurately reflect completed work and/
or fulfill statutory and contractual requirements,
including any specific record keeping requirements
is defined in EPA Order 2160. This document
defines requirements and responsibilities for
record transmittal, distribution, retention,
protection, preservation, traceability. disposition,
and rctric\ ability, and includes the roles and
responsibilities for management and staff. Following
this EPA Order ensures compliance with all statutory,
contractual, and assistance agreement requirements
for records from environmental programs and
provides adequate preservation of key records
necessary to support the mission of the organization.
National Homeland Security Research Center has a
person designated as the Records Liaison.
37 www.epa.gov/records/policy/manual/index.htm
6.2 Project Documentation and Records
Documentation developed on a project-specific basis
is the responsibility of the EPA PI for that project
and is subject to EPA records management policy.
The level of documentation required is dependent
upon the type of project. Specific requirements
are identified in the project QAPP When a project
is completed as determined by the PI, the project
file is archived. This is accomplished by the PI in
conjunction with the Records Liaison.
Because of the type of documents and records that
are created within the Center, procedures are in
place to protect sensitive information from being
shared with unauthorized individuals. Information
regarding NHSRC's security procedures is available
on the intranet at NHSRC a work or by contacting
NHSRC's Information Security Specialist.
6.3 Research Laboratory-Specific Docu-
mentation and Records
Documentation and records that may not be directly
applicable to individual projects, (e.g.. temperature
logbooks, instrument logbooks, standard preparation
logbooks. SOPs, balance calibration logbooks) are
maintained in the location where they are used. All
staff using the equipment or preparing standards
are responsible for recording the appropriate
information. When the logbooks are full or no longer
needed, the logbooks or outdated SOPs are turned
over to Records Liaison for appropriate retention,
storage, and disposition. Personal laboratory
notebooks are the responsibility of the individual.
When full, they may be retained by the individual; or
when no longer needed or wanted by the individual,
they shall be turned over to the Records Liaison.
Table 2 provides the retention schedule for typical
research documents either developed or used by
NHSRC researchers. National Homeland Security
Research Center staff is responsible for follow ing the
procedures for maintaining and retaining important
research files and extramural documents. The
unauthorized disposition is against the law (44U.S.C.
3106). In addition to ORD's research project file
31
-------
retention schedules, NHSRC has a file plan for
other both scientific and administrative records
located on the NHSRC share drive.38
6.4 Software Documentation and
Records
Center personnel are required to maintain software
quality documentation and records consistent with
EPA Office of Environmental Information (OEI)
Policy9 Draft
Retention Time
• Scientific Research Project Files
Related to Rulemaking
501
* 20 years after file
closure — Permanent
transfer to National Archives
• Instrument Logbooks 502
• Scientific Research Project Files 503
R elated to Basic, Exploratory Research
* 5 years after file closure -
Disposable except for
projects that fall under 501
and 503.
* 20 years after file closure
- Disposable
• Research Project Logbooks or Index
R ecords (list of project status)
504
* 2 years after file closure =
Disposable
• Summary Research Projects Status
R eports
506
* 3 years after file closure -
Table 2 Selected Scientific Schedules for ORD Research Activities1
1 National Records Management Policy (NRMP) drafts: intranet.epa.gov/records/schedule/draft/
6.5 NHSRC Quality Documentation and
Records
Any documents (e.g., the NHSRC QMP and other
specific QA documents) developed directly by the
NHSRC DQA, QAM, or at the direction of the
QA staff shall be controlled by NHSRC. Although
the QAMs may keep copies, records of reviews
of QMPs, QAPPs, Facility Manuals, SOPs, final
reports, and journal articles, internal audit reports,
are the responsibility of the PI.
38 Riblic/NHSRC-PUB/NHSRC/NHSRC FILE PLAN 2006.
xls
39 http://www.epa.gov/innpoli8/archived/polman/
32
-------
7.0
Computer Hardware and Software
The Center follow s guidance published by OEFs
and ORD's Office of Science and Information
Management regarding the use of computer
systems, including purchase of computers, purchase
or development of software, design of databases,
records management, security, and data standards.
To assist in the implementation of software and
hardware standards, the Cincinnati location has an
OSIM representative who serves as the local IT
coordinator.
Project-specific requirements for hardware
configurations and for configuration control are
specified by the PI in specific project QAPPs.
Project-specific requirements and plans for software
testing, validation, and documentation, whether
the softw are is developed or purchased from an
outside source, are also described by the PI in the
QAPPs for those projects. Funding packages and
plans for all NHSRC projects involving software
development or purchase or computer purchase
must undergo review and be approved by the
Center Deputy Director to ensure that this guidance
is followed and quality is ensured.
The EPA ORD QA staff has also developed an
abbreviated guidance on model and software
development which is available from the QA staff,
and under QAPP requirements for these projects are
presented in Appendix B.
Once proper planning has been performed (as
described in Section 3.2.4), it is important that the
implementation of the required procedures be done.
It is the responsibility of the EPA PI to ensure that
the required procedures are properly implemented.
33
-------
This page left intentionally blank.
34
-------
8.0
Implementation of Work Processes
8.1 QAPP Development, Review, and
Approval
The EPA PI develops a QAPP based cm NHSRC
guidance (see Section 3.2.6). The QAPP must
be reviewed and approved by the DQA or QAC
before the project begins. A research project
begins as soon as the PI is prepared to collect data/
information to support Agency decisions or for
inclusion in research products (e.g.. paper and
electronic reports, journal articles, symposium/
conference papers, extended abstracts, computer
products/software/models/databases, or scientific
data). QA comments are documented in a memo to
the PI. with a copy forwarded to the Pi s first line
supervisor.
8.2 SOP Development
In some instances, procedures for sampling,
analysis, or other quality activity (e.g.. standards
preparation) are routine and are used for more than
one project. Oftentimes, it is necessary that more
than one person perform a procedure. In these
cases, an SOP should be written. SOPs are also
recommended for special or critical operations.
SOPs can make the development of QAPPs more
efficient, since they can be attached or referenced.
The need for the development of an SOP is
determined by the EPA PI or DQA or QAC. SOPs
must be reviewed and approved by the DQA OR
QAC and approved by an EPA person other than
the author.
It is recommended that SOPs contain the following
sections as appropriate:
1.0
Scope and Application
2.0
Method Summary
3.0
Sample Preservation. Containers.
Handling, and Storage
4.0
Reagents
5.0
Equipment/Apparatus
6.0
Procedure
7.0
Calculations
8.0
Quality Assurance/Quality Control
9.0
References
8.3 QAPP/SOP Implementation
It is the responsibility of the Pis to ensure that the
general procedures utilized throughout the Center
are implemented during data collection activities.
It is the responsibility of the EPA PI to ensure
that the required procedures specified during
the planning process for a specific project are
implemented during data collection activities. This
is done by personally observing the procedures
being performed or by requesting that the QA staff
perform an audit of the project activities outlined in
a QAPP and/or SOP. In either case, deviations from
project requirements must be documented, along
with the corrective action performed. Corrective
action must be performed as soon as possible to
minimize any negative impacts on data quality. If
data quality is affected by any deviations, this must
be discussed in any project report/paper.
8.4 QAPP/SOP Revisions
During the course of a project, it may be necessary
to revise QAPPs and/or SOPs. Revisions are
required whenever a significant change in the
plan or procedure occurs. Over time, it may also
be necessary to revise QAPPs and/or SOPs to
ensure they are still applicable for the work being
performed. QAPPs and project-related SOPs need
to be reviewed on a yearly basis for long term
projects. Center SOPs need to be reviewed every
two years. Revisions (as necessary) in the form of
an addendum or fully revised document must be
made as soon as a significant change is identified.
Significant revisions to a QAPP/SOP must be
reviewed and approved by the DQA or QAC. It
must then be verified that the revised procedures
are being implemented properly, as discussed in
Section 8.3. Copies of the most current document
must be made available to the appropriate project
personnel.
35
-------
All previous document versions must be archived.
Table 3 provides information regarding who has
the responsibility for revising, distributing, and
archiving both the Center's QAPPs and SOPs.
Document
Revision
Responsibility
Distribution of Revised
QAPP/SOP
Archival of Previous
Versions
QAPP
PI
PI
PI
Project-related
SOP
PI
PI
PI
Center SOP
DQ A or designee
DQ A or designee
DQA or designee
Table 3 Distribution and archival of QAPPs/SOPs
-------
9.0
Assessment and Response
The Center conducts a variety of assessments at the
NHSRC-wide and project-specific level to provide
an increased understanding of the program or
system being examined, and to provide a basis for
improving such programs or systems. The follow ing
includes: a review of the various assessment
techniques, plans for appropriate response, and
the process by which management determines the
assessment activities appropriate for each level.
9.1 Assessment Techniques and
Practices
Basic assessment categories were described in
Section 3.3.5. In NHSRC. peer review is a separate
function from QA. Program peer reviews are not
performed by QA staff unless directed by the
NHSRC Director or Deputy Director. The NHSRC
QA staff responds to issues regarding data quality
identified during the peer review process and
provided to the QA staff.
9.2 Types of Assessments
A description of the four types of assessments is
provided in this section.
9.2.1 Self-Assessment of a Quality System
A quality system self-assessment is a qualitative
assessment of a quality system "s operations by
those immediately responsible for overseeing and/
or performing the work to establish whether the
prevailing quality structure, policies, practices, and
procedures are adequate for ensuring that the type
and quality of results needed are obtained.
9.2.2 Independent Assessment of a
Quality System
An independent quality system assessment is
a qualitative assessment of a quality system's
operations by someone other than the group
performing the work to establish whether the
prevailing quality structure, policy, practices, and
procedures are adequate for ensuring that the type
and quality of results needed are obtained.
9.2.3 Technical Self-Assessment
A technical self-assessment is the evaluation
process used by those immediately responsible for
overseeing and/or performing the work to measure
the performance or effectiveness of a technical
system and its elements with respect to documented
specifications and objectives. Such assessments may
include qualitative and quantitative evaluations.
9.2.4 Independent Technical Assessment
An independent technical assessment is the
evaluation process used by someone other than
the group performing the work to measure the
performance or effectiveness of a technical system
and its elements with respect to documented
specifications and objectives. Such assessments may
include qualitative and quantitative evaluations.
9.3 Roles and Responsibility of the As-
sessors
The role of the assessor(s) is to provide information
to the NHSRC PI as to the best practices and
to clearly point out errors observed in the
implementation of the Q APP Some of the errors
might be in technical procedures, in data archival or
analysis, in project management or reporting, or in
any aspect of a project that impacts the data quality.
9.3.1 Capabilities and Authority
Assessors should be thoroughly familiar with QA
practices, policies, and procedures. The optimal
assessment team consists of a Q A professional, a
person with expertise in the technical area being
evaluated, and. if there are complex issues regarding
experimental design or data analysis, a statistician.
In many cases, a QA staff member can fill more than
one role.
Assessors evaluate NHSRC projects under the
authority of. and with the permission of. the NHSRC
PI and NHSRC management. Assessors do not have
the direct authority to change project procedures or
to alter project goals.
9.3.2 Conflict of Interest
The DQA is responsible for ensuring that audit
personnel have no real or perceived conflict
of interest, and have no direct involvement or
responsibility for the work being assessed. The
37
-------
selection of auditors shall ensure objectivity and
impartiality of the audit process. At the discretion
of the DQA, the auditors might be asked to sign
a Conflict of Interest disclosure form prior to
participating in the audit.
9.3.3 NHSRC-Wide Assessment and
Response Procedures
The NHSRC DQA is responsible for planning,
scheduling and/or conducting the assessments
shown in Table 4. The NHSRC DQA may be
assisted in the performance of assessments by
Center researchers who possess the expertise
necessary to conduct the assessment. At the
conclusion of each assessment, the assessment
report shall include a description of the type of
corrective actions required to resolve any findings
of nonconformance as described in EPA quality
documents.
on-site portion of the audit. The general format for
assessment reports is:
o Cover page (assessment identification)
o Summary (brief discussion of findings, those
issues that require corrective action)
o Assessment procedures
o Assessment results (detailed accounting of
findings and other observations)
o Discussion (impact of findings)
o Appendices (checklists, protocols, other data)
Draft assessment reports are generated by the
assessor within two weeks of the audit. Reports are
sent as drafts to the NHSRC PI and to the auditee,
for correction of factual errors and addition of any
pertinent comments or responses to audit findings.
Three weeks are generally allowed for this response
and review.
Self-Assessment of NHSRC Quality System
NHSRC QMP review
yearly, as part of QAARWP
Review of NHSRC Quality System
yearly, as part of QAARWP
Independent Assessment of the Application of the NHSRC Quality System
Quality System Audit
every 3 years
Self-Assessments and Independent Assessment
Technical System Audits
as needed or requested by senior management
Quality System Audit of Programs
as directed by senior management or Program Manager
PEs, ADQ,
are conducted as scheduled in planning documents or
as requested by management
Surveillances
Surveillances are conducted as described in Section
3.3.5.5
Table 4 Types and Frequency of QA Assessment
NOTE: The conduct ofTSAs will probably be performed very infrequently by the DQA. Technical Systems Audits conducted by the
DOA intend to focus mainly on larger projects involving more than one NHSRC location.
9.4 Assessment Reports and
Corrective Action
For TSAs, PEs and ADQs, on-site assessments are
followed up by a debriefing to the auditee and PI
(if available) at the end of the on-site portion of the
evaluation. If the PI is not present at the debriefing,
then assessments are also followed up by a verbal
or brief written summary to the NHSRC PI on
the first working day after the completion of the
At the conclusion of each assessment, the
assessment shall include a description of the type of
corrective actions required to resolve any findings
of nonconformance as described in EPA quality
documents
9.5 Assessment Response Procedures
As mentioned in Section 9.5, the assessment reports
are submitted to the PI. For an extramural project,
the draft reports are distributed to the PI, the PFs
-------
immediate supervisor, and to the auditee. It is the
responsibility of the PI to ensure that the auditee
addresses all of the findings and prepares a response
to findings. The final reports are also distributed to
the NHSRC PI. the Pi s immediate supervisors, and
to the auditee.
For intramural projects, the draft reports are
distributed to the PI and the Pi s immediate
supervisor. It is the responsibility of the NHSRC
PI to resolve audit findings. If findings can not be
resolved by the NHSRC PI. the Pi s immediate
supervisor" assistance is requested. The file on a
given audit is maintained in active status until all
findings are resolved and documented. Audit reports
are then archived in the Pi s project files.
For surveillances, a checklist serves as the report
to the supervisor in charge of the laboratory area. A
copy of the checklist is retained by the DQA. and
another copy should be given to the PI of the area.
9.6 Audit Close-out
The final step of the process is called the audit
close-out. The NHSRC quality assurance staff is
responsible for ensuring that audit findings are
addressed and that specific corrective actions have
been implemented. When a dispute concerning audit
close-out can not be resolved by the Pi s immediate
supervisor, the issue will be resolved by the Deputy
Director for Management and/or the Associate to
Center Director.
39
-------
This page left intentionally blank.
40
-------
10.0
Quality Improvement
Policies and procedures described in this manual
were developed to continuously improve NHSRC
quality system and to document QA planning
and implementation. Two levels of quality
improvement processes combine to evaluate the
effectiveness of the NHSRC quality system and
ensure continual quality improvement. At the
QA Program level, QSAs discussed in Section
10.4 are performed by the NHSRC DQA. Any
problems or areas where quality improvement can
be made are identified and reported to the NHSRC
Director, who is responsible for implementing the
necessary corrective action. At the Project QA
level, the document reviews and audits discussed
in Section 10.5 are conducted to evaluate proposed
procedures and their implementation. Any problems
or areas where quality improvement can be made
are identified and reported to the EPA PI, who
is responsible for implementing the necessary
corrective action. In addition, all project personnel
are responsible for quality improvement activities
with respect to their particular roles.
Finally, it is the responsibility of the DQ A and
QAC to monitor the quality procedures which
are implemented on a day-to-day basis. This is
done via audits, discussions with researchers, and
involvement in the implementation of the quality
system.
If possible, the cause of any identified problem
will be determined before corrective action is
determined. The corrective action shall be planned,
documented, agreed upon, and implemented to
minimize the effect on program/project quality.
To ensure that the current QA practices effectively
accomplish the goals of the NHSRC quality
system, the DQA and QAC meet on a weekly
basis to discuss possible modifications to in-
house Q A procedures. If new procedures need
to be incorporated into the quality system, the
development will be led by the Director of Quality
Assurance with the assistance of key personnel,
determined on a case-by-case evaluation.
10.2 Quality Assurance
Tracking System
The Center's tracking systems are used to monitor
QARF submissions, and QAPP approvals. The
tracking system provides real-time status of QAPPs
that are missing, pending, or approved. This
information is used to send reminders to the Pis
regarding the status of their QA documentation.
In addition, the Pis are required to submit their
products for QA review via the NHSRC « work
intranet site.
10.3 Center-wide Opportunities
for Improvement
The Center strives for continuous improvement
of its processes, services and products by
encouraging personnel to identify areas for
improving our overall quality system. The
quality staff periodically will solicit input from
the staff regarding QA needs and/or satisfaction.
This information will be used to develop training
modules.
10.1 Quality System Opportunities
for Improvement
The DQA will review the Quality Management
Plan annually to determine if the document is still
relevant to the Center's mission and reflects current
practices.
41
-------
This page left intentionally blank.
42
-------
11.0
Supplemental References
American National Standards Institute (ANSI)/
ASQC- E4-1994, Specifications and Guidelines for
Quality Systems for Environmental Data Collection
and Environmental Technology Programs, 1994,
Milwaukee, WI.
U.S.EPA 2000. Quality Manual for Environmental
Programs, CIO 2105 (formerly 5360.1, A2),
US EPA Quality Staff. Office of Environmental
Information (OEI), Washington. D C.
U.S.EPA 2006. Requirements for Quality
Management Plans EPA QA/R-2. EPA/240/B-
01/002, OEI. Washington. D C.
U.S.EPA 2006. Requirements for Quality Assurance
Project Plans EPA QA/R-5. EPA/240/B-01/003,
OEI. US EPA, Washington. D C. (http://www.epa.
gov/quality/qs docs/r5 final.pdf)
U.S.EPA 1992. System Life Cycle Management
Guidance, US EPA Office of Solid Waste and
Emergency Response. Washington. D C.
U.S.EPA 1984. Records Management Manual (EPA
2160), US EPA National Records Management
Program, Washington. D C.
U.S.EPA. 2006. Guidance for the Data Quality
Objectives Process. EPA QA/G-4. EPA/600/R-
96/055, OEI. Washington. D C.
U.S. EPA 2006.Data Quality Assessment: A
Reviewer's Guide. EPA QA/G9-R. EPA/240/B-
06/002, OEI. USEPA. Washington. D C. (http://
www.epa.gov/QUALITY/qs-docs/g9r-final.pdf)
U.S.EPA.2006. Data Quality Assessment: Statistical
Methods for Practitioners. EPA QA/G-9S, EPA/240/
B-06/003. 2006, OEI. USEPA. Washington. D C.
(http://www.epa.gov/quality/qs-docs/g9s-final.pdf)
U.S.EPA 2000. Quality Manual for Environmental
Programs, CIO 2105-P-01-0. OEI. USEPA.
Washington. D C. (http://www.epa.gov/irmpoli8/
ciopolicy/2105-P-01 -0.pdf)
U.S.EPA 2007. Records Management Manual, 2007
NRMP, USEPA, Washington. D C.
(http://vvwvv.epa.gOv/records//policy/indcx.htm
http://vvvvvv.epa.gOv/records//policy/manual/index.
htm)
U.S.EPA. 2006. Guidance on Systematic Planning
Using the Data Quality Objectives Process.
QA/G-4. EPA/240/B-06/01. 2006, OEI. USEPA.
Washington. D C. (http://www.epa.gov/quality/qs-
docs/g4-final .pdf)
U.S.EPA. 2002. Overview of EPA Quality System
for Env ironmental Data and Technology. EPA/240/
R-02/003. 2002, OEI. USEPA. Washington. D C..
(http://www.epa.gov/quality/qs-docs/overv ievv-
final.pdf)
U.S.EPA. 2000.Policy and Program Requirements
for the Mandatory Agency w ide Quality System.
CIO 2105.0, 2000, OEI. USEPA. Washington. D C.
(http://www.epa.gov/irmpoli8/ciopolicy/2105-0.
pdf)
43
-------
This page left intentionally blank.
44
-------
Appendix A:
NHSRC Minimum Requirements for
Center Rsearch Activities
The following requirements are to be implemented
by NHSRC researchers/center personnel to ensure
that data generated for publication are of adequate,
documented quality. Additional requirements
established for a specific project (as specified in
QAPPs, SOPs. or facility manuals) supersede these
minimum requirements.
SECTION 1.0, CENTER RESEARCH
DOCUMENTATION
Center Research Documentation may be made in
laboratory notebooks maintained by the researcher
or in logbooks maintained in the laboratory.
1.1 Research activities shall be sufficiently
documented to allow reconstruction of
those activities. Documentation shall
include the background and the objectives
of the activities, as well as a detailed
account of the activities performed.
1.2 Research documentation shall include
project identification, date, and researcher
identification. The use of bound notebooks
with pro-numbered pages is recommended.
Other forms of documentation may be
used (e.g., p re-printed forms, electronic
records) as long as research activities can
be reconstructed. The PI is responsible
for ensuring that research activities arc
adequately documented.
1.3 Center research documentation shall be
in ink and use the single line method of
correcting entries with the date and initials
of the person making the correction.
1.4 Upon project completion, project-specific
research documentation shall be maintained
in the project file, or the project file shall
identify where the documentation is stored.
1.5 Research documentation which applies
to multiple projects (e.g., equipment and
instrument logbooks) shall be maintained
near the applicable equipment/instrument.
SECTION 2.0, SAMPLE STORAGE
2.1 Refrigerators used to store samples shall be
monitored for temperature daily. Sample
storage refrigerator temperature monitoring
shall be documented.
2.2 Samples shall be maintained to ensure
their integrity. For example, samples
that require refrigeration (as specified in
a project-specific QAPP/SOP) shall be
refrigerated prior to analysis if the analysis
is not performed immediately and returned
to refrigeration immediately following
analysis, until it is determined that
additional analyses are not needed.
2.3 Samples shall be stored away from
standards, samples, and other materials
which could potentially cross-contaminate
them. For example, aqueous/soil samples
for volatile organic compounds should
not be stored in the same refrigerator as
concentrated volatile organic compound
standards.
SECTION 3.0, EQUIPMENT CALIBRATION
AND MAINTENANCE
3.1. Balance calibration shall be checked
periodically in the range of use (daily with
use is recommended unless justification
for less frequent checking is documented
in a QAPP. SOP. or facility manual).
Balance calibration checks shall be
documented. Documentation shall include
the balance identification, date, calibrator's
identification, and calibration check data.
45
-------
Analytical balances will be serviced yearly
and have their calibrations re-certified
against ANSI/ASTM class 1 weight.
3.2 Analytical instrumentation (e.g.. GCs. GC7
MSs, I CPs. and A As) shall be calibrated
prior to use with standards of known
and documented uncertainty traceable
to a recognized standard organization
if applicable. Initial calibration shall be
verified using a standard from a different
source than that used for the initial
calibration. (If this second source check is
not done, justification shall be documented
in a QAPP, SOP. or facility manual.)
Calibration shall be checked periodically
during use. Analytical instrumentation
calibration, calibration verification, and
calibration checks shall be documented.
Documentation shall include the
instrument identification, date, calibrator's
identification, and calibration data. 1
3.3 Prior to project initiation, the PI shall
ensure the accuracy of supporting
equipment, as applicable (e.g.. flow meters,
thermometers/thermocouples, variable
pipettes). Applicable documentation shall
be maintained.
3.4 If temperature is critical, refrigerators,
ovens, incubators, and constant temperature
baths used during the implementation of
analytical procedures (e.g.. incubator for
microbiological tests) shall be monitored
to ensure that the required temperature is
maintained. Temperature monitoring shall
be documented.
3.5. All analytical instruments shall be properly
maintained. Maintenance procedures
shall be documented and kept near the
instrument. Documentation shall include
the instrument identification, date, analyst
identification, and the maintenance
performed.
SECTION 4.0, STAND OPERATING
PROCEDURES (SOP)
4.1 SOPs shall be prepared to document
analytical procedures (if a standard
procedure is not implicitly followed) that
will be performed routinely or that will be
implemented by multiple personnel.
4.2 SOPs shall be maintained in the laboratory
area where the procedure is implemented.
SECTION 5.0, STANDARD PREPARATION
5.1 Standard preparation shall be documented.
Documentation shall include the date,
analyst identification, the identity of
the stock/intermediate standard used,
preparation procedures, and applicable
expiration dates.
5.2 Standards shall be stored to maintain their
integrity.
SECTION 6.0, QUALITY ASSURANCE AND
QUALITY CONTROL CHECKS
6.1 QA/QC checks shall be performed as
defined in applicable SOPs/QAPPs/facility
manuals.
6.2 When QA/QC checks are not defined in
SOPs/QAPPs/facility manuals, positive
controls (e.g.. blank spikes, matrix
spikes), negative controls (e.g.. blanks),
and replicates (e.g.. duplicates) shall be
performed periodically to demonstrate the
accuracy and precision of a method for
each unique matrix.
6.3 Method detection limits (MDLs) shall be
determined when results below the low
calibration standard will be reported.
46
-------
SECTION 7.0, DATA REVIEW
SECTION 8.0, DATA STORAGE
7.1 For each analysis, the analyst shall review
results for QC checks performed to
determine compliance with acceptance
criteria specified in the applicable QAPP,
SOP, or facility manual. If acceptance
criteria are not met. the analyst shall
perform corrective action as required by the
applicable QAPP, SOP, or facility manual.
7.2 Data generated shall be reviewed by a
second technical person for a representative
sample for each method performed (10% is
recommended). (Note that this may not be
possible for very specialized analyses). If
problems are identified, additional review
will be performed to determine the extent of
the problem. Review shall be documented.
Documentation of research activities, including
analytical data, shall be maintained as required
by EPA's record management policies.
47
-------
This page left intentionally blank.
48
-------
Appendix B:
QAPP Requirements and Guidance
REQUIREMENTS AND GUIDANCE
For Category I (enforcement, litigation, or
projects involving human subjects) and Category
11 (development of environmental regulations or
standards). NHSRC's requirements are those listed
in EPA Requirements for Quality Assurance
Project Plans, EPA QA/R-5, EPA/240/B-
01/003, March 2001. Guidance for Quality
Assurance Project Plans, EPA QA/G-5,
EPA/240/R-02/009, December 2002 may be
used to help address these requirements. (These
documents are available at http://www.epa.gov/
qualitv/qa docs.html)
QAPP requirements for Category III (Applied
Research) and Category IV (Basic Research) are
presented in this appendix. Guidance for Quality
Assurance Project Plans, EPA QA/G-5, EPA/240/R-
02/009, December 2002 referenced above may be
used to help address the requirements listed for
Category III and IV QAPPs; however, only the
requirements listed in this appendix need to be
addressed.
Project QAPP; however, only the requirements
listed in this appendix need to be addressed.
QAPP requirements for Sampling and Analysis
Projects. Methods Development Projects, and
Software and Data Management Projects are also
presented in this appendix. Requirements for
modeling projects are currently under development.
(See your DQA OR QAC if you need these
requirements.)
For projects involving design, construction, and/
or operation of environmental technology, the
requirements in Part C of "Specifications and
Guidelines for Quality Systems for Environmental
Data Collection and Environmental Technology."
ANSI/ASQC- E4-1994, American Society for
Quality Control. Milwaukee, WI, January 1995 will
be followed.
The requirements for QMPs submitted to NHSRC
in response to solicitations are also presented in this
appendix.
QAPP requirements Secondary Data Research
Projects are presented in this appendix. Guidance
for Quality Assurance Project Plans, EPA
QA/G-5, EPA/240/R-02/009. December 2002
referenced above may be used to help address the
requirements listed for Secondary Data Research
49
-------
QAPP REQUIREMENTS FOR SAMPLING
AND ANALYSIS PROJECTS
A sampling and analysis activity or project is
typically defined as a study performed to generate
data to either monitor parameters on a routine basis
or to characterize a particular population for later
studies. The follow ing requirements should be
addressed as applicable.
SECTION 1.0, PROJECT DESCRIPTION AND
ORGANIZATION
1.1 The purpose of the study shall be clearly
stated in the sampling and analysis plan
(SAP).
1.2 Responsibilities and points of contact for
each organization shall be identified in the
SAP. This should include identification
of key personnel and/or organization(s)
responsible for sample collection and
custody, analytical and/or process
measurements, data reduction, report
preparation, and quality assurance.
SECTION 2.0, SAMPLING
2.1 Sampling points for all measurements (i.e.,
analytical, physical, and process, including
locations and access points) shall be
identified in the SAP whenever possible. If
the specific locations cannot be identified
at the time of plan generation, discuss the
documentation and/or communication
mechanism(s) for ensuring adequate
information is captured to later identify
sampling points.
2.2 The anticipated sampling frequency (e.g..
how many sampling events and how often
events occur), the number of sample types
(e.g.. metals. VOCs. SVOCs). and the
minimum number of samples of each type
taken at each event shall be provided.
2.3 The expected measurements (i.e., specific
analytes) planned for each sample type
shall be summarized.
2.4 If applicable, know n site-specific factors
that may affect sampling procedures shall
be described.
2.5 If applicable, any site preparation (e.g..
sampling device installation, sampling port
modifications) needed prior to sampling
shall be described.
2.6 Each sampling procedure (including a list
of equipment needed and the calibration
of this equipment as appropriate) to be
used shall be discussed or referenced.
Maintenance requirements/procedures (as
appropriate) must also be addressed in this
section.
2.7 If compositing or splitting of samples is
planned, the applicable procedures shall be
described.
2.8 A list of sample quantities to be collected,
and the sample amount required for each
analysis, including QC sample analysis,
shall be specified.
2.9 Containers used for sample collection,
transport, and storage for each sample type
shall be described.
2.10 Sample preservation methods (e.g.,
refrigeration, acidification) shall be
described.
2.11 Requirements for shipping samples shall be
described.
2.12 Holding time's requirements shall be noted.
2.13 Procedures for tracking samples in
the laboratory and for maintaining
chain-of-custody when samples are shipped
shall be described. COC procedures shall
be described to ensure that sample integrity
is maintained (labeling, seals, records).
50
-------
2.14 Information to be recorded and maintained
by field personnel shall be discussed.
SECTION 3.0, TESTING AND
MEASUREMENT PROTOCOLS
3.1 Each analytical method to be used shall be
referenced. This includes EPA-approved and
other validated nonstandard methods.
3.2 If applicable, modifications to EPA-approved
or other validated nonstandard methods shall
also be described.
SECTION 4.0, QA/QC CHECKS
4.1 The SAP shall list and define all calibrations
and QC checks and/or procedures used for
the project, both field and laboratory as
needed.
4.2 For each specified calibration. QC check,
or procedure, required frequencies and
acceptance criteria shall be included.
SECTION 5.0, DATA REDUCTION AND
REPORTING
5.1 Data reduction procedures specific to
the project, and also specific to each
organization, shall be summarized.
5.2 The reporting requirements (e.g.. units,
reporting method [e.g., wet or dry]) for each
measurement and matrix shall be identified.
SEC TION 6.0, REPORTING REQUIREMENTS
The deliverables expected from each organization
responsible for field and/or analytical activities shall
be described.
51
-------
SECTION 2.0, EXPERIMENTAL APPROACH
2.1 Describe the process, site, facility,
apparatus, and/or environmental system to
be tested.
2.2 Describe all known or pre-established
test conditions and variables, including
replicate experimental runs.
2.3 Describe the planned approach (statistical
and/or non-statistical) for evaluating project
objectives (i.e., data analysis).
SECTION 3.0, SAMPLING AND
MEASUREMENT APPROACH AND
PROCEDURES
3.1 Complete a table similar to the following
to summarize the experimental sampling
strategy to be used.
Sample/Measurement
Location
Matrix
Measurement
Frequency
Experimental
QC1
Total No. Samples
'OC samples generated during experiment, as applicable (e.g., blanks, replicate samples, spikes)
3.2 Complete a table similar to the following to summarize the experimental sampling and analytical
procedures to be used.
Matrix
Measurement
Sampling/
Measurement
Method1
Analysis
Method1
Sample
Container/
Quantity of
Sample
Preservation/
Storage
Holding
Time(s)2
lProvide details in text, as necessary, if standard method or SOP cannot be referenced
2Both to extraction and analysis, if applicable
QAPP REQUIREMENTS FOR BASIC
RESEARCH PROJECTS
A basic research project is a study performed to
generate data used to evaluate unproven theories,
processes, or technologies.
SECTION 1.0, PROJECT OBJECTIVES AND
ORGANIZATION
1.1 State the proj ect obj ectives.
1.2 Identify the responsibilities of all project
participants (e.g., QAPP preparation,
sample collection and analyses, data
reduction/validation/analysis, report
preparation, QA).
-------
SECTION 4.0, QA/QC CHECKS
Complete a table similar to the following to summarize QA/QC checks.
Matrix
Measurement
QA/QC Check1
Frequency
Acceptance Criteria
Corrective Action
1 Include all OA/OC checks (experimental and analytical, as applicable) for accuracy, precision, detection limits, mass balance (e.g
matrix spikes, lab control samples, blanks, replicates, surrogates)
SECTION 5.0, DATA REPORTING
Describe data reduction procedures specific
to the project.
SECTION 6.0, REFERENCES
Provide references to methods and germane
prior publications.
IN ADDITION, WHEN APPLICABLE...
• If bulk sample(s) will be collected in the
field for use in laboratory experiments,
include applicable information from Section
2.0 of OAPP Requirements for Sampling and
Analysis Projects.
• List all project-specific target analytes (i.e.,
when a class of compounds is specified in
the table).
• Indicate if reporting is on a wet or dry
weight basis (solid matrices only).
• Describe the method used to establish
steady-state conditions.
• Describe how sampling equipment is
calibrated.
• Describe how cross-contamination between
samples is avoided.
• Describe the procedures used to collect
representative samples.
• Describe sample packing and shipping
procedures.
• Describe instrument calibration procedures
and acceptance criteria if not included in a
referenced method or SOP.
-------
QAPP REQUIREMENTS FOR APPLIED
RESEARCH PROJECTS
An applied research project is a study to
demonstrate the performance of technologies under
defined conditions. These studies are often pilot- or
field-scale. The follow ing requirements should be
addressed as applicable.
SECTION 1.0, PROJECT DESCRIPTION AND
OBJECTIVES
1.1 The purpose of study shall be clearly stated.
1.2 The process, site, facility, and/or
environmental system to be tested shall be
described.
1.3 Project objectives shall be clearly stated
and identified as primary or non-primary.
SECTION 2.0, PROJECT ORGANIZATION
2.1 Key points of contact for each organization
involved in the project shall be identified.
2.2 All QA staffs and their relationship in the
organizations (i.e., location within each
organization) shall be identified with
evidence that the Q A staff is independent of
project management.
2.3 Responsibilities of all other project
participants and their relationship to other
project participants shall be identified,
meaning that organizations responsible for
planning, coordination, sample collection,
sample custody, measurements (i.e.,
analytical, physical, and process), data
reduction, data validation, and report
preparation shall be clearly identified.
2.4 A distribution list shall be provided to
facilitate the distribution of the most recent
current version of the QAPP to all the
principal project participants.
SECTION 3.0, EXPERI MENTAL APPROACH
3.1 The general approach and the test
conditions for each experimental phase
shall be provided. The statistical methods
that will be used to evaluate the data (i.e.,
ANOVA, or summary statistics) should be
identified.
(NOTE: As deemed appropriate to
the project by the PI. the information
requested in Sections 3.2, 3.3, and 3.4
may be presented here or in Section 4; the
information requested in Sections 3.5 may
be presented here or in Section 5; and the
information requested in Sections 3.6 may
be presented here or in Section 7.)
3.2 The sampling strategy shall be included
and evidence must be presented to
demonstrate that the strategy is appropriate
for meeting primary project objectives, i.e.,
a description of the statistical method or
scientific rationale used to select sample
sites and number of samples shall be
provided.
3.3 Samp 1 ing/monitoring points for all
measurements (i.e., including locations and
access points) shall be identified.
3.4 The frequency of sam p 1 i ng/mon i to ri ng
events, as well as the numbers for each
sample type and/or location shall be
provided, including QC and reserve
samples.
3.5 All measurements (i.e., analytical
[chemical, microbiological, assays|.
physical, and process) shall be identified
for each sample type or process, and
project-specific target analytes shall be
listed and classified as critical or noncritical
in the QAPP.
3.6 The planned approach (statistical and/
or non-statistical) for evaluating project
objectives shall be included.
SECTION 4.0, SAMPLING PROCEDURES
4.1 Whenever applicable, the method used to
establish steady-state conditions shall be
described.
-------
4.2 Known site-specific factors that may affect
sampling/monitoring procedures shall be
described.
4.3 Any site preparation needed prior to
sampling/monitoring shall be described.
4.4 Each sampling/monitoring procedure to
be used shall be discussed or referenced.
If compositing or splitting samples, those
procedures shall be described.
4.5 For samples requiring a split sample for
either QA/QC purposes or for shipment
to a different laboratory, the QAPP shall
identify who is responsible for splitting
samples, and where the splitting is
performed (e.g.. field versus lab).
4.6 If samp 1 ing/monitoring equipment is used
to collect critical measurement data (i.e.,
used to calculate the final concentration
of a critical parameter), the QAPP shall
describe how the sampling equipment is
calibrated, the frequency at which it is
calibrated, and the acceptance criteria for
calibration or calibration verification, as
appropriate.
4.7 If sam p 1 i ng/m on i to ri ng equipment is
used to collect critical measurement
data, the QAPP shall describe how cross-
contamination between samples is avoided.
4.8 The QAPP shall include a discussion of
the procedures to be used to assure that
representative samples are collected.
4.9 A list of sample quantities to be collected,
and the sample amount required for each
analysis, including QC sample analysis,
shall be specified.
4.10 Containers used for sample collection,
transport, and storage for each sample type
shall be described.
4.11 The method for uniquely identifying each
sample shall be described.
4.12 Sample preservation methods (e.g..
refrigeration, acidification), including
specific reagents, equipment, and supplies
required for sample preservation shall be
described.
4.13 Holding time requirements shall be noted.
4.14 Procedures for packing and shipping
samples shall be described.
4.15 Procedures to maintain chain-of-custody
(e.g., custody seals, records) during transfer
from the field to the laboratory, in the
laboratory, and among contractors and
subcontractors shall be described to ensure
that sample integrity is maintained.
4.16 Sample archival requirements for each
relevant organization shall be provided.
SECTION 5.0, TESTING AND
MEASUREMENT PROTOCOLS
5.1 Each measurement method to be used
shall be described in detail or referenced.
Modifications to EPA-approved or similarly
validated methods shall be specified.
5.2 For unproven methods, verification data
applicable to expected matrices shall be
included in the QAPP meaning the QAPP
shall provide evidence that the proposed
method is capable of achieving the desired
performance.
5.3 For measurements which require a
calibrated system, the QAPP shall include
specific calibration procedures applicable
to each project target analyte, and the
procedures for verifying both initial
and continuing calibrations (including
frequency and acceptance criteria, and
corrective actions to be performed if
acceptance criteria are not met).
55
-------
SECTION 6.0, QA/QC CHECKS
6.1 At a minimum, the QAPP shall include
quantitative acceptance criteria for QA
objectives associated with accuracy,
precision, detection limits, and
completeness for critical measurements
(process, physical, and analytical, as
applicable) for each matrix.
6.2 Any additional project-specific QA
objectives shall be presented, including
acceptance criteria. This includes items
such as mass balance requirements.
6.3 The specific procedures used to assess
all identified QA objectives shall be fully
described.
6.4 The QAPP shall list and define all other
QC checks and/or procedures (e.g., blanks,
surrogates, controls) used for the project,
both field and laboratory.
6.5 For each specified QC check or procedure,
required frequencies, associated acceptance
criteria, and corrective actions to be
performed if acceptance criteria are not met
shall be included.
SECTION 7.0, DATA REPORTING, DATA
REDUCTION, AND DATA VALIDATION
7.1 The reporting requirements (e.g.. units,
reporting method [wet or dry]) for each
measurement and matrix shall be identified.
7.2 The deliverables expected from each
organization responsible for field and
laboratory activities shall be listed.
7.3 Data reduction procedures specific to
the project, and also specific to each
organization, shall be summarized.
7.4 Data validation procedures specific to each
organization used to ensure the reporting
of accurate project data to internal and
external clients shall be summarized.
7.5 Data storage requirements for each
organization shall be provided.
7.6 The product document that will be prepared
for the project shall be specified (e.g..
journal article, final report). The contents
of this document can be referenced to
a NHSRC or program-specific QMP, if
appropriate.
SECTION 8.0, ASSESSMENTS
8.1 The QAPP shall identify all scheduled
audits (i.e., both technical system audits
[TSAs] and performance evaluations [PEs])
to be performed, who will perform these
audits, and who will receive the audit
reports.
8.2 The QAPP shall provide procedures
that are to be followed that will ensure
that necessary corrective actions will be
performed.
8.3 The responsible party (-ies) for
implementing corrective actions shall be
identified.
SECTION 9.0, REFERENCES
References shall be provided either in the body of
the text as footnotes or in a separate section.
-------
QAPP REQUIREMENTS FOR RESEARCH
MODEL DEVELOPMENT AND
APPLICATION PROJECTS
A research model project is a study performed to
develop a new model or apply an existing model
to provide information to support non-regulatory
environmental research or decision-making. A
QAPP must be submitted at the beginning of a
research model development or application project.
The QAPP should specify the quality requirements
needed to ensure the quality of the results produced
by the model. The recommended format for
research model development QAPPs is presented
below; guidance for completing each section is
provided in italics. (For model application projects,
a smaller subset of requirements needs to be
addressed, as appropriate; e.g.. information on code
development would not be included.) If data w ill
be generated to develop or calibrate the model, a
separate QAPP is needed and should address the
requirements applicable to the type of research
project (e.g.. basic research, applied research). If
a model will be developed to support regulatory
environmental decision-making, additional
requirements may apply.
SECTION 1.0, PROJECT DESCRIPTION
1.1 Discuss the scope and purpose of the
model. Provide a brief statement of the
scope and purpose. The specific problem
which needs to be addressed should be
discussed, including the intended users of
the model.
1.2 Identify the project's objectives.
Discuss the specific objectives for this
project, including the expected product
and a timetable for completion.
1.3 Identify the roles and
responsibilities of all project
participants and support facilities.
Identify project personnel and
key support facilities (including
computer facilities). Discuss the
duties/responsibilities for each.
An organizational chart can be
used to show lines of authority
and communication.
SECTION 2.0, MODEL DESCRIPTION
2.1 Discuss the model parameters, including
the theoretical approach for the model and
the mathematical relationship between
input and output variables.
Provide an overview of the model
parameters, including:
• model origin and its original
purpose, if applicable
• parameters and variables
• the algorithms and equations that
have been developed to support
the model theory , along w ith the
sources of the algorithms
• spatial extent (individual, group,
population)
• spatial resolution (location
independent/dependent,
dimensionality)
• temporal extent (length of
modeling period)
• temporal resolution (time step)
• model structure (e.g.. stochastic
vs. deterministic, structural
framework).
2.2 Discuss any initial assumptions regarding
model development/application. Initial
assumptions made during model
development should be identified.
57
-------
2.3 Specify required sources for model
databases and any requirements for these
data (e.g., quality, quantity, spatial, and
temporal applicability). If data sources
are not currently known, describe the
criteria used to identify sources.
The purpose of assessing data quality is
to evaluate, to the extent possible, the
reliability of the existing data base(s).
Procedures for determining precision,
accuracy, representativeness, completeness,
and comparability of existing data should
be summarized. Specific parameters to be
discussed include:
• source of data and criteria for
acceptance or rejection
• any modifications from existing
data
• data format, maintenance, and
archiving.
SECTION 3.0, MODEL DEVELOPMENT
3.1 Discuss requirements for code
development. Quality Assurance
procedures for code development should
include complete record keeping of the
model development and of modifications
made in the code. Required records
include:
¦ parameter values and
sources
¦ changes and verification of
changes made in code
¦ output of model runs and
interpretation
¦ assumptions.
If any modifications are made to the model
coding, the code should be tested again;
all QA procedures for model development
should again be applied, including accurate
record keeping and reporting.
The code documentation should include:
• model specifications
• model description
• flow charts
• description of routines
• data base description
• source listing
• error messages.
3.2 Discuss computer requirements for both
hardware and software.
Identify computer requirements, including:
• programming language
(FORTRAN. BASIC, etc.) and
ANSI! standard
• model portability
• memory requirements
• required hardware/software for
application
• data standards for information
storage and retrieval (refer
to Office of Environmental
Information guidance.
When appropriate, a review of existing
software should first be considered to
determine capability for implementing the
new model.
3.3 Discuss how the code will be verified.
The objective of the code verification
process is to check the correctness and
accuracy of the computational algorithms
used to solve the governing equations and
to assure that the computer code is fully
operational.
The inspection of the computer code is
part of the model review process. In this
inspection, attention is given to the manner
in which modern programming principles
have been applied with respect to code
structure, compliance with programming
standards, efficient use of programming
languages, and internal documentation.
This step may reveal programming or logic
errors that are difficult or impossible to
detect in verification runs.
58
-------
Describe the requirements for model
documentation. Model documentation
is defined as the information recorded
during the design, development, and
maintenance of the model, in order to
explain pertinent aspects, including
purposes, methods, logic, relationships,
capabilities, and limitations. It is the
principle instrument of communication
used by the model author, the model user,
and the system operator.
Good documentation includes a description
of (some of these may have been discussed
previously): the equations on which the
model is based;
• the underlying assumptions
• the boundary conditions that can be
incorporated in the model
• the method used to solve the
equations
• limiting conditions.
The documentation may also include:
• user's guide (electronic or paper)
• source code
• instructions for preparing data files
• example problems complete with
input and output
• programmer's instructions
• computer operator's instructions
• report of the initial code
verification
• documentation of significant
changes to the model
• procedures for maintenance and
user support, if applicable.
SECTION 4.0, MODEL CALIBRATION
Model calibration is defined as the
process of refining the model to achieve
a desired degree of correspondence
between the model output and actual
observations of the environmental system
that the model is intended to represent.
Model development is an evolutionary
process responding to new research
results, developments in technology, and
changes in user requirements. Model
calibration needs to follow this dynamic
process and should be applied each time
the model is modified.
4.1 Discuss how the model will be calibrated.
Identify the type and source of data (e.g..
new data, existing data, professional
judgment, expert opinion elicitation) that
will be used to calibrate the model. If data
sources are not currently know n, describe
the criteria used to identify sources.
Describe any requirements for the data that
will be used to calibrate the model.
Calibration data requirements with
respect to quality, quantity, and
spatial and temporal applicability
should be specified, as applicable.
Specify criteria which need to be met
for the difference between predicted
and observed data during model
calibration.
4.2
4.3
-------
The acceptance criteria which need to be
met for the difference between predicted
and observed data should be specified.
The statistical methods to be used (e.g.,
goodness-of-fit, regression analyses) should
also be discussed. If criteria cannot be
specified, this should be discussed.
SECTION 5.0, MODEL ASSESSMENT
(VALIDATION) AND APPLICATION
5.1 Discuss the assessments planned to ensure
the acceptability of model outputs.
This element of the QAPP documents
the types of assessments to be performed
throughout the various stages of model
development and application, the purpose
of each assessment and the specific
model features that each assessment is to
address, and the expected periods of time
in which the assessments will take place.
Details regarding how the assessments
will be performed and by who need to
be provided. The specific assessments
are based on a clear understanding and
statement of the purpose of the model
and the accuracy of the model outputs
needed (predictions).
In general, this QA Project Plan
element specifies the follow ing types of
information:
• a description of the assessment/
oversight strategies and schedule
of assessment activities, including
the order in which the assessments
will be conducted and how the total
set of assessments is structured
to provide a complete and
comprehensive oversight;
• a description of how each
assessment will be planned and
conducted and;
• the organizations and individuals
that are expected to participate
in assessments, including peer
reviews and;
• the information expected success
criteria, and documentation for
each assessment.
Additional guidance on assessments is
provided in EPA QA/G-5M ("www.epa.gov/
ciualitv/qa docs.html).
5.2 Identify any restrictions on the use of the
model. Restrictions on model application
should be outlined. Categories of
restrictions include:
1. assumptions
2. parameter values and sources
3. boundary and initial conditions
4. validation/calibration of the model
output and interpretation of model
runs
If any of these items has been presented
and discussed previously, inclusion here
is not necessary.
SECTION 6.0, REFERENCES
Provide references to methods and applicable
publications.
EPA, Environmental Research Laboratory. Quality
Assurance Guidelines for Modeling Development
and Application Projects. November 1991.
EPA QA/G-5M. 2002. Guidance for Quality
Assurance Project Plans for Modeling, van der
Heijde. P.K.M. 1989.
Quality Assurance and Quality Control in
Groundwater Modeling. IGWMC Groundwater
Modeling Publications. Holcomb Research
Institute. Butler University, Indianapolis. IN:25.
60
-------
QAPP REQUIREMENTS FOR PROJECTS
USING SECONDARY DATA
2.1 The sourcc(s) of the secondary data must be
specified.
A secondary data project involves the gathering and/
or use of existing environmental data for purposes
other than those for which they were originally
collected. These secondary data may be obtained
from many sources, including literature, industry
surveys, compilations from computerized databases
and information systems, and computerized or
mathematical models of environmental processes.
For these projects, a QAPP shall be prepared to
include the requirements identified below. If primary
data will also be generated as part of the project,
then the information below can be incorporated into
the associated QAPP to address the secondary data.
The following requirements should be addressed as
applicable.
SECTION 1.0, PROJECT OBJECTIVES,
ORGANIZATION, AND RESPONSIBILITIES
1.1 The purpose of study shall be clearly stated.
1.2 Project objectives shall be clearly stated.
1.3 The secondary data needed to satisfy
the project objectives shall be identified.
Requirements relating to the type of data,
the age of data, geographical representation,
temporal representation, and technological
representation, as applicable, shall be
specified.
1.4 The planned approach for evaluating project
objectives, including formulas, units,
definitions of terms, and statistical analysis,
if applicable, shall be included.
1.5 Responsibilities of all project participants
shall be identified, meaning that key
personnel and their organizations shall be
identified, along with the designation of
responsibilities for planning, coordination,
data gathering, data analysis, report
preparation, and quality assurance, as
applicable.
SECTION 2.0, SOURCES OF SEC ONDARY
DATA
2.2 The rationale for selecting the source(s)
identified shall be discussed.
2.3 The sources of the secondary data will be
identified in any project deliverable.
SEC TION 3.0, QUALITY OF SEC ONDARY
DATA
3.1 Quality requirements of the secondary
data must be specified. These requirements
must be appropriate for their intended use.
Accuracy, precision, representativeness,
completeness, and comparability need to be
addressed, if applicable. (If appropriate, a
related QAPP containing this information
can be referenced.)
3.2 The procedures for determining the quality
of the secondary data shall be described.
3.3 If no quality requirements exist, this
shall be stated in the QAPP. If no quality
requirements exist or if the quality of the
secondary data will not be evaluated by EPA,
the QAPP shall require that a disclaimer be
added to any project deliverable to indicate
that the quality of the secondary data has
not been evaluated by EPA for this specific
application. The wording for the disclaimer
shall be defined.
SEC TION 4.0, DATA REPORTING, DATA
REDUCTION, AND DATA VALIDATION
4.1 Data reduction procedures specific to
the project shall be described, including
calculations and equations.
4.2 The data validation procedures used to
ensure the reporting of accurate project data
shall be described.
4.3 The expected product document that will
be prepared shall be specified (e.g.. journal
article, final report).
61
-------
QAPP REQUIREMENTS FOR METHOD
DEVELOPMENT PROJECTS
A method development project is typically needed
in situations for which there exists no standard or
known method, or when an existing method needs
to be modified to meet a project-specific need. The
following requirements should be addressed as
applicable.
SECTION 1.0, BACKGROUND
A description of the situation that requires the
generation of a new or modified method shall be
clearly stated. Why are we doing this?
SECTION 2.0, SCOPE AND APPLICATION
The scope and application of the method shall
be clearly stated. Specifically, to what matrices,
conditions, will this method apply for this project'.'
What detection limits and/or practical quantitation
limits are needed'.' How is this method intended to
be used in the future (e.g.. research only, potential
regulatory usage)'.'
SECTION 3.0, PROJECT ORGANIZATION
Responsibilities of all project participants shall
be identified, meaning that key personnel and
their organizations shall be identified, along with
the designation of responsibilities for planning,
coordination, sample collection, measurements (i.e.,
analytical, physical, and process), data reduction,
data validation (independent of data generation),
data analysis, report preparation, and quality
assurance.
SECTION 4.0, EXPERI MENTAL APPROACH
INCLUDING SAMPLING AND ANALYTICAL
SPECIFICATIONS
4.1 A description of the test(s) to be conducted
in order to support the development of the
method shall be included. All known or
preestablished test conditions and variables
shall be provided.
4.2 All planned measurements (i.e., analytical
[chemical, microbiological, assays|.
physical, and process) shall be identified,
and project-specific target analytes shall be
listed.
4.3 Any known restrictions/specifications for
sampling (e.g.. collecting soil samples
from a site or water samples from a port)
or subsampling (e.g.. mixing sample
before taking subsample for analysis) shall
be documented. Include specifications
for: type and size of sample containers;
amount of sample needed for preparation
and analysis; preservation; holding times;
representativeness; compositing; QC
samples; etc.
4.4 The type of instrumentation that will
be used and any required instrument
conditions shall be documented. Include
a discussion of calibration and calibration
verification including frequency,
acceptance criteria, and corrective action to
be taken if acceptance criteria are not met.
SECTION 5.0, QA/QC CHECKS
Any planned QC checks and criteria that must be
met for the method to be considered successful
shall be specified. QC checks may include spikes,
replicates, blanks, controls, and surrogates.
Note: For chemical methods, quality control
procedures to determine the precision, accuracy,
and method detection limit should be described.
For microbiological methods, positive and negative
control procedures should be described.
SECTION 6.0, METHOD VERIFICATION
The tests that will be used to verify the method's
performance once it's been developed shall be
specified.
SECTION 7.0, REPORT
The report for a successful method development
project will be a method written in a format
appropriate for the application e.g.. SW-846 for
RCRA applications. Standard Methods for bacteria
in drinking water, a SOP for a specific application
(with supporting method performance data
appended).
SECTION 8.0, REFERENCES
References shall be provided either in the body of
the text as footnotes or in a separate section.
-------
QAPP REQUIREMENTS FOR SOFTWARE
AND DATA MANAGEMENT PROJECTS
SECTION 3.0, FUNCTIONAL
REQUIREMENTS
Types of projects to which this guidance applies
include the following: software development,
software/hardware systems development, and data
base design and maintenance, and data validation
and verification systems. The QAPP requirements
for software development in this appendix do
not mandate a particular method for software
development. Project managers should choose
software development and QA methods best suited
to their individual projects within the parameters set
forth here. Table B-l provides a set of alternative
QAPP elements for situations in which the elements
applicable to measurement projects are not
appropriate. The applicability of different elements
is based on (1) the QA category and (2) the size or
complexity of the task. Projects that involve both
measurement and software/systems development
should have plans addressing all applicable QA
elements. Main issues to consider for inclusion in a
QAPP for software and data management are listed
in the follow ing sections. Additional guidance for
software and data management projects is available
from the DQA OR QACs.
SEC TION 1.0, PROJEC T DESCRIPTION and
APPROVAL
This section should provide an overview of the
project, its intended uses, quality objectives,
schedules and appropriate milestones, information
about the hardware and operating systems, and
planning documents.
The EPA Technical Lead Person (PI) shall be
responsible for obtaining signatures of appropriate
project participants on the signature page of the QA
plan, documenting agreement to project objectives
and the approach for evaluating these objectives.
SEC TION 2.0, PROJEC T ORGANIZATION
AND RESPONSIBILITIES
This section should discuss all important intramural
and extramural project personnel and should show
the relationship between the development team and
the personnel responsible for QA and testing.
This section should provide a list of the most
important functions that the software system must
address. This section can also state any quantitative
or qualitative data quality objectives (DQOs) that
might apply to the software.
SEC TION 4.0, SYSTEM DESIGN OVERVIEW
(HIGH LEVEL DESIGN)
A brief description of the system design is all that
is necessary in the QAPP. if additional design
documentation is planned.
SEC TION 5.0, DETAILED DESIGN
Complex projects and those w ith significant
defensibility requirements should have a detailed
design document.
SEC TION 6.0, IMPLEMENTATION
Written standard operating procedures (SOPs)
for software development should be provided for
extremely large and complex software projects. The
internal checks applied during development should
also be described.
SEC TION 7.0, TESTING
The QAPP should outline the testing strategy to be
used.
SEC TION 8.0, DATA VALIDATION AND
VERIFICATION
The QAPP must describe the means for checking
the correctness of outputs.
SEC TION 9.0, CHANGE CONTROL AND
CONFIGURATION MANAGEMENT
This section should describe the procedures for
controlling and documenting all significant changes
to software and hardware.
-------
SECTION 10.0, AUDITS AND REVIEWS
This section should describe planned assessments,
including performance evaluation audits (PEAs),
technical systems audits (TSAs), quality systems
audits (QSAs). and audits of data quality ( ADQs).
Additional types of reviews applicable to these
projects include peer reviews and beta testing.
SECTION 11.0, MAINTENANCE AND USER
SUPPORT
Where software or data generated by the project
will be distributed outside NHSRC. maintenance
and user support must be addressed.
SECTION 12.0, SYSTEM DOCUMENTATION
AND ARCHIVING
Documentation is required for software projects in
all QA categories. Table B-2 gives documentation
requirements by QA Category.
SECTION 13.0, QA PROGRESS REPORTS TO
MANAGEMENT
System development QA and QC results and plans
should be reported regularly, particularly in projects
in Categories I and 11 and where contractually
required.
64
-------
Tabic B-l QAPP Elements according to the QA Category
QAPP Element
Title/Signature Page
Category Applicability
Table of Contents
1. Project Description
a. Background
b. Intended Application for Software
c. Quality Objectives for Software
d. Scope of Work
e. Schedule and Milestones
f. Facilities Description
g. Experimental/Test Matrix Design
h. Planning Documents
2. Project Organization and I, II
Responsibilities
3. Functional Requirements
4. System Design Overview I, II
5. Detailed Design I, II
6. Implementation I, II
a. Development of SOPs
b. QC for Implementation
7. Testing I, II
a. Individual Module Tests
b. Integration Tests
c. System Tests
d. Retesting after Changes
e. Acceptance Testing (if applicable)
f. Beta Testing (if applicable)
8. Data Validation and Verification I, II
9. Change Control and Configuration Management
10. Audits and Reviews I, II
11. Maintenance and User Support
12. System Documentation and Archiving
13. QA Progress Reports to Management I
IV*
IV*
, IV
, IV
, IV
, IV
IV*
IV
*These elements may not be applicable for all projects in the specific category.
65
-------
Tabic B- 2 Recommended Documentation for Archiving by QA Category
Document
OA Category
QA Project Plan
I, II, III, IV
Requirements Document
I, II, 111*
Design Document
I, II, 111*
Coding Standards or SOPs
I, II, III*
Source Code with In-line Comments
(archived)
I, II, III, IV
User's Manual
I, II, 111
Command Summary or Instructions for Use
(in lieu of a formal user's manual)
IV
Maintenance Manual or Installation Instructions
(if source code is distributed outside EPA)
(if source code is not distributed)
I, II, III, IV
I, if. III*
Data Dictionary
I, II, III
Testing and Validation Procedures and Results
I, II, III*
Backup Source Code and Build Procedures
on Computer-readable Media
I, II, III, IV
* Project Officer and QA staffs option
-------
Appendix C:
NHSRC Quality Assurance
Review (QARF) Forms
NHSRC QUALITY ASSURANCE REVIEW FORM
FOR EXTRAMURAL ACTIONS
Name :
Division:
Project ID Number: Security Classification:
I. General Information
a. Select Action Type:
In-house Research 0 (Skip to Section II) or Extramural Research 0
b. Select all that apply: Vehicle Type:
0 New Vehicle (must complete Section IVa (before award) and IVb (after award))
0 Existing Vehicle (must complete Section IVb (after award))
Existing Vehicle No
0 Contract
0 Sole Source
0 Simplified Acquisition
0 Assistance Agreement
0 I AG"
0 CRADA"
II. QA related Information
a. QA Category
0 1 (established Q A requirements for projects involving enforcement, litigations or human subjects)
02 (established Q A requirements for projects supporting the development of environmental regulations
or standards)
0 3 (established Q A requirements for projects involving applied research or technology evaluations)
If you are processing an I A G or CRADA. the responsibility for QA must be negotiated within the agreement.
The PI in consultation with the QAMs in the various organizations must agree on, and document, which
organization will take the lead for QA, the names of the QAM and PI from each organization, and the QA
requirements that will be adhered to during the agreement. Include this info in the IAG/CRADA package.
0 Work Assignment No. _
0 Delivery/Task Order No.
0 Modification No.
0 Other:
Attachment #1 to the Statement of Work
Revision 2. March 2008
NHSRC
67
-------
0 4 (established QA requirements for projects involving basic research or preliminary data gathering
activities)
b. Project type (Choose all that apply):
0 Applied Research
0 Basic Research
0 Secondary Data
0 Design/Construction/Operation of Environ. Technology
0 Model Development
0 Sampling and Analysis
0 Method Development
0 Software Development and Data Management
Descriptive Title:
III. Scope of Work
a. Does this extramural action involve the collection, generation, use, and/or
reporting of environmental data; the design, construction, and operation of 0 YES 0 NO
environmental technologies; or development of software, models, or methods?
(If ANo@ then skip to Section IV, and sign the form.)
b. Will the SOW or any subsequent work assignments or task orders involve 0 YES 0 NO
any cross-organizational efforts within EPA? If so, which organization will
take the lead for QA?
c. Has a QAPP already been approved for the activities specified in the SOW? 0 YES 0 NO
If yes, please provide the necessary information.
1. Provide the title, date or revision number, and date of QA approval:
2. Does the QAPP require any revision by the contractor*? 0 YES 0 NO 0 N/A
d. Is an applicable QAPP in the process of being prepared, revised, or approved
by EPA personnel for future use by the contractor? 0 YES 0 NO 0 N/A
(QA approval must be obtained before the contractor can start work.)
1. Provide the expected title and approximate date for submission to QA staff for approval:
IV. QA Documentation Options: For solicitations, complete items 1-4; for all actions other than solicitations,
complete items 3-4. All documentation specified under "Other" must be defined in the NHSRC Quality
Management Plan and be consistent with requirements defined in EPA CIO 2105-P-01-04i (formerly
*
The term "contractor " applies loosely here, such that as applicable, this term can also mean Aawardee@,
AcooperatorQ and/or AgranteeQ. Likewise, the term AcontractQ includes Aagreements@ and other vehicles.
41 EPA Quality Manual for Environmental Programs, CIO 2105-P-01-0, 2000, OEI, USEPA, Washington, DC.
http://www.epa.gov/irmpoli8/ciopolicv/2105- xjf
Attachment #1 to the Statement of Work
Revision 2. March 2008
NHSRC
68
-------
known as 5360 Al). For all items checked below, there must be adequate information in the SOW (or its
appendices) for the offeror to develop this documentation. Where applicable, reference a specific section of
the SOW. (R-2 refers to EPA Requirements for Quality Management Plans (OA/R-2) (EPA/240/B-01/002,
03/20/01) and R-5 refers to EPA Requirements for Quality Assurance Pro ject Plans fOA/R-5) (EPA/240/B-
01/003, 03/20/01). Copies of these documents are available at http://www.epci.eov/quciUty/qa docs, html. )
«t. Before Award Documentation
1. Documentation of an organization's Quality System: Either OQMP developed in
accordance with R2 or 0 Other:
Combined documentation of an organization's Quality System and application of QA and
QC to the single project covered by contract: Either developed in accordance with 0 R2
and R5 or 0 Other:
2. Programmatic QA Project Plan developed in accordance with 0 R5 or
0 Other:
Application of QA and QC activities to the single project covered by contract: Either
OQA Project Plan developed in accordance with R-5 or 0 Other:
Not applicable.
b. After Award Documentation
3. Documentation of an organization's Quality System: Either 0 QMP developed in
accordance with R2 or 0 Other:_
Combined documentation of an organization's Quality System and application of QA and
QC to the single project covered by the contract: Developed in accordance with either
0 R2 and R5 or 0 Other:
Not applicable.
4. Documentation of the application of QA and QC activities to applicable project(s): Either
developed in accordance with 0 R-5; 0 Supplement to the follow ing Programmatic QA
Project Plan ;
or 0 Other:
Programmatic QA Project Plan with supplements for each specific project, developed in
accordance with:
Existing documentation of the application of Q A and QC activities will be used: Either 0
Documentation developed pre-award; 0 Documentation will be identified in individual
Statements of Work or 0 Documentation identified in Section of the Statement of
Work.
V. Signature Block: The signatures below verify that the Statement of Work (SOW) has been reviewed to
ascertain the necessary QA and QC activities required to comply with EP A Order 5360.1 A2, that the COR
understands these requirements, and that the COR will ensure that the quality requirements indicated on the
previous pages of this form are incorporated into all associated SOWs. (Sign/date below, obtain a concurrence
Attachment #1 to the Statement of Work
Revision 2. March 2008
NHSRC
69
-------
signature from the QA Staff, and submit the form along with the other extramural action documentation.)
NHSRC Technical Lead Person
(PI)
Date
NHSRC QA Staff Member
Date
70
Attachment #1 to the Statement of Work
Revision 2. March 2008
NHSRC
-------
Appendix D:
NHSRC QA Requirements/Definitions List
NHSRC QA Requirements/Definitions List
EPA's Quality System Web site: http://www.epa.gov/quality/qs-docs/r5-final.pdf
Category Level Designations (determines the level of QA required):
Category I: Research that directly and/or immediately supports specific Agency rule-making, enforcement,
regulatory, or policy decisions. This category may also include research of significant national interest, such
as tasks that might be monitored by the Administrator. The QAPP shall address all elements listed in
EPA Requirements for QA Project Plans, http://www.epa.gov/quality/qs-docs/r5-final.pdf
Category II: Research of high programmatic relevance that, in conjunction with other ongoing or planned
studies, is expected to provide complementary support of Agency rule-making, regulatory, or policy
decisions. The QAPP shall address all elements listed in EPA Requirements for QA Project Plans,
http ://www. epa.gov/qual ity/qs-d ocs/r5-fi nal.pdf
Category III: Projects involving applied research or technology evaluation; method validation studies.
Category IV: Basic, exploratory, conceptual research to study basic phenomena or issues that typically result
in a peer-reviewed journal article.
Project Types:
Applied Research Project - pertains to a study performed to generate data to demonstrate the performance of accepted
processes or technologies under defined conditions. These studies are often pilot- or field-scale. The QAPP shall address
all requirements listed in AQAPP Requirements for Applied Research Projects@ from Appendix B of the NHSRC QMP.
Basic Research Project - pertains to a study performed to generate data used to evaluate unproven theories, processes,
or technologies. These studies are often bench-scale. The QAPP shall address all requirements listed in AQAPP
Requirements for Basic Research Projects@ from Appendix B of the NHSRC QMP.
Design, Construction, and/or Operation of Environmental Technology Project - pertains to environmental technology
designed, constructed and/or operated by and/or for EPA. The QAPP shall address requirements in Part C of A
Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology,@
ANSI/ASQC E4-1994, American Society for Quality Control, Milwaukee, Wl, January 1995. (Please contact a member of
the QA Staff for further information.)
Method Development Project - pertains to situations where there is no existing standard method, or a standard method
needs to be significantly modified for a specific application. The QAPP shall address all requirements listed in AQAPP
Requirements for Method Development Projects@ from Appendix B of the NHSRC QMP.
Model Development Project - includes all types of mathematical models including static, dynamic, deterministic, stochastic,
mechanistic, empirical, etc. The QAPP shall address all requirements listed in AQAPP Requirements for Modeling
Projects@ (Requirements still in development. Please contact a member of the QA Staff for further information.)
Sampling and Analysis Project - pertains to the collection and analysis of samples with no objectives other than to provide
characterization or monitoring information. The QAPP shall address all requirements listed in AQAPP Requirements for
Sampling and Analysis Projects@ from Appendix B of the NHSRC QMP.
Secondary Data Project - pertains to environmental data collected from other sources, by or for EPA, that are used
for purposes other than those originally intended. Sources may include: literature, industry surveys, compilations
from computerized databases and information systems, and computerized or mathematical models of environmental
processes. The QAPP shall address all requirements listed in AQAPP Requirements for Secondary Data Projects@ from
Appendix B of the NHSRC QMP.
Software Development and Data Management Project - pertains to software development, software/hardware systems
development, database design and maintenance, data validation and verification systems. The QAPP shall address all
requirements listed in AQAPP Requirements for Software Development Projects;/ from Appendix B of the NHSRC QMP.
71
-------
Definitions:
Environmental Data - For EPA, environmental data include information collected directly from measurements, produced
from software and models, and compiled from other sources such as data bases or the literature.
Incremental Funding - Incremental funding is partial funding, no new work.
NHSRC's Quality System Specifications for Extramural Actions - These requirements ty pically pertain to
single project efforts. The five specifications are:
(1) a description of the organization's Quality Sy stem (QS) and information regarding how this QS is
documented, communicated and implemented;
(2) an organizational chart showing the position of the QA function:
(3) delineation of the authority and responsibilities of the QA function;
(4) the background and experience of the QA personnel who will be assigned to the project; and
(5) the organization's general approach for accomplishing the QA specifications in the SOW.
Quality Assurance (QA) - Quality assurance is a system of management activities to ensure that a process, item, or
service is of the type and quality needed by the customer. It deals with setting policy and running an administrative system
of management controls that cover planning, implementation, and review of data collection activities and the use of data in
decision making. Quality assurance is just one part of a quality system.
Quality Assurance Project Plan (QAPP) - A QAPP is a document that describes the necessary quality assurance, quality
control, and other technical activities that must be implemented to ensure that the results of the work performed will satisfy
the stated performance criteria. A QAPP documents project-specific information.
Quality Control (QC) - Quality control is a technical function that includes all the scientific precautions, such as
calibrations and duplications, that are needed to acquire data of known and adequate quality.
Quality Management Plan (QMP) - A QMP is a document that describes an organization's/program's quality system in
terms of the organizational structure, policy and procedures, functional responsibilities of management and staff, lines of
authority, and required interfaces for those planning, implementing, documenting, and assessing all activities conducted.
A QMP documents the overall organization/program, and is primarily applicable to multi-year, multi-project efforts. An
organization's/program's QMP shall address all elements listed in the ARequirements for Quality Management Plans@ in
Appendix B of the NHSRC QMP.
Quality System - A quality system is the means by which an organization manages its quality aspects in a systematic,
organized manner and provides a framework for planning, implementing, and assessing work performed by an
organization and for carrying out required quality assurance and quality control activities.
Substantive Change - Substantive change is any change in an activity that may alter the quality of data being used,
generated, or gathered.
Supplemental Funding - Supplemental funding is additional funding of new work.
Technical Lead Person (PI) - This person is technically responsible for the project. For extramural contract work, the PI is
typically the contracting officer's representative (COR). For intramural work, the PI is typically the Principal Investigator.
NHSRC PRODUCT QA/QC V ERIFICATION REPORT
72
-------
Appendix E:
NHSRC Product QA/QC
Verification Report Form
Attach to all products when submitted for Quality Assurance (QA) review
Date Submitted: EPA PI:
Project No.:
Product Title:
Product Type: o Journal Article o Computer Product/Software/Model/Database
o Symposium Paper/Conference Paper o Scientific Data
o Extended Abstract o Guidance
o Report (Published, Unpublished, Internal)
o Other
Date QA Review Needed:
Quality Assurance Project Plan (QAPP) STATUS:
Was a QAPP prepared and reviewed by Quality Assurance Manger (DQA OR QAC), prior to collecting
data/information included in product? YES o or NO o
If yes, QA ID No. /Review Date:
If no, explain:
DATA QUALITY SUMMARY:
Please mark the following as appropriate:
o All project QA requirements were met (i.e., all QA/QC checks specified in the QAPP were
performed and acceptance criteria were met).
o Some project QA requirements were not met. Product quality IS NOT adversely affected. (Consult
DQA OR QAC if needed.)
o Some project QA requirements were not met. Product quality IS adversely affected. (Consult DQA
OR QAC if needed.)
o Not Applicable. Explain:
Discussion of Data Limitations:
Signed By: Date:
(EPA PI
73
-------
This page left intentionally blank.
74
-------
Appendix F:
Requirements for Quality
Management Plans (QMPs)
REQUIREMENTS FOR QUALITY 1.2
MANAGEMENT PLANS (QMPs)
This Quality Management Plan (QMP)
requirements list is applicable to multi-year, multi-
project efforts and is based on EPA guidance (http://
\Y\YVY.epa.gov/quality/qs-docs/r2-final.pdf). A
QMP defines an organization's quality system and
documents policies and procedures that will be used
to meet customer's quality needs.
Note (1): If a requirement is not applicable, state
"not applicable" and provide an explanation
regarding why. Note (2): To eliminate redundancy,
reference to other sections of the QMP is
permissible. Note (3): Ensure that the QMP clearly
designates the personnel responsible for performing 1.3
each procedure described in the QMP.
SECTION 0.0, QMPAPPROYAL
An approval page for the signatures of the senior
accountable manager, senior line management (as 1.4
appropriate) and the QA staff of the organization(s)
that are part of this program level quality system
needs to be provided. Signatures must be obtained
prior to submitting the QMP for Agency review.
SECTION 1.0, QUALITY SYSTEM
MANAGEMENT AND ORGANIZATION 1.5
1.1 Program Quality Policy. State the "quality
policy." established and implemented by
program management, which ensures that
this environmental program produces the
type and quality of results needed and
expected.
Program Organizational Structure and
Communication. Describe functional
responsibilities (including QA personnel),
levels of accountability, authority, and
communication for each organization
that is part of this program level quality
system. Demonstrate that the QA staff*s)
is/are independent of groups generating,
compiling, and evaluating environmental
data. Include a discussion of how disputes
regarding quality system requirements. QA/
QC procedures, assessments, or corrective
actions are resolved. Note: It may be
helpful to include organizational charts in
addition to a narrative discussion.
Technical Activities. Briefly describe all
technical activities (e.g., basic research,
technology evaluation, modeling,
technology construction) that are supported
by the quality system.
Communicating the Program's
Quality System. Describe how program
management ensures that the quality
management plan is communicated to and
understood by those who are required to
follow it (e.g., training, meetings).
Resources. Describe program management
procedures for allocating resources (human
and financial) to implement the quality
system (including, but not limited to.
personnel training, quality system audits).
75
-------
1.6 Authority to Stop Work for Safety and
Quality Considerations. Define who has
the authority to stop unsafe work, or work
of inadequate quality.
1.7 Management Assessment of Quality
System Adequacy. Describe how program
management assesses and documents
quality system adequacy. Include frequency
of assessment, description of assessment
(purpose, types of activities reviewed), and
possible response actions.
SECTION 2.0, QUALITY SYSTEM AND
DESCRIPTION
2.1 Quality System Elements. Generally
describe the principal components (or
"tools") comprising the quality system
and how they are used to implement the
quality system. This includes (but is not
limited to): QMPs; QA project plans,
standard operating procedures; and audits/
assessments. Include how and when each
component is applied to individual projects
and tasks.
2.2 Quality Management Plan Reviews
and Revisions. Describe procedures for
updating quality system documentation.
SECTION 3.0, PERSONNEL
QUALIFICATION AND TRAINING
3.1 Personnel Training and Qualification
Procedures. Describe how program
management ensures that all personnel
performing work (including subcontractors,
if applicable) are trained and qualified
to perform work prior to initiating work.
In addition to formal education, include
specific on-the-job training for technical
and management personnel (e.g.. lab. field,
health and safety, management. QA).
3.2 Formal Qualifications and Certifications
for Specialized Activ ities. Identify when
formal qualification or certification is
required.
3.3 Training Documentation. Describe how
program management ensures that required
training is performed and documented.
3.4 Evidence of Personnel Job Proficiency.
Describe how objective evidence of
personnel job proficiency is documented
and maintained.
3.5 Re-Training. Describe how the need for
re-training is evaluated.
SECTION 4.0, PROCUREMENT OF ITEMS
AND SERVICES RELATED TO TECHNICAL
ACTIVITIES
4.1 Procurement Planning and Control.
Describe procedures for planning and
controlling the procurement of items
and services (e.g.. subcontractor who
provides analytical support, subcontractor
who provides drilling support, calibrated
sampling equipment).
4.2 Procurement Technical and Quality
Requirements. Describe procedures for
ensuring that procurement documents
(e.g.. formal contract with subcontractor,
purchase order for equipment) clearly
describe the item or service needed and
the associated technical and quality
requirements (including a quality system
consistent with EPA requirements when
applicable). Include a discussion of when
procurement documents will require
the supplier to furnish a demonstrated
capability to furnish items and services that
meet all requirements and specifications.
4.3 Procurement Document Specification
of Verifying Supplier's Conformance.
Describe procedures for ensuring that
procurement documents specify how the
supplier's conformance to customer's
requirements will be verified.
4.4 Procurement Document Review. Describe
procedures for the internal review of
procurement documents to ensure accuracy
and completeness.
4.5 Rev iew of Changed Procurement
-------
Documents. Describe procedures for
ensuring that changed procurement
documents receive the same level of
internal review and approval as the original
documents.
4.6 Review of Procured Items and Services.
Describe how procured items and services
arc reviewed to ensure compliance with
requirements and specifications.
SECTION 5.0, DOCUMENTS AND REC ORDS
5.1 Records Management Procedures.
Describe records management procedures
from preparation to disposal, including
maintenance (protection from damage
and deterioration), storage (including
accessibility), and retention (including
disposition in accordance with statutory or
contractual requirements). Describe how
these records management procedures are
controlled and maintained. Include printed
and electronic records. Identify which
documents/electronic records arc included
under these record management procedures
(e.g., QAPP, data package, electronic data,
laboratory notebooks, chain-of-custody
forms).
5.2 Document Control. Identify which
documents require control (e.g., technical
manuals, operating procedures. QAPPs,
SOPs. final reports). Describe procedures
for document review and approval (internal
to the organization and external, e.g.,
client), and revision (before and after
submission to client). Include procedures
for distribution, replacement of previous
document versions, and for ensuring that
obsolete documents arc no longer used.
SECTION 6.0, COMPUTER HARDWARE
AND SOFTWARE
This section applies to computer hardware and
software operations that directly impact the quality
of the results of environmental programs (both
developed and purchased) including: design, design
analysis, data handling, data analysis, modeling of
environmental processes and conditions, operations
or process control, and data bases.
6.1 Conformance to User and EPA
Requirements. Describe procedures
for ensuring computer software and
computer hardware/software configurations
meet user's requirements and conform
to applicable EPA requirements (e.g.,
Y2K compliance, security [protection
from physical loss of data], and privacy
| protection from unauthorized use of data]).
6.2 Configuration Testing. Describe
procedures for testing computer hardware/
software configurations prior to use
to ensure technical requirements and
quality expectations are met; include
how the results of configuration tests are
documented and maintained.
6.3 Configuration Change Assessment.
Describe procedures for assessing changes
to hardware/software configurations;
include how changes are evaluated based
on the impact on technical and quality
objectives of the program.
6.4 Re-Testing and Re-Documentation.
Describe procedures for re-testing and
re-documentation when components are
changed (creating a new configuration)
or when program requirements change
(bringing capability of the configuration
into question).
In addition to environmental data collection
activities. Sections 7-10 must also be addressed
for the design, construction, and operation of
environmental technologies; use of secondary data
(i.e., use of environmental data); and development/
modification of mathematical models (i.e., use
and/or generation of environmental data), when
applicable.
SEC TION 7.0, PROJEC T PLANNING
7.1 Planning and Documenting the
Generation, Acquisition, and Use of
Environmental Data. Describe procedures
for planning and documenting all work
involving the generation, acquisition,
and use of environmental data (and the
design, construction, and operation of
environmental technologies as applicable).
77
-------
Identify types of planning documents
generated (e.g., work plans, QAPPs) and
summarize their purpose.
7.2 Identifying and Documenting Type and
Quality of Environmental Data Needed.
Describe how the type and quality of
environmental data needed (including
secondary data, mathematical models,
and the quality of technology design) are
identified. Describe how this information is
documented (e.g., DQO Process, QAPP).
7.3 Including Key Users, Customers, and
Technical Staff in Planning. For each
applicable technical activity, describe
procedures for involving the key users and
customers of the data (and technology or
model), in addition to the technical staff,
during project-specific planning.
7.4 Reviewing and Approv ing Planning
Documents. Describe procedures used
for review and approval (internal to
organization and external, e.g., client) of
planning documents prior to initiation of
work. Reference to Section 5.0 may be
applicable.
SECTION 8.0, IMPLEMENTATION OF
WORK PROCESSES
8.1 Implementation of Work According
to Planning Documents. Describe
procedures (e.g., meetings, documentation)
for ensuring that all work is performed
according to approved planning and
technical documents.
8.2 Standard Operating Procedures
Documentation. Describe the process
for documenting standard operating
procedures. Describe how standard
operating procedures should be written so
that they are easily understood by the user
and contain sufficient detail and clarity.
SECTION 9.0, PROJECT ASSESSMENT AND
RESPONSE
For the purposes of this section, audits and
assessments are synonymous.
9.1 Planning Project Assessments. Describe
how, in the planning stage, management
determines the appropriate type of
assessment activity (e.g., technical systems
audit, performance evaluation audit) for a
particular project.
9.2 Assessment Planning and Procedures.
Describe the process used to ensure
assessments are performed according to
approved written procedures.
9.3 Assessment Personnel Qualifications.
Describe procedures for ensuring that
assessments are performed by qualified
personnel. Assessors shall be capable
of assessing technical requirements and
other procedures specified in the planning
document.
9.4 Assessor Responsibility and Authority
to Stop Work. Describe the assessor's
responsibility and authority to stop work.
Describe conditions under which a stop
work order may be needed.
9.5 Assessment Documentation, Reporting,
and Review. Describe assessment
documentation, reporting, and review
procedures. Include procedures for
documenting and reporting assessment
results to management. Include a discussion
of how the assessor reports the impact of
a negative assessment result on planned
operations. For each procedure, include a
time line for completion.
9.6 Assessment Responses and Follow-
up Action. Describe procedures for
documenting assessment responses and
how corrective action occurs. Include how
follow-up is performed to ensure action
78
-------
was taken. For each procedure, include a
time line for completion.
SECTION 10.0, ASSESSMENT AND
VERIFICATION OF DATA USABILITY
10.1 Assessing, Verifying, and Qualifying
Data. Describe procedures for assessing,
verifying/validating, and qualifying
data obtained from environmental data
operations according to their planned
intended use. including secondary data
and mathematical modeling. (Describe
procedures for assessing and verifying the
performance of environmental technology
for its intended use.)
10.2 Expressing and Documenting
Limitations on Data. Describe procedures
for expressing and documenting (in print,
or electronically) any limitations on this
intended data use (and/or technology
performance).
10.3 Providing Independent Review of Data-
Containing Project Reports. Describe
procedures for the independent review of
project reports containing data, or reports
containing the results of environmental
data operations (and/or technology
performance), to confirm that the data or
results are presented correctly. Reference to
Section 5.0 may be applicable.
10.4 Management Approval of Reports.
Describe procedures for obtaining
management approval of these reports
prior to release, publication, or distribution.
Reference to Section 5.0 may be
applicable.
SECTION 11.0, QUALITY SYSTEM
IMPROVEMENT
11.1 Quality Improvement Process. Describe
the quality improvement process used to
continuously develop and improve the
quality system. Include procedures such as
communication with and among customers
and suppliers; staff identification of
problems and solutions.
11.2 Preventing, Detecting and Correcting
Quality System Problems. Describe
procedures used to prevent, detect and
correct quality system problems.
11.3 Response Actions. Describe procedures for
planning, documenting, and implementing
response actions to quality system
problems.
79
-------
This page left intentionally blank.
80
-------
Appendix G:
Routing Slip for Extramural Packages
ROUTING SLIP FOR NHSRC EXTRAMURAL PACKAGES
Project Number:
Project Title:
Type (choose one)
O IAG (new) Interagency Agreement with:
O Cooperative Agreement (new) with:
O Contract (new)
O Work Assignment Name of Contractor:
O R&D Funded Small Purchases
O Incremental Funding
O Contract O Cooperative Agreement O Interagency Agreement
O Task order under GSA schedule
O Other (specify):
ROUTING / APPROVALS:
Initials
PI / Originator:
Reviewer: Please
mark review status
Date
NHSRC Division Director:
~ Reviewed
NHSRC Extramural Management
Specialist:
~ Reviewed
NHSRC Security Officer:
~ Reviewed
QA Officer:
~ Reviewed
Center Deputy Director:
~ Reviewed
Funds Control Officer:
Send original funded package to:
n Contract specialist
n IAG specialist
n Contract level COR
n Return to originator
n Other:
(name, if known)
(name, if known)
(name, if known)
COMMENTS:
81
------- |