United States
Environmental Protection
Agency
Research Triangle Park, NC 27711
EPA454/R-01-011
Office of Air Quality
Planning and Standard
Office of Air and Radiation
r/EFft
Quality Management Plan
For the PM Supersites
Program
-------
Foreword
The following document is a Quality Management Plan (QMP) for the environmental data operations of
the PM Supersites Research Monitoring Program. The Office of Air Quality Planning and Standards
(OAQPS) staff developed this QMP to outline the roles of organizations involved in the Supersites
Particle Monitoring Program.
This QMP was generated using the EPA Quality Assurance (QA) regulations and guidance as described
in EPA QA/R-2, EPA Requirements for Quality Management Plans and the accompanying document,
EPA QA/G-2, Guidance for Developing, Reviewing and Implementing Quality Management Plans. All
pertinent elements of the QMP regulations and guidance are addressed in this document.
-------
Acknowledgments
This QMP is the product of the EPA Office of Air Quality Planning and Standards. The following
individuals are acknowledged for their contributions.
Principle Authors
Dennis Mikel, OAQPS-EMAD-MQAG, Research Triangle Park, North Carolina
Michael Papp, OAQPS-EMAD-MQAG, Research Triangle Park, North Carolina
Dr. Les Hook, Oak Ridge National Laboratory, Oak Ridge, Tennessee
Dr. Sigurd Christensen, Oak Ridge National Laboratory, Oak Ridge, Tennessee
Reviewers
Office of Air Quality Planning and Standards
Jo Ann Rice, Dr. Richard Scheffe, Dr. Marc Pitchford, Joe Elkins, Michael Jones
Office of Research and Development
Dr. Paul Solomon, Las Vegas, Nevada
Parsons Corporation
David Bush
National Oceanic and Atmospheric Administration
Jeffrey West
College of Engineering, Center for Environmental Research and Technology
Dennis Fitz, David Gemmill
Comments and questions can be directed to:
Dennis Mikel, OAQPS, RTP, NC mikel.dennisk@epa.gov
111
-------
Acronyms and Abbreviations
AIRS
APTI
ASTM
CAA
CFR
DQA
DQOs
EDO
EMAD
EPA
FIPS
IMPROVE
LAN
LIMS
MQAG
MQOs
MSR
NAAQS
NAMS
NAREL
NARSTO
OAQPS
PC
PE
QA/QC
QA
QAC
QAFR
QAO
QAPP
QS
QSSC
QMP
R&IE
SLAMS
SOP
TSA
Aerometric Information Retrieval System
Air Pollution Training Institute
American Society for Testing and Materials
Clean Air Act
Code of Federal Regulations
data quality assessment
data quality objectives
Environmental Data Operation
Emissions, Monitoring, and Analysis Division
Environmental Protection Agency
Federal Information Processing Standards
Interagency Monitoring of Protected Visual Environments
local area network
Laboratory Information Management System
Monitoring and Quality Assurance Group
measurement quality objectives
management system review
National Ambient Air Quality Standards
national air monitoring station
National Air and Radiation Environmental Laboratory
formerly North American Research and Stratospheric Transport of Ozone
Office of Air Quality Planning and Standards
personal computer
performance evaluation
quality assurance/quality control
quality assurance
quality assurance coordinator
quality assurance final report
quality assurance officer
quality assurance project plan
quality staff
NARSTO Quality Systems Science Center
quality management plan
Radiation and Indoor Air
state and local monitoring stations
standard operating procedure
technical system audit
IV
-------
Tables
Number Description Page No.
2 i QAPP Elements Applicable to Various Programs ^
2 2 Types of Assessments '
3' \ Core Ambient Air Training Courses ^
Assessment Schedule ^
-------
Figures
Number Description Section Page
1.1 Overview of the National Fine Particle Program 1.1 1
1.2 Phase I and II Supersites 1.2 2
1.3 Supersites Project Management Overview 1.2 3
1.4 Flow diagram illustrating major program stages and review 1.2 4
1.5 Supersites implementation general structure 1.3 7
6.1 Data Flow Diagram 6.1 3
VI
-------
Project: Speciation QMP
Revision No :1
Date:9/13/01
Page 1 of 1
Quality Management Plan Identification and Approval
The attached QMP for the Supersi.be Research Monitoring Program is hereby itscomm-sncted for
approval and commits tlie resources -and personnel Co follow the elements described within.
Office of Air Quality Plauning and Standards
1) Signer*; ^LgOr
-------
Table of Contents
Section
Foreword
Acknowledgments
Acronyms and Abbreviations
Tables
Figures
Title and Approval Page
Table of Contents
1. Management and Organization
1.1 Supersites Program Background
1.2 Roles and Responsibilities
1.3 Principle EPA Coordinators
1.4 Research Groups
2. Quality System and Description
2.1 Supersites QA Applications
2.2 Quality Management Plan
2.3 Data Quality Objectives
2.4 Quality Assurance Project Plans
2.5 Standard Operating Procedures
2.6 Assessments
3. Personnel Qualifications and Training
3.1 Personal Qualification
3.2 Training
3.3 Certification
4. Procurement of Items and Services
4.1 Sources of Funds
4.2 Procurement of Items
4.3 Procurement of Services
4.4 Assistance Agreements
4.5 EPA Exclusive Versus Discretionary Functions
5. Records and Documentation
5.1 Documents Hierarchy and Process
5.2 Document Responsibilities
5.3 Deposition and Storage of Documents and Records
5.4 Deposition of Reports
6. Computer Software and Hardware
6.1 Computer Systems Description
7. Planning and Implementation of Work Process
7.1 Project Goals and Objectives
7.2 Initial Planning and Conceptualization
7.3 Key Planning Personnel
7.4 Other Planning Activities
Page Revision Date
11
iii
iv
v
vi
1/1
1/2
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
1/9
3/9
6/9
8/9
1/7
1/7
2/7
2/7
2/7
6/7
111
1/3
1/3
3/3
1/10
2/10
2/10
4/10
4/10
1/4
2/4
3/4
4/4
1/3
1/6
3/6
5/6
5/6
2
3
o
J
o
J
3
1
1
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
1
1
1
1
1
1
1
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
09/13/01
08/08/01
08/08/01
08/08/01
08/08/01
-------
Section
8. Implementation of Work
8.1 Implementation Roles
9. Data Quality Assessments
9.1 Program Assessment Techniques
9.2 Reports to Management
9.3 Planning, Training and Authority
10. Data Quality Improvement
10.1 Quality Improvement Process
10.2 Quality Improvement Assurance
Glossary
Contact List
Appendix A - Procurement Policy Notice
Page Revision
1
1/2 1
1
1/4 1
2/4 1
3/4 1
1
1/3 1
3/3 1
1/4 1
1/1 1
1/11 1
Date
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
08/08/01
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 1 of 9
1.0 Management and Organization
The purpose of this section is to document the overall policy, scope, applicability, and
management responsibilities of the "Supersites" Program's quality system. The section will
provide a brief description of the Supersites Program and the organization and management of
the programs as it relates to the quality assurance aspects.
1.1 Supersites Program Background
The Clean Air Act (CAA) requires EPA to revise or update the air quality standards based on
review of the latest scientific information on known and potential human health effects
associated with Particulate Matter (PM) levels found in the ambient air. In fulfilling this
obligation, the EPA reviewed the air quality criteria, National Ambient Air Quality Standards
(NAAQS) for PM and epidemiological evidence that shows an association between ambient
concentrations of PM and a range of serious health effects. Based on the results of its review, the
EPA revised and promulgated
new primary standards for the
fine fraction of PM (i.e., particles
with aerodynamic diameters less
than or equal to [a nominal] 2.5
m, referred to as PM2.s) and the
-15 Daily Sites' x
'x (
-50 Trends ^s
(Mass & Components)
-50 Sites
(Coordinated With SS)
-200 Sites (Sips)
8 Enhanced Airsheds
Koutme
Speciation
-150
Mass Sampling
850 Required
-1050 FRM
(Compare to
NAAQS)
-200
Continuous
Figure 1.1 Overview of the National Fine Particle Network
regulatory requirements for
monitoring the chemical
composition of these particles. In
response to this promulgation,
EPA has instituted a PM2.5
network.
Figure 1.1 illustrates the overall
national fine particle network.
The network is divided into three
tiers. The base is composed of the mass sampling network, for which the mass fine paniculate
data is collected for comparison to the national ambient air quality standards (NAAQS). The
second tier, the Speciation Trends Network (STN) is intended to monitor and gather data on the
chemical makeup of fine particles. These STN samplers will be placed at various national air
monitoring stations (NAMS) and State and local air monitoring stations
(SLAMS) across the Nation. The top tier, the "Supersites" Program is a grant based research
program "designed to conduct special, detailed chemical and physical characterization studies in
geographic areas with a range of characteristic PM2.5 source-receptor and health risk situations1"
This series of analytes is very similar to those measured within the Interagency Monitoring of
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 2 of 9
Protected Visual Environments (IMPROVE) program. In addition, several STN monitors will be
placed at IMPROVE locations (or visa versa) in order to ascertain whether there are statistical
and chemical links between these two national networks.
The goals of the Supersites Program took shape at a public PM Measurements Research
Workshop held in Chapel Hill, N.C. on July 22 and 23, 1998. To commence the PM Supersites
Program, EPA selected two initial sites: Atlanta GA and Fresno CA. These sites, henceforth
referred to as Phase I Supersites, were non-competitively selected by virtue of ongoing and
planned research activities, which align with those of the PM Supersites Program, and
characteristics of the two airsheds. Six additional sites, and the Fresno Phase I site, henceforth
referred to as Phase II Supersites, were competitively selected cooperative agreements awarded
in January 2000. Figure 1.2 identifies both phase U Supersites and I. the Supersites will address
objectives in three major areas:
1) SIPs.... Support development of State Implementation Plans (SIP's) through improved
understanding of source-receptor relationships leading to improved design, implementation,
and tracking of control strategy effectiveness in the overall PM program;
2) health effects and exposure.....development of monitoring data and samples to support
health and exposure studies to reduce uncertainty in National Ambient Air Quality
Standards setting and to enable improved health risk assessments; and
ew York
Baltimore
I Phase I
I Phase II
O Both Phases
Figure 1.2 Phase I and H Supersites
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 3 of 9
3)methods testing.... comparison and evaluation of emerging sampling methods with
routine techniques to enable a smooth transition to advanced methods.
Additional background information on the Supersites program can be found on the Ambient
Monitoring and Technology Website (AMTIC) , (http://www.epa.gov/ttn/amtic/Supersites.html).
1.2 Roles and Responsibilities
Program Management, Organization and Review
Figure 1.3 provides an overview of program management that will establish the communications
and accountability essential for program planning, coordination and implementation.
Project Organization
{Sponsors (Supersites and releva
EPA(OAQPS, ORD); NOAA; DO
State/Locals; ndustry
1 i
Ext. Research Coordinatio
CENR; NARSTO/HEI;
NAS/CASAC
( Planning J (
rtstudies)1 Executive Leads
EPAOARandORDAAdelegates
X
" EPA Steering
-, EPA staff (OAR,
* r^
Committee
NERL, NHEERL)
Princioal EPA Technical Coordinators
OAR designate
ORD designate (s)
Participating Research Groups and Contractors
Data ^N /Field ^\( °nl^sis
Management ^/\. Measurements/ X.^ ^S
Science Oversight
MAS Zf'
* ' FP Subcommittee \
^^^ (CASAC) y
( QA 7
Figure 1.3 Supersites Project Management Overview
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 4 of 9
Executive Leads
OAQPS and Office of Research and Development (ORD) will share in the overall administration
and management of the program. The Assistant Administrators of both Offices and their
designates will be accountable for all program objectives, including the integration of science
research sponsored and conducted by EPA with the Supersites program.
Supersites Coordination Committee
The Coordination Committee will extend beyond EPA to sponsors of related programs in other
Federal agencies, industry and State and local agencies. The role of this Committee is to provide
a forum for coordination and leveraging of resources by establishing and maintaining a dialogue
among the members collectively who share similar needs and interests. In addition, the
Coordination Committee would provide a valuable resource in reviewing Supersites plans and
assessing progress.
Program Stagt-s and Rtvitw Cycles
Figure 1.4 Flow Diagram Illustrating Major Program Stages and Review
Science Oversight
The Supersites represent an important component to foster greater integration across several
science research programs. The National Academy of Sciences Committee on Research
Priorities for Airborne Particulate Matter clearly has expressed a desire to see comprehensive
science planning. The Technical Subcommittee on Fine Particle Monitoring of the Clean Air
Scientific Advisory Committee (CASAC) (hereafter referred to as the Subcommittee} is
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 5 of 9
reviewing the Supersites Program and will provide advice and consultation. Program execution
involves a sequence of activities starting with conceptualization, design and planning, and
measurement deployment, with necessary reviews and assessments that feed back into program
design. The proposed role of the Subcommittee within this sequence of events is shown
schematically in Figure 1.4. Each of the major stages is also outlined briefly below.
Following Subcommittee review in 1998, EPA established formal internal and external planning
and design teams. Internally, EPA established a planning team composed of atmospheric
science, regulatory and health effects and exposure specialists. In parallel, invitations were
mailed to other Federal and State/local agencies and private industries active in relevant research
to participate in a broader External Coordination Workgroup. EPA staff will be responsible for
developing program plans and working with the external committee at a partnership level by
providing early drafts and conducting meetings on an as needed basis. The design approach was
based on developing a measurements strategy responsive to key questions (science and
regulatory) and scientific hypotheses, taking advantage of the PM Measurements Workshop
Report. EPA will be responsible for establishing and managing all administrative tasks related to
program funding.
External Research Coordination
The active work with the External Coordination Committee is one of several steps taken to
optimize measurement resources across different organizations. The Subcommittee will be
requested to review more detailed plans as part of the decision approval process.
Accordingly, the Supersites program will be responsive to advice generated by other venues
explicitly dealing with larger science integration issues.
EPA Steering Committee
Figure 1.5 provides a more detailed view of the program described in Figure 1.3 starting with the
EPA Steering Committee. The EPA Steering committee is made up of program managers and
science leads in OAQPS, NHEERL and NERL. This committee takes the advice and
recommendations from the various stakeholder groups and provides internal EPA technical
direction for the Supersites program technical coordinators. The Steering Committee builds
consensus within EPA on technical direction and helps establish/prioritize resources to
implement the program.
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 6 of 9
1.3 Principal EPA Technical Coordinators
Internal EPA project management and technical coordinator teams that include regulatory,
atmospheric sciences, health effects and exposure specialists will deal with resource
management, communications, and technical issues. The principal coordinators are responsible
for ensuring the implementation of the program. The specific duties of the coordinators as they
relate to the development and implementation of the quality system are described below.
1.3.1 EPA Program Manager - R. Scheffe
Dr. Richard Scheffe has oversight of the entire Supersites program. Dr. Scheffe interacts at all
levels of the Supersites Program. His duties include:
implementing and overseeing the EPA policy throughout this program ;
communicating the goals of the program with the technical leads of the program;
interacting directly with the Supersites Principal Investigators (Pis);
communicating the progress of the program with management.
1.3.2 EPA Project Officer - M. Jones
Mr. Jones is responsible for monitoring performance and ensuring compliance with agreement
terms and conditions for each Supersites Project. His primary duties include:
* review and approve progress reports and other deliverables;
> maintain all programmatic, fiscal, technical deliverable, and communication records;
* review and approve/ recommend approval (as appropriate) requests for changes to budget,
schedule, work plan, and key personnel;
* conduct site visit(s) for each project (programmatic / fiscal / technical).
1.3.3 EPA Technical Leads - P. Solomon and M. Pitchford
Drs. Solomon and Pitchford are the technical coordinators for the Supersites program. As such,
they work closely with the Pis of the Supersites program. They are responsible for the technical
aspects of this program. As such, their responsibilities are:
> coordinate with the PI the types of instruments at each Supersites;
> coordinate meetings between Pis and EPA personnel;
> provide technical guidance to the Pis and the other technical leads.
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 7 of 9
1.3.4 Quality Assurance Coordinator - D. Mikel
Mr. Mikel will oversee the quality assurance aspects of the Supersites Program. His primary
responsibility is to ensure that a quality system is in place for each Supersites. Additional
responsibilities include:
> implementing and overseeing the OAQPS QA policy throughout this program ;
> assisting in solving QA-related problems at any level of the program;
> ensuring that an approved QAPP is in place for all environmental data operations associated
with the program.
> work with the Project QA manager to ensure that technical systems audits, audits of data
quality, and data quality; assessments occur within the appropriate schedule and conducting
or participating in these audits;
> coordinate the QA Supersites Tele-conference group.
The QA Coordinator (QAC) has the authority to carry out these responsibilities and to bring to the
attention of the project officer, program manager or technical leaders any issues related to these
responsibilities.
Principal EPATechnical
Coordinators
EPA Steering
Committee
J. Bachmann (OAQPS)
R. Scheffe (OAQPS)
J. Vandenberg (NHEERL)
J. Vickery (NERL)
EPA Project Officer:
M. Jones (OAQPS)
EPA Technical Leads:
P. Solomon (NERL)
M. Pitchford (OAQPS)
Quality Assurance:
Dennis Mikel (OAQPS)
1
1
Data Manaqement:
Les Hook (DOE/ORNL)
Sigurd Christensen
(DOE/ORNL)
Research Groups
Los Angeles CA
C. Sioutas -TL
D. Bush - QA
Fresno C A
J. Watson -TL
D. Fitz - QA
St Louis MO
J. Turner -TL
J. Watson - QA
Houston TX
D. Allen -TL
G. McCaughey - QA
Baltimore M D
J. Ondov - TL
P. Hopke - QA
Pittsburgh PA
S. Pandis - TL
B. Wittig - QA
New York NY
K. Demerjian - TL
V. Mohnen - QA
Figure 1.5 Supersites Implementation General Structure
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 8 of 9
1.3.5 Data Management - L. Hook and S. Christensen
Drs. Christensen and Hook are responsible for working with all of the Supersites data managers
and ensuring that they archive the data in the NARSTO Permanent Data Archive. Due to the
nature of the data collected, EPA's Aerometric Information Retrieval System (AIRS) does not
have neither capability nor the temporal flexibility to house the data from the Supersites. Section
6 will highlight the details of this archive. In addition, their responsibilities include:
> leading the data management working group;
> leading the technical discussions of efforts to develop consistent metadata (e.g., variable
naming, units, methods, and flags);
> ensuring that all data that are entered into the NARSTO archive have been quality assured;
* attending annual Supersites meetings and updating the Supersites community on data
management issues.
1.4 Research Groups
Each Supersites is made up of a number of research scientists performing the environmental data
collection activities as described in the Supersites specific grant proposal. Figure 1.5 identifies the
technical leads, Pis and the QA Manager (QAM) for each Supersites. Both these individuals
provide a focal point for the coordination of the research activities at the Supersites
Research Group Principle Investigators
The Pis responsibilities include:
> ensuring research scientists fulfill their obligations for development of QAPPs for their
research environmental data operations;
* ensuring communication between the research group and the EPA technical QAC;
* assisting the research group QA lead in coordinating QA activities;
> approving and implementing the Supersites QAPP for which he/she is responsible.
Research Group QA Manager
The Research Group QAMs will have primary responsibility to ensure that a quality system is
developed and implemented for the Supersites he/she is responsible. The QAM's responsibilities
include:
-------
Project: Supersites QMP
Element No: 1
Revision No:3.0
Date: 9/13/01
Page 9 of 9
* developing and approving the Supersites QAPP for which he/she is responsible prior to
implementing environmental data operations;
* ensuring research scientists are aware of their obligations for development and
implementation of QAPPs for there research environmental data operations;
> establishing communications with the QAC and reporting progress on QA activities;
* providing internal technical systems audits and assessments of researchers efforts;
> ensuring that all Standard Operating Procedures (SOPs) are reviewed and finalized before the
start of the program.
References
1. Allbritton, D. and D. Greenbaum. 1998. Atmospheric Observations: Helping Build the
Scientific Basis for Decisions Related to Airborne Paniculate Matter. Report of the PM
Measurement Research Workshop, Chapel Hill, North Carolina, 22-23-July. Prepared by
Health Effects Institute and the Aeronomy Laboratory of the National Oceanic and
Atmospheric Administration. Health Effects Institute, Cambridge, MA.
http://www.al.noaa.gov/wwwhd/pubdocs/PMMRW.pdf
-------
Project: Supersites QMP
Element No: 1
Revision No:2.0
Date: 9/13/01
Page 1 of 7
2.0 Quality System Description
A quality system is defined as a structured and documented management system describing the
policies, objectives, principals, organizational authority, responsibilities, accountability, and
implementation of an organization for ensuring quality in its work processes, products (items),
and services. The quality system provides the framework for planning, implementing, and
assessing work performed by the organization and for carrying out required QA and QC. This
section will describe the quality system applications used by the organization to implement
effective quality assurance activities.
2.1 Supersites QA Application
In order to meet its stated mission using environmental data, the Supersites PI must implement a
QA program that assures that the data can be used for its intended purpose. The following
elements will assist in the assurance of data quality and will be described in the following
sections.
QA management plans;
Management systems reviews;
Data quality objectives process;
QA project plans;
Standard operating procedures;
Data quality assessments.
Various reviews to determine the successful application of QA in Supersites will be discussed in
Section 9 and 10.
2.2 Quality Management Plans
The Quality Management Plan (QMP) is part of the mandatory Agency-wide policy requires that
all organizations performing work for EPA develop and operate management processes and
structures for assuring that data or information collected are of the needed and expected quality
for their intended use. The QMP describes the quality system in terms of the organizational
structure, functional responsibilities of management and staff, lines of authority, and required
interfaces for those planning, implementing, and assessing activities involving the Supersites
Environmental Data Operation (EDO). This document represents the QMP for the Supersites
program. The QMP will reside on the Air Monitoring Technology Information Center (AMTIC)
web site for easy access to all Supersites cooperators. A hardcopy will also be filed with the
OAQPS Document Control Manager. Approval for the QMP will include the OAQPS QA
Manager, Mr. Joe Elkins and the Supersites QAC Mr. Dennis Mikel.
-------
Project: Supersites QMP
Element No: 1
Revision No:2.0
Date: 9/13/01
Page 2 of 7
2.3 Data Quality Objectives (DQOs)
Data quality objectives (DQOs) are qualitative and quantitative statements derived from the DQO
process that clarify project technical and quality objectives, define the appropriate type of data,
and specify tolerable levels of potential decision errors that will be used as the basis for
establishing the quality and quantity of data needed to support decisions.
As described in section 1, the Supersites will address objectives in three major areas:
* SIPs.: support development of State Implementation Plans through improved understanding
of source-receptor relationships leading to improved design, implementation, and tracking of
control strategy effectiveness in the overall PM program;
* health effects and exposure: development of monitoring data and samples to support health
and exposure studies to reduce uncertainty in National Ambient Air Quality Standards setting
and to enable improved health risk assessments; and
* methods testing: comparison and evaluation of emerging sampling methods with routine
techniques to enable a smooth transition to advanced methods.
The goals of each Supersites are described on the AMTIC Supersites web page
(http://www.epa.gov/ttn/amtic/ssproiec.html.). In addition, each Supersites contains a number of
sub-objectives which are carried out by various research organizations participating in the
Supersites. OAQPS identifies the types of projects occurring in the Supersites Program as
category 3 (see section 2.1.4). Category 3 projects do not require a formal DQO process but do
require a determination of the quality of data needed for decision making. The quality of data will
be defined in the QAPPs that are submitted for each Supersites project.
2.4 QAPPs
The QAPP is a formal document describing in comprehensive detail the necessary QA/QC, and
other technical activities that must be implemented to ensure that the results of work performed
will satisfy the stated performance criteria (DQOs).
The quality assurance policy of the EPA requires every Environmental Data Operation (EDO) to
have written and approved QAPPs prior to the start of the EDO. It is the responsibility of the
Research Groups participating in the Supersites Program to adhere to this policy. The technical
lead and QA lead identified for each Research Group are responsible for assuring adherence to
this EPA QA Policy and for approving the respective Supersites QAPP.
Each Supersite will produce one QAPP, incorporating all sub-projects. Due to the number of sub-
projects in each Supersite the QAC will not review and approve SOPs. The research group QA
lead will be responsible for assuring that SOPs are developed for each sub-project in accordance
with the approved QAPP.
-------
Project: Supersites QMP
Element No: 1
Revision No:2.0
Date: 9/13/01
Page 3 of 7
QAPPs are secured in a file by the grant ID number with the EPA Project Officer.
2.4.1 Categories of QA Project Plans
OAQPS will utilize a four-tiered project category approach to its QA Program in order to
effectively focus QA. This approach was originally developed by the U.S. EPA, Air and Energy
Engineering Research Laboratory (AEERL) and published by the EPA Risk Reduction
Engineering Laboratory, Cincinnati, Ohio (EPA/600/9-89/087). Category I involves the most
stringent QA approach, whereas Category IV is the least stringent. The following definition of
the categories are quoted from the document listed above:
Category I Projects
Projects include EDOs that directly support rulemaking, enforcement, regulatory, or
policy decisions. They also include research projects of significant national interest, such as
those typically monitored by the Administrator. Category I projects require the most detailed
and rigorous QA and QC for legal and scientific defensibility. Category I projects are typically
stand-alone; that is, the results from such projects are sufficient to make the needed decision
without input from other projects.
Category II Projects
Projects include EDOs that complement other projects in support of rulemaking regulatory, or
policy decisions. Such projects are of sufficient scope and substance that their results could be
combined with those from other projects of similar scope to provide necessary information for
decisions. Category n projects may also include certain high visibility projects as defined by
EPA management
Category III Projects
Projects include EDOs performed as interim steps in a larger group of operations. Such
projects include those producing results that are used to evaluate and select options for interim
decisions or to perform feasibility studies or preliminary assessments of unexplored areas for
possible future work.
-------
Project: Supersites QMP
Element No: 2
Revision No: 1.0
Date: 9/13/01
Page 4 of 7
Category IVProjects
Projects involving EDOs to study basic phenomena or issues, including proof of concepts,
screening for particular analytical species, etc. Such projects generally do not require
extensive detailed QA/QC activities and documentation. The number of elements required
for each category is reduced as one proceeds from category I to IV as illustrated in Table
2-1.
OAQPS has designated the Supersites program as a category 3 project. The statement of
the category will be placed on the QAPP signature and approval page, which will include
signatures of the QAPP preparer, Task Lead, QA Lead and the QAC.
Table 2-1 QAPP Elements Applicable to Various Categories
QAPP Element
Project Management
A 1 Title and Approval Sheet
A2 Table of Contents
A3 Distribution List
A4 Project/Task Organization
A 5 Problem Definition/Background
A6 Project/Task Description
A7 Quality Objectives and Criteria for Measurement Data
A9 Special Training Requirements/Certification
A 1 0 Documentation and Records
Measurement and Data Acquisition
Bl Sample Process Design
B2 Sampling Methods Requirements
B3 Sample Handling and Custody Requirements
B4 Analytical Methods Requirements
B5 Quality Control Requirements
B7 Instrument Calibration and Frequency
B8 Inspection/Acceptance Requirements for Supplies and
Consumables
B9 Data Acquisition Requirements
BIO Data Management
Assessment and Oversight
Cl Assessments and Response Actions
C2 Reports to Management
Data Validity and Usability
D 1 Data Review, Validation, and Verification Requirements
D2 Validation and Verification Methods
D3 Reconciliation and User Requirements
Category
Applicability
I, II, III, IV
I, II, III
I, II,
I, II, III
I, II, III
l' II' III, IV
I, II, III, IV
I
I, II, III
I, II, III, IV
I, II, III,
I, II, III
I, II, III, IV
I, II, III, IV
I, II, III
I, II
I, II, III
I, II
I, II, III
I, II, III
I, II, III
l'll'
I, II, III
-------
Project: Supersites QMP
Element No: 2
Revision No: 1.0
Date: 9/13/01
Page 5 of 7
2.4.2 QAPP Review and Approval
QAPPs are reviewed and approved in accordance with EPA QA/R5, Requirements for
Quality Assurance Project Plans for Environmental Data Operations. Copies of this
document are available from the EPA Quality Staff (QS) Web site
(http://www.epa.gov/quality I/). EPA Quality Staff encompasses all EPA staff that proves
Quality Assurance for the agency. This includes Headquarters, OAQPS, Office of
Research and Development (ORD) and the Regional Offices.
This document identifies and defines the elements that must be addressed in all formal
QAPPs. The NARSTO QAPP model1 was used as the template of all QAPPs for this
program. This was a decision made by the Supersites QA Work Group, since many of the
scientists were familiar with the format and had used the NARSTO format previously.
The NARSTO QAPP format is similar to the EPA QA format, but is organized differently.
All required sections of the EPA QA format are represented in the NARSTO format.
Review of the QAPP must include QAPP preparer, Research Group Task Lead and QA
Lead and the QAC. Mr. Dennis Mikel, the QAC for the Supersites Program, will review
and approve each QAPP for the required elements and the soundness of the QA/QC. The
QAC will attempt to review QAPPs within 30 working days of submission. The QAC
will provide written comments on each element. Through the QAPP review process, the
QAC will determine whether the QAPP can be approved, and if not, will identify those
elements requiring revision. If the QAPP requires revision, it will be sent back to the
author. The revisions, which may be included in the QAPP or as an addendum, must be
reviewed and approved by the QAC. All QAPP reviews are secured in a file by the grant
ID number with the EPA Project Officer.
Conditional Approvals
OAQPS does not encourage the use of conditional approvals; therefore, QAPPs may be
conditionally approved only by the QAC. Conditional approval is defined as a QAPP that
demonstrates that a quality system is in place and operational and that critical elements of
the QAPP are provided in enough detail to allow the reviewer to determine that the data
collected under the QAPP will be documented and of sufficient quality to meet the
program data quality objectives.
QAPP Revision
Any revisions required to the original QAPP can be included in a second or subsequent
revision or an addendum. However, sometimes the scope of a project can change which
may have the potential to affect the quality of the data. If these changes affect the
collection of environmental data, an addendum to the approved QAPP must be submitted
-------
Project: Supersites QMP
Element No: 2
Revision No: 1.0
Date: 9/13/01
Page 6 of 7
that describes the changes and the appropriate QA/QC techniques necessary to meet the
DQOs. The QAC must approve the changes.
QAPP Archive
Upon completion of the Supersites Program, QAPPs will be filed with the OAQPS
Document Control Officer (DCO) who will identify the document with a unique document
control number (see section 5). All original copies of the QAPPs and any subsequent
revisions will be secured by the DCO.
2.5 Standard Operating Procedures (SOPs)
Standard operating procedures (SOPs) are written documents that detail the method for an
operation, analysis, or action with thoroughly prescribed techniques and steps. SOPs are
protocols for all routine activities, especially those that are involved in the EDOs, which
generally involve repetitious operations performed in a consistent manner.
SOPs should ensure consistent conformance with organizational practices, serve as
training aids, provide ready reference and documentation of procedures, reduce work
effort, reduce error occurrences in data, and improve data comparability, credibility, and
defensibility. They should be sufficiently clear and written in a step-by-step format to be
readily understood by a person knowledgeable in the general concept of the procedure.
Guidance for SOP development can be found in QS document entitled Guidance for the
Preparation of Standard Operating Procedures (SOPs) EPA QA/G-6. Copies of this
document are available at the QS Website (http://www.epa.gov/qualityl/qa_docs.html).
SOPs must be written prior to the start of an EDO. The Research group QA Manager will
be responsible for ensuring SOPs are developed. SOPs for data collection methods must
be included in QAPPs either by reference, by inclusion of the actual method or be attached
as an appendix. In general, approval of SOPs occurs during the approval of the QAPP.
Any change in a SOP during the EDO should be documented and filed by the research
lead. SOPs will be reviewed during Technical Systems Audits (TSAs).
-------
Project: Supersites QMP
Element No: 2
Revision No: 1.0
Date: 9/13/01
Page 7 of 7
2.6 Assessments
There are several assessments tools that will be implemented by the QA system. Please
see Chapter 9.1 for details. Assessments will be performed as the program begins and on
a periodic basis after January 2001. Table 2-2 lists the types of assessments, assessor and
assessment frequency.
Table 2-2 Assessments
Assessment Type
Technical Systems Audits
Performance Audits
Network Review
Performance Evaluations
QAPP Review and Approval
Data Quality Assessment
Assessor
Project Level QA Managers
Project Level QA Manager
EPA- OAQPS-QA Coordinator
EPA -ORIA-NAREL Lab
EPA - OAQPS-QA Coordinator
Project Level QA Managers
Frequency
At the beginning of the project
At the beginning of the project/annually
after the first year
At the beginning of the program
Throughout the life of the program
Before projects begin
After data collection phase
Refere nee
1. NARSTO Quality Planning Handbook, November 23, 1999, Oak Ridge National
Laboratory, http://cdiac.esd.ornl.gov/programs/NARSTO
-------
Project: Supersites QMP
Element No: 2
Revision No: 1.0
Date: 9/13/01
Page 1 of 3
3.0 Personal Qualifications and Training
This section will discuss the process put in place to provide training for the Supersites
program. This chapter will outline the process involved and training available for air
monitoring professionals.
The process of training the personnel whom will be the involved in the Supersites program
will vary. The Supersites projects will be employing many research and well-known
methods. For instance, Federal Reference Method PM2.5 instrument will be employed at
many locations. With these, State and Local agency professionals will be operating the
instruments. For the research type of instruments, interns and graduate students from
various colleges and universities will be operating the monitors. It is the responsibility of
each Supersites project PI and QAM to provide training for all personnel involved in their
projects. OAQPS provides numerous satellite classes and on-your-own courses that are
free to air monitoring individuals. All persons working on this program are encouraged to
take these courses.
3.1 Personal Qualifications
Each Supersites program will make every effort to provide training to all who participate
in this program. Personnel assigned to the Supersites Program should meet the
educational, work experience, responsibility, personal attributes, and training requirements
for their positions. Although OAQPS can provide training to all agencies, it cannot
require the Supersites projects or any contractors to send their staff to EPA training
courses. During the TSAs, the QAM for each project or its contractor will review records
on personnel qualifications and training. All agencies should maintain these records in
personnel files and will be accessible for review during audit activities.
3.2 Training
Appropriate training is made available to persons supporting the Supersites program,
commensurate with their duties. Such training may consist of classroom lectures,
workshops, tele-conferences, and on-lhe-job training.
Over the last 2 years, a number of courses have been developed in cooperation with EPA
for personnel involved with ambient air monitoring and quality assurance aspects. Formal
QA/QC training is offered through the following organizations:
> Air Pollution Training Institute (APT!) http://www.epa.gov/oar/oaq.apti.htrnl
> Air & Waste Management Association (AWMA) http://awma.org/epr.htm
-------
Project: Supersites QMP
Element No:3
Revision No: 1.0
Date: 9/13/01
Page 2 of 3
> American Society for Quality Control (ASQC)
http://www. asqc. org/products/educat.html
" EPA Institute
> EPA Quality Staff (QS), http://www.epa.gov/qualityll
The courses mentioned below are open to all air monitoring personnel. EPA strongly
encourages all state and local agencies and contractors to take these courses. Table 3.1
presents a sequence of core ambient air monitoring and QA courses for ambient air
monitoring staff, and QA managers. The suggested course sequences assume little or no
experience in QA/QC or air monitoring. Persons having experience in the subject matter
described in the courses would select courses according to their appropriate experience
level.
Table 3.1 Core Ambient Air Training Courses
Sequence
1*
2*
3*
4*
5*
6*
7*
8*
9
10
11
*
*
*
*
*
Course Title (SI= self instructional)
Air Pollution Control Orientation Course (Revised), SL422
Principles and Practices of Air Pollution Control, 452
Orientation to Quality Assurance Management
Introduction to Ambient Air Monitoring (Under Revision), 81:434
General Quality Assurance Considerations for Ambient Air Monitoring (Under
Revision), SL471
Quality Assurance for Air Pollution Measurement Systems (Under Revision), 470
Data Quality Objectives Workshop
Quality Assurance Project Plan
Atmospheric Sampling (Under Revision), 435
Analytical Methods for Air Quality Standards, 464
Chain-of-Custody Procedures for Samples and Data, SL443
Data Quality Assessment
Management Systems Review
Beginning Environmental Statistical Techniques (Revised), SL473A
Introduction to Environmental Statistics, SL473B
Statistics for Effective Decision Making
AIRS Training
Source
APTI
APTI
QS
APTI
APTI
APTI
QS
QS
APTI
APTI
APTI
QS
QS
APTI
APTI
ASQC
OAQPS
* Courses recommended for QA Managers
-------
Project: Supersites QMP
Element No:3
Revision No: 1.0
Date: 9/13/01
Page 3 of 3
3.3 Certification
No certificates are required for this program.
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 1 of 10
4.0 Extramural Agreements and Procurement of Items and
Services
OAQPS must ensure that the items and services it acquires are procured within EPA
regulations, are delivered in a timely fashion, and are within the required specifications.
The following sections will provide general information on OAQPS procurement
procedures and provide personnel involved in the Supersites Program with the a
description of the requirements
4.1 Source of Funds
4.1.1 State Assistance Grants: Many SLAMS and NAMS will support the Supersites
Program by operation of ancillary sites in the SLAMS/NAMS network. The source of
funds is SectionlOS and eventually Section 105 State Assistance Grants (STAG). Every
year, funds will be allocated to the State and local air monitoring organizations to operate
the PM2.5 Federal Reference and Speciation Program. Funds are allocated to the EPA
Regions who then allocate them to the State, local or Tribal agencies. These agencies then
follow their own procurement policies to get the monitoring accomplished.
A portion of the STAG funds are allocated back to OAQPS for two activities
1 National speciation monitor contract- OAQPS set up a national contract to facilitate
the purchase of speciation monitors
2 Analytical laboratory contract- OAQPS set up a national contract to perform all the
filter preparation and analyses and reporting activities.
Each year OAQPS will submit a request for the appropriate allocation of funds for these
activities based on the number of monitors being implemented (or planned) for that fiscal
year.
4.1.2 OAQPS Internal funds: Each year OAQPS plans the activities it will pursue in the
upcoming fiscal year. The OAQPS speciation monitoring and QA leads will work with
various work groups and cooperators to prioritize the use of the environmental program
management (EPM) funds. These funds may be used to purchase capital equipment or for
contracting.
OAQPS, through the Memorandum of Agreement with the Office of Radiation and Indoor
Air lab will provide contract funds to these labs. The use/ allocation of the funds will be
negotiated during fiscal year planning.
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 2 of 10
4.2 Procurement of Items
In EPA, only contracting officers (COs) are authorized to procure items and services,
unless it is an impress fund transaction approved by the CO prior to the originators
purchase of the item. The Federal Government is not bound by any commitments made
by other than authorized personnel.
Requests for purchases begin at the yearly planning stages of the Speciation Program for
the EPM funds. Purchases by contractors must be identified in the project scope of work
for such purchases. All items should be identified and specifications that meet the
government's minimum needs should be detailed. These specifications will be referred to
during the procurement process and will assure that the OAQPS requestor receives the
proper item and reduces the chances of purchase delays or incorrect purchases because of
inadequate product specifications.
4.3 Procurement of Services
Two types of mechanisms are primarily used to procure services, contracts and assistance
agreements (grants, cooperative agreements, etc.). As mentioned in section 4.1, COs are
the only individuals who can obligate funds.
When procuring services, one should follow the same basic procedure used for the
procurement of items. There are certain activities that are of a policy- and decision-
making nature that should remain the sole authority of EPA. The CMD should be
contacted during the initial planning of the PR to discuss specific requirements for the
procurement.
The Project Officer (PO) states the service that will be delivered, measures the quality of
the service, and accepts the service. When a level-of-effort contract is the vehicle used in
procuring services, the work assignment manager (WAM) provides the technical expertise
for the work assignment and assumes responsibility for the QA requirements assigned to
the PO. Two major tools to ensure that adequate service is provided are a well-defined
statement of work (SOW) and a QAPP that includes reviews (audits).
The QAM or DQAO assists in this activity by providing knowledge and guidance on the
QA requirements and aspects of any potential project. The QAM or DQAO will also
approve the QA review form that is discussed in the next section.
4.3.1 Contracts: Contracts are used when the government derives sole benefit from a
particular product or service. Contracts can be specific and can require a degree of lead-
time for development. Depending upon the scope of the service, QA attributes can be
developed that must be adhered to under the terms and agreements of the contract. Any
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 3 of 10
EPA initiated contracts are required to use some type of QA form to determine if the
contract will require EDO and therefore requires a QMP, a QAPP assessments and reports.
After the form is completed it must be reviewed by the (WAM/PO) and a QA officer. The
form must be kept in the official contract file.
The Federal Acquisition Regulations, Title 48 of the Code of Federal Regulations, was
recently amended to address contract quality systems requirements on a government-wide
basis. The new FAR clause at 52.246-11, Fligher-Level Quality Requirement, allows a
Federal agency to select a voluntary consensus standard as the basis for its quality
requirements for contracts and allows tailoring of the standard to more effectively address
specific needs or purposes. Based on this FAR clause, EPA has selected ANSI/ASQC E4,
Specifications and Guidelines for Quality Systems for Environmental Data Collection and
Environmental Technology Programs, as the basis for its environmental quality
requirements and has tailored this standard to ensure that contractors demonstrate
conformance to this national standard. The background and application of the new
procurement policy as it relates to QA is included in Appendix A must be followed.
Due to these changes, 48 CFR 1546, a quality regulation that applies only to EPA, will be
removed from the Code of Federal Regulations. The tailoring language allowed by 52
CFR 246-11 and pertinent requirements in 48 CFR 1546 will be included in the EPA
Directive 1900, Contracts Management Manual. This procurement policy notice is being
issued to ensure an orderly transition from 48 CFR 1546 to EPA Directive 1900 and
contains tailoring language allowed by 52 CFR 246-11. It is in effect until the revisions to
Directive 1900 are completed
Whenever the government enters into a contract, it is entitled to receive quality service. In
order to define and measure this quality, the WAM/PO must develop a SOW that will
accurately define the minimum acceptable requirements for the service or product.
Methods used to determine quality (audits, quarterly interviews, random inspections, etc.)
should be explained prior to project implementation so that the supplier will understand
how quality will be assessed.
Part of the procurement process of certain types of large contracts include the use of a
technical evaluation panel (TEP). When this form of contracting mechanism is used to
solicit contracts in which a significant percent of the cost (> 25%) includes EDO, the TEP
must include a QA representative, if possible, a representative from the group/branch
processing the contract. Part of the TEP responsibilities will include rating each potential
contractor against a standard set of criteria. A portion of these criteria can include various
assessments such as on-site audits and the analysis of performance evaluation materials.
Prior to the solicitation for bid, it must be determined what proportion of the TEP rating
will be allocated to QA assessments. It is suggested that a minimum of 5% of the overall
TEP rating be allocated to QA.
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 4 of 10
Depending upon the type of contract used to acquire a service, different types of QA
methods for determining the quality of product or output may be used. However, in all
cases, documentation is essential. POs/WAMS are responsible for documenting quality
on a regular basis. EPA personnel must be aware of the "personal services" type of work
characterized by an employer-employee relationship between government and contractor
employees. These contracts are illegal in EPA. Personal services conflicts arise when
government employees assume the right to instruct, supervise, or control a contractor's
employee in how he or she performs work. It is the contractor's right to hire and
terminate, to assign, and to organize and implement tasks as the contracting organization
deems appropriate. OAQPS may tell the contractor what to do within the terms and
agreements of the contract, but not how to do it.
4.4 Assistance Agreements
Assistance agreements are used when both parties (EPA and the group providing the
service) derive benefit out of the service. This usually occurs with grants or cooperative
agreements where universities or states derive benefits from participating in EDOs. QA
requirements are developed for all assistance agreements that include EDOs. OAQPS
follows guidelines developed in the EPA Assistance Administration Manual (EPA-5 700).
Assistance agreement SOWs are usually developed jointly. However, once the SOW is
completed, the parties must also agree on the quality standards for assuring the product or
service. It is the responsibility of the WAM/PO to be knowledgeable of the EPA QA
policy and to represent these standards during the development of the projects SOW.
Special conditions are usually included in assistance agreements. The PO will list the
conditions to which project participants must adhere. One of these conditions relates to
QAPPs. Any assistance agreement that includes EDOs must include the following
statement:
A quality assurance project plan must be submitted within 90 days of this
agreement and/or 30 days prior to commencement of any EDOs.
Implementation dates will be adjusted based upon the above conditions.
Costs associated with data collection are not allowable costs until the quality
assurance project plan is submitted, nor will costs be reimbursed until the
quality assurance program plan is approved.
4.5 EPA Exclusive Versus Discretionary Functions
The following information comes directly from EPA Quality Manual for Environmental
Programs 5360.
Many quality system activities involving environmental data operations are inherently
governmental functions and must be performed only by EPA personnel or by personnel explicitly
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 5 of 10
authorized by EPA based on statute, regulation, or by the terms of an extramural agreement. Such
representatives may include other governmental personnel and with specific authorization,
contractor personnel. When quality management tasks are performed by a contractor, the contract
must be appropriately managed and must remain under the control of the authorized EPA
contracting representatives. EPA cannot use cooperative agreements or grants to provide quality
management activities such as QA and QC services for EPA because it is an inappropriate use of
financial assistance (Office of General Counsel memorandum, August 2, 1994).
This section describes the quality management tasks necessary to comply with the Order and
identifies those tasks that may be performed by non-government personnel under appropriate
management controls.
Two types of quality management functions are described:
Exclusively EPA Functions - inherently governmental work which must be performed
only by responsible EPA officials, including the QA Managers (QAMs), or authorized
EPA representatives.
Discretionary Functions - activities that may be performed either by EPA personnel or by
non-EPA personnel under the specific technical direction of and performance monitoring
by the QA Manager or other responsible EPA or Government official under an approved
contract, work assignment, delivery order, task order, etc.
In the situations involving the other associated functions, there may be instances involving
sensitive contracting services, advisory and assistance services, and vulnerable contracting
practices as defined by the Federal Acquisition Regulations, Office of Federal Procurement Policy
(OFPP), and the EPA Contracts Management Manual (EPA Order 1900). Such situations are
identified by italicized text in the following sections. In addition, management approval of
services contracts as defined by OFPP Letter 93-1 must be obtained for many of the associated
tasks.
Technical direction or other instructions to an extramural organization, relating to performance of
an extramural agreement, shall be provided only by authorized EPA or other Government
representatives in accordance with the terms of the applicable extramural agreement. Only
authorized EPA or other Government representatives are to provide direction or instructions to an
extramural organization providing quality systems support for environmental programs. This is to
avoid such actions as:
the providing of directions or instructions that are inconsistent with the terms of an
extramural agreement,
unauthorized access to confidential business information (CBI), or
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 6 of 10
unauthorized access to information that may allow an extramural organization to gain an
unfair competitive advantage.
4.5.1 Mandatory Quality Management Tasks and Descriptions: This section describes the
activities and tasks integral to an effective quality system. These tasks are required to implement
EPA Order 5360.1 CHG1.
Manage and Coordinate the Quality System
Exclusively EPA functions that must be performed by EPA QA personnel include:
managing the day-to-day implementation of the mandatory quality system.
acting as liaison between the organization and the QS on matters of QA policy.
coordinating with senior management the development of and preparation of the
organization's Quality Management Plan.
coordinating with senior management changes to the Quality System as needed to assure
its continued effectiveness and assisting in reporting the results annually to management
and to QS in the QA Annual Report and Work Plan.
managing organization resources designated for the quality system.
maintaining records of pertinent quality system activities performed by the organization.
Review and Approve Procurement and Financial Assistance Documents for QA Requirements
Exclusively EPA functions that must be performed by EPA QA personnel include:
reviewing procurement and financial assistance documents (e.g., statements of work,
scopes of work, applications for assistance, funding requests, and purchase requests) to
confirm any need for QA requirements, providing any necessary special language or
conditions for such QA requirements, and approving by signature the appropriate Quality
Assurance Review Form.
participating directly or indirectly in the solicitation or agreement review process to advise
the Project Officer on the suitability of the offer or quality system or quality
assurance/quality control (QA/QC) approach for the particular project.
reviewing work assignments, delivery orders, and task orders to certify that appropriate
QA/QC requirements have been established and that the necessary instructions are being
communicated to the contractor to carry out the required QA/QC tasks. Approving by
signature appropriate Quality Assurance Review Form (EPA Order 1900, Chapter 2).
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 7 of 10
Review and Approve QA Planning Documents
Exclusively EPA functions that must be performed by EPA QA personnel or their authorized
EPA representative include:
reviewing Quality Assurance Project Plans (QAPPs) for all projects, work assignments,
delivery orders, task orders, grants, cooperative agreements, and interagency agreements
involving data acquisition, data generation, and/or measurement activities that are
performed on behalf of EPA.
approving all QAPPs for implementation in all applicable projects, work assignments,
delivery orders, task orders, grants, cooperative agreements, and interagency agreements
performed on behalf of EPA.
coordinating the correction of deficient QAPPs with the Project Officer and his/her
management.
Discretionary functions that may be performed by either EPA personnel or non-EPA personnel
include:
reviewing, at the specific technical direction of the QAM, QA Project Plans and other QA-
related planning documents, such as sampling and analysis plans, Data Quality
Objectives (DQO) specifications, etc., and providing specific substantiated
recommendations to the QAM on the adequacy of the QA approach in meeting the
criteria provided by the QAM. (The reviews should identify specific technical
deficiencies in the planning documents.)
Track and Report Quality System Deliverables
Exclusively EPA functions that must be performed by EPA QA personnel or their authorized
EPA representative include:
tracking critical quality system deliverables for the organization and make periodic reports
to senior management on the status of reporting actions and deliverables.
Discretionary functions that may be performed by either EPA personnel or non-EPA personnel
include:
compiling/logging administrative and management information including turnaround
times to correct deficient QAPPs, responses to audits (e.g., responses and corrective
actions), and quality reviews of final reports.
Manage Contractor Support Work Assignments, Delivery Orders, and Task Orders
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 8 of 10
Exclusively EPA functions that must be performed by EPA QA personnel include:
serving as the Contracting Officer Representative (for example, Project Officer, Work
Assignment Manager, or Delivery Order Project Officer) for specific QA support
contracts, work assignments, delivery orders, and task orders.
Plan and Conduct Management Assessments
Exclusively EPA functions that must be performed by EPA QA personnel include:
planning, directing, and conducting assessments of the effectiveness of the quality system
being applied to environmental data operations and reporting results to senior
management. Such assessments may be conducted using the Management Systems
Review (MSR) process.
coordinating with senior management any revision of the quality system as necessary
based on the findings of the assessment.
Discretionary functions that may be performed by either EPA personnel or non-EPA personnel
include:
providing technical support to the EPA QAM in the planning phase of management
assessments. (Such activities are limited to the assembly and compilation of background
information and data, guidance documents, technical reports, etc., available in the public
domain, for use by EPA in designing the assessment goals and specifications.)
Plan and Conduct Technical Assessments
Exclusively EPA functions that must be performed by EPA QA personnel or their authorized
EPA representative include:
planning and directing with the responsible EPA project officials the implementation of
periodic technical assessments of ongoing environmental data operations to provide
information to management to assure that technical and quality objectives are being met
and that the needs of the customer are being satisfied. Such assessments may include
technical systems audits, surveillance, performance evaluations, and data quality
assessments.
determining conclusions and necessary corrective actions (if any) based on the findings of
the assessments.
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 9 of 10
Discretionary functions that may be performed by either EPA personnel or non-EPA personnel
include:
performing technical assessments of environmental data producing activities, both
intramural and extramural (on-site and off-site) according to a specific plan approved by
the QAM. Preparations for such assessments may include the acquisition or development
of audit materials and standards. Results (findings) are summarized, substantiated, and
presented to the QAM or authorized EPA representative.
A determination of whether an authorized Agency representative should accompany a
contractor's personnel should be made on a case-by-case basis only after coordination
between the responsible organization and contracting officer. Such coordination should
include consideration of the purpose of the accompaniment and clear definition of the
Agency representative's role and responsibility during the contractor's performance of
the audit or technical assessment to avoid the appearance of a personal services
relationship.
Prepare and Present QA Training Materials and Courses
Exclusively EPA functions that must be performed by EPA QA personnel or their authorized
EPA representative include:
developing and presenting detailed guidance and training for QA/QC activities based on
interpretation of Agency-wide requirements and guidance.
Discretionary functions that may be performed by either EPA personnel or non-EPA personnel
include:
providing or coordinating quality-related training for the organization in special skill
areas identified by the Agency and not generally available to the organization.
providing allowable technical and/or logistical assistance in preparing and presenting
quality-related technical training (within the Agency's implementation of special
management and control measures and the constraints of potential for conflict of interest,
of revealing confidential business information, or of appearing to be interpreting or
representing Agency policy).
Review and Approve Final Reports for Quality Documentation
Exclusively EPA functions that must be performed by EPA QA personnel or their authorized
EPA representative include:
-------
Project: Supersites QMP
Element No: 4
Revision No: 1.0
Date: 9/13/01
Page 10 of 10
establishing criteria for the acceptability of quality documentation in the organization's
published papers and reports; that is, defining what is required for an adequate discussion
of the quality of the project results and the usability of the information reported.
approving for publication those papers and reports that meet the defined criteria.
Discretionary functions that may be performed by either EPA personnel or non-EPA personnel
include:
conducting a substantiated technical review of all reports produced by the organization
using the qualitative and quantitative specifications obtained from the DQO process or
other criteria provided by EPA. This quality review complements the peer review
process.
4.5.2 Non-Mandatory Quality Management Tasks and Descriptions: This section describes
other activities and tasks integral to an effective quality system. They are not explicitly required
to implement EPA Order 5360.1 CHG 1, but if implemented, they must be implemented as
described below.
-------
Project: Supersites QMP
Element No: 5
Revision No: 1.0
Date: 9/13/01
Page 1 of 4
5. 0 Records and Documentation
The responsibility of record keeping falls upon OAQPS, ORD and the individual Supersites
Projects and their contractors. For this program, there are number of documents and records that
need to be retained. A document, from a record management perspective, is a volume that
contains information, which describes, defines, specifies reports, certifies, or provides data or
results pertaining to environmental programs. As defined in the Federal Records Act of 1950 and
the Paperwork Reduction Act of 1995 (now 44 U.S.C. 3101-3107), records are: "...books, papers,
maps, photographs, machine readable materials, or other documentary materials, regardless of
physical form or characteristics, made or received by an agency of the United States Government
under Federal Law or in connection with the transaction of public business and preserved or
appropriate for preservation by that agency or its legitimate successor as evidence of the
organization, functions, policies, decisions, procedures, operations, or other activities of the
Government or because of the informational value of data in them..." EPA-OAQPS and ORD
will adhere to this guideline. Section 5.1 will illustrates the process that will be implemented for
storing documents and records. Since many agencies are involved, their documentation storage
capabilities and processes will differ; however, at a minimum, all documents and records for this
program will be securely stored. For more information on document control and storage, please
see the individual agency QAPPs.
5.1 Document Hierarchy and Process
This section will outline the hierarchy of the documentation and illustrate the review process for
the major documents created for this program.
5.1.1 Hierarchy: The Clean Air Act (CAA) and EPA Order 5360.1, July 1998 are the
overarching documents for this program. As such, all authority to create programs and allocate
funds is given in these documents. EPA Order 5360.1 gives the EPA authority to require all
agencies that accept federal funds to create QMPs, QAPPs and Network Plans. OAQPS has the
authority to require, review, comment and withhold funds if these requirements are not met. The
order of hierarchy follows:
> The Code of Federal Regulation, through the CAA and Order 5360.1 are the
overarching authority.
> The QMP encompasses the entire program. All agencies, OAQPS, ORD and the
individual Supersites projects will adhere to the requirements and guidelines in the
QMP. The QMP discusses the roles of each agency.
* The QAPPs for individual agencies will govern that agency. The agency must adhere
to the statements made in their QAPP.
> The Network Plan will outline how the network will be implemented and document
the location of each sampler with all ancillary data.
-------
Project: Supersites QMP
Element No: 5
Revision No: 1.0
Date: 9/13/01
Page 2 of 4
5.1.2 Document Creation and Review Process
5.1.2.1 QMP
The QMP for this program was generated by OAQPS-EMAD-MQAG. It has the overarching
authority over all QAPPs, Network Plan and all other ancillary documents. This document has
undergone thorough review by OAQPS, ORD, and the Supersites Project Pis and QA Managers.
5.1.2.2 QAPPs
The individual Supersites Pis and QA Managers to describe their process of assuring the quality
of the data write the QAPPs. OAQPS reserves the authority to review, make comments and
approve the individual QAPPs.
5.1.2.3 Network Plan
OAQPS requests that all Supersites project take electronic photographs of each site in the cardinal
directions. These will be forwarded to OAQPS with all other siting data. OAQPS will create
electronic resources that will include the following:
> Electronic photos of the sampler in place;
* Electronic photos of the area in all cardinal directions;
* Maps of the area showing local sources (if known);
> Coordinates of the location generated by Geographic Positioning Systems. The
standard is +/- 10 meters.
This data will be compiled and placed in an accessible electronic database and distributed and
stored by OAQPS. Any parties that wish to review the network will be able to obtain this data
expeditiously.
5.1.2.4 Other Documents
The responsibility of all other documents is detailed in the next section.
5.2 Documentation Responsibilities
5.2.1 OAQPS -EMAD
This division has oversight of the Supersites. As such, the documents that must be controlled and
stored are under the jurisdiction of the Project Officer, who has the responsibility of storing and
archiving all records that pertain to the requisition and deposition of contracts.
-------
Project: Supersites QMP
Element No: 5
Revision No: 1.0
Date: 9/13/01
Page 3 of 4
5.2.2 Supersites
Principle Investigators- the Pis are responsible for the oversight of the field and laboratory
documents and implementation of the QAPP. As such, he/she is responsible for the storage of all
records and documents generated by the labs or field.
Quality Assurance Managers - The QA Managers are responsible for the archiving of all QA
related documents created during any assessments by the manager or contractor in accordance
with the QAPP.
Individual Investigators - The individual research investigators are responsible for the oversight
of the their instrument documentation. All calibration or maintenance data, notes, field
information is their responsibility. As such, he/she is responsible for the storage of all records
and documents generated by their field operations in accordance with their SOPs.
5.3 Deposition and Storage of Documents and Records
This section will address the deposition, storage accessibility, and protection of documents and
records. It is noted that the persons filling the roles mentioned above are responsible for the
documents and record that they generate. These agencies will take full responsibility for the
deposition of these records. Please note that all records and documents will be made available
for review and scrutiny upon request for up to 5 years after the data were generated.
5.3.1 Field notebooks
Notebooks will be utilized for recording results of field audits. Dates, times, field conditions,
temperature, pressure and flow rates will be recorded. Each investigator will archive all field
logs. Any computer-generated logs will be downloaded to their headquarters.
5.3.2 Lab Notebooks
Notebooks will also be issued for the laboratory. These notebooks should be uniquely numbered
and associated with the Supersites program. One notebook will be available for general
comments/notes; others will be associated with, the temperature and humidity recording
instruments, the refrigerator, calibration equipment/standards, and the analytical balances and all
instruments used for this program. Laboratory notebooks review and archiving are the
responsibility of the individual investigators or researchers. All logs must be maintained for at
least 5 years after the data are generated.
-------
Project: Supersites QMP
Element No: 5
Revision No: 1.0
Date: 9/13/01
Page 4 of 4
5.3.3 Chain of Custody Forms
Original Chain of Custody forms will be archived by the individual investigators.
5.3.4 Other Documents
All other documents must be stored according to their QAPP.
5.3.5 Electronic data collection
In order to reduce the potential for data entry errors, automated systems will be utilized where
appropriate and will record the same information that is found on data entry forms, such as the
output of the Fine Particle Federal Reference Method or Speciation samplers. Safe and secure
handling and storage of electronic information must be assured through good data administrative
practices, including periodic data backups, as described in their QAPP.
5.4 Deposition of Reports
5.4.1 Data Reporting Package/Archiving and Retrieval
All the information, electronic and written, will be retained for 5 years from the date the grantee
submits its final expenditure report unless otherwise noted in the funding agreement. However, if
any litigation, claim, negotiation, audit or other action involving the records has been started
before the expiration of the 5-year period, the records will be retained until completion of the
action and resolution of all issues which arise from it, or until the end of the regular 5-year period,
whichever is later. For example, any data collected in calendar year 2001 (1/1/01 - 12/31/01)
will be retained until, at a minimum, January 1, 2006, unless the information is used for litigation
purposes.
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 1 of 3
6.0 Computer Software and Hardware
There is an increasing dependence upon computers and computer related hardware in the
collection of environmental data. Indeed, all environmental programs within and outside of the
EPA use computers extensively to collect, store, validate and analyze environmental data. This
section will outline briefly what computer systems will be employed throughout the Supersites
program. This chapter will also describe the roles and responsibilities for system hardware and
software.
6.1 Computer System Descriptions
6.1.1 EPA-OAQPS
The QMP and Network Plan will be archived in Research Triangle Park, North Carolina. All
communications and hardcopy information will also be housed at the EPA facility in Research
Triangle Park, North Carolina. The QMP, final Supersites QAPPs will be posted on the EPA-
OAQPS website, AMTIC.
6.1.2 Supersites
Individual Supersites will develop, operate, and maintain their computer systems as outlined in
their individual site's QAPP or data management plan.
6.1.3 NARSTO
While participating in the flow of information from Supersites projects to the NARSTO
Permanent Data Archive, NARSTO will employ three computer systems. Supersites data and
metadata will be entered into, stored, and processed by the Data and Information Sharing Tool
(DIST), an FTP site, and the NARSTO Quality Systems Science Center processing system.
Please see Figure 6-1.
Data and Information Sharing Tool (DIST): DIST is a web-based index and clearinghouse of
atmospheric measurement and chemistry data and metadata, made available by the NARSTO
program. The DIST enables small groups of investigators to share project data in a secure
environment and also provides data from numerous sources to the at-large research community.
The data available through DIST may include measurement data, model outputs, images, and
other information of interest to the atmospheric research community. Data are indexed using
consistent metadata categories to support searching by fields such as project, location, date,
keyword, and investigator. Data providers can easily enter metadata and add links to their data in
this Web-based tool. The DIST is a key component in the flow of data from projects to the
NARSTO Permanent Data Archive (PDA) with output capabilities that facilitate metadata and
data archiving. The DIST can be accessed at http://cdiac.esd.ornl.gov/programs/NARSTO/.
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 2 of 3
The overall DIST system is robust in design and efficient in implementation. Users will find a
convenient interface, similar to many existing web-based products, that includes a file manager
and good help files. The system is implemented using Internet standards, including XML, and
supports international metadata standards, including FGDC and Z39.50. DIST is based on
commercial off-the-shelf software with many value-added improvements and is part of the ORNL
Mercury Consortium, a group of independent data projects that work together to share
improvements and reduce individual costs.
Supersites FTP site: The Supersites shared-access FTP site has two areas for accessing data files.
One area is publicly accessible as an anonymous login site (ftp://narsto.esd.ornl.gov) and the
other is an internal area with login and password limited access to directories for each Supersites
project and working group. System administrators will distribute login names and passwords,
create subdirectories and move files as directed by site users, and maintain and periodically
backup the site. Appropriate security and access disclaimers are distributed to users and posted
on the site.
Quality Systems Science Center processing system: This system accepts Supersites data provided
in the Data Exchange Standard format, checks the format, calculates summary statistics,
assembles documentation, and transmits the data to the NARSTO Permanent Data Archive
(PDA). The Data Exchange Standard is documented in the NARSTO Data Management
Handbook (http://cdiac.esd.ornl.gov/programs/NARSTO/narsto.html#qsnip), and also in Excel 97
templates designed to support creating these files. Files with related data are grouped into data
sets for processing and archiving. A QSSC "Read and Verify" code reads the files in each data
set, reproduces each file with an added section containing summary statistics for data in the file,
produces files helpful in documenting the data set for archiving, verifies conformity to key
provisions of the Data Exchange Standard, and produces a QA report indicating any deviations
from the Standard that were found. If deviations were found, this QA report is sent back to the
data originator so issues can be resolved. When the data set is complete and ready,
documentation is assembled, and the data set is sent to the PDA for archiving.
NARSTO Permanent Data Archive (PDA): The PDA is maintained at the Langley Distributed
Active Archive Center (DAAC) and operated by the NASA Langley Research Center, Hampton,
Virginia. NARSTO data are maintained as part of their permanent data collection and are
available to the public at no charge through a convenient Internet ordering system
(http://eosweb.larc.nasa.gov/).
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 3 of 3
Supersites Data Flow
Data
Collection
System
Principal
Investigator
Data Set Metadata
Project
Data
System
Project Data
Coordinator
Data Set Metadata
NARSTO QSSC
Data Processing System
Data Files in
DES1 or
other format
Data Files in
DES format2
Data Files in DES format2
Data Set Documentation
1 NARSTO Data Exchange Standard Format
2 Special formats permitted
3 Data and Information Sharing Tool
NARSTO
Permanent Data
Archive
Figure 6-1. Data Flow Diagram
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 1 of 6
7.0 Planning and Implementation of Work Process
7.1 Project Goals and Objectives
This section outlines planning and implementation procedures that were employed in the
Supersites program. This program has several diverse agencies that are interacting at several
levels. Therefore, to ensure that the work is being performed and that the quality of the data is
acceptable, clear communication must be employed for this program. The following sections
outline how this is accomplished.
7.1.1 Program Objectives
The program addresses the objectives in three major areas:
> SIPs: support development of State Implementation Plans (SIPs) through improved
understanding of source-receptor relationships leading to improved design,
implementation, and tracking of control strategy effectiveness in the overall PM
program;
> health effects and exposure: development of monitoring data and samples to support
health and exposure studies to reduce uncertainty in National Ambient Air Quality
Standards setting and to enable improved health risk assessments; and
> methods testing: comparison and evaluation of emerging sampling methods with
routine techniques to enable a smooth transition to advanced method?,.
These objectives are broad in scope and presented the challenge of developing specific data
quality objectives within a National program responsive to many disciplines. Based on the
original funding rationale, each of the Supersites study areas provided some support for
implementation questions. Some of the sites added objectives related to research on health,
exposure, and methods testing. Thus, while some aspects of the program were common to all
locations, others, including duration, measurement frequency, and indicators measured may vary
with specific objectives at differing locations. The Measurements Workshop Report (see Chapter
1 reference) provided numerous examples of overlapping data needs across diverse science
disciplines that typically exhibit very limited interaction. A simple example includes the daily
collection of chemically speciated data that assist both air quality model evaluations and exposure
studies. Clearly, windows of opportunity exist for optimizing the use of environmental data to
respond effectively to seemingly disparate objectives. An organized approach to building specific
study objectives must be followed to ensure needs are met and resources optimized. Targeted
program objectives were developed by:
* starting from test hypotheses and questions that are generated by an integrated program
planning team;
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 2 of 6
* utilizing site/time based objectives where certain locations and study periods are
optimized for specific topic areas; e.g.: specific airsheds optimized for source receptor
and air quality model evaluation of specific airsheds optimized to support
epidemiological and exposure studies;
> emphasizing methods testing early, then transitioning to other objectives within a single
airshed; including discrete or intensive sampling periods optimized to address specific
test hypotheses;
> requiring all investigators to follow existing quality assurance protocols in the
development of QAPPs which includes requirements for developing DQOs. Optimizing
objectives by location or time does not preclude some level of support at all locations to
SIPs, health effects and exposure studies and methods testing, given the multiple uses of
similar data.
7.1.2 Program Principles
EPA staff adhered to the following organizational and guiding principles derived from
the PM Measurements Workshop Report (Chapter 1 reference) in developing an overarching
strategy for implementing the program:
be designed as a "learning" rather than a "measurement" program;
provide consistent and comparable, but not necessarily identical, measurements across the
sites and the nation; be an investment that leverages the largest possible number of other
governmental and private investments;
have analysis and evaluation built in from the start;
organize the measurements approach by asking; what are the major questions and hypotheses;
what should be measured; where and when should the measurements.
The Supersites program must be flexible to adjust to and accommodate the unique needs of
different research disciplines by planning across scientific disciplines (health effects, exposure
and atmospheric science measurement needs) and regulatory agencies. Results must be developed
in a timely manner to assist development of SIPs which are required as early as 2005, and review
of the PM standard which is to be completed in 2002 and again in 2007. Therefore, program
deployment is following a dual track staging with an initial establishment of two sites in 1999 and
a gradual full site deployment accomplished in 2002. The rationale for this dual track deployment
is to test technical and organizational elements of the program early in order to aid the
optimization of the full program, and allow adequate planning and design so that the full program
can provide the most relevant support for a mix of regulatory and research based needs.
Program planning and design to date consisted of the planning meeting, and report writing by the
steering committee and attendees related to the PM Measurements Workshop, along with internal
EPA meetings involving regulatory, atmospheric sciences, health effects and exposure specialists.
More formal planning and design with a coordination group started the beginning of 1999. EPA
staff recommended the establishment of two initial sites located in Atlanta, Georgia and
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 3 of 6
Fresno/Bakersfield, California in mid-1999, which operated from 2 years or longer. Initial
objectives for these sites was oriented toward source-receptor characterizations and testing non-
routine monitoring methods and establishing logistical procedures, including assessment of
resource needs, that will benefit subsequent deployment in other locations.
Preliminary feedback from the initial sites (Atlanta and Fresno) was factored into subsequent
design of the full program, which benefited from more integrated planning among science
disciplines and regulatory groups. Selection of remaining site locations was completed in the last
calendar quarter, 1999 so that local agencies and universities could take into account the
availability of Supersites in deploying their chemical speciation network. EPA staff recommends
the deployment of the remaining six sites commencing in mid-2000.
7.2 Initial Planning and Conceptualization
The development of this plan was discussed in the introduction above.
7.2.1 Program Planning and Design
Following Subcommittee review scheduled for November 30, 1998, EPA established
formal internal and external planning and design teams. Internally, EPA established a planning
team composed of atmospheric science, regulatory and health effects and exposure specialists. In
parallel, invitations were mailed to other Federal and State/local agencies and private industries
active in relevant research to participate in a broader External Coordination Workgroup. EPA
staff were responsible for developing more detailed program plans and working with the external
committee at a partnership level by providing early drafts and conducting meetings on an as
needed basis. The design approach was based on developing a measurements strategy responsive
to key questions (science and regulatory) and scientific hypotheses, taking advantage of the PM
Measurements Workshop Report. EPA also was responsible for establishing and managing all
administrative tasks related to program funding. The active work with the External Coordination
Committee is one of several steps (see Section 7) taken to optimize measurement resources across
different organizations. The Subcommittee requested to review more detailed plans as part of the
decision approval process.
7.2.2 Program Execution
EPA manages program resources that result in funding vehicles to research groups and
contractors that conduct much of the work. The actual work is being performed principally by
university and other non-profit research groups with support as needed by contractor
organizations. EPA assigned Technical Coordinators to the program to work closely with Project
Pis. The limited number of Supersites locations demanded that a thoughtful and objective
selection process be established. The initial assumptions underlying selections included the ability
to capture unique airsheds in populated areas roughly defined through a combination of air
chemistry, source distribution and geographical/meteorological characteristics. The following
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 4 of 6
selection criteria, which again draw on the PM Workshop Report, guided the selection of study
areas.
> High concentrations of PM in unique and prototypical "airsheds": known or
expected "high" concentration areas that will approach or exceed the PM NAAQS and
affect substantial exposure to populations (serves SIPs and health effects and exposure).
In the aggregate, these airsheds should reflect locations with varying meteorological,
source composition and atmospheric properties, to allow for more comprehensive
stressing of sampling methods, more sound statistical design for exposure/health
research, and capture areas for varying dominance/mix of sources/atmospheric processes,
including concentration regimes that approach the standard.
> Existence of ongoing/planned advanced monitoring: availability of existing advanced
field studies with an established expert monitoring support infrastructure to increase the
chance of success, and leverage environmental measurement resources (serves
predominantly SIPs). However, "under-served" locations lacking a historically strong
support infrastructure would benefit from advanced measurements, and test the ability to
start up a sophisticated measurement program. When viewed in the aggregate as a group
of airsheds, a desirable balance of well-served, complemented with historically "under-
served" locations provide potential rewards toward expansion of widespread
measurement capability.
* Ongoing and planned health effects and exposure research studies: Studies that
benefit from Supersites measurements and foster greater coordination between
measurements, atmospheric scientists and health and exposure science communities. EPA
staff recommended that two sites in 1999 located in Atlanta, GA and Fresno/Bakersfield,
CA be established. Both of these locations are likely to exhibit high PM levels, are
associated with planned or ongoing major field sampling programs with expert technical
personnel, and represent diverse airsheds (e.g., east versus west; predominant high sulfate
versus high nitrate; predominant summer versus winter episodes). Moreover, it was
imperative that the initial sites offer a high success probability to increase the usefulness
of data early in the program.
These early needs included testing and intercomparisons of emerging sampling methods to
expedite application to other areas, data to support EPA's review of the PM standard and to
elucidate source-receptor relationships for SIPs. Atlanta and Fresno provided excellent
opportunities for conducting health effects and exposure research studies in the near and long
term. Furthermore, both locations served as models for coordinating across university groups,
industry and State/local agencies.
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 5 of 6
7.3 Key Planning Personnel
7.3.1 Program Manager
The Program Manager has the responsibility to make the final decision on the implementation of
the program. He has the following responsibilities:
* meet with the expert panel or/or CAS AC to review the progress of the program;
* direct OAQPS personnel listed below;
* review the progress of the program and assure that it is moving forward as recommended
by the expert panel.
7.3.2 Program Officer
The Program Officer is the person who performs the following planning activities:
identify program schedules;
writes the level of effort proposals;
oversees the implementation of program from a technical perspective.
7.3.3 Quality Assurance Coordinator
The QAC is responsible for the QA planning for the program. He is responsible for:
> overseeing the overall QA for the program;
> assess any data obtained from sources outside of the EPA that did not use approved
QAPPs;
7.4 Other Planning Activities
The following activities will facilitate the success of the program.
7.4.1 Communication
OAQPS has the overall responsibility for the Supersites program. As such, the agency must
assure that each agency within the program receives the goods, services and technical knowledge
to perform their duties. In addition, all parties must be made aware of events and deadlines. Part
of this is clear communication amongst all agencies. The following methods will be used to
impart information to ensure proper planning.
-------
Project: Supersites QMP
Element No:7
Revision No:2.0
Date: 9/13/01
Page 6 of 6
7.4.2 Tele-communications
Tele-conferencing is an extremely useful tool to impart information and ensure that the planning
process is moving forward. Drs. Les Hook and Sigurd Christensen lead the tele-conference
working group for the data processing issues. This working group consists of OAQPS, NARSTO
and Supersites data base managers. Drs. Hook and Christensen have guided this working group
by informing the group concerning the development of the NARSTO data archiving process, data
formatting, and metadata issues.
In addition, a working group formed in spring of 2000 to bring together the quality system for the
Supersites program. This QA working group is led by Mr. Dennis Mikel, who is the Supersites
QAC. The QA workgroup consists of OAQPS and ORIA and Supersites QA managers.
Dr. Joellen Lewtes, EPA ORD in Seattle, Washington leads the Organic Analysis Workgroup that
has been overseeing the issues related to Organic Carbon research and analysis.
Dr. Peter McMurray, led the Supersites Size Distribution Committee, which summarized and
compared the different measurements for particles and aerosols.
7.4.3 Internet
EPA supports and maintains the AMTIC web site on the Word Wide Web. The address for the
Supersite Program is http://www.epa.gov/ttn/amtic/ssproiec.html. Guidance, special
announcements and related documents are posted on this website. These documents can be
downloaded from the File Transfer Protocol (FTP) areas of the web site. In addition, the EPA
and all of the agencies involved in this program have electronic mail (email) capabilities, by
which information can be transmitted and all affected parties can be informed of meetings and
special events.
-------
Project: Supersites QMP
Element No:8
Revision No: 1.0
Date: 9/13/01
Page I of 2
8.0 Implementation of Work
Each Supersites group will develop a QAPP as described in Section 2.4.2 of this QMP. Since
each Supersites group has developed their own QAPPs, ultimately each agency is responsible for
the implementation of their program. This section will outline the individuals in each agency that
will be required to implement the work.
8.1 Implementation Roles
8.1.1 Principal Investigator
The Pis are responsible for all work to be performed. This includes:
* ensuring that work is being performed according the approved QAPP;
> development and implementation of procedures;
> development of special or "critical" techniques that might deviate from the normal good
laboratory practices; and
> ensuring that quality assured data are transferred to the NARSTO QSSC in a timely
manner.
8.1.2 Quality Assurance Managers
The QAMs oversee through internal TSAs and review of data, that procedures are being followed
as specified by the project QAPP and SOPs. In addition, the QA managers must also:
identify operations needing SOP's;
help prepare the procedures by writing and revising the QAPP;
review and approve SOP's before they are implemented;
provide new tools to the monitoring or laboratory staff that may enhance or increase the
productivity of the operation;
work with the PI in approving changes to procedures;
revise the QAPP to remove obsolete techniques and keep up-to-date procedures available
to field and laboratory staff;
verify that changes made in the field, through TSAs, are performed as prescribed in the
QAPP and SOP's;
perform (or through a contractor) TSA and DQA.
-------
Project: Supersites QMP
Element No: 8
Revision No: 1.0
Date: 9/13/01
Page 2 of 2
8.1.3 Individual Investigators
The investigators oversee their particular project or instrument. Any contractors or students
working with or for them must follow the specified SOPs and QAPPs. In addition, the
investigators must also:
> prepare the procedures and SOPs for their instrument;
> review and approve procedures before they are implemented;
> provide training to their students or contractors before the instruments are operated;
> work with the PI and QA manager in approving changes to procedures;
* validate collected data and work with Data Manager to process data for site data system
and archiving; and
* actively engage other researchers in analyzing data.
8.1.4 Data Manager
The Data Manager works with the project team to facilitate the data management, review, analysis,
and archiving process. In addition, the Data Managers must also:
* design, develop, and maintain a site data system suitable for site data acquisition,
validation, and analysis activities;
* assist investigators in collecting data and metadata, quality assuring data, formatting data
files, and providing data sets to the NARSTO QSSC for archiving; and
> participate in Supersites Data Management Working Group activities.
-------
Project: Supersites QMP
Element No: 9
Revision No: 1.0
Date: 9/13/01
Page 1 of 4
9.0 Data Quality Assessments
This section describes the quality-related activities necessary to support the Supersites program for
acquisition, validation, assessment, and reporting.
9.1 Program Assessment Techniques
Assessment is an all-inclusive term used to denote any of the following: TSAs, performance
audits, data quality assessments(DQAs), performance evaluations, Network Reviews and QAPP
reviews. Definitions for each of these activities can be found in the Glossary. Table 10.1 provides
information on the assessment type, assessor and frequency.
Table 9.1 Assessment Schedule
Assessment Type
Technical Systems Audits
Performance Audits
Network Review
Performance Evaluations
QAPP Review and Approval
Data Quality Assessment
Assessor
Project Level QA Managers
Project Level QA Manager
EPA- OAQPS-QA Coordinator
EPA -ORIA-NAREL Lab
EPA - OAQPS-QA Coordinator
Project Level QA Managers
Frequency
At the beginning of the project
At the beginning of the project/annually
after the first year
At the beginning of the program
Throughout the life of the program
Before projects begin
After data collection phase
9.1.1 Technical System Audit
The individual project QA managers for each project or their contractors will perform the TSAs.
QA mangers must not be involved in the routine data collection or analysis program. The results
of the audits will be submitted to the QA manager who will review the results and institute
corrective action if needed. The TSA results will then be incorporated into the QA final report
(QAFR) that will be submitted to OAQPS.
9.1.2 Performance Audits
Individual project QA managers for each project or their contractors will perform performance
audits. The audits should commence at the beginning of the program in order to detect any
problems with the instruments. Any deviations from the Measurement Quality Objectives (MQOs)
as stated in the project QAPP will be reported to the project QA manager. Corrective actions will
be performed as stated in the QAPP. Performance audits for the criteria pollutants are performed
annually. The results will then be incorporated into the QA final report (QAFR) that will be
submitted to OAQPS.
-------
Project: Supersites QMP
Element No: 9
Revision No: 1.0
Date: 9/13/01
Page 2 of 4
9.1.3 Network Review
The EPA-EMAD-MQAG office will perform the network assessment in the first year of the Phase
II project. The EPA QAC will request that all project QA managers or Pis submit electronic
photographs to the MQAG. These pictures will be compiled to create the Network Plan. Maps
will be generated through Geographic Information Systems (GIS) programs and will be saved to
computer hard drives.
9.1.4 Performance Evaluations
EMAD-MQAG is the lead agency for Performance Evaluations (PEs). The MQAG will be
working with the Office of Indoor Air (ORIA) National Air and Radiation Environmental
Laboratory (NAREL). NAREL will submit PEs to a majority of the laboratories during the first
year of the study. The results from the PEs will then be submitted to MQAG-QAC who will share
this information with the QA managers of each Supersite. This information will be included in the
QAFRs.
9.1.5 QAPP Review and Approval
EMAD-MQAG is the agency that will review and approval all QAPP. These will be submitted to
the MQAG office with enough time for a thorough review prior to project start. Comments will be
sent back to the submitting project for clarification and re-submission. When MQAG QA
personnel are satisfied with the QAPP, then final approval will be determined. Data collected prior
to QAPP approval must be flagged accordingly.
9.1.6 Data Quality Assessment
Each project QA manager is tasked with writing a QAFR for their individual project. The QAFR
will consist of the results of the performance and technical systems audits. In addition, the QA
Managers will include their assessment of the data collected as stated in their QAPPs. Precision,
Bias and accuracy data will be presented as stated in the QAPPs. The equations used in the
QAPPs will be utilized to access the data sets. This will be submitted within two years after the
end of the project to EMAD-MQAG. The QAFR will evaluate the data using the MQOs that are
stated in the QAPP. EMAD-MQAG will create a final program QAFR that will summarize the
quality of the Supersites experiments.
9. 2 Reports to Management
As stated in Section 9.1, each of the QA managers is required to submit a QAFR for each project.
The QAFR will evaluate the data using the MQOs that are stated in the QAPP. EMAD-MQAG
will create a final program QAFR that will summarize the quality of the Supersites experiments.
-------
Project: Supersites QMP
Element No: 9
Revision No: 1.0
Date: 9/13/01
Page 3 of 4
9.3 Planning, Training and Authority
The following sections will discuss process of planning, training and the authority of those whom
will be performing assessments.
9.3.1 Planning
The QMP is the first step towards having an effective planning process. This QMP will outline
how assessors for this program will plan, schedule and implement assessments. At the beginning
of the year, those who have been assigned to perform assessments will set out their tentative
schedule for assessments. This schedule will first be submitted to the Pis and QA managers, who
can modify schedule. After management approval, the schedule is submitted (by email) to the
agencies that will be assessed. Usually, one month before the assessment, the agency to be
assessed is notified by telephone of the exact dates and times. At this time, the assessment form
(TSA forms) is submitted to the agency to be assessed (in writing or via email). This allows the
agency the time to review the forms and gather the information needed to be presented to the
assessors. This has a two-fold objective: it allows those to be assessed knowledge of what will be
required and it can minimize the time that assessors are in the field and that managers and
scientists are away from their other duties.
9.3.2 Training
Training is essential to assessors in two ways: the assessor needs to understand the process by
which data are generated, without this knowledge the assessment may be inadequate, and in order
to communicate clearly with the agency that is being assessed, the assessor must be competent.
Training fills these needs. A part of training that is not seen or documented is the fact that those
chosen for assessment should have experience in the field in which they are assessing. Although
most QA criteria and theory are universal, understanding the process by being experienced in
working in that field is essential. It is the responsibility of the QA manger of each individual
Supersites project to provide training for the assessment team. If the project decides to hire a
contractor to perform assessments, the project PI and QA manager should have the confidence that
this contractor can fulfil their duties as described in the QAPP and this QMP.
9.3.3 Authority
All personnel that are chosen to conduct assessments to this program have the authority to do so
through the EPA. OAQPS has the overall responsibility and authority over this program. It
delegates this authority to perform assessments to all agencies/contractors that perform such duties.
All personnel in this capacity have the right and responsibility to:
-------
Project: Supersites QMP
Element No: 9
Revision No: 1.0
Date: 9/13/01
Page 4 of 4
identify problems;
Identify and cite noteworthy practices that may be shared with others to improve the quality of
their operations;
propose recommendations for resolving quality problems;
independently confirm implementation and effectiveness of solutions;
report these finding to the Supersites PI or QA manager.
Reports of assessments are discussed in section 9.2.
9.3.4 Disputes
Occasionally, findings in an assessment report may be disputed by the researcher/investigator
assessed. Any disputes that are announced should first be handled by the Supersites PI or QA
manager. If this fails to satisfy the situation, then OAQPS has the final authority to make a
decision concerning a dispute.
-------
Project: Supersites QMP
Element No: 10
Revision No: 1.0
Date: 9/13/01
Page 1 of 4
10.0 Quality Improvement
This section will outline planning and implementation procedures that will be employed for
improving the quality of the program. OAQPS and the QA managers from each Supersite have the
responsibility to improve the quality of the program over an unspecified period of time. There can
be no set dates on when this improvement can or will occur, however, OAQPS and the Supersites
QA managers will make every effort to improve the system during the life of the Supersites
program.
10.1 Quality Improvement Process
This section will outline the process flow of the quality improvement paradigm.
10.1.1 Assessment
The assessments that are planned for the Supersites program are detailed in section 9.1 of this
QMP. Once the assessment agency has completed an assessment, a report will be sent to the QA
manager of the Supersites, who will review the results. Depending on the assessment report and
the assessment scheme detailed in the QAPP, action may be deemed necessary and an assessment
loop will be initiated.
10.1.2 Assessment Report
The assessment report will state the who, what, where and when of the assessment. The report will
highlight the findings of the assessment. The QA manager will contact investigator where findings
may be outside of the MQOs or requirements of the QAPP.
10.1.3 Response
The investigator has the right to respond in writing, or email. All responses will be reviewed by
the assessor, QA manager and PI and will respond in kind. If any disputes arise from the
assessment this will be dealt as detailed in section 9.3.4 of this QMP. In addition, the EPA has
electronic mail (email) capabilities, by which information can be transmitted and all affected
parties can be informed of meetings and special events.
10.1.4 Final Assessment Report
The final assessment report (QAFR) will be sent to OAQPS and the assessed agency. This report
will highlight the findings of the assessment and recommendations.
-------
Project: Supersites QMP
Element No: 10
Re vision No: 1.0
Date: 9/13/01
Page 2 of 2
10.1.5 Review, Compilation and Analysis
Once OAQPS has received the final assessment reports from all of the Supersites, the EMAD
QAC will compile the information and analyze the data. Any disputes concerning the assessments
will be finalized at that time. The QAC will check the assessment reports to for outstanding issues
raised by the reports. If any have not been resolved, then the QAC may recommend flagging of
the data for a particular parameter.
10.1.6 OAQPS-QAFR
The QAFR will be the final report. The report will highlight the major findings of the Supersites
assessment and recommendations will be made in this report. In addition, the results from the
NAREL PE will also be included into the QAFR.
10.2 Quality Improvement Assurance
The OAQPS-QAFR and the QAFRs from each Supersite will assess the quality of the Supersites
data set. Once the QAFRs are issued, the report will be sent to NARSTO for archive into the
Supersites data area. Deficiencies that are not addressed will be noted in the reports. Any
university, government or public researcher will be advised to read the QAFRs before they use the
data in any analysis. This assures that the researchers understand the limitations of the data that
they will use and should act responsibly in presenting their results to the scientific community.
-------
Project: Speciation QMP
Glossary
Revision No: 1.0
Date: 9/13/01
Page 1 of 4
GLOSSARY OF QUALITY ASSURANCE AND RELATED TERMS
Activity An all-inclusive term describing a specific set of operations of related tasks to be performed,
either serially or in parallel (e.g., research and development, field sampling, analytical operations,
equipment fabrication), that, in total, result in a product or service.
Assessment The evaluation process used to measure the performance or effectiveness of a system and its
elements. As used here, assessment is an all-inclusive term used to denote any of the following: audit,
performance evaluation (PE), management systems review (MSR), peer review, inspection, or surveillance.
Audit (quality) A systematic and independent examination to determine whether quality activities and
related results comply with planned arrangements and whether these arrangements are implemented
effectively and are suitable to achieve objectives.
Audit of Data Quality (ADQ) A qualitative and quantitative evaluation of the documentation and
procedures associated with environmental measurements to verify that the resulting data are of acceptable
quality.
Certification The process of testing and evaluation against specifications designed to document, verify,
and recognize the competence of a person, organization, or other entity to perform a function or service,
usually for a specified time.
Collocated samples Two or more portions collected at the same point in time and space so as to be
considered identical. These samples are also known as field replicates and should be identified as such.
Computer program A sequence of instructions suitable for processing by a computer. Processing may
include the use of an assembler, a compiler, an interpreter, or a translator to prepare the program for
execution. A computer program may be stored on magnetic media and referred to as "software," or it may
be stored permanently on computer chips, referred to as "firmware." Computer programs covered in a
QAPP are those used for design analysis, data acquisition, data reduction, data storage (databases),
operation or control, and database or document control registers when used as the controlled source of
quality information.
Corrective action Any measures taken to rectify conditions adverse to quality and, where possible, to
preclude their recurrence.
Data Quality Assessment (DQA) The scientific and statistical evaluation of data to determine if data
obtained from environmental operations are of the right type, quality, and quantity to support their intended
use. The five steps of the DQA Process include: 1) reviewing the DQOs and sampling design, 2)
conducting a preliminary data review, 3) selecting the statistical test, 4) verifying the assumptions of the
statistical test, and 5) drawing conclusions from the data.
Data Quality Objectives (DQOs) The qualitative and quantitative statements derived from the DQO
Process that clarify study's technical and quality objectives, define the appropriate type of data, and specify
tolerable levels of potential decision errors that will be used as the basis for establishing the quality and
quantity of data needed to support decisions.
Data reduction The process of transforming the number of data items by arithmetic or statistical
calculations, standard curves, and concentration factors, and collating them into a more useful form. Data
reduction is irreversible and generally results in a reduced data set and an associated loss of detail.
Design The specifications, drawings, design criteria, and performance requirements. Also, the result of
deliberate planning, analysis, mathematical manipulations, and design processes.
-------
Project: Speciation QMP
Glossary
Revision No: 1.0
Date: 9/13/01
Page 2 of 4
Document Any written or pictorial information describing, defining, specifying, reporting, or certifying
activities, requirements, procedures, or results.
Environmental data Any parameters or pieces of information collected or produced from
measurements, analyses, or models of environmental processes, conditions, and effects of pollutants on
human health and the ecology, including results from laboratory analyses or from experimental systems
representing such processes and conditions.
Financial assistance The process by which funds are provided by one organization (usually
governmental) to another organization for the purpose of performing work or furnishing services or items.
Financial assistance mechanisms include grants, cooperative agreements, and governmental interagency
agreements.
Finding An assessment conclusion that identifies a condition having a significant effect on an item or
activity. An assessment finding may be positive or negative, and is normally accompanied by specific
examples of the observed condition.
Independent assessment An assessment performed by a qualified individual, group, or organization that
is not a part of the organization directly performing and accountable for the work being assessed.
Inspection The examination or measurement of an item or activity to verify conformance to specific
requirements.
Management Those individuals directly responsible and accountable for planning, implementing, and
assessing work.
Management system A structured, nontechnical system describing the policies, objectives, principles,
organizational authority, responsibilities, accountability, and implementation plan of an organization for
conducting work and producing items and services.
NARSTO Program NARSTO is a public/private alliance, whose membership spans
government, the utilities, industry, and academe throughout Mexico, the United States, and
Canada. The NARSTO mission is to plan, coordinate, and facilitate comprehensive, long-term,
policy-relevant scientific research and assessment of primary and secondary pollutant species
emitted, formed, transformed, and transported in the troposphere over the North American
continent. Member organizations support the mission through participation in workshops and
meetings, financial support, or contribution of in-kind resources. Atmospheric research and
assessment initiatives that support the mission may request to become NARSTO Technical
Programs. Technical Programs agree to follow certain quality assurance and data management
guidelines and send their data to the NARSTO Permanent Data Archive.
NARSTO QSSC The NARSTO Quality Systems Science Center (QSSC) at the Oak Ridge
National Laboratory assists NARSTO projects by consulting, reviewing quality system planning
documents, and advising about data management issues. The QSSC is responsible for archiving
data at the NARSTO Permanent Data Archive.
Organization A company, corporation, firm, enterprise, or institution, or part thereof, whether
incorporated or not, public or private, that has its own functions and administration.
-------
Project: Speciation QMP
Glossary
Revision No: 1.0
Date: 9/13/01
Page 3 of 4
Organization structure The responsibilities, authorities, and relationships, arranged in a pattern,
through which an organization performs its functions.
Procedure A specified way to perform an activity.
Process A set of interrelated resources and activities that transforms inputs into outputs. Examples of
processes include analysis, design, data collection, operation, fabrication, and calculation.
Project An organized set of activities within a program.
Quality The totality of features and characteristics of a product or service that bears on its ability to meet
the stated or implied needs and expectations of the user.
Quality Assurance (QA) An integrated system of management activities involving planning,
implementation, assessment, reporting, and quality improvement to ensure that a process, item, or service is
of the type and quality needed and expected by the client.
Quality Assurance Project Plan (QAPP) A formal document describing in comprehensive detail the
necessary quality assurance (QA), quality control (QC), and other technical activities that must be
implemented to ensure that the results of the work performed will satisfy the stated performance criteria.
The QAPP components are divided into four classes: 1) Project Management, 2) Measurement/Data
Acquisition, 3) Assessment/Oversight, and 4) Data Validation and Usability. Guidance and requirements
on preparation of QAPPs can be found in EPA QA/R-5 and QA/G-5.
Quality Control (QC) The overall system of technical activities that measures the attributes and
performance of a process, item, or service against defined standards to verify that they meet the stated
requirements established by the customer; operational techniques and activities that are used to fulfill
requirements for quality. The system of activities and checks used to ensure that measurement systems are
maintained within prescribed limits, providing protection against "out of control" conditions and ensuring
the results are of acceptable quality.
Quality improvement A management program for improving the quality of operations. Such
management programs generally entail a formal mechanism for encouraging worker recommendations with
timely management evaluation and feedback or implementation.
Quality management That aspect of the overall management system of the organization that determines
and implements the quality policy. Quality management includes strategic planning, allocation of
resources, and other systematic activities (e.g., planning, implementation, and assessment) pertaining to the
quality system.
Quality Management Plan (QMP) A formal document that describes the quality system in terms of the
organization's structure, the functional responsibilities of management and staff, the lines of authority, and
the required interfaces for those planning, implementing, and assessing all activities conducted.
Quality system A structured and documented management system describing the policies, objectives,
principles, organizational authority, responsibilities, accountability, and implementation plan of an
organization for ensuring quality in its work processes, products (items), and services. The quality system
provides the framework for planning, implementing, and assessing work performed by the organization and
for carrying out required quality assurance (QA) and quality control (QC).
Requirement A formal statement of a need and the expected manner in which it is to be met.
-------
Project: Speciation QMP
Glossary
Revision No: 1.0
Date: 9/13/01
Page 4 of 4
Round-robin study A method validation study involving a predetermined number of laboratories or
analysts, all analyzing the same sample(s) by the same method. In a round-robin study, all results are
compared and used to develop summary statistics such as interlaboratory precision and method bias or
recovery efficiency.
Self-assessment The assessments of work conducted by individuals, groups, or organizations directly
responsible for overseeing and/or performing the work.
Specification A document stating requirements and referring to or including drawings or other relevant
documents. Specifications should indicate the means and criteria for determining conformance.
Standard Operating Procedure (SOP) A written document that details the method for an operation,
analysis, or action with thoroughly prescribed techniques and steps and that is officially approved as the
method for performing certain routine or repetitive tasks.
Technical review A documented critical review of work that has been performed within the state of the
art. The review is accomplished by one or more qualified reviewers who are independent of those who
performed the work but are collectively equivalent in technical expertise to those who performed the
original work. The review is an in-depth analysis and evaluation of documents, activities, material, data, or
items that require technical verification or validation for applicability, correctness, adequacy, completeness,
and assurance that established requirements have been satisfied.
Technical Systems Audit (TSA) A thorough, systematic, on-site qualitative audit of facilities,
equipment, personnel, training, procedures, record keeping, data validation, data management, and
reporting aspects of a system.
Vendor Any individual or organization furnishing items or services or performing work according to a
procurement document or a financial assistance agreement. An all-inclusive term used in place of any of
the following: seller, contractor, subcontractor, fabricator, or consultant.
Verification Confirmation by examination and provision of objective evidence that specified
requirements have been fulfilled. In design and development, verification concerns the process of
examining a result of a given activity to determine conformance to the stated requirements for that activity.
-------
Project: Speciation QMP
Contact List
Revision No: 1.0
Date: 9/13/01
Page 1 of 1
Contact List
The following list is a compilation of contacts for the Supersites Program
Contact
Rich Scheffe
Dennis Mikel
Jeffrey West
Michael Clark
Paul Solomon
Marc Pitchford
Michael Jones
Les Hook
Sigurd Christensen
John Ondov
John Watson
David Allen
Constantinos Sioutas
KenDemerjian
Spyros Pandis
Jay Turner
Agency
OAQPS-EMAD-MQAG
EPA-NARSTO
NOAA (NARSTO)
ORIA-NAREL
ORD-Las Vegas
OAQPS-EMAD-Las Vegas
OAQPS-EMAD-MQAG
DOE (NARSTO)
DOE (NARSTO)
University of Maryland
Desert Research Institute
University of Texas
University of California
University of Albany
Carnegie Mellon University
Washington University
Phone Number
919-541-4650
919-541-5511
919-541-4635
334-270-7069
702-798-2280
702-798-0432
919-541-0528
865-241-4846
865-574-7394
301-405-1859
775-674-7046
512-475-7842
213-740-0603
518-437-8705
412-268-3531
314-935-5480
Email
scheffe.richfgjepa. gov
mikel.denniskfgtepa. gov
west.j effrey epa. gov
clark.michaelfgtepa. gov
solomon. paul@epa. gov
marcp(g).dri.edu
jones. mikefSlepa. gov
Hooklafgtornl. gov
swc@ornl.gov
i ondovfg) warn, umd. edu
Johw@,dri.edu
Allen@che.utexas.edu
SioutasfSlalmaak.usc.edu
Kldfgjasrc. cestm.albanv.edu
Spyros@.andrew. emu, edu
Jrturner@,seas. wustl.edu
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date: 9/13/01
Appendix A - Procurement Policy Notice
For Contracting Officer's Representatives
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date: 9/13/01
Procurement Policy Notice
For Contracting Officer's Representatives
1. Background
The Federal Acquisition Regulations, Title 48 of the Code of Federal Regulations, was recently
amended to address contract quality systems requirements on a government-wide basis. The new
FAR clause at 52.246-11, Higher-Level Quality Requirement, allows a Federal agency to select a
voluntary consensus standard as the basis for its quality requirements for contracts and allows
tailoring of the standard to more effectively address specific needs or purposes. Based on this
FAR clause, EPA has selected ANSI/ASQC E4, Specifications and Guidelines for Quality
Systems for Environmental Data Collection and Environmental Technology Programs, as the
basis for its environmental quality requirements and has tailored this standard to ensure that
contractors demonstrate conformance to this national standard.
Due to these changes, 48 CFR 1546, a quality regulation that applies only to EPA, will be
removed from the Code of Federal Regulations. The tailoring language allowed by 52 CFR 246-
11 and pertinent requirements in 48 CFR 1546 will be included in the EPA Directive 1900,
Contracts Management Manual. This procurement policy notice is being issued to ensure an
orderly transition from 48 CFR 1546 to EPA Directive 1900 and contains tailoring language
allowed by 52 CFR 246-11. It is in effect until the revisions to Directive 1900 are completed.
2. Application
This procurement policy notice applies to all Contracting Officer's Representatives, that is, all
Project Officers, Deputy Project Officers, Regional Project Officers, Zone Project Officers,
Delivery Order Project Officers, Work Assignment Managers, and Task Order Managers.
This procurement policy notice applies to all solicitations; task orders, work assignments, and
other statements of work for contracts (including simplified procurement acquisitions) that
involve environmentally related measurements (i.e., the collection and use of environmental
data1 and the design, construction, and operation of environmental technologies). Examples of
environmentally related measurements are contained in Attachment 1.
Environmental data are defined as any measurements or information that describe environmental
processes, location, or conditions; ecological or health effects and consequences; or the performance of
environmental technology. For EPA, environmental data include information collected directly from measurements,
produced from models, and compiled from other sources such as data bases or the literature.
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date: 9/13/01
3. General Requirements
Although this procurement policy notice applies solely to contracts, EPA requires that all
recipients of funds (i.e., contractors, grantees, etc.) for work involving environmentally-related
measurements comply with the American National Standard ANSI/ASQC E4-1994,
Specifications and Guidelines for Quality Systems for Environmental Data Collection and
Environmental Technology Programs. To demonstrate conformance to this standard, EPA
requires all recipients submit two types of documentation:
1. Documentation of the organization quality system (usually called a Quality
Management Plan), and/or
2. Documentation of the application of quality assurance (QA) and quality control
activities to a project-specific effort (usually called a Quality Assurance Project
Plan).
Use of existing quality system documentation, such as documentation that a company is ISO
9000 certified, may be acceptable alternatives.
For small contracts, these two documents may be combined into a single document that describes
the organization's quality system and the application of this system to the work performed under
the contract. This can only be done with permission of the EPA QA Manager who will identify
which elements should be addressed in this combined document.
Some contracts may cover activities of a program that are to be conducted at multiple locations
or over a long period of time; for example, a large monitoring program that uses the same
methodology at different locations. In this case, a Programmatic Quality Assurance Project Plan
may be used to describe, in a single document, the general, common activities that are not site- or
time-specific but are applied throughout the program. Project-specific information is then added
to the approved Programmatic Quality Assurance Project Plan on a project-specific basis.
4. Directions for Pre-Award and Post-Award Activities
STEP 1. After consultation with the QA Manager (or the appropriate QA personnel2),
complete the QA Review Form (as described in Section 2.5 of the Contracts
Management Manual) and obtain the concurrence signature of the QA Manager.
Appropriate QA personnel are defined in each EPA organization's Agency-approved Quality Management
Plan. For simplicity, the use of the term QA Manager will refer to both the QA Manager and other approved QA
personnel.
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date: 9/13/01
If QA requirements are not applicable to the procurement (indicated on the QA
Review Form), the remaining Steps do not apply.
STEP 2. With the assistance of the QA Manager, determine what quality standards apply.
Generally, ANSI/ASQC E4-1994 applies to the majority of EPA's work;
however, standards other than ANSI/ASQC E4-1994 may also apply.
STEP 3. If ANSI/ASQC E4-1994 applies, identify (with the assistance of the QA
Manager) whether the contract work will consist of:
A. A single project,
B. Multiple projects with different activities, or
C. Multiple projects with similar activities.
A If the contract work consists of a single project, you must require one of the
following:
1.Before Award:
After Award:
2. Before Award:
After Award:
3. Before Award:
After Award:
4. Before Award:
After Award:
A Quality Management Plan
A Quality Assurance Project Plan for the contract
(Note: These are the default requirements.)
QA Manager-specified documentation3
A Quality Management Plan and a Quality
Assurance Project Plan for the contract
QA Manager-specified documentation3
A Joint Quality Management Plan/Quality
Assurance Project Plan for the contract
A Joint Quality Management Plan Quality
Assurance Project Plan for the contract
None
QA Manager-specified documentation is defined in an EPA organization's Agency approved Quality
Management Plan. This documentation must be consistent with Agency requirements defined in EPA Order 5360
(May 2000).
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date: 9/13/01
B. If the contract work consists of multiple projects with different activities.,
you must require one of the following:
1. Before Award: A Quality Management Plan
After Award: A Quality Assurance Project Plan for each
applicable project
(Note: These are the default requirements.)
2. Before Award: QA Manager-specified documentation3
After Award: A Quality Management Plan and a Quality
Assurance Project Plan for each applicable project
C. If the contract work consists of multiple projects with similar activities,
you must require one of the following:
1. Before Award: A Quality Management Plan
After Award: A Quality Assurance Project Plan for each
applicable project
(Note: These are the default requirements.)
2. Before Award: A Quality Management Plan
After Award: A Programmatic Quality Assurance Project Plan for
the program (contract) and a project-specific
supplement to the Programmatic Quality Assurance
Project Plan for each applicable project
3. Before Award: A Quality Management Plan and a Programmatic
Quality Assurance Project Plan for the program
(contract)
After Award: A project-specific supplement to the Programmatic
Quality Assurance Project Plan for each applicable
project
For each of the three cases (single project, multiple projects with different
activities, or multiple projects with similar activities), the default requirements are
listed as the first option (1). These requirements should be used unless the QA
Manager concurs otherwise.
STEP 4. For each type of documentation identified in STEP 4, identify (with the assistance
of the QA Manager) whether the documentation should be prepared in accordance
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date: 9/13/01
with the standard EPA requirements [i.e., EPA Requirements for Quality
Management Plans (QA/R-2) and EPA Requirements for Quality Assurance
Project Plans (QA/R-5)} or whether other EPA-approved equivalent requirements
will be used. The standard EPA requirements should be used unless the QA
Manager concurs otherwise.
STEP 5. If additional standards apply besides ANSI/ASQC E4 -1994, identify (with the
assistance of the QA Manager) what documentation is required to determine
conformance to these standards.
STEP 6. Provide the Contracting Officer with a list of documentation required before and
after award (from cases A, B, and C in STEP 3) and if applicable, a list of any
equivalent requirements to be used (STEP 4), and the Title, Numbering, Date, and
any documentation required to demonstrate conformance for any additional
standards (STEP 5).
The information that must be submitted to the Contracting Officer is contained in
Attachment 2. It is recommended that you complete this form and provide it to
the Contracting Officer with the QA Review Form (STEP 1).
STEP 7. After award of the contract, if the work consists of multiple projects (cases B and
C in STEP 3), complete a QA Review Form and Section 3 of Attachment 2 for
each statement of work (e.g., work assignment, delivery order, task order).
Include in each applicable statement of work the requirement to submit the quality
documentation needed after contract award. For example, if a project-specific
supplement to the Programmatic Quality Assurance Project Plan is required for
the project described in the statement of work, you must incorporate the
requirement to develop this document into the statement of work.
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date: 9/13/01
5.0 Example of Activities involving Environmentally-Related Measurements
The following are some examples that involve environmentally-related measurements:
Activities that collect data to establish/determine the states/conditions of
environmental or ecological systems and the health of human populations;
Activities that collect data to establish the ambient conditions in air, water, sediments,
and soil in terms of physical, chemical, radiological, or biological characteristics;
Activities that collect data to establish/categorize radioactive, hazardous, toxic, and
mixed wastes in the environment and to establish their relationships with and/or
impact on human health and ecological systems;
Activities that monitor and quantify the waste and effluent discharges to the
environment from processes and operations (e.g., energy generation, metallurgical
processes, chemicals production), during either normal or upset conditions (i.e.,
operating conditions that cause pollutant or contaminant discharges);
Activities that use environmental data to develop environmental technology for
pollution prevention, pollution control, waste treatment, storage, and disposal, and
waste remediation;
Activities that use environmental data in mapping environmental process and
conditions, and/or human health risk data, etc. (e.g., geological information system);
Activities that generate data from the evaluation of environmental technology used
for pollution prevention; pollution control; waste treatment, storage, and disposal; and
waste remediation;
Activities that generate/collect data to support enforcement and/or compliance
monitoring efforts;
Activities that collect/generate data for the evaluation and/or demonstration of
environmental technology (e.g., treatability and pilot studies);
Activities that investigate and collect data to determine chemical, biological, physical,
or radioactive constituents in environmental and ecological systems, and their
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date: 9/13/01
behavior and associated interfaces in those systems, including exposure assessment,
transport, and fate;
Activities that collect and/or generate data from the development and evaluation of
methods for use in the collection, analysis, and use of environmental data;
Activities that involve the development, evaluation, and use of computers or
mathematical models (and their input data) to characterize environmental processes or
conditions;
Activities that use secondary data (i.e., environmental data that were collected for
other purposes or obtained from other sources, including literature, industry surveys,
compilations from computerized data bases and information systems) for the
development and/or evaluation of computerized or mathematical models of
environmental processes and conditions, and collect/generate data from the process;
and
Activities that collect and/or use environmental data for monitoring/addressing
concerns over the occupational health and safety of personnel in EPA facilities (e.g.,
indoor air quality measurements) and in the field (e.g., chemical dosimetry, radiation
dosimetry).
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date:08/08/01
6.0 Contracts Clause and Tailoring Language Form
Use this form to provide direction to the Contracting Officer on the quality assurance
activities that are required in your solicitation and contract.
1. List any additional quality standards besides Specifications and Guidelines for
Quality Systems for Environmental Data Collection and Environmental
Technology Programs (ANSI/ASQC E-4)
Title:
Numbering:
Date:
Documentation required to determine conformance:
2. a. Check all required documentation required before award of contract:
Documentation
Quality Management Plan
Joint Quality Management
Plan/Quality Assurance
Project PI an
Programmatic Quality
Assurance Project Plan
Other Equivalent:
Specifications
EPA Requirements for Quality Management
Plans (QA/R-2) [dated I4
EPA Requirements for Quality Management
Plans (QA/R-2) [dated 1 and EPA
Requirements for Quality Assurance Project
Plans (QA/R-5) [dated 1
EPA Requirements for Quality Management
Plans (QA/R-2) [dated 1 and EPA
Requirements for Quality Assurance Project
Plans (QA/R-5) [dated 1
[Insert specification]
b. If the standard specifications do not apply, identify equivalent
specifications:
3. a. Select all documentation required after award of contract either at time of
award or upon issuance of a statement of work:
Note: we will fill in this date once the Federal Register Notice is published.
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date:08/08/01
Documentation
Specifications
Due After
Quality Management Plan
EPA Requirements for Quality
Management Plans (QA/R-2)
[dated ]
Award of
contract
Joint Quality Management
Plan/Quality Assurance
Project PI an
EPA Requirements for Quality
Management Plans (QA/R-2)
[dated ] and EPA
Requirements for Quality
Assurance Project Plans (QA/R-5)
[dated ]
Award of
contract
Contract Quality
Assurance Project Plan
EPA Requirements for Quality
Assurance Project Plans (QA/R-5)
[dated ]
Award of
contract
Programmatic Quality
Assurance Project Plan
EPA Requirements for Quality
Management Plans (QA/R-2)
[dated ] and EPA
Requirements for Quality
Assurance Project Plans (QA/R-5)
[dated ]
Award of
contract
Quality Assurance Project
Plan for each applicable
project
EPA Requirements for Quality
Assurance Project Plans (QA/R-5)
[dated ]
Issuance of
statement of
work
Project-specific
supplement to
Programmatic Quality
Assurance Project Plan
EPA Requirements for Quality
Assurance Project Plans (QA/R-5)
[dated ]
Issuance of
statement of
work
Other Equivalent:
[Insert specification]
[Select one]
award of
contract
issuance of
statement of
work
-------
Project: Speciation QMP
Appendix A
Revision No: 1.0
Date:08/08/01
If the standard specifications do not apply, identify equivalent
specifications.
-------
TECHNICAL REPORT DATA
(Please read Instructions on reverse before completing)
1. REPORT NO.
EPA-454/R01-011
2
AND SUBTITLE
Quality Management Plan for the PM Supersites Program
7. AUTHOR(S)
Dennis Mikel, Michael Papp, Les Hook, Sigurd Christensen
9. PERFORMING ORGANIZATION NAME AND ADDRESS
U.S. Environmental Protection Agency
Office of Air Quality Planning and Standards
Research Triangle Park, NC 2771 1
12. SPONSORING AGENCY NAME AND ADDRESS
Director
Office of Air Quality Planning and Standards
Office of Air and Radiation
U.S. Environmental Protection Agency
Research Triangle Park, NC 2771 1
3. RECIPIENT'S ACCESSIONNO.
5. REPORT DATE
09/01
6. PERFORMING ORGANIZATION CODE
8. PERFORMING ORGANIZATION REPORT
NO.
1 0. PROGRAM ELEMENT NO.
1 1 . CONTRACT/GRANT NO.
13. TYPE OF REPORT AND PERIOD COVERED
1 4. SPONSORING AGENCY CODE
EPA/200/04
15. SUPPLEMENTARY NOTES
16. ABSTRACT
The Quality Management Plan outlines the management structure for the Particle Matter Supersites Program.
The guidance document gives details on how the Supersites Program will be implemented. The Supersites
program is a multi-year research program that draws upon EPA OAQPS and Office of Indoor Air and
university research groups throughout the country. This Plan outlines each agency's responsibilities and
authority.
KEY WORDS AND DOCUMENT ANALYSIS
a. DESCRIPTORS
Air Quality Monitoring
Quality Assurance
18. DISTRIBUTION STATEMENT
Release Unlimited
b. IDENTIFIERS/OPEN ENDED TERMS
Air Pollution Control
19. SECURITY CLASS (Report)
Unclassified
20. SECURITY CLASS (Page)
Unclassified
c. COSATI
Field/Group
21. NO. OF
PAGES
22. PRICE
------- |