U.S. Environmental Protection Agency, Region 8
Quality Management Plan
July 30, 2020
<
^ฃDS7"X
A
3B
ฆ*>
z
LU
0
U.S. Environmental Protection Agency, Region 8
1595 Wynkoop St.
Denver, CO 80202
-------
[Page intentionally left blank.]
-------
US EPA Region 0
Quality Management Plan
July 30,2020
Page i
APPROVALS
APPROVALS FOR AGENCY
\/AI If^WM Digitally signed by
VMUOniN VAUGHN NOGA
k | /-\ p* a Date: 2020.08.19
INUOM 13:03:23-04'00'
Vaughn Noga
Chief Information Officer
lx ATIIPPIMP Digitally signed by
t\j-\ i ni_r\M\ii_ katherinechalfant
pUAl CAMT Date 2020 08 12
UMALrAIN I 21 27 02-04001
Katherine Chalfant
Acting Division Director
Enterprise Quality Management Division
Date Signed
Date Signed
-------
US EPA Region 0
Quality Management Plan
July 30,2020
Page ii
APPROVALS FOR REGION 8
Digitally signed by
GREGORY SOPKIN ฐaRtf ฐ0*07ฐN
16:26:04 -06'00'
Gregory Sopkin
Regional Administrator
HPRDA Digitally signed by
DEBRA THOMAS
THOIWIAQ Date: 2020.08.12
I nWIVIMO 15:22:46 -06'00'
Debra H. Thomas
Deputy Regional Administrator
QAMHDA Digitally signed by
0/-\l\IUr\/-\ SANDRA SPENCE
S P E N 0 E Date 2020'08'10
12:37:06 -06'00'
Sandra D. Spence
Director
Laboratory Services and Applied Sciences Division
I imha Digitally signed by LINDA
LINDA HIMMELBAUER
HIMMELBAUER^ff
Linda Himmelbauer
Regional Quality Assurance Manager
Quality Assurance Branch
Date Signed
Date Signed
Date Signed
Date Signed
Digitally signed by
OUIUdUC, Goldade, Mary
IV/lar\/ Date: 2020.08.06
IVIdly 12:16:37-06'00'
Mary Goldade Date Signed
Deputy Regional Quality Assurance Manager
Quality Assurance Branch
-------
Quality Management Plan
US EPA Region 0
Quality Management Plan
July 30,2020
Page iii
Table of Contents
1.0 QUALITY SYSTEM FOUNDATION 1
1.1 Description of the Quality System and Implementation 1
1.2 Mission, Policy and Scope 2
1.3 Definitions of Environmental Data & Environmental Technology 4
1.4 Plan, Do, Check, Act Model in Region 8's Quality System 6
2.1 Quality System Organizational Structure 7
2.2 Region 8 Organization 8
2.2.1 Office of the Regional Administrator 8
2.2.2 Laboratory Services and Applied Science Division (LSASD) 9
2.2.3 Enforcement and Compliance Assurance Division (ECAD) 9
2.2.4 Mission Support Division (MSD) 9
2.2.5 Air and Radiation Division (ARD) 10
2.2.6 Water Division (WD) 10
2.2.7 Superfund Emergency Management Division (SEMD) 10
2.2.8 Land, Chemicals and Redevelopment Division (LCARD) 11
2.2.9 Region 8 Science Council 11
2.3 Responsibilities of EPA Management and Staff 12
2.3.4 Individual Quality System Roles, Responsibilities and Authorities 12
2.4 State Partnerships Roles and Responsibilities 15
2.4.1 Quality Assurance Requirement for State Agencies 15
2.4.2 Requirements for Extending Delegation Authority to States 16
2.4.3 Grantee QA Annual Reviews & Reporting Requirements 17
2.4.4 Region 8 QA Program State Oversight 18
2.5 Tribal Nations Roles and Responsibilities 18
2.5.1 Quality Assurance Requirements for Tribes 19
2.5.2 Requirements for Extending Delegation Authority to Tribes 20
2.5.3 Grantee QA Annual Reviews & Reporting Requirements 22
2.2.4 Region 8 QA Program Tribal Oversight 22
-------
US EPA Region 0
Quality Management Plan
July 30,2020
Page iv
3 SCIENTIFIC INTEGRITY POLICY 23
3.1 Scientific Integrity Policy Overview 23
3.2 Policy Applicability 23
3.3 Policy Implementation 24
3.3.1 Scientific Integrity Committee 24
4 POLICIES FOR FIELD AND LABORATORY COMPETENCY AND ACCREDITATION
24
4.1 Laboratory Competency at Agency Laboratories 2 5
4.2 Organizations Generating Environmental Measurement Data for Agency-funded
Contracts and Agreements 26
4.2.1 Policy Overview 26
4.2.2 Policy Applicability 2 6
4.2.3 Policy Implementation in Region 8 2 6
5 POLICY AND PROCEDURES FOR PROTECTION OF HUMAN RESEARCH SUBJECTS
IN EPA CONDUCTED OR SUPPORTED RESEARCH 29
6 QA FIELD ACTIVITIES PROCEDURE (QAFAP) 29
6.1 Applicability/Scope 30
6.2 Definition of Field Activities 30
6.3 Definition of Field Activities 30
6.4 Training on the QA Field Activities Procedure 32
6.5 Field Implementation Team (FIT) 32
7 PLANNING (PLAN) 33
7.1 Region-wide Planning 33
7.1.1 Internal Strategic Planning 33
7.1.2 External Planning 34
7.2 Program-specific Planning 34
7.3 Project-level Planning 35
7.4 QA Documents in Planning 36
7.5 Quality Management Plans (QMPs) 37
7.5.1 Annual QMP Reviews 38
7.5.2 QMP Tracking System 39
7.6 Quality Assurance Project Plans (QAPPs) 39
7.6.1 Annual QA Document Reviews 41
7.6.2 QA Document Review Tracking System 42
7.7 Uniform Federal Policy for QAPPs 42
7.8 Standard Operating Procedures (SOPs) 43
7.9 Definition and Use of Existing Environmental Data 44
7.10 Systematic Planning Using Data Quality Objectives 45
7.11 Conceptual Models (Conceptual Site Models) 47
7.12 Policy for Geospatial Data 48
-------
US EPA Region 0
Quality Management Plan
July 30,2020
Page v
7.13 Peer Review in Project Planning 49
7.14 Information Quality Guidelines and Pre-Dissemination Review in Project
Planning 50
8 IMPLEMENTATION (DO) 51
8.1 Implementation of Work Processes 51
8.2 Authorities Delegated to the Regional QA Manager (RQAM) 51
8.3 QA Document Approval Authorities 52
8.3.1 RQAM QA Document Approval Authority 52
8.3.2 Delegated QA Approving Officer (DAO) Authority and Responsibilities 54
9 EVALUATION & ASSESSMENT (CHECK) 57
9.1 Assessments 58
9.2 Assessment Tools and Other Resources 59
9.3 Data Quality Indicators 61
9.4 Data Verification & Validation 61
9.5 Data Quality Assessment 62
9.6 Annual Review of QA Documents 63
9.6.1 Annual Review of QMPs 63
9.6.2 Annual Review of QA Project Plans and other QA Documents (QAPPs, SAPs,
FSPs) 64
9.7 Quality Systems Assessments 65
9.7.1 Delegated QA Approving Officer Program 66
9.7.2 States 66
9.7.3 Technical Systems Audits 66
9.8 Information Quality Guidelines and Pre-Dissemination Review Procedures 67
9.8.1 Scope and Applicability of Information dissemination 67
9.8.2 Policy Implementation for IQGs and Pre-Dissemination Reviews 68
9.8.3 Clearance or Office Review Procedures 69
9.8.4 Print, Web and Promotional Product Review Procedures 73
9.8.5 Disclaimers for Disseminated Information 73
9.8.6 Requests for Correction and Reconsideration 75
9.9 Peer Review Policies and Procedures 76
9.9.1 Policy Overview 77
9.9.2 Definitions 77
9.9.3 Region 8 Peer Review Coordinator 79
9.9.4 Other Key Roles and Responsibilities 80
9.9.5 Peer Review Policy Implementation 83
-------
US EPA Region 0
Quality Management Plan
July 30,2020
Page vi
Work ProductPlanning Stage 85
9.9.6 Documentation and Peer Review Record 91
9.9.7 Peer Review Tracking and ReportingScience Inventory 92
9.9.8 National Peer Review Advisory Group 94
9.9.9 Training for Peer Review Requirements 94
10 RESPONSE & QUALITY IMPROVEMENT (ACT) 95
10.1 Quality Improvement 95
10.2 Corrective Actions 96
10.3 Dispute Resolution 96
11 PERSONNEL QUALIFICATIONS, TRAINING and CCOMPETENCE 97
11.1 Core QA Training 98
11.2 Delegated QA Approving Officer Training 99
11.3 Existing Data Training 100
12 PROCUREMENT AND FINANCIAL ASSISTANCE 100
12.1 Procurement of Items & ServicesAcquisitions 101
12.1.1 Statements of Work (SOWs) 101
12.1.2 QA Review Form (QARF) 102
12.1.3 QA Requirements for Solicitations 105
12.1.4 QA Requirements for Contract Actions After Award 107
12.2 Grants and Cooperative Agreements 108
12.2.1 Procedures in IGMS to Document QA Requirements 108
12.3 Interagency Agreements (IAs) 110
13 DOCUMENTATION AND RECORDS MANAGEMENT Ill
14 COMPUTER HARDWARE AND SOFTWARE 113
15 REFERENCES 115
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
vii
Table of Figures and Tables
Figures
Figure 1 Organizational Structure for EPA Region 8's Quality System
Figure 2 Project Life Cycle
Figure 3 QA Document Types
Figure 4 QA Document Approval Authority Hierarchy
Figure 5 QARF Approval Flow Diagram for Pre-Award (Solicitation) Contract
Actions
Figure 6 QARF Approval Flow Diagram for Post-Award Contract Actions
Figure 7 QA Requirements in the Grants Process Flow
Text Boxes
Text Box 1 Region 8 Quality Policy
Text Box 2 Definitions for Environmental Data and Environmental Technology
Text Box 3 Quality Assurance is Everyone's Responsibility
Text Box 4 Approved QAPPs are Required Prior to Initiating Work
Text Box 5 Existing Environmental Data Definition
Text Box 6 Special Considerations for Enforcement Actions
Tables
Table 1
Table 2
Table 3
Current Representatives for Key Positions Identified in the QMP
Assessment Tools and Application
Example Work Products for Peer Review Categories 1-3
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
viii
Acronyms
AA
Assistant Administrator
AMU
Acquisitions Management Unit
ANSI
American National Standard Institute
AOs
Administrative Orders
AOCs
Administrative Orders on Consent Decrees
ASQ
American Society for Quality Control
BRAC
Base Realignment and Closure Act
CERCLA
Comprehensive Environmental Response, Compensation and Liability Act
CFR
Code of Federal Regulations
CIO
Chief Information Officer
CLP
Contract Laboratory Program
CMM
Contracts Management Manual
CO
Contract Officer
COI
Conflict of Interest
COR
Contracting Officer's Representative
CRADA
Cooperative Research and Development Agreement
CSM
Conceptual Site Model
DAO
Delegated QA Approving Officer
DO
Delivery Order
DoD
Department of Defense
DOE
Department of Energy
DPM
Designated Project Manager
DQA
Data Quality Assessment
DQIs
Data Quality Indicators
DQOs
Data Quality Objectives
Deputy RA
Deputy Regional Administrator
EPA
United States Environmental Protection Agency
FAR
Federal Acquisition Regulation
FEM
Forum on Environmental Measurements
FGDC
Federal Geographic Data Committee
FSP
Field Sampling Plan
FUDS
Formerly Used Defense Sites
FUSRAP
Formerly Used Sites Remedial Action Program
GIS
Geographic Information Systems
GPRA
Government Performance Results Act
GPS
Global Positioning System
HISA
Highly Influential Scientific Assessment
HSO
Human Subjects Officer
HSRRO
Human Subjects Research Review Official
IGMS
Integrated Grants Management System
IQG
Information Quality Guidelines
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
ix
IRB Institutional Review Board
IRM Information Resource Management
ISI Influential Scientific Information
ISP Information Systems Program
MSR Management System Review
NARA National Archives and Records Administration
NEPPs National Environmental Performance Partnership Systems
NSDI National Spatial Data Infrastructure
OEI Office of Environmental Information
OMB Office of Management and Budget
ORD Office of Research and Development
OSC On-Scene Coordinator
OSWER Office of Solid Waste and Emergency Response
PDCA Plan, Do, Check, Act Quality Model
PDR Pre-Dissemination Review
PDRGs Pre-Dissemination Review Guidelines
PI Principal Investigator
PIN Procurement Initiation
PO Project Officer
PPA Performance Partnership Agreement
PPG Performance Partnership Grant
PR Purchase Request
PRAG Peer Review Advisory Group
PRP Potentially Responsible Party
PT Performance Testing
PWS Performance Work Statement
QA Quality Assurance
QAARWP Quality Assurance Annual Report and Work Plan
QAFAP Quality Assurance Field Activities Procedure
QAM Quality Assurance Manager
QAPP Quality Assurance Project Plan
QARF Quality Assurance Review Form
QC Quality Control
QMP Quality Management Plan
QSAs Quality System Assessments
RA Regional Administrator
RCRA Resource Conservation and Recovery Act
RFP Request for Proposal
RPs Responsible Parties
RPM Remedial Project Manager
RQAM Regional Quality Assurance Manager
SAM Site Assessment Manager
SAP Sampling and Analysis Plan
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
x
SIO
Senior Information Official
SOP
Standard Operating Procedure
sow
Statement of Work
SPM
State Program Manager
STPC
Science and Technology Policy Council
TEP
Technical Evaluation Panel
TMS
Technical and Management Services
TO
Task Order
TPM
Tribal Program Manager
TSA
Technical Systems Audit
UFP-QAPP
Uniform Federal Policy for Quality Assurance Project Plan
UNC
University of North Carolina
WA
Work Assignment
WAM
Work Assignment Manager
WP
Work Plan
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
1
1 QUALITY SYSTEM FOUNDATION
This section of the Region 8 Quality Management Plan contains the basic introduction of a
quality system as well as the roles and responsibilities for implementation of the Quality System
within Region 8. Closely linked to its implementation is the organizational structure, which
describes how various groups and/or individuals work together within its structure to accomplish
the Region 8 mission and Quality Policy.
1.1 Description of the Quality System and Implementation
EPA Order CIO 2105.0, Policy and Program Requirements for the Mandatory Agency-Wide
Quality System, requires that each EPA Regional Office develop and document a quality system
to ensure that environmental data used to support Agency decisions are scientifically sound,
defensible and of known and documented quality and are of appropriate and adequate quality for
the intended purpose.
The EPA Region 8 Quality System described in this Quality Management Plan (QMP) has been
planned, developed, and documented in accordance with the following:
EPA Order CIO 2105.0; Policy and Program Requirements for the Mandatory Agency-
wide Quality System, May 2000;
EPA Quality Manual for Environmental Programs, CIO 2105-P-01-0, May 5, 2000.
U.S. Environmental Protection Agency Quality Assurance Field Activities Procedure,
CIO 2015-P-02.0;
U.S. Environmental Protection Agency Quality Policy, EPA Order CIO 2106.0, October
20, 2008; and
U.S. Environmental Protection Agency Procedure for Quality Policy, CIO 2106-P-01.0,
October 20, 2008.
The EPA Quality System
The EPA's quality system is based on the Policy and Program Requirements for the Mandatory
Agency-wide Quality System, Order CIO 2105.0, (hereafter called the Order). Specifications for
implementing the Order in EPA organizations are given in EPA Quality Manual for
Environmental Programs, CIO 2105-P-01-0. Specifications for extramural organizations
receiving Agency-funded assistance agreements (grants) are given in Title 2 Code of Federal
Regulations Part 1500.11 (2 CFR 1500.11). Specifications for Agency-funded acquisitions
(contracts) are provided 48 CFR Part 46. Specifications for agreements with other Federal
Agencies must follow applicable Federal Regulations for each Agency and be negotiated into
each agreement. The EPA Requirements for Quality Management Plans (EPA QA/R-2) is
applicable for all organizations producing or using environmental information or technologies.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
2
Although the EPA builds Quality Assurance into its programs, it typically also has a QA Branch
or Program Office in each organization that is chartered with monitoring implementation and
effectiveness of the QA Program.
The EPA quality system complies with Quality Management Systems for Environmental
Information and Technology Programs - Requirements with Guidance for Use, ASQ/ANSIE4-
2014 (formerly Specifications and Guidelines for Quality Systems for Environmental Data
Collection and Environmental Technology Programs, ANSI/ASQC E4-2004, which was
developed by the American National Standards Institute (ANSI) and the American Society for
Quality (ASQC). According to ANSI/ASQ E4, assessments of environmental programs should
be conducted periodically, and the assessment findings should be evaluated to measure the
effectiveness of the programs' quality systems. Because Agency decisions rely on the quality of
environmental data, it is a requirement that the effectiveness of the quality systems that support
the collection and use of environmental data be periodically assessed.
The QMP describes Regional processes and policies for ensuring the quality of environmental
data collected, produced and/or used by or for EPA; other Federal, State, Tribal and local
partners under interagency agreements and financial assistance agreements; contractors funded
by EPA; regulated entities; and potentially responsible parties. In addition, this QMP provides
policy to allied Regional programs responsible for implementing the Agency's Science Integrity
Policy, Policy on Protection of Human Research Subjects in EPA Conducted or Supported
Research, and National Geospatial Data Policy
The Region 8 QMP has been signed and authorized for implementation by Senior Managers in
the Region and by the Office of Mission Support (OMS) Chief Information Officer and
Enterprise Quality Management Division Director. The QMP will be approved for 5 years from
the date of OMS approval. Revisions during that 5-year period may be necessary based on:
periodic internal assessments or annual reviews conducted by Region 8; external assessments
conducted by OMS or other U.S. EPA organizations; and/or significant changes to Region 8's
organization, resources or scope of mission.
1.2 Mission, Policy and Scope
EPA's mission is to protect human health and the environment. To accomplish that mission, EPA
utilizes environmental information from a variety of sources. Almost everything EPA does is
directly dependent upon the collection and use of environmental data as defined in Section 1.3.
The Region 8 Quality Policy (Text Box 1) supported by Region 8's Quality System requires that
all environmental data and information used in Agency decision-making are scientifically sound,
defensible and of known and documented quality and are of appropriate and adequate quality for
the intended purpose. Region 8 is strongly committed to sound science and quality assurance
(QA) practices which will produce environmental data and information of appropriate quality to
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
3
be used for decision-making. This commitment is consistent with the goals of the Agency's
Science Integrity Policy.
Region 8 Quality Policy:
It is Region 8 policy that all environmental data and information collected
and/or used in the process of decision-making are of known and documented
quality, suitable for its intended use, with all aspects of collection and
analysis thoroughly documented; such documentation being verifiable and
defensible. This policy applies to all data collected under environmental
operations and environmental technology activities performed directly by or
for the Region. This includes Federal, State, Tribal and local partners under
interagency and financial assistance agreements; contractors funded by EPA;
regulated entities and potentially responsible parties. The Regional
Administrator, Senior Leadership and managers ensure adequate allocation of
resources (intramural and extramural money, travel and training funds, and
personnel) to achieve the Region's Quality Policy.
Text Box 1. Region 8 Quality Policy
It is EPA policy that all environmental programs performed by EPA or directly for EPA through
EPA-funded extramural agreements shall be supported by individual quality systems that comply
fully with the American National Standard ASQ/ANSI E4:2014, Quality Management Systems
for Environmental Information and Technology Programs.
This QMP applies to all Region 8 programs, activities, grants, contracts and interagency
agreements that collect and/or evaluate environmental data and information, which are used to
make decisions or support actions related to our defined mission and responsibilities.
Quality Assurance Policies
Generally, the following policies apply to all environmental data collection, generation or use
conducted by Region 8 personnel and its contractors, grantees, and interagency agreement
recipients:
Appropriate QA planning documents (e.g., Quality Management Plan (QMP), Quality
Assurance Project Plan (QAPP) or functionally equivalent document, which may include
supplements such as: Sampling and Analysis Plan (SAP), Field Sampling Plan (FSP), or
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
4
Work Plan) are developed and approved for each environmental data collection activity
prior to the initiation of data and information collection.
Intended use(s) and Data Quality Objectives (DQOs) of environmental data and
information are identified prior to collection of the data in the appropriate QA planning
document(s).
Implementation of projects and tasks involving environmental data and information
generation, or use conforms to information provided in approved QA planning
documents.
Oversight of data collection activities is performed by an individual or organization that
is independent of data generation activities and any identified deficiencies are promptly
corrected.
Programs and projects that use existing data or data from secondary sources must have an
approved QAPP (or equivalent QA document). The QA document should specify the
quality system that will be used to determine the suitability of the data for the proposed
use. For more information about the definition and use of existing environmental data,
refer to Section 7.9.
The specifics of each of these requirements are the scope and content of this QMP.
1.3 Definitions of Environmental Data & Environmental Technology
EPA Orders CIO 2105.0 and 2106.0 establish the policy and program requirements for a
mandatory Agency-wide Quality System. When implemented, the Quality System provides the
needed management systems to ensure that environmental data (defined in Text Box 2) used to
support Agency decisions are of adequate quantity and quality for their intended purpose.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
5
Text Box 2. Definitions for Environmental Data and Environmental Technology
Environmental Data is defined as:
Any measurement or information that describes environmental processes, location, or
condition; ecological or health effects and consequences, or the performance of
environmental technology.1
Environmental data include any information collected directly from measurements or
obtained from any other sources (i.e., existing/secondary data) such as those compiled from
databases, data reports, literature, surveys, or produced from models.2
Environmental Technology is:
An all-inclusive term used to describe pollution control devices and systems, waste
treatment processes and storage facilities, and site remediation technologies, and their
components that may be utilized to remove pollutants or contaminants from or prevent them
from entering the environment.2
1 universal definition as defined in ANSI/ASQ E4-2014 and EPA Orders CIO 2105.0/2106.0
2EPA Quality Manual for Environmental Programs (CIO 2105-P-01-0)
As seen above, the definition of environmental data extends beyond collection of primary data
such as field samples or monitoring. Environmental data include environmental technology
as well as all information collected from both primary and secondary sources. Secondary
data or existing data may be obtained from sources that are internal or external to EPA Region 8;
examples include but are not limited to measurements or information obtained from: GPS or
GIS; inspections; models; surveys; other federal, state or local agencies; third party sources,
industry; potentially responsible parties; and so on (See Section 7.9). Refer to Section 9 for more
information about requirements for assessing quality and using existing data in decision-making.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
6
1.4 Plan, Do, Check, Act Model in Region 8's Quality System
Region 8 has adopted the Plan, Do, Check, Act (PDCA) quality model as the foundation of its
Quality System. The PDCA quality model is an iterative four-step approach for managing,
planning, implementing and administering continuous improvement over the lifecycle of any
Region 8 activity, including all work processes and work products.
The elements of the PDCA quality model are summarized below.
PLAN
Establishes the objectives and processes necessary to deliver results in accordance
with the desired output or goals. Planning includes project organization, securing
resources, and roles and responsibilities of participants.
DO
The implementation or execution of the planned activity, following standard
protocols or procedures for direct data acquisition, acquisition of data from
existing or other sources, and modeling.
CHECK
Monitors and measures the process performance through the assessment of the
project to identify any differences or deviations from the implementation of the
plan and the expected results, targets, objectives or goals established in the
planning phase.
ACT
The corrective action taken for significant differences between actual and planned
results. This step includes a root cause analysis to determine where to apply the
changes that will create improvement of the process or product.
Application of the PDCA quality model is appropriate for all Region 8 environmental operations,
administrative activities and extramural agreements. PDCA serves as the theme for this QMP
and its approach is integrated into all aspects of this document.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
7
2 REGION 8 STRUCTURE AND ORGANIZATION
Within Region 8's Organizational Structure, there are defined implementation roles and
responsibilities, authorities, accountability, lines of communication and processes to manage the
quality of environmental data operations conducted by or for EPA. The structure of the Regional
Quality System ensures that appropriate resources are effectively applied to its implementation at
all organizational levels.
2.1 Quality System Organizational Structure
The Region's Organizational Structure, as shown in Figure 1. is headed by the Regional
Administrator (RA), Deputy Regional Administrator (Deputy RA) and their staff. There are
seven Divisions, each headed by a Division Director, Office of the Regional Administrator and
Office of Regional Counsel. The Quality System is overseen by the Regional Quality Assurance
Manager (RQAM), who maintains independence from environmental data operations and is
afforded access to the Regional Administrator and/or Deputy Regional Administrator, if needed.
The independence of the RQAM, QA Staff and other delegated QA representatives is not only
required by national EPA policy, but is vitally important to the Region's implementation of its
Quality System, allowing the RQAM (or delegated QA representatives) the authority to advocate
the importance and relevance of quality in EPA's work. The RQAM (or delegated QA
representatives) is able to serve without any potential conflicts of interest due to his/her location
in the Laboratory Services and Applied Sciences Division (LSASD), outside any sub-group that
collects and/or uses environmental data directly, and with explicit ties to the RA and Deputy RA
to resolve any potential conflict of interest posed by the RQAM's organizational location in
LSASD.
Functionally, Region 8 has a centralized QA Branch. This centralized QA System consolidates
the QA decision-making, assessment (auditing), guidance, and training functions in a central
core, yet allows delegation of authority for day to day QA activities and project-level QA
documents to staff in the various Divisions and Branches.
Also located within the LSAS Division is the Region 8 Laboratory. The Laboratory's QA Officer
oversees implementation of the Laboratory QA activities and maintains independence from data
generation at the Laboratory. The Laboratory QA Officer is afforded access to the Regional QA
Manager, if needed.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
8
Figure 1. Organizational Structure for EPA Region 8's Quality System
MISSION
SUPPORT
DIVISION
REGI
AD MINIS
3NAL
TRATOR
DEPUTY REGIONAL
ADMINISTRATOR
AIR&
RADIATION
DIVISION
LABORATORY
SERVICES AND
APPLIED
SCIENCES
DIVISION
5
WATER
DIVISION
ENFORCEMENT
& COMPLIANCE
ASSURANCE
DIVISION
LAND,
CHEMICALS, &
REDEVELOPMEN
T DIVISION
SUPERFUND &
EMERGENCY
MANAGEMENT
DIVISION
DIVISION DIRECTOR
TECHNICAL
ASSISTANCE
BRANCH
2.2 Region 8 Organization
As seen in Figure 1, Region 8 is structured with seven Divisions, with all Division Directors
reporting to the Regional Administrator. The Divisions are each organized into separate functional
responsibilities. Thumbnail descriptions for each are provided below.
2.2.1 Office of the Regional Administrator
The Office of the Regional Administrator (ORA) provides the overall direction and management
of the Region. ORA is responsible for planning, programming, implementation, control, and
direction of technical and administrative aspects of Region 8 programs and activities. ORA
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
9
provides technical oversight of the Region's Equal Employment Opportunity and Special
Emphasis Programs. Additionally, ORA houses the Region's:
Public Affairs, Tribal Affairs, and National Environmental Policy Act Branches; and
advisors for agriculture, tribal affairs, science, children's health, and environmental justice.
2.2.2 Laboratory Services and Applied Science Division (LSASD)
The Laboratory Services and Applied Science Division (LSASD) provides the applied science
and technology support to the region. The division ensures the quality assurance programs meet
Agency requirements. LSASD is responsible for working with the other regional divisions to
ensure that the environmental data collected, reported and used meets data quality requirements.
LSASD provides integrated field support and laboratory analytical support for Regional
programs. This includes field sampling, investigations, and the field activities associated with
studies in support of Superfund remediation, hazardous waste determinations, air quality
assessment, water quality assessment, ecological condition assessment, and enforcement
activities.
LSASD houses the QA Branch. The QA Branch Chief also holds the role of Regional Quality
Assurance Manager (RQAM).
2.2.3 Enforcement and Compliance Assurance Division (ECAD)
The Enforcement and Compliance Assurance Division (ECAD) is responsible for developing
and implementing Region 8 enforcement and compliance assurance programs and statutes that
EPA administers in Colorado, Montana, North Dakota, South Dakota, Utah, Wyoming, and
Region 8 Tribes.
ECAD works closely with the other Region 8 Divisions, Office of Regional Counsel (ORC),
Criminal Investigations Division (CID), and Department of Justice (DOJ) to deliver a
comprehensive enforcement and compliance assurance program that uses the entire spectrum of
compliance assurance tools available to the Region.
2.2.4 Mission Support Division (MSD)
The Mission Support Division (MSD) provides leadership, support, communications, and
direction to ensure efficient operations vital to EPA and Regional goals. The Director also holds
the role of Senior Resource Official and Senior Information Official (SIO).
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
10
The Division advises the Regional Administrator and Deputy Regional Administrator, senior
leadership, and management on Regional and national policies involving strategic planning,
technical, and resource management issues. The Division includes the Human Resources;
Information Management; Infrastructure; Grants, Acquisition, and Interagency Agreements;
Fiscal Management and Planning Branches.
2.2.5 Air and Radiation Division (ARD)
The Air and Radiation Division (ARD) implements the programmatic aspects of the Clean Air
Act (CAA) within the geographic boundaries of Region 8, except for inspections and
enforcement, which are principally managed by the Enforcement and Compliance Assurance
Division. Under this statute and in accordance with implementing regulations and agency
guidelines, the Division conducts activities to reduce emissions so that air pollution does not
constitute a threat to public health, safety, well-being or the environment.
To carry out its mission, the Division works with other federal agencies, state and local agencies,
tribal governments, the public, and the private sector. The Division coordinates with the Office
of Air and Radiation to ensure national consistency and strives to meet legal deadlines imposed
by the CAA.
2.2.6 Water Division (WD)
The Water Division (WD) manages and implements programs under the Clean Water Act
(CWA) and the Safe Drinking Water Act (SDWA) to protect, preserve, and enhance water
resources, and works closely with states and local communities to protect drinking water sources,
and to ensure that estuaries, rivers, streams, and lakes are healthy, vital resources for
communities. This is accomplished by:
Conducting monitoring;
Relying on water quality standards, pollution load allocations for streams; permits for
stream discharges, public health standards, and required testing for drinking water safety;
Providing financial and technical assistance to states and localities.
The WD also encourages citizen involvement in developing and carrying out major estuary, local
watershed and drinking-water protection plans.
2.2.7 Superfund Emergency Management Division (SEMD)
The Superfund and Emergency Management Division (SEMD) is responsible for activities
regarding the Oil Pollution Act (OPA) and the Comprehensive Environmental Response,
Compensation and Liability Act (CERCLA), commonly known as Superfund. SEMD
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
11
determines goals, priorities and objectives for the Superfund and oil programs; oversees the work
of federal responsible party cleanups; and concurs on military base closure property transfers.
SEMD also:
Coordinates the discovery, assessment, remediation, and removal of, and assists with
enforcement of abandoned hazardous waste sites;
Develops strategies to reuse Superfund sites, and
Responds to emergency-including terrorism-situations that pose an immediate threat to
human health and the environment.
2.2.8 Land, Chemicals and Redevelopment Division (LCRD)
The Land, Chemicals, and Redevelopment Division (LCRD) administers environmental statutes
that protect our air, water and land environments, as well as the public's right to know about
hazardous chemicals in their community. It:
Protects the air from asbestos emissions;
Protects water and land from inefficient hazardous waste practices;
Conserves energy and natural resources through recycling and recovery;
Protects the public from pesticide misuse or contaminants;
Protects children from the hazards of lead-based paint and other environmental
contaminants;
Supports the redevelopment of Brownfields; and
Regulates underground storage tanks containing petroleum or hazardous substances.
2.2.9 Region 8 Science Council
The Regional Science Council is composed of scientists, engineers, and technical specialists who
represent a variety of disciplines and who work on behalf of the Agency to strengthen the use of
science in the Region. This self-directed, volunteer group meets to address important regional
science issues, assists with regional problem solving, and helps to enhance science capacity in
Region 8. The Council meets monthly with meetings being open to EPA and on-site federal staff.
The Region 8 Science Council's vision is an EPA where management and staff recognize the
need for and are fully committed to incorporating scientific and technical excellence in the
decision-making process. The Regional Science Council Leadership Team is comprised of the
Chairperson, the Vice-Chair, the Management Champion, and the Regional Science Liaison to
ORD. The EPA Region 8 Science Council Charter (2017) explains the purpose and operations of
the council. The mission of the Council is to ensure continued enhancement of science capacity
in the region by:
Serving as a resource to staff and managers to assist with integrating sound science in
meeting program goals and building credibility in the decision-making process;
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
12
Enhancing the region's ability to provide new and existing employees with the scientific
and technical skills they will need as their careers develop;
Enhancing communication and coordination on science activities among program offices
and with Regional leadership; and
Serving as a clearinghouse for science activities in the region so that redundancy is
minimized.
2.3 Responsibilities of EPA Management and Staff
In accordance with EPA Order CIO 2105.0, overall responsibility for the Regional Quality
System resides with the Regional Administrator. The responsibility for developing and
documenting Regional QA policies, procedures and guidance; overseeing the implementation
and assessment of the Regional Quality System; and providing QA training has been delegated to
the Regional Quality Assurance Manager. The RQAM is located in the Laboratory Services
and Applied Sciences Division and is independent of environmental data operations. On issues
relative to the Region's Quality System, the RQAM is afforded access to the Regional
Administrator and/or Deputy Regional Administrator, if needed.
The responsibility to implement the Region 8 QMP rests with all Regional staff and managers
involved in generation or use of environmental data (Text Box 3).
Text Box 3. Quality Assurance is Everyone's Responsibility
All Regional staff and managers are responsible for implementing
Region 8's Quality Management Plan.
2.3.4 Individual Quality System Roles, Responsibilities and
Authorities
Although implementation of the Region 8 Quality Systems resides with all Regional staff and
managers, the following positions have key roles and authorities.
Regional Quality Assurance Manager
The RQAM has responsibility for oversight and implementation of Regional and Agency QA
policies. The primary focus of this oversight is to ensure that appropriate QA practices are
implemented for the Region as described in this QMP. The specific functions of the RQAM are:
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
13
Oversees implementation of the Regional Quality System;
Develops and conducts annual reviews of the Regional QMP;
Prepares Quality Assurance Annual Reporting to OMS EQMD as requested;
Reviews and approves Quality Management Plans (QMPs) and QA Documents for
Region 8;
Represents Region 8 on Agency, Interagency and National QA policy issues;
Participates in contract awards as prescribed by contracting regulations;
Performs management and technical audits of Regional and extramural programs;
Develops and provides training to Regional personnel on QA policies and practices as
needed and as resources allow;
Provides similar training upon request to States and Tribes within Region 8; and
Serves as the Information Quality Guidelines (IQG) Officer for the Region.
Managers and Supervisors in Region 8
QA is an integral part of every Region 8 Program that is associated with the collection or use of
environmental data and information. Managers and supervisors are responsible for providing and
managing adequate resources to ensure that all environmental data in support of their respective
programs are of known quality, defensible and adequate for the intended use. EPA Order CIO
2105.0 requires that the performance agreements of all supervisors, senior managers, and
appropriate staff have critical element(s) that are commensurate with the quality management
responsibilities assigned by the QA Orders and this QMP.
Designated Project Manager
A Designated Project Manager (DPM) is a term describing any Region 8 employee (staff or
manager) with immediate managerial, administrative or technical control of a project, program or
extramural agreement involving generation or use if environmental data and information. The
DPM is a role responsible for ensuring the Region 8 Quality System is employed for all activities
involving environmental data, such as: specifying the quality of the environmental data required
for each project, securing the appropriate approvals of the respective project-level or program-
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
14
level QMP/QAPP, ensuring environmental data that are generated/used are defensible and of
known quality, and requesting the appropriate resources needed from management.
DPMs may have a variety of job titles or functions; some examples are:
On-Scene Coordinator (OSC)
Remedial Project Manager (RPM)
Site Assessment Manager (SAM)
Project Manager
Work Assignment Manager (WAM)
Contracting Officer's Representative
(COR)
Grant Specialist
Grant Project Officer
State Program Manager (SPM)
Tribal Program Manager (TPM)
Program Specialist
Task Manager
Contracting Officer (CO)
Contract Specialist
Contract Project Officer
Other Roles/Positions Identified in the QMP
Some key positions, mentioned in this QMP, are assigned to individuals within Region 8. The
names of these individuals are listed in Table 1.
Table 1. Current Representatives for Key Positions Identified in the QMP
Region 8 Position
Representative
Contact Information
Deputy Scientific Integrity Official Deputy
Regional Administrator
Deb Thomas
thomas. debrah(a),eDa. gov
303-312-6298
Laboratory Services and Applied Sciences
Division Director
Sandra Spence
soence. sandra(a),eDa. gov
303-312-6947
Regional Quality Assurance Manager
Information Quality Guidelines Officer
Linda Himmelbauer
himmelbauer. linda(a>epa. gov
303-312-6020
Regional Science Advisor
Peer Review Coordinator
Human Subjects Officer
A1 Basile
basile.alfredfo!eoa.gov
303-312-6551
Deputy Regional Quality Assurance Manager
Quality Assurance Field Activities Procedure Lead
Mary Goldade
goldade. mary(a>epa. gov
303-312-7024
Laboratory QA Officer
Brian Eisenhauer
eisenhauer.brian(a>epa.gov
303-462-9504
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
15
2.4 State Partnerships Roles and Responsibilities
State agencies (Colorado, Montana, North Dakota, South Dakota, Utah and Wyoming) are
partners with U.S. EPA Region 8 environmental programs and are vital to achieving the Region's
primary mission. U.S. EPA Region 8 enlists the cooperative participation of its State agencies in
achieving the Region's mission through financial assistance in the form of grants, cooperative
agreements (including primacy agreements), and performance partnership agreements (PPAs).
2.4.1 Quality Assurance Requirements for STATE AGENCIES
States are required to conform to applicable QA requirements as specified in 2 CFR 1500.11
Quality Assurance. The regulation states:
Quality assurance applies to all assistance agreements that involve environmentally
related data operations, including environmental data collection, production or use.
Recipients shall develop a written quality assurance system commensurate with the
degree of confidence needed for the environmentally related data operations.
If the recipient complies with EPA's quality policy, the system will be presumed to be in
compliance with the quality assurance system requirement. The recipient may also
comply with the quality assurance system requirement by complying with American
National Standard ASQ/ANSI E4:2014: Quality management systems for environmental
information and technology programs.
The recipient shall submit the written quality assurance system for EPA review. Upon
EPA's written approval, the recipient shall implement the EPA-approved quality
assurance system.
EPA Quality Policy is available at: http://www.epa.gov/qualitv.
The standards required in this section, are incorporated by reference with the approval of
the Director of the Federal Register under 5 U.S.C. 552(a) and 1 CFR part 51.
Each State agency, which conducts environmental data operations and receives financial
assistance from Region 8, shall submit a QMP or other appropriate QA documentation for EPA
Region 8's approval. Each State agency, in accord with the respective grant term and condition,
is afforded the flexibility to:
Submit a QMP for EPA review and approval; or
Submit Quality Assurance Project Plans (QAPPs) for EPA review and approval.
Any State desiring delegated QA authority to review and approve QA documents (e.g., QAPPs)
must document all Quality System components required in a QMP by EPA Requirements for
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
16
Quality Management Plans QA/R-2, (2000) reissued May 2006, which is then reviewed and
approved by the Regional Quality Assurance Manager. Region 8 encourages States to obtain this
delegation of authority as part of the Performance Partnership Agreement (PPA).
2.4.2 Requirements for Extending Delegation Authority to
States
The RQAM has authority to delegate the Agency-wide Quality System to States, provided the
States have documented their quality system with a Quality Management Plan that meets the
minimum QA requirements in EPA Order 2105.0. State Agencies, having obtained QA delegated
authority by Region 8, can review and approve their project-level QA documents such as QAPPs
providing the respective program allows such a delegation.
Following Region 8's approval of a State agency's QMP or appropriate QA document, the State
agency is still required to ensure that project-level QA documents (QAPP or other appropriate
document) are reviewed and approved prior to initiating environmental data operations for
programs encompassed by the approved QMP or QA document. Each State Agency shall ensure
that such project-level environmental data operations are systematically planned, documented,
and assessed.
Some programs such as Superfund pre-remedial, remedial and removal programs require by
regulation that U.S. EPA review and approve all QMPs and QAPPs. Additionally, Clean Air Act
air monitoring programs require by regulation that the U.S. EPA review and approve all QMPs
and QMP and QAPP pairs for Primary Quality Assurance Organizations (PQAOs) that support
environmental data operations. In these instances, final QA document approval cannot be
delegated to State agencies.
State agencies which have not been delegated QAPP review and approval authority or where
programs are prohibited by regulation, shall submit QAPPs or other appropriate QA
documentation to Region 8 for approval. For situations where EPA funding of environmental
programs is not covered by an approved State QMP (e.g., EPA funding of local entities) QAPPs
must be submitted to EPA for review and approval.
Required QMP Elements for State Delegation
The State must address the following quality system components in a QMP to have delegated
QA authority to review and approve QAPPs.
A quality assurance manager (QAM), or person assigned to an equivalent position, who
functions independently of direct environmental data generation, model development, or
technology development responsibility.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
17
A Quality Management Plan (QMP), which documents the organization's quality policy,
describes its quality system, identifies the environmental programs to which the quality
system applies, and which is implemented following approval by the organization's
executive leadership and the RQAM.
Sufficient resources to implement the quality system defined in the approved QMP.
Assessments of the effectiveness of the quality system at least annually.
Submittal to the EPA Region 8 RQAM an annual report for the organization that
summarizes the previous year's QA and quality control (QC) activities and outlines the
work proposed for the current year.
Use of a systematic planning approach based on the scientific method to develop
acceptance or performance criteria for all work covered by EPA Order CIO 2105.0. (See
Section 3.8 of OA/R-2. EPA Requirements for Quality Management Plans, where use of
QA/G-4 (2000), EPA Guidance for the Data Quality Objectives Process is suggested).
Approved QAPPs, or equivalent documents defined by the QMP, for all applicable
projects and tasks involving environmental data with review and approval having been
made by the authorized representative defined in the QMP. QAPPs must be approved
prior to any data gathering work or use, except under circumstances requiring immediate
action to protect human health and the environment or operations conducted under police
powers.
Assessment of existing data, when used to support Agency decisions or other secondary
purposes, to verify that they are of sufficient quantity and adequate quality for their
intended use.
Implementation of Agency-wide Quality System requirements in all applicable EPA-
funded agreements.
Implementation of corrective actions based on assessment results.
Appropriate training, for all levels of management and staff, to assure that QA and QC
responsibilities and requirements are understood at every stage of project
implementation.
2.4.3 Grantee QA Annual Reviews & Reporting Requirements
Each State agency shall submit an annual report of its Quality System activities to EPA Region
8.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
18
The annual report shall summarize its review of the State agency quality system, including:
Review of the QMP and revision/update, if necessary;
Listing of QA training provided within the State agency for the previous year including
training provider, dates of training, and number of attendees;
Identification of all QA documents including status (current/reviewed/approved) and
whether the QA document is for new or continuing projects;
Reporting of all assessments conducted in the previous year and the corrective actions
taken to resolve any findings; and
Reporting if there are real or perceived risks or vulnerabilities in the quality system that
need to be addressed or if support is needed from EPA Region 8.
Refer to Section 9.6 of the QMP for more information about annual QA document review
requirements.
2.4.4 Region 8 QA Program State Oversight
Region 8's oversight responsibilities for State Agency quality systems include:
Review of updates and revisions of State Agency QA documentation;
Review of State Agency annual reports on quality system implementation;
Development, assessment and concurrence of QA goals in State agency work plans,
PPAs, etc.;
Conducting periodic quality system assessments (QSAs) to assess the implementation
and effectiveness of the approved quality system;
Oversight of any corrective actions identified in QSAs or other oversight activities; and
Reporting State QA activities to OMS EQMD Headquarters in the Region 8 QA annual
reporting if requested.
2.5 Tribal Nations Roles and Responsibilities
Region 8 works in partnership with twenty-seven (27) tribal governments to advance the
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
19
protection of human health and the environment in tribal lands and enlists participation by the
tribal governments through financial assistance in the form of cooperative agreements and
performance partnership grants (PPGs). Through direct implementation of these grants and
agreements, Region 8's oversight functions take several forms: programmatic, technical,
financial and quality assurance. Additionally, the Region 8 Laboratory provides
analytical services to the Tribes. Therefore, direct implementation is carried out by several
environmental and enforcement programs and includes support by Region 8 programs.
2.5.1 Quality Assurance Requirements for Tribes
Tribes are required to conform to applicable QA requirements as specified in 2 CFR 1500.11
Quality Assurance. The regulation states:
Quality assurance applies to all assistance agreements that involve environmentally
related data operations, including environmental data collection, production or use.
Recipients shall develop a written quality assurance system commensurate with the
degree of confidence needed for the environmentally related data operations.
If the recipient complies with EPA's quality policy, the system will be presumed to
follow the quality assurance system requirement. The recipient may also comply with the
quality assurance system requirement by complying with American National Standard
ASQ/ANSI E4:2014: Quality management systems for environmental information and
technology programs.
The recipient shall submit the written quality assurance system for EPA review. Upon
EPA's written approval, the recipient shall implement the EPA-approved quality
assurance system.
EPA Quality Policy is available at: http://www.epa.gov/quality.
The standards required in this section, are incorporated by reference with the approval of
the Director of the Federal Register under 5 U.S.C. 552(a) and 1 CFR part 51.
Each Tribe, which conducts environmental data operations and receives financial assistance from
Region 8, shall submit a QMP or other appropriate QA documentation for EPA Region 8's
approval. Each Tribe is afforded the flexibility to:
Submit a QMP for EPA review and approval; or
Submit Quality Assurance Project Plans (QAPPs) for EPA review and approval.
Where resources and/or capacity are/is limited, a tribal government may reference the Region 8
QMP as its governing document for Quality System requirements. In addition, Region 8 has
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
20
provided flexibility to tribal governments to document their quality requirements for individual
extramural agreements in QAPPs. Tribal governments receiving financial assistance from EPA
shall meet Region 8 QA requirements outlined in this QMP as well as respective program goals
and requirements. All QAPPs must be reviewed and approved by the RQAM (or Delegated QA
Approving Officer) prior to implementation of the work described therein.
Any Tribe desiring delegated QA authority to review and approve QA documents (e.g., QAPPs)
must document all Quality System components required in a QMP by EPA Requirements for
Quality Management Plans QA/R-2, (2000) reissued May 2006, which is then reviewed and
approved by the Regional Quality Assurance Manager. Region 8 encourages Tribes to obtain this
delegation of authority as part of the Performance Partnership Grant (PPG).
2.5.2 Requirements for Extending Delegation Authority to
Tribes
The RQAM has authority to delegate the Agency-wide Quality System to Tribes, provided the
Tribe has documented their quality system with a Quality Management Plan that meets the
minimum QA requirements in EPA Order 2105.0. Tribal Agencies, having obtained QA
delegated authority by Region 8, can review and approve their project-level QA documents such
as QAPPs providing the respective program allows such a delegation.
Following the U.S. EPA Region 8's approval of a QMP or appropriate QA document, the Tribal
agency is still required to ensure that project-level QA documents (QAPP or other appropriate
document) are reviewed and approved prior to initiating environmental data operations for
programs encompassed by the approved QMP or QA document. Each Tribal agency shall ensure
that such project-level environmental data operations are systematically planned, documented,
and assessed.
Some programs such as Superfund pre-remedial, remedial and removal programs require by
regulation that U.S. EPA review and approve all QMPs and QAPPs. Additionally, Clean Air Act
air monitoring programs require by regulation that the U.S. EPA review and approve all QMPs
and QMP and QAPP pairs for Primary Quality Assurance Organizations (PQAOs) that support
environmental data operations. In these instances, final QA document approval cannot be
delegated to Tribal agencies.
Tribal agencies which have not been delegated QAPP review and approval authority or where
programs are prohibited by regulation, shall submit QAPPs or other appropriate QA
documentation to Region 8 for approval.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
21
Required QMP Elements for Tribal Delegation
The Tribe must address the following quality system components in a QMP to have delegated
QA authority to review and approve QAPPs.
A quality assurance manager (QAM), or person assigned to an equivalent position, who
functions independently of direct environmental data generation, model development, or
technology development responsibility.
A Quality Management Plan (QMP), which documents the organization's quality policy,
describes its quality system, identifies the environmental programs to which the quality
system applies, and which is implemented following approval by the organization's
executive leadership and the RQAM.
Sufficient resources to implement the quality system defined in the approved QMP.
Assessments of the effectiveness of the quality system at least annually.
Submittal to the EPA Region 8 RQAM an annual report for the organization that
summarizes the previous year's QA and quality control (QC) activities and outlines the
work proposed for the current year.
Use of a systematic planning approach based on the scientific method to develop
acceptance or performance criteria for all work covered by EPA Order CIO 2105.0. (See
Section 3.8 of OA/R-2. EPA Requirements for Quality Management Plans, where use of
QA/G-4 (2000), EPA Guidance for the Data Quality Objectives Process is suggested).
Approved QAPPs, or equivalent documents defined by the QMP, for all applicable
projects and tasks involving environmental data with review and approval having been
made by the authorized representative defined in the QMP. QAPPs must be approved
prior to any data gathering work or use, except under circumstances requiring immediate
action to protect human health and the environment or operations conducted under police
powers.
Assessment of existing data, when used to support Agency decisions or other secondary
purposes, to verify that they are of sufficient quantity and adequate quality for their
intended use.
Implementation of Agency-wide Quality System requirements in all applicable EPA-
funded agreements.
Implementation of corrective actions based on assessment results.
Appropriate training, for all levels of management and staff, to assure that QA and QC
responsibilities and requirements are understood at every stage of project
implementation.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
22
2.5.3 Grantee QA Annual Reviews & Reporting Requirements
Each Tribal agency with an EPA-approved QMP shall submit an annual report of its Quality
System activities to EPA Region 8.
The annual report shall summarize its review of the State agency quality system, including:
Review of the QMP and revision/update, if necessary;
Listing of QA training provided within the State agency for the previous year including
training provider, dates of training, and number of attendees;
Identification of all QA documents including status (current/reviewed/approved) and
whether the QA document is for new or continuing projects;
Reporting of all assessments conducted in the previous year and the corrective actions
taken to resolve any findings; and
Reporting if there are real or perceived risks or vulnerabilities in the quality system that
need to be addressed or if support is needed from EPA Region 8.
Refer to Section 9.6 of the QMP for more information about annual QA document review
requirements.
2.5.4 Region 8 QA Program Tribal Oversight
Region 8's oversight responsibilities for Tribal agency quality systems include:
Review of updates and revisions of Tribal agency QA documentation;
Review of Tribal agency annual reports on quality system implementation;
Development and assessment of QA goals in Tribal agency Performance Partnership
Grants (PPGs), grant workplans, etc;
Conducting periodic quality system assessments (QSAs) to assess the implementation
and effectiveness of the approved quality system;
Oversight of any corrective actions identified in QSAs or other oversight activities; and
Reporting Tribal QA activities to OMS EQMD Headquarters in the Region 8 QA annual
reporting if requested.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
23
3 SCIENTIFIC INTEGRITY POLICY
Science is the backbone of EPA's decision-making, and the Agency's ability to pursue its
mission to protect human health and the environment depends upon the integrity of the science
on which it relies. In 2012, EPA established a Scientific Integrity Policy3 to ensure that sound
science drives Agency decision making. This policy provides a framework intended to ensure
scientific integrity throughout EPA and promote scientific and ethical standards, including
quality standards; communications with the public; the use of peer review and advisory
committees; and professional development. It also describes the scope and role of a standing
committee of Agency-wide scientific integrity officials to implement this policy.
3.1 Scientific Integrity Policy Overview
To promote scientific integrity throughout the Agency, EPA's Scientific Integrity Policy
addresses four specific areas:
The culture of scientific integrity at the EPA,
Public communications,
The use of peer review and Federal Advisory Committees, and
Professional development of government scientists.
In addition, the policy establishes the Scientific Integrity Committee, chaired by the Agency's
Scientific Integrity Official, to implement this policy.
3.2 Policy Applicability
When dealing with science, it is the responsibility of every EPA employee to conduct, utilize,
and communicate science with honesty, integrity, and transparency, both within and outside the
Agency. All Agency employees, regardless of grade level, position or duties and including
scientists, managers, and political appointees, are required to follow this policy when engaging
in, supervising, managing, or influencing scientific activities; communicating information in an
official capacity about Agency scientific activities; and utilizing scientific information in making
Agency policy or management decisions. In addition, all contractors, grantees, collaborators and
student volunteers of the Agency who engage in scientific activities are expected to uphold the
standards established by this policy and may be required to do so as part of their respective
agreements with the EPA.
3 In this document, "science" and "scientific" are expansive terms that refer to the full spectrum of scientific
endeavors, e.g., basic science, applied science, engineering, technology, economics, social sciences, and statistics.
The term "scientist" refers to anyone who collects, generates, uses, or evaluates scientific data, analyses, or
products.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
24
3.3 Policy Implementation
The policy establishes a Scientific Integrity Committee for policy implementation and to provide
an annual report. In advance of completing the annual report, the Scientific Integrity Committee
will conduct an Agency-wide annual meeting on scientific integrity that will include the
involvement of senior EPA leadership, reports from offices and programs, and an opportunity for
input from the EPA scientific community.
3.3.1 Scientific Integrity Committee
The Agency has appointed a Scientific Integrity Official to champion scientific integrity
throughout the Agency. The Scientific Integrity Official chairs a standing committee of Deputy
Scientific Integrity Officials representing each EPA Program Office and Region. These senior-
level employees provide oversight for the implementation of the Scientific Integrity Policy at
EPA, act as liaisons for their respective Programs and Regions, and are available to address any
questions or concerns regarding this policy.
The Agency's Scientific Integrity Committee is charged with implementing, reviewing, and
revising, as needed, policy governing the four specific areas of scientific integrity described in
Section 3.1. Region 8's Deputy Scientific Integrity Official provides representation on this
Committee.
4 POLICIES FOR FIELD AND LABORATORY COMPETENCY AND
ACCREDITATION
EPA's Forum on Environmental Measurements (FEM) is a standing committee of senior
managers established to develop policies to guide the Agency's measurement community in:
Validating and disseminating methods for sample collection and analysis;
Ensuring that monitoring studies are scientifically rigorous, statistically sound, and yield
representative measurements; and
Employing a quality systems approach that ensures that the data gathered and used by the
Agency is of known and documented quality.
The FEM is engaged in a number of activities to implement its mission and reports to the
Agency's Science Policy and Technology Council (SPTC). In recent years, the SPTC has issued
several policies developed by the FEM related to field and laboratory competency requirements.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
25
These are:
Policy Directive on Assuring the Competency of Environmental Protection Agency
Laboratories, dated March 10, 2004;
Policy to Assure Competency of Laboratories, Field Sampling and Other Organizations
Generating Environmental Measurement Data under Agency-funded Acquisitions, FEM-
2011-01; and
Policy to Assure Competency of Organizations Generating Environmental Measurement
Data Under Agency-funded Assistance Agreements, FEM-2012-02.
4.1 Laboratory Competency at Agency Laboratories
EPA Order CIO 2105 and CIO 2106 establish the policy and program requirements for a
documented Quality System. This requirement extends to Agency laboratories and is affirmed in
the Policy Directive on Assuring the Competency of Environmental Protection Agency
Laboratories to assure the quality of data generated by all the laboratories operated by EPA,
including Agency owned, contractor operated facilities.
All EPA laboratories, including the Region 8 Laboratory and its contractor-operated facilities
must:
Document the laboratory quality system;
Document adherence to the laboratory quality system through periodic independent
assessments;
Participate in inter-laboratory performance evaluations;
Obtain and maintain independent, third-party accreditation for applicable laboratory
operations (e.g., environmental sample analysis, microbiology, etc.).
For additional Region 8 Laboratory competency information, please refer to the U.S. EPA
Region 8 Environmental Laboratory Quality Assurance Manual, QSP-001 and/or contact the
Region 8 Laboratory Quality Assurance Officer to obtain copies of the laboratory's
accreditation, independent laboratory assessment reports, etc.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
26
4.2 Organizations Generating Environmental Measurement Data for
Agency-funded Contracts and Agreements
The Agency's Policy to Assure Competency of Organizations Generating Environmental
Measurement Data Under Agency-Funded Acquisitions #FEM-2011-01 and the Agency's Policy
to Assure Competency of Organizations Generating Environmental Measurement Data Under
Agency-Funded Assistance Agreements #FEM-2012-02 require organizations that generate
environmental data through measurement (e.g., laboratory and/or field activities) to submit
documentation of their competency to do so.
4.2.1 Policy Overview
Organizations performing environmental analysis or field activities for the Agency shall
demonstrate their competencies and qualifications prior to performing such analyses and field
activities. Where accreditation or certification is available for those fields of analysis,
organizations may submit documentation of existing accreditations or certifications.
Accreditation/certification to an international consensus standard, a state accreditation or
certification program acceptable to EPA, or the contracted laboratory's participation in the EPA
Contract Laboratory Program for the applicable fields of analysis are appropriate documentation
of competency. Accreditation/certification granted by an organization that accredits
environmental data operations for those fields of analyses shall be valid at the time of award and
must be sustained through the life of the period of performance. If accreditation/certification or
other change in competency is suspended or revoked at any time during the period of
performance, the contractor or grantee must notify the EPA contract officer and project officer
immediately to ensure any potential impact to the scope of work being performed is addressed
accordingly.
4.2.2 Policy Applicability
This policy applies to all EPA programs (e.g., Programs Offices, Laboratories) and personnel
responsible for evaluating, issuing, and/or managing Agency acquisitions and agreements such
as purchases, contracts and contract work assignments/statements of work/task orders/etc. The
scope of the policy encompasses all acquisitions and agreements that either originate in Region 8
or for which QA oversight of the contract is delegated to Region 8.
4.2.3 Policy Implementation in Region 8
Region 8 COs, POs, and CORs are responsible for ensuring Region 8 QA requirements for these
policies are incorporated into all contract and grant actions (e.g., before award and after award of
contract). Other CO, PO, and COR responsibilities related to implementation of this policy
include, but are not limited to:
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
27
Before enacting any Action:
Require the appropriate QA documentation and record of a contractor's or grantee's
competency to generate environmental measurement data (e.g., laboratory and/or field
sampling and measurements);
Consult with the QA Branch, as necessary, prior to enacting any contract or grant action
to ensure all Region 8 QA requirements are met for all acquisitions and agreements that
generate environmental measurement data and information;
Evaluate and assure the contractor's competency to generate the environmental
measurement data and information; and
Obtain approval and signature from the RQAM for all QA Review Forms (QARFs) prior
to processing any contract actions (e.g., both pre-award and/or post-award of contract);
At any time:
Alert the QA Branch of changes in a contractor or grantee's competency status within 15
days of the change.
Additional requirements for acquisition activities, not related to the Agency Policy to Assure
Competency of Organizations Generating Environmental Measurement Data Under Agency-
Funded Acquisitions #FEM-2011-01, are found in Section 12.1 of this QMP.
Tools for Assessing Competency
Accreditation or certification is one tool that may be useful in evaluating competency, but there
are other tools that may be used in evaluating competency of a contractor or grantee to generate
environmental data. This combination of tools may be used in addition to and/or instead of
accreditation/certification to assess competencydepending upon the Program-specific
requirements for which environmental data will be generated.
The FEM provides extensive guidance on the range of laboratory/field competency evaluation
tools available for use. Some considerations for evaluating competency, other than
accreditation/certification are described below. Whatever considerations are chosen, the
evaluation of each criteria must be documented in the POs' and CORs' project files.
Instrumentation. Determine if the organization possesses all the equipment needed to
analyze samples for your specific project. Since many analytical methods include
optional equipment and procedures, it is important to ensure that the equipment needed
for your project is available in those cases. For large projects (e.g., many samples over a
short time frame), determine the availability for redundant instrumentation in the event
that their primary instrument fails.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
28
Sampling equipment. Determine the availability of sampling equipment. Some
organizations own all the equipment to collect samples, while other organizations may
rent or lease specialized sampling equipment for the duration of a project. Request copies
of all relevant SOPs for the work that will be accomplished.
Method sensitivity and reporting practices. Determine if the contractor has
demonstrated sensitivity and reporting practices that meet the specific project needs. A
competent laboratory must be able to provide an analyte-specific table of the sensitivity
of its analytical methods, describing their application of Method X in Matrix Y under
ideal conditions. They must also describe their routine reporting practices for results,
including their practices to censor results below a concentration (e.g., censor below some
reporting limit, quantitation limit, or detection limit).
Capacity and experience. Determine the organization's capacity and experience for
testing the quantity of samples as well as the expected sample matrices. Evaluate an
organization's demonstrated throughput capacity. Even organizations with
accreditation/certification may not collect specific types of samples or perform a given
analysis very often, therefore, consider their capacity to collect or analyze all your
samples in the required time frame. Determine if there is demonstrated experience with
your matrices of interest. For example, not all solid matrices are the same, such that a
laboratory with extensive experience in soil analysis may not be familiar with analyses of
sediment samples for the same analytes, or may not be familiar with soil types from other
geographic regions (e.g., calcareous soils from the arid Southwest are very different from
sandy loams from the East Coast).
Staff redundancy. Determine the organization's staffing capacity to complete the work
as scheduled.
Past performance. Request the organization to provide the names and examples of past
clients who can attest to the organization's past performance for similar work activities.
Proficiency Testing Samples. Obtain and review the organization's results for relevant
proficiency testing (PT) samples {i.e., relevant meaning that the methods and matrices are
similar to the project of interest) to see if the organization can produce acceptable results.
For projects of particularly critical significance or with very high visibility, consider
taking a further step:
o Providing relevant PT samples to the laboratory for analysis prior to contract
award, before submitting any field samples from your project, and/or periodically
during the contract period.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
29
On-site Evaluations. Conduct an in-depth on-site evaluation of the organization prior to,
or during the course of, the project, whether that involves sampling or laboratory
analyses.
5 POLICY AND PROCEDURES FOR PROTECTION OF HUMAN RESEARCH
SUBJECTS IN EPA CONDUCTED OR SUPPORTED RESEARCH
All research involving human subjects proposed by EPA staff or EPA supported researchers must be
approved by the EPA Human Subjects Research Review Official (HSRRO) before human subject's
work may begin. The HSRRO's responsibility is to ensure that all research studies supported by EPA
are in compliance with EPA regulations concerning research with human subjects. All human
research studies at the EPA must be approved by the HSSRO before work can begin.
The DPM in consultation with Region 8's Human Subjects Officer (HSO), first must determine
whether the study activity is human subjects research. This determination is preferably
completed during the time of grant pre-award and award during design and planning of the study,
at the time when the study objectives and involvement of human subjects is defined.
6 QA FIELD ACTIVITIES PROCEDURE (QAFAP)
The EPA QA Field Activities Procedure [QAFAP] (CIO 2105-P-02.0) was issued by the
Agency's Enterprise Quality Management Division (EQMD) to promote national consistency
among the Agency's field activities. The Procedure requirements are based on best practices for
field activities as determined by EPA field groups, EPA quality requirements, and concepts of
quality management systems established by the International Organization for Standardization
(ISO 17025). The QAFAP provides a consistent, coordinated approach to implementation for
field activities, and describes formal procedural requirements that are a part of EPA's Policy and
Program Requirements for the Mandatory Agency-wide Quality System (EPA Order CIO 2105).
The EPA QA Field Activities Procedure is comprised of ten elements that establish a quality
management system for field activities and include both sampling and non-sampling activities.
The ten elements are: Personnel and Training, Document Control, Records Management, Sample
and Environmental Data Management, Field Documentation, Field Equipment, Planning Field
Inspections and Investigations, Reports, Internal Audits, and Corrective Action. Each element of
the QAFAP is an important and integral component of the whole, which when used together,
provides the foundation for a Field Quality Management System and helps to ensure the
scientific integrity of data collection.
Region 8 has implemented all aspects of the QAFAP including procedures and maintains a
compendium of administration and technical SOPs for field activities.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
30
6.1 Applicability/Scope
It is Region 8's policy that the EPA personnel who are involved in field activities implement and
comply with the EPA QA Field Activities Procedure (CIO 2105-P-02.0) as implemented through
the Regional Field Quality Management System. The Region 8 Field Quality Management
System is outlined in nine Regional Overarching QAFAP SOPs. Assessments will be evaluated
using the QAFAP Assessment Checklist which has been modified for Region 8.
The Regional Overarching QAFAP SOPs are living documents and represent current best
approach for implementation of the QA Field Activities Procedure. Region 8 fosters a culture of
continued improvement so that as new tools, systems, or procedures are updated or refined, the
SOPs should be revised to reflect the system changes.
6.2 Definition of Field Activities
Field Activities are defined as activities requiring the collection or generation of observations,
samples, or data in support of EPA programs, Executive Orders, regulations, or environmental
laws, at a site or location (facility, water body, wetland, etc.) Field Activities include, but are not
limited to:
Planning and conducting on-site inspections of the following sites: facility permit
operations, maintenance practices, self-monitoring practices, field recordkeeping
practices and field sampling/measurement practices for gathering data/potential evidence
for all EPA programs; and
Planning and carrying out field studies/investigations/evaluations for gathering and
developing data/potential evidence, including, but not limited to, field observations
(including photographs), field measurements, sample collection, and field engineering
evaluations for EPA programs that conduct ambient and compliance monitoring and
other comprehensive studies/evaluations (both short and long term).
6.3 Definition of Field Activities
Key roles and responsibilities related to compliance with the QAFAP are listed below:
EPA Administrator. The Administrator promotes the use of consistent field operations
as an integral component to mission assurance through the best management practices
outlined in the QAFAP and that support EPA's field activities.
Office of Environmental Information's Enterprise Quality Management Division
(EQMD). The EQMD is responsible for oversight of the QAFAP and conducts Quality
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
31
System Assessments (QSAs) of EPA organizations to evaluate their conformance with
the Agency's Quality Policies and Procedures, including the QAFAP.
Regional Administrator (RA) is responsible to ensure that policies and procedures to
support field activities are developed and implemented for Region 8 and dedicate
sufficient resources to support the effective implementation of the QAFAP. The RA has
re-delegated these responsibilities for the QAFAP to appropriate senior managers, the:
Deputy Regional Administrator and the Region 8 Senior Information Official.
Additionally, Region 8 Management has designated the Field Operations Lead with
defined responsibility and authority for determining needs and prioritizing the
development of policies, procedures, and guidance to ensure the consistency of Region 8
field activities and for ensuring that the Region 8 Field Quality Management System is
implemented.
Deputy Regional Administrator (DRA) and Senior Information Official (SIO) are
responsible for dedicating sufficient resources to support the effective implementation of
the QAFAP. The DRA is the approving authority for the Regional Overarching QAFAP
Standard Operating Procedures (SOPs).
Regional QA Manager is responsible to assure that the appropriate aspects of the
QAFAP is documented in the Region's Quality Management Plan and any additional
needed procedures. The RQAM shall oversee Region 8's implementation of the QAFAP.
Field Operations Lead has the responsibility and authority for ensuring that the Region
8 Field Quality Management System is implemented and followed by administering
Quality System Assessments (QSAs) to evaluate conformance with the QAFAP in
Region 8 organizations that are involved in field activities. The Field Operations Lead is
the approving authority for Regional-wide QAFAP controlled documents such as,
technical SOPs, checklists, forms, etc.
Deputy Division Directors are responsible to respond to any requests for information
from the QA Branch such as progress reports or other "data calls" related to
implementation and compliance with the QA Field Activities Procedure. Additionally,
programs and units within their Division must participate in internal or external
assessments, as requested.
Branch Chiefs have ultimate responsibility for compliance with the QAFAP, the
Regional Overarching QAFAP SOPs, and Field Quality Management System for all field
activities and audits planned and/or conducted within their respective programs. They are
also responsible to provide auditors to serve on internal audit teams.
Managers/Supervisors are responsible for compliance with the QAFAP, the Regional
Overarching QAFAP SOPs, and Field Quality Management System for all field activities
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
32
or audits planned or conducted within their respective Branch or Section. They are also
responsible to provide auditors to serve on internal audit teams, as requested. They are
the approving authority for programmatic controlled documents for the QAFAP.
EPA Region 8 Personnel who plan or conduct field activities are responsible for being
trained in and complying with the QAFAP, the Regional Overarching QAFAP SOPs, and
Field Quality Management System.
Field Implementation Team (FIT) Members are responsible to: attend FIT meetings
and represent their Branch/Section/Team, serve as leaders and advocates to fully
implement the QAFAP requirements within his/her group; encourage
managers/supervisors to include the QAFAP as a standing item during
Branch/Section/Team meetings, work through implementation challenges within their
group and as part of the FIT; and perform or participate in internal and external audits, as
requested.
6.4 Training on the QA Field Activities Procedure
EPA Region 8 managers and staff involved in field activities shall be trained on the QA Field
Activities Procedure on an annual basis. Individuals are responsible to document completion of
annual training, which is comprised of two elements: 1) completion of QAFAP training; and 2)
review and acknowledgement of the Regional QAFAP Overarching SOPs. In addition,
individuals should document review and acknowledgement of any QAFAP programmatic SOPs
within their respective programs. For information about documenting training, refer to the
Regional QAFAP Overarching SOP: Personnel & Training for Field Activities (R8FQPOL-005).
6.5 Field Implementation Team (FIT)
The Field Implementation Team (FIT) is a cross-organizational forum established to
institutionalize and sustain consistent field operations in Region 8 by implementing the QAFAP
into every group that is involved in field activities. The FIT's Team Lead is the Field Operations
Lead or designate. The team charter is available on the FIT SharePoint site.
The FIT is an Advisory Team whose members:
Serve as leaders and advocates to fully implement the QAFAP requirements within
his/her group;
Work through implementation challenges within their group and as part of the FIT;
Attend FIT meetings and represent their group;
Encourage managers/supervisors to include the QAFAP as a standing item during
Branch/Section/Team meetings; and
Perform or participate in internal and external audits, as requested.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
33
The FIT Guiding Principles are to sustainably implement the QAFAP with a consistent approach
and flexible application, and to leverage national, Regional and other resources whenever
possible.
7 PLANNING (PLAN)
A major goal of Region 8's Quality System and the Plan, Do, Check, Act (PDCA) quality model
is to promote effective planning for the collection, analysis and processing of environmental
information and/or data. Quality planning must occur at three levels to ensure that such data
meets Regional programmatic and quality goals. Planning should occur at the Region-wide,
Program-specific and Project levels.
7.1 Region-wide Planning
Careful annual planning is necessary to the success of the Region 8 Quality System. In order for
management to budget adequate resources to implement the quality system for an organization,
estimates of the quality system workload are needed. Moreover, management must give priority
to quality system resource needs with respect to other mission requirements; this planning may
come in several forms, including installation of the Region 8 QMP. Region-wide planning
activities occur both internally and with our extramural partners and contractors.
7.1.1 Internal Strategic Planning
Region 8 conducts annual strategic planning, which is the foundation for establishing all
programmatic priorities and environmental data operations. Most Regional work activities are
mandated by statute, regulations, or policy and are tracked. The primary components used in
annual planning are the anticipated annual budget, guidance from the various EPA Headquarters
program offices, and meetings held by the Regional Administrator with Senior Leadership early
in the fiscal year to set Regional priorities. The Deputy RA allocates resources to each Division
for the management and operation of specific programs based on the Region's anticipated
budget.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
34
Planning for QA must be fully integrated into this annual planning process. Support from the QA
Branch helps the Region meet Agency Government Performance Results Act (GPRA) goals,
program goals, and Agency commitments.
7.1.2 External Planning
Region 8 personnel such as DPMs are expected to coordinate planning for environmental data
operations with appropriate organizations (e.g., government agencies, and/or academic and
private organizations) to ensure annual grant work plans and/or contractor work plans are in
place. Close coordination and planning are essential to ensure that data are: (1) of enough quality
to support the intended use; and (2) obtained in the most efficient manner.
The primary vehicles for annual planning for extramural agreements are the budget process,
GPRA, U.S. EPA Agency and Regional strategic planning, and Performance Partnership
Agreements (PPAs)/National Environmental Performance Partnership System (NEPPs). Based
on the budget for the Region and guidance from the program managers in EPA Headquarters, as
well as Regional, State, Tribal identification of environmental issues, the Regional Administrator
and the Division Directors establish the Regional fiscal year priorities for use in Regional, State
and Tribal program planning, and develop a Regional plan for the fiscal year. This plan
establishes overall goals, priorities for resource utilization and distribution of the Regional
budget.
The end result of the above efforts is the establishment of overall operating plans for the Region
to meet the goals within each program based on state, tribal, regional, and other available
resources. The planning for QA must be fully integrated into this planning process and may be
documented in several forms, including State/Interagency QMPs, Tribal QMPs and/or QAPPs,
and contractor QMPs. Any specific QA requirements are included in the PPA's or work plans,
and a condition related to completing these requirements is included in the associated grant.
7.2 Program-specific Planning
All Regional environmental data operations conducted in support of environmental programs are
covered by this QMP, though not all require the same level of QA. The program is responsible
for establishing, and updating when necessary (e.g., initiation of a new program, incorporating
major statutory changes) the minimum quality assurance required to achieve program
compliance.
For many ongoing environmental monitoring programs, the National Program Offices at EPA
Headquarters have established standard QA/QC requirements. These QA/QC requirements may
be documented in a generic Program QMP or similar QA document. In these cases, the Region
shall defer to these national program documents. Any modifications or deviations from these
documents shall be documented.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
35
In the absence of existing National Program QMPs, Regional Program Offices are encouraged,
as appropriate, to create Program QMP(s) or a similar QA document to establish standard
programmatic QA/QC requirements. The appropriateness of installation of a Program QA
document is determined on a case by case basis by the DPM in coordination with the RQAM.
Any Regional Program QMPs must be reviewed and approved by the RQAM and must align
with the Region 8 Quality System described herein.
QA Branch Annual Planning
Annual systematic planning for the QA Branch ensures that resources are used efficiently to
accomplish the Region's QA activities.
The RQAM provides periodic updates to the LSAS Division Director and Deputy Director on
QA Branch work plan accomplishments and briefs senior managers about QA Branch work plan
activities and status twice annually (e.g., mid-year and end-of-year timeframes). Region 8 QMP
issues may also be documented in annual Federal Management Financial Integrity Act (FMFIA)
reporting.
Region 8 will provide reporting to OMS EQMD as requested.
73 Project-level Planning
Planning at the project level is paramount for all projects and tasks involving environmental data
operations that are conducted by or for the Region. Refer to Region 8's Project Life Cycle
diagram (Figure 2) to visualize the role planning has in the overall project life. There are four
components to the Project Life Cycle: Project Planning, QAPP Development, Implementation
and Assessment, and Assessment and Decision-making. The colored bands in the life cycle
roughly represent the relative time spent for each of the four stages. As seen, time spent in
Project Planning can be as much (or more) than time spent during Implementation.
Each project must employ a systematic planning process (described in Section 7.10) which
results in the development of a conceptual model (Section 7.11\ a sampling network design,
generation of appropriate data quality indicators, selection of measurement and analytical
methodologies, standard operating procedures, qualification and use of secondary data, records
management, assessment activities, data validation and assessment approaches, etc. sufficient in
detail to support defensible decision-making. EPA Order 2105.0 requires that the results of the
systematic planning process be documented in a Quality Assurance Project Plan (QAPP) and
approved by the RQAM (or Delegated QA Approving Officer) prior to implementation. The only
exception to this requirement shall be for environmental data operations that require immediate
action to protect human health and the environment or operation conducted under police powers.
In these cases, the required documentation must follow shortly after the action or operation.
Refer to Sections 7j5 and 7/7 for more detail about QAPP requirements.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
36
The DPM is responsible for ensuring that a systematic planning process is used and documented
and that all organizations and/or parties who contribute to the quality of the environm ental
project or use the results are identified and participates in the planning process. Guidance and
technical support for using a systematic planning process are available from the RQAM and the
Quality staff, upon request.
Figure 2. Project Life Cycle
7.4 QA Documents in Planning
EPA recognizes two key documents for use in planning QA activities: Quality Management
Plans (QMPs) and Quality Assurance Project Plans (QAPPs). QAPPs are sometimes also
referred to as Sampling and Analysis Plans (SAPs) or Field Sampling Plans (FSPs).
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
37
These QA documents are described in further detail in the next sections. Also, refer to Figure 3.
QA Document Types.
Figure 3. QA Document Types
The Quality Management Plan (QMP) describes
processes and procedures at the organizational
level, management and staff functional
responsibilities and line of authority. Include all QMP
elements (QA/R-2).
The Quality Assurance Project Plan (QAPP) is the
document that details the who, what, when, where,
why and how of the project objectives (data quality
objectives). Include all QAPP elements (QA/R-5 and
QA/G-5).
The Field Sampling Plan (FSP) provides the field
sampling details for the sampling event(s).
The Sampling and Analyses Plan (SAP) provides the
field and analytical details for the sampling event(s).
Standard Operating Procedures (SOPs) describe
specific procedures (e.g., sampling techniques or
field analytical methodologies) used. These are
attached to the approved QA Document(s).
Note:
FSPs, SAPs, and SOPs are subsets of a QAPP
FSPs and SAPs may be addendums to a previously
approved QAPP
or
a stand alone document, if all QAPP elements are
included (QA/R-5 and QA/G-5)
QA Document Types
Organization
to&
Project
7.5 Quality Management Plans (QMPs)
QMPs describe the broad view of an organization's quality system, including management/staff
responsibilities, organization structure, and lines of authority. An approvable QMP must address
all the required QMP elements. Guidance for preparing QMPs is found in EPA QA/R-2 (2000),
EPA Requirements for Quality Management Plans, reissued May 2006.
A Program-level or project-level QMP may be appropriate for projects that are complex in nature
(e.g., multiple projects of similar construct, emergency response activities, several operable units
under investigation at the same time, etc.) to establish standard programmatic QA/QC
requirements. The appropriateness of installation of a Program-level or project-level QMP is
determined on a case-by-case basis by the DPM in coordination with the RQAM. Any Regional
Program QMPs must be reviewed and approved by the RQAM and must align with the Region 8
Quality System described herein.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
38
QMPs are approved for a period of no longer than 5 years. Prior to its expiration, a QMP must be
revised and/or reissued to Region 8 for review and RQAM approval. All QMPs considered by
Region 8 shall be reviewed and documented using the Region 8 QMP review Crosswalk which is
based on the required elements in QA/R-2 Requirements for Quality Management Plans (2000),
reissued May 2006.
7.5.1 Annual QMP Reviews
All QMPs shall be reviewed annually by the organization that originated the QMP. Reviews
must be documented. The Region 8 QMP Review Crosswalk shall be used to document the
review.
This review assessment must include an evaluation of the effectiveness of the QMP. The process
of developing and annually updating the QMP provides an opportunity for management and staff
to review and clarify roles and responsibilities, to address problem areas, and to acknowledge
successes. Any changes to the organization's quality system must be documented by the
designated individual in organization's QMP as well as on the Region 8 QMP Review
Crosswalk. If the annual review reveals no changes to the QMP, this should be noted on the
Region 8 QMP Review Crosswalk and reported in the organization's annual QA report to Region
8.
If minor revisions are necessary, the QMP's organization shall document those in the annual
report, which is submitted to the RQAM. Major revisions in quality systems or organizational
structure require that the QMP be revised and resubmitted by the organization for review and
approval by the RQAM. The completed crosswalk must accompany the approved QMP. These
companion documents must be maintained by the DPM in accord with the Regional records
management requirements (Section 13.0). The DPM is responsible for ensuring all versions of
QMPs are controlled and distributed, their versions tracked, and older versions are removed from
circulation.
QMP approvals are valid for a period not to exceed five (5) years4'5. Providing the organization
continues to receive funding and/or produce work for EPA beyond the QMP's operating period,
the organization shall submit an updated QMP (prior to expiration date) for review and approval
by the RQAM. A revised QMP may be submitted at any time, but is required under certain
conditions including:
Expiration of the five-year life of the approved QMP;
Any major change to the organization's quality system;
A significant change in the organization's mission;
Change in delegation status in the program.
4 EPA Order CIO 2105.0, pg. 7.
5 EPA Quality Manual for Environmental Programs (CIO 2105-P-01-0), May 5, 2000; pg. 3-3.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
39
Annual Reporting of QMPReviews
The organization shall include the results of their annual QMP review in their annual QA report
to the RQAM. For the QMP reviews, the report shall include the completed Region 8 QMP
Review Crosswalk, a summary of any changes to the QMP as a result of the review, the date of
the review, and the original QMP approval date (i.e., date of EPA's RQAM signature). The
annual QA report shall be provided to the RQAM. For extramural agreements, the EPA
Contracting Officer and Project Officer are responsible for providing the annual QA report to the
RQAM.
7.5.2 QMP Tracking System
Although the QA Branch tracks all QMPs that are reviewed and/or approved by the RQAM, the
DPM must also maintain an accurate complete copy of the QMP. The QA Branch tracks internal
programmatic QMPs as well as those developed by contractors and extramural parties (court
order respondents, etc.). Information tracked includes hard and electronic copies of the QMP:
title, review status, and approval date. Copies of all QMPs approved by Region 8 are maintained
according to records management requirements. (Section 13.0).
7.6 Quality Assurance Project Plans (QAPPs)
All work funded or conducted by EPA that involves the acquisition of environmental data or
information generated from direct measurement activities, collected from other sources {i.e.,
existing data), or compiled from computerized databases and information systems shall be
performed in accordance with an approved QAPP or an equivalent QA planning document (Text
Box 4). Equivalent documentation is determined on a case-by-case basis in consultation with the
RQAM and is based upon the project quality objectives and the intended use of the data. The
QAPP must be developed using a systematic planning process (Section 7.10). The results of the
systematic planning process (Section 7.10) must be documented in a QAPP. DPMs are
responsible for ensuring that QAPPs are developed and approved prior to project initiation for all
projects under their authority. Guidance for preparing QAPPs is found in EPA QA/R-5 (2001),
EPA Requirements for Quality Assurance Project Plans, reissued May 2006.
To be approvable, a QAPP must address all the QAPP elements outlined in QA/R-5. The RQAM
has the authority to define any special requirements beyond those listed in EPA QA/R-5. If a
QAPP element does not apply to the project, the element must be included, and an explanation
provided for why it does not apply. A QAPP is sometimes referred as a Field Sampling Plan
(FSP) or Sampling and Analysis Plan (SAP), which is acceptable, if the QA document contains
all the required QAPP elements. On occasion, an SAP or FSP may not include all the required
QAPP elements; however, in this case the SAP or FSP must be appended to a previously
approved companion QAPP. In this case, all documents must be reviewed as a single QAPP; the
DQOs within the companion QA documents must remain consistent with the original QAPP.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
40
Text Box 4. Approved QAPPs are Required Prior to Initiating Work
QAPPs must be approved prior to any data gathering work or use, except
under circumstances requiring immediate action to protect human health and
the environment or operations conducted under police powers.
EPA Order 2105.0 (6.a.7).
QAPP Approval Authority
All environmental data operations conducted by EPA or funded by EPA must have an approved
QAPP in place prior to the initiation of data collection. The only exception to this requirement is
for environmental projects that require immediate action to protect human health and the
environment or operations conducted under police powers. The RQAM has the authority and
responsibility to approve all extramural and intramural environmental data operation QAPPs,
unless delegated as described in this QMP. Refer to Section 8.3.2 for delegated approval
authority. In cases with circumstances requiring immediate action (Text Box 4) and preparation
of the QAPP or functionally equivalent document is delayed, the QAPP must be approved within
30 days of the incident date.
Implementing and Revising QAPPs
The DPM shall ensure that the QAPP is implemented as approved. The DPM is responsible for
ensuring all versions of the QA document are controlled and distributed, the versions tracked,
and older versions are removed from circulation. The RQAM reserves the right to expire a
QAPP, SAP, FSP or similar document that is not reviewed and updated annually.
It is essential that project QA documents such as a QAPP and any companion documents to the
QAPP such as SAPs or FSPs be kept current and that all personnel involved in the work have
easy access to a current version of the QAPP. During the approval period of a QA document
when changes affect the scope, implementation, or assessment of the outcome, the plan shall be
revised to keep project information current. The DPM, with the assistance of the RQAM,
determines the impact of any changes on the technical and quality objectives of the project.
Although the approved QAPP must be implemented as prescribed; it is not inflexible. During the
period when an approved QAPP is being implemented, because of the complex and diverse
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
41
nature of environmental data operations, changes to original plans may be needed. A QAPP may
be modified and amended at any time. Changes may be either:
Substantive (i.e., significant) modifications (e.g., changes in project scope,
organizational restructuring, laboratory, sampling parameters or procedures, analytical
methods, project/data quality, objectives, or use of the data) must be documented in either
an amended or revised QAPP, as appropriate, and undergo a review and approval process
similar to that of the original QAPP prior to implementation of the changes. A revised
QAPP (or QAPP Addendum) must be accompanied by a completed Region 8 QA
Document Review Crosswalk when it is submitted to the RQAM (or designee) for review
and approval. Only after the revision has been received and approved shall the change be
implemented.
Non-substantive (minor) QAPP modifications may be necessary to address changing
site/project conditions and to ensure project objectives are met. Such changes must be
documented and approved by the DPM prior to implementation and reported to the
RQAM as part of the project report (or annual QA document review if applicable).
The RQAM shall arbitrate disputes regarding whether a change is minor or substantive.
7.6.1 Annual QA Document Reviews
For projects or programs (having an approved QAPP) that extend beyond one year, the DPM
and/or authoring organization must review the existing approved QAPP at least annually to
ensure it continues to be appropriate for the project. Expired QAPPs are not eligible for annual
review; instead the expired QAPP must be submitted for review as a new or revised QA
document. The RQAM reserves the right to expire a QAPP, SAP, FSP or similar document that
is not updated annually.
The annual review process must include a comparison to the negotiated statement of work,
workplan, or similar document to confirm that consistency with the QA Document. Any changes
to the organization's quality system must be documented in the revised QA document (QAPP,
SAP and/or FSP) as well as on the Region 8 QA Document Review Crosswalk.
The annual review must also ensure and document the continuing competency of laboratories
performing upcoming project work. The review must verify all URLs (web links) in the QAPP.
To determine if an annual review is applicable, the following apply:
If the annual review of a QAPP identifies one or more substantive change to the QAPP or
project is required, the QAPP must be revised, submitted to the RQAM (or designee)
with a completed QA Document Review Crosswalk, and reviewed and approved as that
of the original document. The annual review process should be terminated as soon as a
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
42
substantive change is identified. Instead, the preparation of a new or revised QAPP is
necessary.
If the annual review shows that neither the project nor the QAPP require substantive
change, the DPM shall identify all non-substantive changes made to the QAPP during the
previous year on the QA Document Review Crosswalk. For extramural agreements, the
EPA Contracting Officer and Project Officer are responsible for providing the annual QA
report.
The Annual QAPP Review crosswalk must identify all changes from the previous year's
QAPP in the comments section. If the annual review reveals no changes to the QA
document(s), this should be noted on the Crosswalk and reported in the organization's
annual review report. This shall be accompanied by a new signature page, and any
updates or changes pages and sent to the RQAM (or designee) for review and approval.
Once approved, the signature page, crosswalk and other accompanying documents
become a part of the QAPP and document the extension of the QAPPs effective period.
7.6.2 QA Document Review Tracking System
Document review databases developed by the QA Branch are used to monitor and track the
status of reviews or approvals of QA planning documents. The Microsoft Excel databases and
Max.gov databases track each document from initial submittal through one or more iterations to
final approval. If an approved document is amended or revised, the revised document is assigned
a new record. A copy of the approved QA documents must be maintained in accordance with the
Regional records management requirements (Section 13.0).
7.7 Uniform Federal Policy for QAPPs
The Uniform Federal Policy for Quality Assurance Project Plans (UFP-QAPP) Manual was
developed as a joint initiative among several government agencies, EPA, Department of Defense
(DoD) and Department of Energy (DOE), to establish a standardized QAPP format for use on all
Federal facility hazardous waste sites. Originally issued in 2005, the UFP-QAPP was developed
to provide procedures and guidance for consistently implementing the nation consensus standard,
American National Standard ANSI/ASQ E4-2004, for the collection and use of environmental
data at Federal Facilities. The ANSI/ASQ 2004 is now updated to the American Society for
Quality/American National Standard Institute (ASQ/ANSI) E4 2014.
The Office of Solid Waste and Emergency Response (OSWER) Directive 9272.0-17, June 2005,
requires the use of the UFP-QAPP at Federal facilities where environmental data are collected.
For purposes of the OSWER Directive, Federal facility projects include all projects for which a
Federal agency (e.g., DoD, DOE) or its components are responsible. This includes all Base
Realignment and Closure Act (BRAC) related projects and those Formerly Used Defense Sites
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
43
(FUDS) or Formerly Used Sites Remedial Action Program (FUSRAP) related projects where the
U.S. Army Corps of Engineers is the lead project manager.
The UFP-QAPP template may also be applied to non-Federal facilities where environmental data
collection related to hazardous waste investigations occurs for the purpose of cleanup under the
Comprehensive Environmental Response, Compensation and Liability Act (CERCLA) program
and the Resource Conservation and Recovery Act (RCRA) corrective action program. The UFP-
QAPP is also designated for data collection related to the active management of hazardous waste
generated by RCRA facilities. Thus, the UFP-QAPP may be used if the site under investigation
meets these conditions.
Additionally, contracts that support CERCLA may require the UFP-QAPP format.
EPA Region 8 recognizes that adherence to the UFP-QAPP Manual will result in full compliance
with QA/G-5 and QA/R-5 for environmental data collection efforts under CERCLA and RCRA
at Federal or non-Federal facilities. The DPM is responsible for ensuring that the UFP-QAPP is
completed in accordance to the UFP-QAPP Manual and Region 8 policies. Specifically, the
UFP-QAPP must adhere to the QAPP documentation (i.e., completion of the Region 8 QA
Document Review Crosswalk requirements for implementation, approvals, annual reviews,
records managements, etc. as outlined in this QMP.
7.8 Standard Operating Procedures (SOPs)
The use of standard operating procedures (SOPs) serves to ensure comparability across programs
and individual environmental data operations. SOP use is encouraged as a means of ensuring that
routine or repetitive activities, processes or procedures are performed consistently and with the
level of quality needed to support the respective project or program goals. SOPs may describe
technical and administrative operational elements. Whatever the context, SOPs shall thoroughly
describe steps and techniques for the activity and will be sufficiently clear to be understood by a
person with knowledge in the general concept of the procedure or process. Any limitation on the
use or applicability of a specific SOP shall be documented in the SOP itself. Guidance for
preparing SOPs is found in EPA QA/G-6, Guidance for Preparing Standard Operating
Procedures and is available on EPA's Quality website (http://www.epa.gov/region8/qa/).
The DPM is responsible for applying the scientifically and technically appropriate SOPs that
meet project, regulatory and EPA decision-making goals. To avoid duplication, the DPM is
responsible for reviewing exiting Region 8 SOPs that are approved for use to determine if any
are suitable as written for the DPMs purposes. If the DPM deems preparation of a new SOP
appropriate, the DPM must ensure that the SOP is reviewed and approved (with signature) prior
to distribution.
Document control procedures must be followed for the control documents such as SOPs,
checklists, templates, forms and include the appropriate document and revision numbers,
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
44
maintain copies of all versions, and remove older versions from distribution. Refer to the
QAFAP Overarching Document Control SOP (R8FQP-010) for information about document
control procedures for Regional field activities. Refer to Region 8's SOP writing SOPs (R8FQP-
001) and SOP templates for Administrative and Technical SOPs: R8FQPTemplate-001 and
R8FQPTemplate-002, respectively.
7.9 Definition and Use of Existing Environmental Data
Although not a new concept, the term existing (sometimes referred as "secondary")
environmental data may seem new to many data users in Region 8. In brief, existing
environmental data are not collected through direct measurement (termed primary data) but are
obtained from an existing source. Refer to Text Box 5 for the complete definition of existing
environmental data (Text Box 5). As seen, existing environmental data are those obtained from
any secondary source, which may be internal or external to EPA. A large existing source is data
generated during previous EPA investigations as well as obtained from other government
agencies, industries, GIS, databases, surveys, literature searches, etc.
Text Box 5. Existing Environmental Data Definition
Existing (Secondary) Environmental Data is defined as:
Any measurements or information that describe environmental processes,
location, or conditions; ecological or health effects and consequences; or the
performance of environmental technology obtained from other sources
(internal or external) such as those compiled from databases, data reports,
literature, surveys, or produced from models.
Universal definition as defined in ANSI/ASQ E4-2004 and EPA Orders CIO 2105.0/2106.0
EPA Quality Manual for Environmental Programs (CIO 2105-P-01-0)
It may be surprising to realize that existing (secondary) data is typically the lion's share of data
used in decision-making and/or designing new data collection efforts. Because of the key role
existing data has in decision-making or in new study design at EPA, the quality of the
existing data needed for its use must be documented in a QAPP, just as is required for
collection of primary data.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
45
Use of Existing Data Requires a QAPP
Prior to its use, environmental data obtained from existing (secondary) sources shall be evaluated
to ensure a level of quality that is commensurate with its intended use. The process for obtaining,
evaluating and using the existing data must be documented in a project-level QAPP which shall:
Identify the types of data needed for project implementation or decision-making;
Describe the intended use of the data;
Define the acceptance criteria for use of the data;
Specify any limitation on the use of the data; and
Identify the individual(s) responsible for evaluating and qualifying the data.
For those projects which involve the compilation and use of environmental data from existing
(secondary) sources exclusively {i.e., there will be no direct environmental data generation
performed to accomplish the project), a project-level QAPP is required. The level of detail for
this QAPP will differ from that for a direct environmental data generation project. Assistance
with determining the appropriate elements for a QAPP for projects involving existing data
sources may be provided by the QA Branch, upon request.
The DPM is responsible for ensuring a QAPP is prepared for projects involving use of existing
environmental data.
7.10 Systematic Planning Using Data Quality Objectives
The hallmark of all successful projects, studies and investigations is a planned data collection
process that is conducted following the specifications given by an organization's Quality System.
EPA Orders CIO 2105.0 and 2106.0 require that before information or data are generated or used
for Agency-funded or regulated environmental programs and projects, a systematic planning
process must occur during which performance or acceptance criteria are developed for the
collection, evaluation, or use of these data. For this reason, systematic planning is a key
component of EPA's Quality System.
Systematic planning is a process based on the widely accepted "scientific method" and includes
concepts such as objectivity of approach and acceptability of results. The process uses a
common-sense approach to ensure that the level of documentation and rigor of effort in planning
is commensurate with the intended use of the information and the available resources. The
systematic planning approach includes well-established management and scientific elements that
result in a project's logical development, efficient use of scarce resources, transparency of intent
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
46
and direction, soundness of project conclusions, and proper documentation to allow
determination of appropriate level of peer review.
EPA's approach to systematic planning is the Data Quality Objectives (DQOs) process, which is
outlined in the guidance document Systematic Planning Using the Data Quality Objectives
Process QA/G-4. The DQO Process is a stepwise approach that guides managers or staff to a
resource-effective plan for the acquisition of environmental data. It is both flexible and iterative,
and applies to both decision-making (e.g., compliance/non-compliance with a standard) and
estimation (e.g., ascertaining the mean concentration level of a contaminant). The DQO Process
is used to establish performance and acceptance criteria, which serve as the basis for designing a
plan for collecting data of enough quality and quantity to support the goals of the study. Use of
the DQO Process leads to efficient and effective expenditure of resources; consensus on the type,
quality, and quantity of data needed to meet the project goal and/or decision; and the full
documentation of actions taken during the development of the project.
The DPM is responsible for ensuring that (1) the DQO Process (or a functionally equivalent
systematic planning process) is used and documented; and (2) all organizations and/or parties
who contribute to the quality of the environmental project or use the results are identified and
participates in the planning process.
Team Approach
Region 8 encourages the team approach for those projects for which expertise is not available
within the Program managing the project. Each DPM is encouraged to involve as many
professionals on his/her site team as relevant to site or project issues. The DPM retains the
primary responsibility to ensure the environmental data under his/her auspices meet the
objectives of the project/program.
Involving Data Users in the Planning Process
DPMs are strongly encouraged to include data users (including, but not limited to: states, tribes,
PRPs, toxicologists, hydrologists, GIS specialists, modelers, enforcement personnel, permit
writers, community groups or the general public) in the development of DQOs, QAPPs and other
QA planning documents.
Elements of a systematic planning approach should include:
Identification and involvement of the project manager, sponsoring organization and
responsible official, project personnel, stakeholders, scientific experts, etc. (i.e., all
customers and suppliers);
Description of the project goal, objectives and questions/issues to be addressed;
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
47
Identification of project schedule, budget, milestones, and any applicable requirements
(e.g., regulatory requirements, contractual requirements);
Identification of the type of data needed and how the data will be used to support the
project's objectives;
Determination of the quality of data needed and specification of the performance criteria
for measuring quality;
Description of how, when, and where the data will be obtained including existing data
and identification of any constraints on data collection;
Specification of needed QA and QC activities to assess the quality performance criteria
(e.g., QC samples for both the field and laboratory, audits, technical assessments,
performance evaluations, etc.);
Description of how the acquired data will be analyzed (either in the field or the
laboratory), evaluated (i.e., QA review, validation, verification), and assessed against its
intended use and the quality performance criteria.
7.11 Conceptual Models (Conceptual Site Models)
EPA Order CIO 2105.0 requires that before environmental data or information are collected or
used on Agency-funded or regulated environmental programs and projects, a systematic planning
process must occur. EPA's systematic planning process is the DQO Process, described in Section
7.10. requires the use of a conceptual model, often referred as a project lifecycle conceptual site
model (CSM). Per EPA's DQO Guidance (QA-G4), it is critical to carefully develop an accurate
conceptual model of the environmental problem, as the model summarizes information that is
currently known, how it relates to the project's goals, and serves as the basis for all subsequent
inputs and decisions.
A CSM is a comprehensive graphical and written summary of what is known or hypothesized
about environmental contamination at a site and the relationships among key site information
that are pertinent to decision-making. It depicts known or potential:
Sources of contamination
Contaminants
Movement of contamination through the environment
Media that are contaminated or may become contaminated
Exposure scenarios and human health or ecological receptors
The planning team will typically begin by developing a conceptual model of the problem, which
summarizes the key environmental release, transport, dispersion, transformation, deposition,
uptake, and behavioral aspects of the exposure scenario which underlies the problem.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
48
Additionally, the CSM may also present or be used to develop potential benchmarks or action
levels for the program or project. The conceptual model is an important tool for organizing
information about the current state of knowledge and understanding of the problem, as well as
for documenting key theoretical assumptions underlying an exposure assessment.
It is important to identify theories and assumptions underlying the conceptual model to ensure
adequate transparency. If the problem is complex, the team may consider breaking it into more
manageable pieces, which might be addressed by separate studies. Priorities may be assigned to
individual segments of the problem and the relationship between the segments examined.
The DPM is responsible for ensuring that the DQO Process (or functionally equivalent
systematic planning process), including development of a conceptual site model, is applied and
documented whenever environmental data are collected or used.
Tools for developing a CSM are available on the Region 8 QA Program website.
7.12 Policy for Geospatial Data
Geospatial data provide EPA with the capacity to spatially locate, identify, and assess aspects of
the environment critical to program operations. The National Geospatial Data Policy, CIO 2131
establishes principles, responsibilities, and requirements for collecting and managing geospatial
data used by Federal environmental programs and projects within the jurisdiction of the EPA.
This Policy applies to all EPA organizations, grantees, agents working on behalf of EPA, tribes,
localities and partner states of EPA who directly or indirectly design, develop, compile, operate,
or maintain EPA information collections developed for environmental program support. By
reference, the National Geospatial Data Policy includes within it the commitment to implement
the requirements specified by the National Spatial Data Infrastructure (NSDI) and to abide by the
guidelines and data standards of the Federal Geographic Data Committee (FGDC).
There are a number of different types of projects that utilize geospatial information. Some
projects involve the development of new techniques and/or new underlying spatial data to
answer specific questions. Others utilize Geographic Information Systems (GIS) to manage
existing data, while still others utilize GIS to manage newly collected data or a combination of
both old and new data. GIS is a collection of computer hardware, software, and geographic data
designed to capture, store, update, manipulate, analyze, and display geographically referenced
data.
Projects require an approved QAPP prior to generation or use of geospatial data. Examples of
this are projects that involve in-field data collection of new spatial data (e.g., Global
Positioning System points or real-time mobile mapper) or development of new procedures
to support environmental decisions (i.e., GPS coordinates during a wetlands survey).
All QAPPs that include or require geospatial or GIS data generation or use shall conform to the
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
49
following policies and guidance. DPM should consult with a Region 8 Data Coordinator (or
designee) prior to the QAPP approval to ensure the proper geospatial data are collected.
CIO 2131 -P-01 -0, Procedure for Geospatial Metadata Management;
OMB Circular A-16, Coordination of Geographic Information, and Related Spatial Data
Activities;
Guidance for Geospatial Data Quality Assurance Project Plans.
The DPM is responsible for ensuring that generation and use of geospatial data meets the policy
requirements described herein.
7.13 Peer Review in Project Planning
Peer review is a critical part of the environmental data generation process that enhances the
credibility and integrity of scientific and technical documents and ensures the quality of products
disseminated by Region 8. Peer reviews are documented reviews of scientific and technical work
products by qualified individuals (or organizations) that are independent of those who performed
the work but are collectively equivalent in technical expertise. A peer review is performed to
ensure that activities are technically adequate, competently performed, properly documented, and
satisfy established technical and quality requirements. The peer review is an in-depth assessment
of the assumptions, calculations, extrapolations, alternate interpretations, methodology,
acceptance criteria, and conclusions pertaining to specific work and of the documentation that
supports them.
To be most effective and efficient, peer review of a scientific or technical work product must be
incorporated into the up-front planning of any decision or action that will be made or taken based
on the work product. The planning for peer review activities should occur at the time of overall
project planning and includes obtaining the proper resource commitments and establishing
realistic schedules to accommodate the peer review process. Peer review is not restricted to the
nearly final version of a report or similar work product. In fact, peer review of the planning
documents such as QAPPs, study designs or research plans can often be extremely beneficial to
avoid fundamental and costly errors in the study design. Proper advance planning by the project
or program manager is essential to ensuring a positive and seamless peer review experience.
The peer review requirements are different depending on the nature of the work product and/or
relevant statutory requirements. Whatever the nature of the work product, peer review is
encouraged and expected for all scientific and technical information that is intended to inform or
support Agency decisions. General guidelines for informal and formal peer review are outlined
in the EPA Peer Review Handbook, 4th edition. Specific peer review requirements for the
various types of work products are summarized in Section 9.9 of this QMP.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
50
7.14 Information Quality Guidelines and Pre-Dissemination Review in
Project Planning
EPA's Information Quality Guidelines (IQGs) describe EPA's policy and procedural guidance for
ensuring and maximizing the quality of information the Agency disseminates. Thus, the IQGs
are integral to the Regional Quality System for ensuring the quality of EPA's data products and
information. "Information" generally includes any communication or representation of
knowledge or position/policy such as facts or data in any medium or form. This includes
"preliminary" information that EPA has endorsed or adopted as well as conclusions or facts
drawn from or based upon other existing information. This QMP incorporates by reference all
definitions, principles, policies and procedures found in EPA's IQGs: Guidelines for Ensuring
and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by
the Environmental Protection Agency.
To be most effective and efficient, both the IQGs and pre-dissemination review should be
incorporated into the up-front planning for any information that will be disseminated by Region
8. That is, the plans for and process of pre-dissemination review must be addressed in the
program's and/or project's QAPP or other QA document(s).
During the planning process and prior to information release, staff should inform their direct
supervisor of information products that are being developed for dissemination as described in the
IQGs. Questions regarding applicability of the IQGs to a particular information product can be
directed to the Region 8 Information Quality Guidelines Officer. The Region 8 QA Branch
Chief serves as the IQG Officer for Region 8. Supervisors are responsible for ensuring that
information subject to the IQGs is identified before it is disseminated by their group. Supervisors
should also determine what mechanism for reviewing the information is appropriate before it is
disseminated. Each DPM is responsible for ensuring that the IQGs are applied appropriately to
their respective projects. Refer to Section 9.8 for specific IQGs requirements.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
51
8 IMPLEMENTATION (DO)
The implementation phase or Do component of the PDCA quality model is certainly the most
visible and active portion of the Project Life Cycle and may represent about one quarter of the
total time spent on the project (Figure 2). Successful execution of the project is paramount to
ensuring generation and use of defensible data.
8.1 Implementation of Work Processes
Complete application of the Planning Process (Section 7) paves the way
to successful and efficient implementation of environmental data
operations. Implementation occurs at all three levels: Regional,
Program and project-level. Thus, assurance that work processes are
implemented in accord with EPA Order 2105.0 requires Region 8 managers and staff to
implement the planning, review and approval, assessment, and reporting activities outlined
throughout the QMP.
8.2 Authorities Delegated to the Regional QA Manager (RQAM)
EPA Order 2105.0 (formerly 5360.1 A2) cites the U.S. EPA Delegation of Authority Chapters 1-
41 Mandatory Quality Assurance Program, 1200 TN 496 01/28/2000 which specifies that the
Assistant Administrator for the U.S. EPA Office of Environmental Information is authorized to
develop, coordinate and direct the implementation of the Agency-wide quality system. The
implementation of the Agency-wide quality system includes U.S. EPA program offices,
laboratories, Regional Offices as well as extramural organizations such as contractors and
grantees, which generate or use environmental data on behalf of U.S. EPA.
The Assistant Administrator for OMS is designated as the Agency Senior Management Official
for Quality and has delegated this authority to the Director, Enterprise Quality Management
Division, as the central management authority for this program.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
52
The senior manager of each U.S. EPA organization is responsible for ensuring that their
organization complies with requirements for the Agency-Wide Quality System. Further, the
Senior Manager for Quality is required to authorize a Quality Assurance Manager (QAM) to
oversee the implementation of the quality system. In Region 8, the Regional Administrator is that
designated official.
Region 8 has a centralized quality system, authorized by the Regional Administrator, who has
delegated QA responsibilities and authority to the Regional Quality Assurance Manager
(RQAM) through the Division Director of the Region 8 Laboratory Services and Applied
Sciences Division (LSASD).
8.3 QA Document Approval Authorities
EPA Order 2105.0 authorized OEI (now OMS) to develop, coordinate and direct the
implementation of the Agency-wide quality system, thereby delegating OEI the authority to
approve Region 8's QMP. The RQAM delegations include the authority to approve QA
documents for Region 8. The RQAM has exercised authority to delegate limited approval
authority for some project-level QA documents to Region 8 personnel. Delegated QA
Approving Officers (DAOs). For more information about DAOs and their approving authority,
refer to Section 8.3.2.
8.3.1 RQAM QA Document Approval Authority
The RQAM retains authority for approving the following QA Documents:
Federal, State, Tribal QMPs;
Primary Quality Assurance Organization QMPs and QAPPs;
Region 8 Program-level QA Documents;
QMPs developed by contractors and grantees using EPA funds;
Interagency Agreement QA Documents;
QA Documents prepared by EPA personnel (i.e., not prepared by contractors or
grantees);
QA Documents for EPA-lead sampling events not involving grant or contract support for
field sampling;
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
53
Enforcement actions, as appropriate. Refer to the individual enforcement action for
information about EPA's QA document approval authority (See Text Box 6); and
QA Documents currently in review with the Region 8 QA Branch.
In addition, the RQAM retains authority for approving QA Review Forms (QARFs) for all
contract/procurement actions unless a Division-level QMP is approved that does not restrict
national delegations. Figure 4 illustrates the hierarchy of QA document approval authorities. For
more information about QA requirements for contract actions, see Sectionl2.1.
Text Box 6. Special Considerations for Enforcement Actions
Special Considerations for Enforcement Actions:
For enforcement actions such as Administrative Orders (AOs) or Administrative
Orders on Consent Decrees (AOCs), QA Document approval authority may differ
from guidelines presented here. Refer to the individual Order for information about
EPA's QA document approval authority.
Note: It is recommended that OA requirements are specified in each Order.
Additionally, it is recommended that each Order specify the EPA person(s)
responsible for approving OA documents prepared by the responsible parties or
potentially responsible parties (RPs or PRPs).
A
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
54
Figure 4. QA Document Approval Authority Hierarchy
OEIOffice of Environmental Information
AAAssistant Administrator
RQAMRegional Quality Assurance Manager
SAPSampling and Analysis Plan
QAPPQuality Assurance Project Plan
QMPQuality Management Plan
FSPField Sampling Plan
SOPStandard Operating Procedure
8.3.2 Delegated QA Approving Officer (DAO) Authority and
Responsibilities
EPA Region 8 personnel working in the Program Offices are versed in their respective regulatory
requirements and/or familiar with pertinent project goals necessary to support program decisions.
It is in Region 8's best interest to leverage this knowledge base. As such, when certain conditions
are met, the RQAM may delegate authority to an individual, to approve project-level QA
documents such as QAPPs, FSPs and SAPs. DAOs may also review and approve QAPPs
associated with Interagency Agreements only if the Interagency Agreement has an EPA-
approved QMP.
The Delegated QA Approving Officer (DAO) represents the RQAM by reviewing, approving
and signing a project-level QA document. Signing a QA document is an Agency action;
therefore, DAOs must adhere to certain requirements. The requirements for eligibility to obtain
and maintain delegated approval authority are outlined below.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
55
Candidates for Delegated QA Approving Officer Status
To become a DAO, the candidate must:
Complete Delegated QA Approving Officer Training and/or other training as required by
the RQAM;
Receive first line supervisor concurrence to receive DAO status;
Send the RQAM an email requesting delegated authority for project-level QA document
approval and indicate the supervisor's concurrence (e.g., copy the individual's
supervisor);
Receive notice of DAO status from the RQAM; and
Agree to abide by DAO Agreement Statements for review, reporting and oversight.
A DAO's authority to approve project-level QA documents begins the date of the RQAM's
email documentation awarding delegation. DAO authority will remain in effect until or unless
the DAO:
Leaves Region 8 (e.g., retirement or transfer to another EPA location);
Voluntarily requests to have his/her DAO status rescinded and has supervisor
concurrence;
Does not maintain DAO training requirements; and/or
Does not abide by the DAO Agreement Statements including responding to data calls.
DAO Agreement Statements
To maintain project-level QA document approving authority, the DAO agrees to:
Ensure appropriate EPA-approved project-level QA document(s) are in place prior to
collection, generation or use of environmental data or environmental technology, except
under circumstances requiring emergency response immediate action to protect human
health and the environment or operations conducted under police powers;
Review the project-level QA documents using the Region 8 QA Document Review
Crosswalk located on the QA Program website (http://www.epa.gov/region8/qa);
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
56
Maintain independence of the data generation activities; review only those project-level
QA documents that were prepared by a grantee/contractor (not the Approving Officer).
Track all QA documents that are reviewed, noting whether or not the documents were
approved. Use the DAO tracking tool available on the QA Program website
(http://www.epa.gov/region8/qa);
Maintain a file of all reviewed and/or approved QA documents and their companion
Crosswalks;
For QA documents that are deemed approvable by the DAO, sign and date the project-
level QA documents and create an electronic file (PDF) of the entire document, including
all attachments and the completed signature page;
Report project-level QA document review and approval status to the RQAM, as
requested;
For emergency response action excepted from having EPA-approved project-level QA
document(s) in place prior to collection, generation or use of environmental data or
environmental technology, provide a copy of required documentation to the RQAM, as
requested;
Provide information as requested by the RQAM to support periodic quality systems
assessments (QSAs) of the project-level QA document reviews and approvals; and
Complete any training for Delegated QA Documents Approving Officers as required by
the RQAM. (Section 11.2).
DAO Signatory Approvals on Project-level QA Documents
When approving a project-level QA document, the DAO must include the title of Delegated QA
Approving Officer on the signature line in addition to any other titles desired by the DPM (e.g.,
Remedial Project Manager, Site Assessment Manager, On Scene Coordinator, etc.)
A final note: if a team has more than one team member who is a certified DAO, only one person
may be designated as the Delegated QA Approving Officer for the project. That person alone has
authority to act as DAO for the project.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
57
DAO Tracking and Reporting Requirements
Delegated QA Approving Officers are responsible for tracking and maintaining copies of all QA
documents that they review and/or approve. Periodically (not more frequently than quarterly),
the DAO will provide the RQAM with the following information:
OA Document Review Statusa list of documents in review or already reviewed
including review type (new document or annual review), approval status (approved or not
approved) and approval date, if approved.
QA Documents Reviewedelectronic copies of all QA documents reviewed in pdf
format. All approved QA documents must include the approval signatures & dates; and
Review Crosswalkelectronic copies of completed Region 8 QA Document Crosswalk
for each accompanying QA document.
If the review/approval status for a particular QA document has not changed since the last
reporting period, it is not necessary to report on that QA document status until or unless its status
changes.
9 EVALUATION & ASSESSMENT (CHECK)
The EPA Orders CIO 2105.0 and CIO 2106.0 require all Agency employees involved in
environmental data activities and any extramural organization or individual conducting
environmental activities in support of EPA activities (e.g., grantees, contractors, etc.) to comply
with the Agency's Quality System requirements. To implement and maintain the Quality System,
the EPA Orders require an annual assessment of the effectiveness of the Region 8 Quality
System and implementation of corrective actions based on the assessment results. Evaluation and
assessments are the Check component of the PDCA quality model. Corrective actions taken as a
result of the assessments are governed under the Act portion of the PDCA quality model (Section
10.0Y
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
58
9.1 Assessments
EPA Orders 2105.0 and 2106.0 require the Region to assess the QA and QC activities applied to
environmental programs conducted by or for EPA. An assessment is an all-inclusive term used to
describe an evaluation process used to measure the performance or effectiveness of a system and
its elements. Some examples of assessments include: audit, inspection, management systems
review, peer review, performance evaluation, quality system assessment, technical audit and
surveillance, among others.
Region 8 uses a multi-faceted approach to evaluate and assessing quality systems:
Quality Systems Assessments (QSAs)
Technical Systems Audits (TSAs)
Annual Review of QA Documents (QMPs, QAPPs, SAPs, FSPs)
Implementation of the Information Quality Guidelines
Data Quality Indicators
Data Validation and Verification
Data Quality Assessments
Peer Review
Overall, the outcome of any assessment is expected to:
Identify strengths and weaknesses;
Cause corrective actions to be taken to alleviate problems;
Facilitate the initiation of changes to enhance the QA program;
Serve as a vehicle for providing technical assistance;
Enhance awareness and understanding of QA/QC policies and procedures; and
Provide a measurement of the effectiveness of QC in assuring the quality of data.
Any assessment for environmental data activity must outline how and at what frequency the
assessment will be conducted. Each planning document must describe: the process for
implementing and documenting the required assessments; the required assessor qualifications;
the process for monitoring and documenting the corrective actions; and the requirements for
reporting results to management.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
59
Frequency and Assessment Selection Criteria
It is the DPM's responsibility to assign the types and identify the frequency of assessments
that are germane to their environmental data activities (generation or use). The selection
criteria may change with project, program and/or activity within a project or program. The
DPM will ensure that the QA planning documents outline the assessments needed for each
environmental data activity.
Criteria for Personnel Conducting Assessments
Personnel conducting an assessment must be independent of work being assessed. Assessors
must have sufficient experience and training to conduct a particular assessment. Assessment
personnel must be granted sufficient authority, access to programs and personnel (staff and
manager), access to documents and records, and organizational freedom to conduct the
assessment.
9.2 Assessment Tools and Other Resources
Selection of the appropriate assessment tool depends upon whether a quality system or technical
system is being assessed. Technical system assessment tools are also dependent upon the stage of
the project being assessed. Table 2 outlines the types of assessment tools that may be used.
Table 2. Assessment Tools and Application
Assessment Focus
Appropriate Tool
Comments
Quality Systems
Quality Systems Assessment
(QSA)
Assesses conformance to a
documented quality system
through collection of
information and documented
evidence of implementation.
Planning
QA Project Plan Review; Site
scoping meeting and/or visit
QAPP Review; Peer Review
Sampling/Field Activities
Technical Systems Audit
Field Audit
Analysis
Technical Systems Audit
Laboratory (Field or Fixed)
Audit; Proficiency Testing
Sample
Data Evaluation/Assessment
Data Audit
Data audit inspection; Data
verification and validation
Data Usability
Data Quality Assessment
Data Quality Assessment;
Usability Report; Peer
Review
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
60
The Region employs a number of tools to facilitate assessment of QMP implementation. A few
examples of these tools are described below.
QMP Structure and Format Communication Tool
The Region is committed to providing easy access to the QMP in the most useful and effective
format. The Region 8 QMP is available on the Region 8 Internal Website.
Internal Quality Systems Reviews Assessment Tool
The Region uses a variety of tools (i.e., quality system assessments, technical systems audits,
etc.) to ensure that the procedures documented in this QMP are being implemented. An effective
Quality System requires periodic assessment to determine if the system is operating as designed
and to establish a basis for corrective action. Assessments are the principal means to determine
compliance with established QMPs and QAPPs. These independent audits, reviews and
assessments evaluate the conformance of the Region's Quality System with the procedures
described in this QMP.
Regional QA Training Information Sharing Tool
The QA Branch provides necessary QA training as resources permit. One of the goals of the
training program is to inform Regional staff of QA regulations and standards. The RQAM is
responsible for assessing the Region's QA training needs.
Quality Assurance Internal Processes Assessment Tools
The QA Branch has many processes and tracking systems that provide feedback on the
effectiveness of the region's Quality System and the implementation of the QMP. These
processes include Quality Assurance Review Forms (QARF), QAPP Reviews, QA Training
Program, Authority Delegation, Peer Review, and State QMP oversight program.
Quality Process Improvement A Tool for Change
The QA Branch encourages managers and staff to adopt a culture of "continuous improvement",
a process by which to proactively address vulnerabilities and to enhance efficiency. Process
improvement is incorporated as a core organizational element of the Region's culture and
philosophy and looks to correct systemic problems, improve consistency, streamline processes,
re-engineer ineffective work procedures, and customize quality tools. Management and staff are
encouraged to establish communications among themselves and with customers and suppliers to
explore areas for improved service. EPA personnel are expected to identify areas for process
improvement and to actively participate in problem solving.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
61
9.3 Data Quality Indicators
Data Quality Indicators (DQIs) are an integral part of systematic planning and development of
data quality objectives (DQOs). Data quality indicators are the quantitative statistics and
qualitative descriptors used by the data user(s) to interpret the degree of the data's acceptability
or utility. The principal DQIs are precision, bias (or accuracy), representativeness, comparability,
completeness and sensitivity.
Any project conducted by or for EPA that involves the collection, generation, use, and reporting
of environmental data requires an approved Quality Assurance Project Plan that addresses all of
the DQIs for generation of both field and laboratory data. The project DQIs are used to identify
and select field and laboratory test methods, whose measurement quality objectives can meet the
data quality objectives of the project. For projects that involve collection of existing data, the
QAPP must describe the measurement quality objectives required for inclusion of data in the
project.
The DPM is responsible for ensuring all projects involving collection, generation, use, and
reporting of environmental data under their authority have approved QA documents describing
data quality indicators that are appropriate for the programmatic or project's data quality
objectives.
9.4 Data Verification & Validation
Data verification and data validation are essential, prerequisite assessments conducted before an
overall data quality assessment (DQA) can be completed for a project's environmental data.
These assessments are necessary in order for data users to understand the potential qualitative
and quantitative biases of environmental data and how decision-making may be impacted as a
result of the biases. Each is defined as follows:
Data verification is the process of evaluating the completeness, correctness and
conformance/compliance of a specific data set against method, procedural, and/or contractual
specifications. The goal of data verification is to ensure and document that the reported results
reflect what was actually done in contrast to required methodology, procedures or other
specifications. Data verification may be performed by personnel involved with the field
collection of samples or data, the laboratory generating the analytical data, or by an external data
verifier. Although laboratories can verify laboratory measurements were collected according to
methods, procedures, and contractual specifications, the organization responsible for the project
should verify that data met the project specifications and permit authorization of payment for the
work. Field sampling and measurement data must also be evaluated for completeness,
correctness and conformance/compliance against method, procedural, or contractual
specifications.
Data validation is an analyte- and sample- specific process which extends the evaluation of data
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
62
beyond method, procedure, and/or contractual compliance to determine the quality of a specific
data set relative to the end use. The goal of data validation is to evaluate whether the goals
established by data quality indicators during the project planning phase have been achieved. Data
validation is typically performed by person(s) independent of the project activity. The
appropriate degree of independence will be determined on a program specific basis. At a
minimum, the individual(s) conducting the validation should not belong to the same
organizational unit with immediate responsibility for producing the data set.
Another process, data review, may precede both data verification and data validation. Data
review is defined as an internal examination of data, conducted by field, laboratory or data
management personnel, to ensure that data have been recorded, transmitted and processed
correctly. Data review may include checks for errors in transcription, calculation, reduction and
transformation as well as completeness of sampling information or losses of samples or data.
Personnel performing data verification and validation should have professional knowledge of
principles (e.g., chemical, biological, etc.), theories, practices and established methods, statistical
techniques commonly used in quality control, data assessments, and data management practices.
Extensive knowledge of the principles and practices of quality assurance and familiarity with the
project-specific data quality indicators is also necessary. EPA QA/G-8 Guidance on
Environmental Data Verification and Data Validation, November 2002, reissued January 2008
provides general guidance on data verification and validation processes. Examples of method-
specific data validation guidance include the U.S. EPA Contract Laboratory Program (CLP)
National Functional Guidelines for organic and inorganic analyses.
All project-level QA planning documents {i.e., QAPPs, SAPs, FSPs) must clearly describe the
specific processes, including SOPs for data verification and data validation for environmental
data generated or existing data used during the project and also identify individuals or
organizations responsible for the completion of the data verification and data validation. Quality
documents containing statements such as "data will be validated" do not sufficiently describe the
data verification and data validation process. Data review, data verification and data validation,
as defined above, are not usually provided in conjunction with analytical services unless they are
specifically requested and required in the QAPP or other QA documents.
DPMs are responsible for ensuring that the data received from either internal or external sources
are verified and validated prior to proceeding to data quality assessment. The specific procedures
and title(s) of the individual(s) responsible for data verification and validation shall be included
in the project's QA document, including requirements for documenting and reporting the results
of the data verification and validation.
9.5 Data Quality Assessment
A Data Quality Assessment (DQA) is the scientific and statistical evaluation of data to
determine if data obtained from environmental data operations are of the right type, quality, and
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
63
quantity to support their intended use. The DQA is a required element of all QAPPs and
completes the data life cycle by providing the assessment needed to determine if the planning
objectives were achieved, a key step prior to making Agency decisions.
DQAs are designed to examine one or more of the components of the data, namely: planning
(design), generation (methods), and usability (quality). EPA's guidance documents Data Quality
Assessment: A Reviewer's Guide, QA/G-9R and Data Quality Assessment: Statistical Methods
for Practitioners, QA/G-9S may be used to conduct DQA. The EPA DPM is responsible for
ensuring a DQA to determine whether the data may be used for their intended purpose. The level
of effort for a particular DQA will be commensurate with the project objectives and intended use
of the data.
9.6 Annual Review of QA Documents
To implement and maintain Region 8's Quality System, EPA Orders CIO 2105.0 and 2106.0
require an assessment of the effectiveness of the quality system annually; administration of
corrective actions based on the assessment results is also required6. Region 8, under the direction
of the RQAM, complies with the requirement for an annual assessment of QA Documents
through mandatory annual review of all QA documents, such as:
Quality Management Plans (QMPs)
Quality Assurance Project Plans (QAPPs)
Sampling and Analysis Plans (SAPs)
Field Sampling Plans (FSPs)
Refer to Section 9.6.1 for further detail about annual review requirements for QMPs and Section
9.6.2 for annual reviews of QAPPs, SAPs, and FSPs.
Annual Review of QMPs
All QMPs must be reviewed annually by the organization that originated the QMP. The annual
review shall be documented using the Region 8 QMP Review Crosswalk, which is available on
the Region 8 QA Branch website (http://www.epa.gov/region8/qa). This assessment must
include an evaluation of the effectiveness of the QMP. The process of developing and annually
updating the QMP provides an opportunity for management and staff to review and clarify roles
and responsibilities, to address problem areas, and to acknowledge successes. Any changes to the
organization's quality system must be documented by the author in the organization's QMP as
well as on the Region 8 QMP Review Crosswalk. If the annual review reveals no changes to the
QMP, this should be noted by the author on the Region 8 QMP Review Crosswalk and reported
in the organization's annual QA report to Region 8.
6 EPA Order CIO 2015, pp. 5 & 6, respectively.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
64
QMP approvals are valid for a period not to exceed five (5) years7'8. Providing the organization
continues to receive funding and/or produce work for EPA beyond the QMP's operating period,
the organization shall submit an updated QMP (prior to expiration date) for review and approval
by the RQAM. A revised QMP may be submitted at any time, but is required under certain
conditions including:
expiration of the five-year life of the approved QMP;
any major change to the organization's quality system;
a significant change in the organization's mission;
Annual Reporting of QMP Reviews
The organization shall include the results of their annual QMP review in their annual QA report
to the RQAM. For the QMP reviews, the report shall include the completed Region 8 QMP
Review Crosswalk, a summary of any changes to the QMP as a result of the review, the date of
the review, and the original QMP approval date {i.e., date of EPA's RQAM signature). The
annual QA report shall be provided to the RQAM. For extramural agreements, the EPA
Contracting Officer and Project Officer are responsible for providing the annual QA report to the
RQAM.
Annual Review of QA Project Plans and other QA Documents (QAPPs, SAPs, FSPs)
It is essential that project QA documents such as the QAPP and any companion documents to the
QAPP such as SAPs or FSPs be kept current. When changes affect the scope, implementation, or
assessment of the outcome, the plan shall be revised to keep project information current. The
DPM, with the assistance of the QA Manager, determines the impact of any changes on the
technical and quality objectives of the project. When a substantive change is warranted, the
originator of the QAPP shall modify the QAPP to document the change and submit the revision
for review and approval9.
At minimum, all QA documents (QAPPs, SAPs, FSPs) must be reviewed annually by the Project
Manager10 (DPM). The annual review must be documented using the Region 8 QA Document
Review Crosswalk. Any changes to the organization's quality system must be documented in
the revised QA document (QAPP, SAP and/or FSP) as well as on the Region 8 QA Document
Review Crosswalk. If the annual review reveals no changes to the QA document(s), this should
be noted on the Region 8 QA Review Crosswalk and reported in the organization's annual QA
report to Region 8. QA documents associated with a grant are valid for a period not to exceed the
life of the grant cycle {i.e., typically 1-2 years).
7 EPA Order CIO 2105.0, pg. 7.
8 EPA Quality Manual for Environmental Programs (CIO 2105-P-01-0), May 5, 2000; pg. 3-3.
9 EPA Quality Manual for Environmental Programs (CIO 2105-P-01-0), May 5, 2000; Section 5.2.2.
10 EPA Quality Manual for Environmental Programs (CIO 2105-P-01-0), May 5, 2000; Section 5.2.2.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
65
Annual Reporting of QA Document Reviews
The organization shall include the results of their annual QA document review in their annual
QA report to the RQAM. For the QA document reviews, the report shall include the completed
Region 8 QA Document Review Crosswalk, a summary of any changes to the QAPP, SAP,
and/or FSP as a result of the review, the date of the annual review, and the original QA
Document approval date {i.e., date of EPA's RQAM or Delegated QA Approving Officer's
signature). The annual QA report shall be provided to the RQAM. For extramural agreements,
the EPA Contracting Officer and Project Officer are responsible for providing the annual QA
report to the RQAM.
9.7 Quality Systems Assessments
EPA Orders 2105.0 and 2106.0 require the Region to assess the QA and QC activities applied
to environmental programs conducted by or for EPA. A Quality Systems Assessment (QSA) is
a qualitative evaluation of a data collection operation and/or organization(s) to establish
whether the prevailing quality management structure, policies, practices, and procedures are
adequate for ensuring that the type and quality of data needed are obtained. They are used to
determine the effectiveness of, and adherence to, the Quality System and the adequacy of
resources and personnel provided to achieve and ensure quality in all activities. Refer to EPA's
Guidance on Assessing Quality Systems, EPA/240/R-03/002 (2003) QA/G-3.
All QA documents (QMPs and QAPPs) for environmental data activity must outline how and at
what frequency the quality system will be evaluated. QMPs provide the blueprint for how the
organization (EPA and/or extramural organizations) will plan, implement, and assess its quality
system for the environmental work to be performed. Project-level QA documents must describe
in comprehensive detail the types of assessments that will be conducted during the life of the
project to ensure that the results of the work performed will satisfy the stated performance
criteria.
Frequency and QSA Selection Criteria
The RQAM (or designee) will conduct QSAs as needed and as resources permit. QSA
selection is based on Regional needs, and may be based upon criteria such as: Division or
Program requests for a QSA; information gaps observed through training events, QA
document reviews, annual QA reports; etc. Some examples of the types of QSAs are described
in the next sections.
Reporting QSA Results
The RQAM will report results of the QSA to OMS EQMD if requested.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
66
9.7.1 Delegated QA Approving Officer Program
The RQAM (or designee) will conduct a QSA of those delegated to review and approve project-
level QA documents as needed and as resources permit. Assessments will center on
documentation supporting individual QA document reviews and approvals completed by
Delegated QA Approving Officers (DAOs). The individuals performing the assessment shall be
independent and knowledgeable in QA document reviews. The assessors will review appropriate
documentation (e.g., QA documents, completed companion Region 8 QA Document Review
Crosswalks, etc.) and evaluate the way the activity is being conducted in comparison to
documented requirements. Commendations for any observed best practices will be noted, and
corrective actions for any DAO will be identified and monitored by the RQAM (or designee).
At minimum, best practices, findings and/or recommendations will be included in DAO trainings
as a teaching tool. Additionally, the QSA and its outcomes will be reported in the RQAM's
annual reporting to OMS EQMD if requested.
9.7.2 States
A QSA is the principle means for determining if a State's quality system is in compliance with
EPA QA requirements, contractual, and regulatory specifications. The RQAM (or designee) will
conduct state QSAs as needed and as resources permit using QA/G-3. The QSA will comprise
some of the following: interviews; reviews of QAPPs and associated QAPP review forms;
reviews of records and documents; and reviews of internal assessment reports. The State's QMP
is the primary document to guide the QSA. The assessments of the State's quality system provide
management with information that is helpful to determine the status of the quality system,
identify "best practices" and suggestions for improvement, as needed. The QSA and its outcomes
will be reported to OMS EQMD if requested.
Upon request, the Region 8 QA Program can assist EPA environmental programs with technical
or program assessments of the State's environmental programs.
9.7.3. Technical Systems Audits
One assessment function used to evaluate the effectiveness of EPA's quality system is a
Technical Systems Audit (TSA). TSAs are focused, thorough, and systematic assessments that
measure the performance of the work at a project level and with respect to the established
technical guidelines, SOPs, and project requirements as defined in QAPPs or other QA
documents. These audits include document reviews (i.e., QAPP review prior to completing a
milestone) and may include qualitative audit of facilities, equipment, personnel, training,
procedures, recordkeeping, data validation, data management, and reporting aspects of
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
67
environmental data generation, use and reporting. EPA's Guidance on Technical Audits and
Related Assessments for Environmental Data Operations (QA/G-7) should be used to assist with
the TSA.
TSAs should be scheduled throughout the project life cycle to identify potential problems early
in the project and allow for timely corrective action. The types of TSAs that may be considered
and applied for a particular project will vary; some options are:
Field Sampling
Field Testing/Monitoring
Split Sampling and Analysis
Data Package Completeness Reviews
Assessment of Data Validation Reports
A project or a program may conduct a TSA at their discretion and/or as required by regulation.
The review findings and corrective actions shall be documented by the project or program
representative in the end-of-year project report. The DPM is responsible for ensuring that the
identified TSA is accomplished and properly documented for a project. The RQAM may request
regional reporting of TSAS.
9.8 Information Quality Guidelines and Pre-Dissemination Review
Procedures
Region 8 is dedicated to the collection, generation and dissemination of high-quality information.
EPA's Information Quality Guidelines (IQGs) describe EPA's policy and procedural guidance for
ensuring and maximizing the quality of information the Agency disseminates. Thus, the IQGs
are integral to the Regional Quality System for ensuring the quality of EPA's data products and
information. This QMP incorporates by reference all definitions, principles, policies and
procedures found in EPA's IQGs: Guidelines for Ensuring and Maximizing the Quality,
Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection
Agency.
9.8.1 Scope and Applicability of Information dissemination
The term "scientific information11" or "information" means factual inputs, data, models,
analyses, technical information, or scientific assessments related to such disciplines as the
behavioral and social sciences, public health and medical sciences, life and earth sciences,
engineering, or physical sciences. This includes any communication or representation of
knowledge such as facts or data, in any medium or form, including textual, numerical, graphic,
cartographic, narrative, or audiovisual forms. This definition includes information that an agency
disseminates from a web page but does not include the provision of hyperlinks on a web page to
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
68
information that others disseminate. The term includes "preliminary" information that EPA has
endorsed or adopted as well as conclusions or facts drawn from or based upon other existing
information. This definition excludes opinions, where the agency's presentation makes clear that
an individual's opinion, rather than a statement of fact or of the agency's findings and
conclusions, is being offered.
Definition of Dissemination11
For the purposes of the OMB Information Quality Guidelines and Peer Review Bulletin,
"dissemination" means Agency initiated or sponsored distribution of information to the public.
Dissemination does not include distribution limited to government employees or agency
contractors or grantees; intra- or inter-agency use or sharing of government information; and
responses to requests for agency records under the Freedom of Information Act, the Privacy Act,
the Federal Advisory Committee Act or other similar law. This definition also does not include
distribution limited to correspondence with individuals or persons, press releases, archival
records, public filings, subpoenas or adjudicative processes.
EPA disseminates information when EPA initiates or sponsors the distribution of
information to the public, as follows:
EPA initiates a distribution of information if EPA prepares the information and
distributes it to support or represent EPA's viewpoint, or to formulate or support a
regulation, guidance, or other Agency decision or position.
EPA initiates a distribution of information if EPA distributes information prepared or
submitted by an outside party in a manner that reasonably suggests that EPA endorses or
agrees with it; if EPA indicates in its distribution that the information supports or
represents EPA's viewpoint; or if EPA in its distribution proposes to use or uses the
information to formulate or support a regulation, guidance, policy, or other Agency
decision or position.
Agency-sponsored distribution includes instances where EPA reviews and comments on
information distributed by an outside party in a manner that indicates EPA is endorsing it, directs
the outside party to disseminate it on EPA's behalf, or otherwise adopts or endorses it.
9.8.2 Policy Implementation for IQGs and Pre-Dissemination
Reviews
The Region 8 Information Quality Guidelines Officer is the RQAM. The IQG Officer is
responsible for overseeing implementation of the IQGs in Region 8.
11 OMB Information Quality Guidelines and Peer Review Bulletin: "Final Information Quality
Bulletin for Peer Review", M-05-03, December 16, 2004.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
69
Region 8 employs a multi-faceted approach to implement EPA's Information Quality Guidelines
and conduct Pre-Dissemination Reviews (PDRs). The key components of Regional
implementation include:
Implementation of the Region 8 Quality System and quality policies outlined in this
QMP;
Identification of information products and selecting the appropriate review process;
Implementation of Clearance or Office Reviews of all "information" prior to
dissemination;
Use of disclaimers in all disseminated information. Example disclaimers are provided
below (Section 9.8.5); and
Peer Review, as appropriate, coordinated by the Region 8 Peer Review Coordinator
(Section 9.9.3).
Key areas of Region 8's Quality System that maximize the quality of environmental information,
including information disseminated by the Agency, include:
use of a systematic planning process which establishes acceptance or performance criteria
prior to the initiation of all projects that involve environmental information collection
and/or use (Section 7.10): and
development and implementation of approved QAPPs (or equivalent documents) for all
applicable projects and tasks involving generation and/or use of environmental data.
The DPM is responsible for ensuring that an approved QAPP is in place prior to generation, use
or reporting of environmental data; identifying appropriate clearance/office reviews;
coordinating with the Information Quality Guidelines Officer and Peer Review Coordinator,
as appropriate; and ensuring that the QAPP describes the necessary review(s) that must be
conducted prior to dissemination of information.
9.8.3 Clearance or Office Review Procedures
The steps in conducting a Clearance or Office Review are:
1. Identifying potential information products that are subject to IQGs;
2. Selecting and Conducting an appropriate review process; and
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
70
3. Documenting the review.
Identifying Potential Information Products Subject to IQGs
The IQGs address the circumstances under which the dissemination of information by EPA may
be subject to its guidelines. Region 8 has created a list of potential information products to aid in
identifying the categories of information products that may be subject to the IQGs. The list is not
exhaustive. The inclusion of an information product on this list does not mean that it is
automatically subject to pre-dissemination review. On the other hand, the exclusion of an
information product does not mean that it would not be subject to the IQGs. When in doubt, ask
your supervisor or the Region 8 Information Quality Guidelines Officer, or the Region 8
Science Council.
General Environmental Information and/or Interpretation of Data
Technical reports
Quality System and other QA documents (QMPs, QAPPs, SAPs, FSPs, etc.)
Technical guidance/assistance documents
Risk assessments and risk assessment guidance
Decision documents (e.g., Records of Decision, etc.)
Publicly accessible databases
Manuscripts for journal articles
Results from environmental monitoring/sampling
Interpretation of existing data
Results from models
Maps, contour maps
Database outputs and displays
Tools
Models
GIS-related products/programs, mapping products/programs
Decision tools
Workshops/Conferences:
Brochures (mailers, announcements)
Presentations
Posters/displays at meetings, conferences, seminars that contain environmental
information
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
71
Outreach Information
Articles
Fact sheets
Booklets, manuals, workbooks
E-mail updates (large e-mailings sent to a targeted audience)
Newsletters
Web pages
Compliance tools/calendars
Citizen guides
Education aids
Mailings to targeted audiences
Two categories of information that rise to the agency level of importance are "influential" and
"assessment" information. Dissemination of this type of information triggers a higher level of
information quality review (Refer also to Peer Review, Section 9.9).
During the planning process and prior to information release, staff should inform their direct
supervisor of information products that are being developed for dissemination as described in the
IQGs. Questions regarding applicability of the IQGs to a particular information product can be
directed to the Region 8 Information Quality Guidelines Officer. Supervisors are responsible
for ensuring that information subject to the IQGs is identified before it is disseminated by their
group. Supervisors should also determine what mechanism for reviewing the information is
appropriate before it is disseminated.
Selecting and Conducting an Appropriate Review Process
Whenever possible, existing review procedures will be used for the pre-dissemination review of
information subject to the IQGs. The following describes various review processes, roles, and
responsibilities for staff and management to implement IQG PDR. One or more types of review
may be appropriate, depending upon the nature and importance of the information product.
Authors and supervisors who are developing an information product subject to IQG should use
best judgment in determining which of the review procedures listed below are most appropriate.
Criteria that management and staff consider when selecting the appropriate review mechanism
include:
Policy implications
Scientific/technical content
Precedent setting, innovative, or emerging issues
-------
Level of controversy or public/political interest
Interagency implications
Level of investment of Agency resources
Crossing multiple jurisdictions
Types of Reviews, Roles and Responsibilities
Authors Review
The primary author has the main responsibility to ensure that the information content of a
product is scientifically sound, meets the project's objectives, and fulfills the objectivity, utility,
and integrity criteria of the IQGs.
In some cases, the primary author may have had others {i.e., chemists, toxicologists, GIS staff,
etc.) prepare all or part of the material required to generate an information product. The technical
support staff also has a role to ensure that the method and assumptions used in the analysis are
appropriate to meet the primary author's stated purpose, that the data sources used are as defined
in the project plan, and that the output produced has provided a reliable rendering of the analysis.
The supervisor of technical support staff is responsible for the quality of their work.
Supervisor and/or Management Review
The direct supervisor of the author is responsible for the quality of the information subject to the
IQGs disseminated by his/her Section/Branch/Division. The supervisor is also responsible for
conformance of information products to applicable EPA policy. The supervisor's review of
disseminated information includes the responsibility to ensure that the information quality is
adequate for its intended use, and has the characteristics of objectivity, utility, and integrity as
defined in the IQGs.
Informal External Reviews
The Region periodically solicits reviews of draft documents and information products from
qualified individuals who are external to the Region. The focus of an external review should be
to ensure that the document or product is scientifically sound. Examples of external reviewers
include experts from:
States
Tribes
Academia
Regional personnel have the responsibility to ensure that external reviewers are aware of any
specific issues that should be a focus of attention and that reviewers understand their overall
US EPA Region 0
Quality Management Plan
July 30, 2020
72
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
73
responsibilities. Products that are reviewed for quality by external reviewers should also be
reviewed by qualified EPA personnel.
Formal Peer Review
Formal peer reviews are conducted and documented in accordance with the Agency's Peer
Review Handbook and Peer Review Policy. Formal peer review is generally considered to be the
highest level of technical quality review. Supervisors are responsible to inform their respective
Assistant Regional Administrators and the Region 8 Peer Review Coordinator about any
information products that are likely to be subject to formal peer review. Any scientific and
technical products that are peer reviewed must also be reviewed by qualified EPA Regional
personnel to verify that IQG requirements are fully met. For more information about Peer
Review procedures, see Section 9.9.
Documenting the IQG/Pre-Dissemination Review
Region 8 requires that all IQG reviews be documented by the author and carried through the
review process. When reviews are completed, reviewers sign/acknowledge the review and the
author documents the review with the IQG Officer. The IQG Officer will sign off to
acknowledge the pre-dissemination review is complete and the product may be processed. The
Region 8 Information Quality Guidelines Officer tracks all reviews using a tracking
spreadsheet.
9.8.4 Print, Web and Promotional Product Review Procedures
A Product Review is required by and conducted through the Region 8 Office of Public Affairs.
All Region 8 print, web and promotional items must go through a Product Review prior to
dissemination. All products are reviewed for the following:
style, format and presentation that is consistent with EPA's standard for content and
communications;
uniform messaging (look, feel and style) that is consistent with Agency-wide standards;
and
adherence to EPA and Federal policies regarding product creation (e.g., green materials,
etc.).
9.8.5 Disclaimers for Disseminated Information
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
74
It is Region 8's policy to include disclaimers when disseminating information for either of the
following conditions:
Distributing draft information prior to completion of pre-dissemination review(s); or
Distributing information that is not generated or sponsored by EPA, its grantees or
contractors.
Refer to the discussion on Peer Review Policies and Procedures (Section 9.9) for disclaimers
required when disseminating written materials that are published by non-EPA entities.
Disclaimers are Required when Distributing Draft Information
Region 8's policy is to distribute information only after a pre-dissemination review is completed.
Although it should not be a common practice, there may be occasions when Region 8 must
distribute information prior to completing the pre-dissemination review. In these cases, the
author must obtain written supervisory approval to distribute information prior to completion of
pre-dissemination review. In addition, the author is responsible for informing the Region 8
Information Quality Guidelines Officer in writing that a product/information will be
distributed prior to pre-dissemination. In that written statement, the author must include the
disclaimer used in the information delivery/dissemination.
Any information that is distributed without a documented and completed pre-dissemination
review must:
Have written approval from the author's supervisor allowing distribution of the
information prior to completing the pre-dissemination review;
Be marked "DRAFT" in several conspicuous locations; and
Include a disclaimer indicating that the information has not undergone pre-dissemination
review, is subject to change and does not represent EPA decisions and/or policy.
Examples of Disclaimers for Use When Distributing Non-EPA Information
There may be cases when EPA distributes information that is not generated and/or sponsored by
EPA, its grantees or contractors. In these cases, all information must include a disclaimer,
conspicuously located, to inform users of that fact. In these cases, the requestor (a Region 8 EPA
employee) must document the request, which is signed by the employee's supervisor and Region
8 IQG Officer and identifies the nature of the information as well as the disclaimer used.
Example disclaimers are provided below, although this list is not exhaustive.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
75
EPA is distributing this information solely as a public service. [Insert name of
information source\ is responsible for the quality of this information. EPA's distribution
of this information does not represent or imply endorsement by EPA.
Links to Websites outside of the EPA Website are provided for the convenience of the
user. Inclusion of information about a Website, an organization, a product, or a service
does not represent endorsement or approval by EPA, nor does it represent EPA opinion,
policy or guidance unless specifically indicated. EPA does not exercise any editorial
control over the information that may be found at any non-EPA website.
Some of the events, articles and websites listed in this email are non-EPA sponsored
events, articles and websites. EPA is listing them for information purposes only and is not
responsible for the content of the websites/articles or the information distributed at a non-
EPA listed event.
The technology descriptions contained on this site including, but not limited to,
information on technology applications, performance, limitations, benefits and costs have
been provided directly by the vendors. No attempt was made to examine, screen or verify
company or technology information. Therefore, EPA has not confirmed the accuracy or
legal adequacy of any disclosures; product performance or other information provided by
the companies and used by EPA on this website.
EPA has not evaluated or verified statements made on this site pertaining to compliance
with federal, state or local regulations, standards, permits or other requirements.
The inclusion of companies and their products [insert tool type] does not constitute or
imply endorsement or recommendation by the EPA.
9.8.6 Requests for Correction and Reconsideration
The IQGs establish two mechanisms that the public may employ to request correction or
reconsideration of information that has already been disseminated. EPA's Office of
Environmental Information (now OMS) manages the administrative mechanisms that enable
affected persons to seek and obtain, where appropriate, correction or reconsideration of
information disseminated by the Agency, if it does not comply with EPA or OMB Information
Quality Guidelines.
Request for Correction
Affected persons may request correction of information if that information does not comply with
the IQGs. The OEI will receive these complaints and forward them to the Region 8 IQG Officer
when the information in question belongs to or involves Region 8. Upon receipt of the request
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
76
for correction from the OEI, the Region 8 IQG Officer will notify the responsible Program(s) or
Offices(s). The Region 8 IQG Officer will track the progress of the request until it has been
resolved and then report resolution to OEI.
Request for Reconsideration
EPA seeks public and stakeholder input on a wide variety of issues, including the identification
and resolution of discrepancies in EPA data and information. Affected persons may request a
reconsideration of EPA's decision on a request for reconsideration if they are dissatisfied with
the decision. The OEI will receive these complaints and forward them to the Region 8 IQG
Officer when the information in question belongs to or involves Region 8. Upon receipt of the
request for reconsideration from OEI, the Region 8 IQG Officer will notify the responsible
Program(s) or Offices(s). The Region 8 IQG Officer will track the progress of the request until
it has been resolved and then report resolution to OEI.
9.9 Peer Review Policies and Procedures
Peer Review is a critical process that enhances the credibility of scientific and technical
documents and ensures the quality of products disseminated by Region 8. Peer reviews are
documented critical reviews of scientific and technical work products by qualified individuals (or
organizations) that are independent of those who performed the work but are collectively
equivalent in technical expertise. A peer review may be either internal or external to EPA and is
performed to ensure that activities are technically adequate, competently performed, properly
documented, and satisfy established technical and quality requirements. The peer review is an in-
depth assessment of the assumptions, calculations, extrapolations, alternate interpretations,
methodology, acceptance criteria, and conclusions pertaining to specific work and of the
documentation that supports them.
To be most effective and efficient, peer review of a scientific or technical work product must be
incorporated into the up-front planning of any decision or action that will be made or taken based
on the work product. The planning for peer review activities should occur at the time of overall
project planning and includes obtaining the proper resource commitments and establishing
realistic schedules to accommodate the peer review process. Peer review is not restricted to the
nearly final version of a report or similar work product. In fact, peer review of the planning
documents such as QAPPs, study designs or research plans can often be extremely beneficial to
avoid fundamental and costly errors in the study design. Proper advance planning by the project
or program manager is essential to ensuring a positive and seamless peer review experience.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
77
9.9.1 Policy Overview
The Agency's Peer Review Policy, Peer Review and Peer Involvement at the U.S.
Environmental Protection Agency states:
"Peer review is encouraged and expected for all scientific and technical
information that is intended to inform or support Agency decisions. Influential
scientific information and highly influential scientific assessments should be peer
reviewed according to the EPA Peer Review Handbook. All Agency managers are
accountable for ensuring that Agency policy and guidance are appropriately
applied in determining if their work products are influential or highly influential,
and for determining the nature, scope and timing of their peer review. For highly
influential scientific assessments external review is the expected procedure. For
influential scientific information intended to support important decisions, external
peer review is the approach of choice. Peer review is not restricted to the nearly
final version of work products, in fact, peer review at the planning stage can often
be extremely beneficial."
The peer review requirements differ somewhat depending on the nature of the work product
and/or relevant statutory requirements. Specific peer review requirements for the various types of
work products are detailed in Section 9.9.5.2.
9.9.2 Definitions
Several definitions related to the peer review process are provided below.
Influential Scientific Information
Influential Scientific Information (1ST) is scientific information that has or will have a clear
and substantial impact on important public policies or private sector decisions. These include
products or scientific information that support major Agency regulatory decisions. An ISI work
product has a major impact, involves precedential, novel, and/or controversial issues, the Agency
has a legal and/or statutory obligation to conduct a peer review, is used to support a regulatory
program or policy position and meets one or more of the following factors:
Establishes a significant precedent, model, or methodology;
Likely to have an annual effect on the economy of $100 million or more, or adversely
affect in a material way the economy, a sector of the economy, productivity, competition,
jobs, the environment, public health or safety, or state, tribal, or local governments or
communities;
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
78
Addresses significant controversial issues;
Focuses on significant emerging issues;
Has significant cross-Agency/interagency implications;
Involves a significant investment of Agency resources;
Considers an innovative approach for a previously defined
problem/process/methodology; or
Satisfies a statutory or other legal mandate for peer review.
Scientific Assessment
One type of scientific information is a scientific assessment, which is an evaluation of a body of
scientific or technical knowledge and typically synthesizes multiple factual inputs, data, models,
assumptions, and/or applies best professional judgment to bridge uncertainties in the available
information. These assessments include, but are not limited to, state-of-science reports;
technology assessments; weight-of-evidence analyses; meta-analyses; health, safety, or
ecological risk assessments; toxicological characterizations of substances; integrated assessment
models; hazard determinations; or exposure assessments. Such assessments often draw upon
knowledge from multiple disciplines.
Highly Influential Scientific Assessment
A Highly Influential Scientific Assessment (HISA) is a scientific assessment that could have a
potential economic impact of more than $500 million in any year, and/or is novel, controversial,
precedent-setting, or has significant interagency interest. A scientific assessment is an evaluation
of a body of scientific or technical knowledge which typically synthesizes multiple factual
inputs, data, models, assumptions, and/or applies best professional judgment to bridge
uncertainties. HIS As are a subset of ISIs.
Other Scientific and Technical Work Products
This category includes all other scientific and technical work products that do not meet the
definition of HISA or ISI.
12 OMB Information Quality Guidelines and Peer Review Bulletin: "Final Information Quality
Bulletin for Peer Review", M-05-03, December 16, 2004.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
79
Work Product
A work product may be any scientific or technical report authored by and/or sponsored by
Region 8 that informs or supports an Agency decision, including (but not limited to): QA
Documents, QAPPs, SAPs, FSPs, study designs, research plans, technical and scientific reports,
books, manuals, abstracts and/or draft manuscripts to be submitted to journals.
9.9.3 Region 8 Peer Review Coordinator
The Region 8 Peer Review Coordinator is the Regional Science Advisor and coordinates and
monitors peer review activities in the Region. The Peer Review Coordinator is the main contact
for Region 8 and can direct interested parties to other persons/contacts in the Region on specific
work products, as appropriate (e.g., Peer Review Leader).
Specific responsibilities of the Peer Review Coordinator are:
General oversight responsibility for the Region's peer review process;
Report peer review activities to the Deputy RA;
Recommend classification of work products (as ISIs, HISAs, Other scientific and
technical work products) and the mechanism(s) of peer review utilized for each of the
work products;
Obtain concurrence from the DRA regarding identification of work products as ISI,
HISA or Other;
Designate a Peer Review Leader to organize a particular peer review;
Provide advice, guidance, and support to the Peer Review Leader in the preparation,
conduct, and completion of the peer review;
Organize and coordinate a Region 8 peer review board, as necessary;
Help mediate difficult issues between the Region and others. If an issue cannot be
resolved, then the Peer Review Coordinator can bring the issue to the attention of the
appropriate level Decision Makers for resolution;
Function as the liaison with ORD and the Science and Technology Policy Council
(STPC) to:
o Represent the Region before the STPC on peer review issues;
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
80
o Advise ORD of any changes in the list of work products and peer review
mechanisms during the annual reporting, and when necessary, at other times;
o Participate in Agency peer review training, workshops, etc., as requested and
disseminate this information to the Region; coordinate and/or present training
within the Region.
Submit information on the Region's peer review plans and activities as needed to
implement the Peer Review Policy;
o Assure that all Science Inventory records for peer review products are accurate
and comply with OMB reporting requirements; and
o Assure the Science Inventory record addresses all issues and reporting
requirements for the annual report to OMB and sign the record, making it
available to the public via the Science Inventory website.
Establish procedures to ensure that the work product peer review documentation (i.e.,
peer review record) is filed and maintained in an appropriate manner; and
Distribute Agency-wide peer review guidance and materials to appropriate Region 8
personnel, as requested.
In cases of highly influential scientific assessments and/or influential scientific information
developed by Region 8 in collaboration with other EPA offices (e.g., Headquarters National
Program Managers and/or ORD), the peer review activities may be coordinated through a
different EPA office or location. In these instances, the peer review as well as the Peer Review
Coordinator, Decision Maker, and Peer Review Leader roles may be assigned to another office
or location. When this occurs, all peer review responsibilities, oversight and tracking are
transferred to that office/location. Region 8 retains no role or responsibility in the peer review
process for that work product except to provide information as requested.
9.9.4 Other Key Roles and Responsibilities
In addition to the Regional Peer Review Coordinator, there are several other key positions of
responsibility in the peer review process. They are detailed here.
Decision Maker
The RA is the ultimate Decision Maker in Region 8 and is accountable for the decisions
regarding the identification of ISIs and HIS As and the mechanism(s) of peer review utilized for
each of the work products developed or sponsored by Region 8. The Decision Maker is
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
81
responsible for ensuring that peer reviews are properly performed and documented. The RA may
designate the LSASD Division Director who, in consultation with the Region 8 Peer Review
Coordinator, recommends to the RA the peer review category and the type of peer review
mechanism needed for HISA or ISI scientific and technical work products. The Decision Maker
commits the resources needed to ensure an appropriate level of peer review.
The Peer Review Coordinator may serve as the front-line Decision Maker for overseeing the
initial steps of the peer review process for ISI or HISA work products but will always consult
with the RA regarding these steps. The Peer Review Coordinator may function as the Decision
Maker for Other scientific and technical work products.
In order to ensure greater independence of peer reviews, it is important to strictly separate the
management of work products from the actual peer review of those work products. Therefore, for
HISA and ISI work products the Decision Maker and the Peer Review Leader for a work product
should not be the same person.
Specific responsibilities of the Decision Maker(s) are:
Classify work products and identify which work products require peer review;
Ensure that sufficient funds are designated in the office's budget request to conduct the
peer review; also ensure that adequate resources and/or extramural management support
are available for the peer review;
Designate the stage(s) of product development where peer review is appropriate;
Ensure that the peer reviews are properly performed and documented;
Consider public comments on the peer review plans and consider and decide if the public
will be asked to nominate potential peer reviewers for influential scientific information
and for highly influential scientific assessments;
For highly influential scientific assessments, decide whether it is feasible and appropriate
to make the draft scientific assessment available to the public for comment at the same
time it is submitted for peer review (or during the peer review process) and whether it is
feasible and appropriate to sponsor a public meeting where oral presentations on
scientific issues can be made to the peer reviewers by interested members of the public.
When public comment on the assessment is sought, provide peer reviewers with
significant comments raised by the public; and
Ensure all relevant issues and comments raised by the peer reviewer(s) are adequately
addressed and documented for the record, and where appropriate, incorporated into the work
product that is used as a basis for decision-making.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
82
Peer Review Leader
The Peer Review Leader is responsible for organizing, conducting, completing and documenting
the peer review for a specific individual work product. Duties of the Peer Review Leader
include, but are not limited to, selecting reviewers or documenting the selection of the reviewers
when reviewers are selected by another party, such as a contractor or the Science Advisory
Board; establishing and maintaining the peer review record; and recommending an approach for
responding to peer review comments.
For HISA and ISI work products, the Peer Review Leader cannot be the Decision Maker.
According to the Peer Review Handbook, the Peer Review Leader can be the project manager for
the work product. However, in order to avoid the appearance of a conflict of interest, the Peer
Review Leader should not be the project manager whenever possible. The Peer Review Leader
will be chosen on a case by case basis depending on the work product needing peer review.
Specific responsibilities of the Peer Review Leader are:
Keep the Peer Review Coordinator and Decision Maker informed of the status of all
projects;
Organize, conduct, complete, and document the peer review following Agency
procedures and in coordination with the Peer Review Coordinator:
o Establish and maintain the peer review record for the specific individual peer
review currently being performed;
o Develop a peer review plan and/or summary information for your peer review
activity and provide it to the Peer Review Coordinator for review and inclusion
into the Science Inventory;
o Establish a realistic peer review schedule that includes considerations for schedule
and outlines all necessary resources;
o Select the peer reviewers in consultation with others involved with the peer
review (e.g., Peer Review Coordinator and Decision Maker) and ensure that
conflict of interest issues are addressed and documented in the peer review record;
o Advise and document peer reviewers of their responsibilities, including
preparation of a peer review report if the product is influential scientific
information or a highly influential scientific assessment.
o
Provide information to the Peer Review Coordinator and Decision Maker
(including all appropriate managers in the Peer Review Leader's chain of
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
83
command) on the charge, profile of peer reviewers, the peer review comments,
and a proposal on how to address the comments.
o Obtain Decision Maker approval on the approach to responding to peer reviewer
comments. Clearly identify any peer review comments for the Decision Maker
that will not be addressed in the agreed upon approach:
ฆ Make the peer review report for influential scientific information and
highly influential scientific assessment publicly available;
ฆ For highly influential scientific assessments, develop the Agency response
to the peer review and make it publicly available.
o Originate and route the Region 8 Peer Review Routing Form to the Peer Review
Coordinator and Decision Maker (including all appropriate managers in the Peer
Review Leader's chain of command) to document completion of all key
milestones in the peer review.
o Provide the Peer Review Coordinator with necessary data (as requested) for
national and regional reporting purposes;
o Archive the peer review record in a manner consistent with the Region 8's records
management procedures (Section 13.0).
9.9.5 Peer Review Policy Implementation
The Peer Review Handbook specifies the peer review process in relation to a work product. A
work product may be any scientific or technical document or report authored by and/or
sponsored by EPA that informs or supports an Agency decision, including (but not limited to):
QA planning documents (QAPPs, SAPs, FSPs), study designs, research plans, technical and
scientific reports, books, manuals, abstracts and/or draft manuscripts to be submitted to journals.
Scientific or technical work products planned for generation and/or generated in Region 8 shall
be classified prior to development. Classification determines whether and what level of peer
review applies to the work product.
Peer review is not restricted to the nearly final version of a report or similar work product. In
fact, peer review of the planning documents such as QAPPs, study designs or research plans can
often be extremely beneficial to avoid fundamental and costly errors in the study design. Proper
advance planning is essential to ensuring a positive and seamless peer review experience. It is the
responsibility of the DPM to plan for, report on and ensure proper conductance of peer review
for all work products developed or sponsored by that DPM. It is also the DPM's responsibility to
inform the Regional Peer Review Coordinator about work products requiring peer review prior
to their development.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
84
Project-level Classification for Peer Review
Due to the vast quantity of planning documents, reports, summaries, and other documents subject
to peer review classification generated each year, Region 8 has adopted the following
implementation policy for work product classification. This policy is subject to change should
EPA's Peer Review Handbook provide further guidance on this topic.
All work conducted in Region 8 shall be monitored to determine whether it is or is emerging as a
HISA or ISI classification. For simplicity, classifications will be assigned at the project-level.
A project may change classification during its lifetime. That is, it could originally be classified as
"Other" and at some point be elevated to ISI or HISA. Because of this, it is important for
managers to receive regular briefings from staff about the technical and scientific content of
project work. Region 8 staff are responsible for reporting the technical and scientific content of
current project work to their supervisors. In turn, supervisors and managers are responsible for
informing their ARA, Deputy ARA and the Peer Review Coordinator of these projects and
anything that may change a project's classification. The decision to classify projects at a HISA or
ISA peer review level is made by the Decision Maker who informs the Peer Review Coordinator
for documentation and tracking.
All work products for any project that is classified as HISA or ISI must undergo a determination
of whether and at what level, if any, peer review is required. Because the scope of each
individual work product for a particular project may be different, each work product is classified
for peer review separately. For example, the QAPP, the resulting scientific report(s) and draft
manuscript(s) for a peer reviewed journal may have different peer review requirements. These
decisions must be documented by the Peer Review Coordinator. The DPM is responsible for
ensuring the required peer review is conducted for all work products.
Peer Review Process for Each Work Product
Generally, the peer review process includes the following for each work product:
Assign the work product classification (1 of 3 Region 8 peer review categories);
Log the work product into the Science Inventory and document its Region 8 peer review
classification;
Create and maintain Peer Review Record;
Define whether the peer review is internal or external to EPA;
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
85
Plan peer review (identify peer reviewers, conflict of interest/confidentiality, peer review
charge, etc.);
Develop peer review plans for any work product that meets ISI and HISA classification;
Compile the peer review materials, including work product, charge, all supporting data,
etc.;
Conduct peer review;
Document peer reviewers' comments and responses; and
Document completion of peer review and log the status into the Science Inventory.
In cases of highly influential scientific assessments and/or influential scientific information
developed by Region 8 in collaboration with other EPA offices (e.g., Headquarters National
Program Managers and/or ORD), the peer review activities may be coordinated by a different
EPA office or location. In these instances, the peer review as well as the Peer Review
Coordinator, Decision Maker, and Peer Review Leader roles may be assigned to another office
or location. When this occurs, all peer review activities, responsibilities, oversight and tracking
are transferred to the respective role at that office/location and Region 8 retains no role or
responsibility in the peer review process for that work product except to provide information as
requested.
Work Prod uctPlanning S ta ge
To be most effective and efficient, peer review of a scientific or technical work product must be
incorporated into the up-front planning of any decision or action that will be made or taken based
on the work product, including the respective QAPP that details the project plans from which the
work product will be generated. The planning for peer review activities should occur at the time
of overall project planning and includes obtaining the proper resource commitments, establishing
realistic schedules to accommodate the peer review process and coordinating early with the
Region 8 Peer Review Coordinator. Proper advance planning by the DPM is essential to ensure a
positive and seamless peer review experience.
Assigning the Peer. Review Category
EPA's Peer Review Handbook discusses the following three peer review categories for scientific
and technical work products. A peer review may be internal and/or external to EPA, depending
on the category of the scientific or technical work product. The 3 categories are:
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
86
1) Highly Influential Scientific Assessment (HISA);
2) Influential Scientific Information (ISI); and
3) Other scientific and technical work products (Other).
Table 3 provides examples of work products for each Category. Refer to the subsequent sections
for the peer review requirements for each Category.
Table 3. Example Work Products for Peer Review Categories 1-3
Peer Review Category
Example Work Products
Highly Influential Scientific Assessments
(Category 1)
QA Planning Documents (QAPPs,
SAPs, FSPs)
Scientific Technical Reports
Decision Documents
Reference Concentrations
New analytical or sampling methods
Novel models
High-profile draft manuscripts
(journals, articles, books, etc.),
scientific findings, book chapters, and
reports that have heightened public or
media interest, or are of interest to
EPA's national program or regional
offices due to policy implications.
Influential Scientific Information
(Category 2)
Other Scientific and Technical Work Products
(Category 3)
Draft manuscripts (journals, articles,
books, etc.) that are not classified as
HISA or ISI and are planned for
submission to non-EPA publications.
All Category 3 work products that appear in non-EPA publications must include a disclaimer and
be logged by the Peer Review Coordinator; therefore, Category 3 work products must be routed
through the Peer Review Coordinator as part of the Clearance/Office Review process.
Additionally, all scientific and technical work products not requiring peer review must be
processed through the Region 8 internal Clearance/Office Review process prior to dissemination
(Refer to Section 9.8.3).
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
87
The Decision Maker is responsible for assigning the Peer Review Category for each work
product, although the DPM is responsible for informing both the Decision Maker and the Peer
Review Coordinator of the work product so that it may be classified.
Reclassification of a Work Product
On occasion, there may be situations that require the Peer Review Category for a work product
to be reclassified. For example, new scientific information and/or changes in project visibility
may result in Region 8 requiring an increased level of peer review for either a single work
product or an entire project. It is the DPM's responsibility to alert the Peer Review Coordinator,
Decision Maker and Peer Review Leader of any conditions that may lead to a change in work
product and/or project classification. Refer to definitions of HIS A, ISI and Other in Section
9.9.2.
Peer Review Expectations
The following provides an overview of peer review expectations for each Peer Review Category.
Decisions regarding peer reviewer make-up (e.g., individual versus panel review); peer review type
(e.g., letter versus panel); timing; scope of the review; selection of reviewers; disclosure and
attribution; public participation; disposition of reviewer comments; and adequacy of prior peer
review are considerations that are situational and should be made on a case-by-case basis. Consult
the Peer Review Handbook (including the OMB Peer Review Bulletin) for more detailed
information and requirements, especially for HISAs and ISIs. Lastly, some peer review activities
are prescribed by a rule-making action or other regulatory requirement. In these cases, also refer
to the specific regulatory requirements for peer review.
Expectations for HISA Work ProductsCategory 1
General expectations for conductance of peer review for any work product meeting the HISA
Category 1 are outlined below. The individual(s) responsible for conducting the activity is/are
listed in the brackets that follow the action item. In addition the Decision Maker and/or the Peer
Review Coordinator have decision-making/oversight responsibilities for these items as described
in Sections 9.9.3 and 9.9.4.
Route the decision document classifying the project or work product to the Peer Review
Coordinator [Peer Review Leader],
Develop a peer review plan early during the product development process, including a
schedule and resource allocations for external peer review. Allow for a 30-day public
comment period for the posted peer review plan [Peer Review Leader],
Enter information on the peer review plan in the Science Inventory [Peer Review
Coordinator],
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
88
Create and maintain the Peer Review Record [Peer Review Leader],
Develop the charge [Peer Review Leader],
Establish external peer review panel and/or other peer reviewer process, including peer
reviewer acquisition and establishing confidentiality agreements and/or conflict of
interest documentation, as necessary [Peer Review Leader],
Compile peer reviewer's materials, including work product, charge, all supporting data,
etc. [Peer Review Leader],
Conduct internal review prior to external peer review [Peer Review Leader],
Update the work product to incorporate internal peer review comments [Peer Review
Leader],
Conduct external peer review, preferably through a panel [Peer Review Leader],
Require the peer reviewers to prepare a written peer review report [Peer Review Leader],
Prepare EPA's written response to the peer review findings explaining inclusion,
exclusion or augmentation of review comments into the work product [Peer Review
Leader],
Finalize the work product in accord with EPA's written response to the peer review
findings [DPM and Peer Review Leader],
Route the documentation to the Peer Review Coordinator (through appropriate chain of
command) for final work product Clearance [Peer Review Leader],
Post the peer review charge, the peer review report, EPA's response to the peer review
comments, and the final product in the Science Inventory [Peer Review Coordinator],
Expectations for IS/ Work ProductsCategory 2
General expectations for conductance of peer review for any work product meeting the ISI
Category 2 are similar to HIS A (above) with two key exceptions. For ISIs, the peer review may
be internal and/or external in nature, and the opportunities for public input along the process may
(or may not) be less. The individual(s) responsible for conducting the activity is/are listed in the
brackets that follow the action item. In addition the Decision Maker and/or the Peer Review
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
89
Coordinator have decision-making/oversight responsibilities for these items as described in
Sections 9.9.3 and 9.9.4.
Route the decision document classifying the project or work product to the Peer Review
Coordinator [Peer Review Leader],
Develop a peer review plan early during the product development process, including a
schedule and resource allocations for external peer review. Allow for a 30-day public
comment period for the posted peer review plan [Peer Review Leader],
Enter information on the peer review plan in the Science Inventory [Peer Review
Coordinator],
Create and maintain the Peer Review Record [Peer Review Leader],
Develop the charge [Peer Review Leader],
Establish the peer reviewer process, including peer reviewer acquisition and establishing
confidentiality agreements and/or conflict of interest documentation, as necessary [Peer
Review Leader],
Compile peer reviewer's materials, including work product, charge, all supporting data,
etc. [Peer Review Leader],
Conduct internal review prior to external peer review [Peer Review Leader],
Update the work product to incorporate internal peer review comments [Peer Review
Leader],
Conduct external peer review, if required [Peer Review Leader],
Require the peer reviewers to prepare a written peer review comments and/or report [Peer
Review Leader],
Prepare EPA's written response to the peer review findings explaining inclusion,
exclusion or augmentation of review comments into the work product [Peer Review
Leader],
Finalize the work product in accord with EPA's written response to the peer review
findings [DPM and Peer Review Leader],
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
90
Route the documentation to the Peer Review Coordinator (through appropriate chain of
command) for final work product Clearance [Peer Review Leader],
Post the peer review charge, the peer review report, EPA's response to the peer review
comments, and the final product in the Science Inventory [Peer Review Coordinator],
For purposes of the OMB Peer Review Bulletin, "dissemination" excludes research produced by
government-funded scientists (e.g., those supported extramurally or intramurally by federal
agencies or those working in state or local governments with federal support) if that information
is not represented as the views of a department or agency (i.e., they are not official government
disseminations). As a result, ISIs that are published as journal articles, posters, etc., are excluded
from the OMB Peer Review Bulletin requirements provided the following disclaimer is included:
The views expressed in this (article/chapter/paper) are those of the author[s] and do
not necessarily represent the views or policies of the U.S. Environmental Protection
Agency.
Expectations for Other Scientific and Technical Work ProductsCategory 3
All draft manuscripts (articles, book chapters, manuals, etc.) that are not classified as ISI or
HISA but will be published in non-EPA sponsored publications are classified as Other Category
3. All Category 3 work products must include a disclaimer in the published material.
Additionally, the work product is recorded in the Science Inventory by the Peer Review
Coordinator, therefore Category 3 work products must be routed through the Peer Review
Coordinator as part of the Clearance/Office Review process.
Route documentation of the Clearance/Office Review process to the Peer Review
Coordinator (through appropriate chain of command) for final work product
Clearance/Office Review [Lead Author],
Provide the Peer Review Coordinator with information about publication date and
citation [Lead Author],
Enter the publication in the Science Inventory [Peer Review Coordinator],
Disclaimer for Review Drafts
In addition to any standard disclaimer needed for the document, the Lead Author should include
the disclaimer listed below for information quality guidelines purposes prior to external review.
This document is distributed solely for the purpose ofpre-dissemination peer review
under applicable information quality guidelines. It has not been formally disseminated by
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
91
EPA. It does not represent and should not be construed to represent any Agency
determination or policy.
Disclaimer for Published Work Products
All work products submitted by a Region 8 employee for publication (e.g., manuscripts, papers,
book chapters, etc.) to a non-EPA publisher must conspicuously include the following disclaimer
in the final publication:
The views expressed in this (article/chapter/paper) are those of the author[s] and do
not necessarily represent the views or policies of the U.S. Environmental Protection
Agency.
Conflict of Interest/Confidentiality
Documentation of any potential or perceived conflict of interest (COI) is required for peer
review of HIS A work products and/or a peer reviewer who receives monetary compensation for
peer review services. And, it is a good practice to document COI for anyone conducting peer
review, regardless of compensation. For this purpose, Region 8 has created a
COI/Confidentiality Agreement form.
9.9.6 DOCUMENTATION AND Peer Review Record
A peer review of a work product is documented in the Peer Review Record. The purpose of the
Peer Review Record is to provide sufficient documentation for someone independent of the peer
review to reconstruct the peer review process, results and decisions.
For each peer review conducted, a Peer Review Record includes, at minimum:
Completed Conflict of Interest and/or Confidentiality Agreement Forms, if applicable;
Draft work product (including supporting data) submitted to peers for review;
Materials and information (including the charge) given to the peer reviewers;
Peer review report, summarizing the peer review findings, containing information about
the peer reviewers (name, affiliation, area of expertise/qualifications, and a statement
concerning potential conflicts and their resolution, if applicable);
Logistical information (times/locations) about the conduct of the peer review;
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
92
Written records, concurred on by the Decision Maker, responding to all peer reviewer
comments, specifying acceptance or rebuttal and/or non-acceptance of peer review
comments; and
Final draft work product that addresses the peer review comments.
The Peer Review Leader is responsible for creating and maintaining the Peer Review Record.
The Peer Review Leader must document that all record contents are included in the Peer Review
Record. The peer review record may be kept with other records relating to the overall project, as
long as it is easily and separately identifiable. The peer review record should be maintained in
accordance with the Agency's record-keeping schedule (Section 13.0). At any time, the Peer
Review Coordinator may request to review all or parts of the Peer Review Record.
If more than one peer review is conducted on a single work product (e.g., both an internal and
external peer review is conducted), then a separate Peer Review Record is created for each peer
review (e.g., one record for the internal review and another for the external review).
Additional Documentation for HISAs and JS/s
Additional documentation and records are required for highly influential science assessments and
influential science information, such as: peer review plan, peer reviewer selection criteria, and
several others. Refer to the Peer Review Handbook for additional documentation requirements
related to peer review of HISA and ISI work products and/or contact the Regional Peer Review
Coordinator for specific requirements.
Documentation for Draft Manuscripts for Submission to non-EPA Sponsored Publication
For draft manuscripts the "final" work product consists of two items, at minimum:
Final draft work product that addresses the peer review comments and which will be
submitted to the journal; and
the published article.
9.9.7 Peer Review Tracking and ReportingScience Inventory
The Science Inventory is a searchable database designed to track and report peer review and
other science activities across the Agency. The database is the single repository for product-
specific peer review reporting and tracking and uses a common reporting form
(http://cfpub.epa.gov/si). Peer review work products are divided into three categories: highly
influential scientific assessments, influential scientific information, and other products. The
Science Inventory is used to automatically populate the Agency's Peer Review Agenda, which is
required by the OMB Peer Review Bulletin.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
93
Tracking Peer Review Activity
If the activity or work product is categorized as influential scientific information or a highly
influential scientific assessment, a peer review plan is prepared by the Peer Review Leader
which summarizes the peer review for use in the Science Inventory.
The Peer Review Leader shall provide the following information to the Peer Review
Coordinator, which addresses the provision of the OMB Bulletin that allows the public to view
and comment on the Agency's peer review plans for these activities or products:
A paragraph including the title, subject and purpose of the activity or product;
An Agency contact to whom inquiries may be directed to learn the specifics of the peer
review plan;
Whether the work product is likely to be influential scientific information or a highly
influential scientific assessment (this will be confirmed by the Decision Maker);
The timing of the review (including deferrals);
Whether the review will be conducted through a panel or individual letters (or whether an
alternative procedure will be employed);
Whether there will be opportunities for the public to comment on the work product to be
peer reviewed and, if so, how and when these opportunities will be provided;
Whether EPA will provide significant and relevant public comments to the peer
reviewers before they conduct their review;
The anticipated number of reviewers (3 or fewer; 4-10; or more than 10);
A succinct description of the primary disciplines or expertise needed in the review;
Whether reviewers will be selected by EPA or by a designated outside organization;
Whether the public, including scientific or professional societies, will be asked to
nominate peer reviewers.
National Reporting
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
94
The Region 8 Peer Review Coordinator will update the Science Inventory roughly twice
annually, which serves as Region 8's reporting requirements to the Office of Research and
Development (ORD). Real-time updates may occasionally be necessary for changes in high
visibility activities such as: a significant change in the timing for the peer review of a high-
profile study or a change in the timing of the public availability of a draft of a highly influential
assessment.
The Deputy Administrator has designated ORD to conduct an annual review of the peer review
plans. As called for in OMB's Peer Review Bulletin, ORD expects to submit a report to OMB by
December 15 of each year. This report will include information concerning the peer reviews
conducted on the highly influential scientific assessments and influential scientific information
during the previous fiscal year. ORD will generate this report from the information obtained
from the Science Inventory database.
Region 8 Reporting
Approximately annually, the Peer Review Coordinator will report to the Deputy RA and the
LSASD Director on Region 8 peer review activity and status, including a summary of any
ISI/HISA work products developed by the Region.
9.9.8 National Peer Review Advisory Group
The Peer Review Advisory Group (PRAG) consists of the Agency's Peer Review Coordinators.
The PRAG functions under the Science and Technology Policy Council (STPC), which
originates in the Office of the Science Advisor. The STPC has created the PRAG to assist in the
implementation of the Agency's Peer Review Policy. The primary role of the PRAG is to provide
interpretation of the policy and to assist the STPC and Agency Offices and Regions in preparing
updates to the Peer Review Handbook.
The Science and Technology Policy Council is responsible for overseeing Agency-wide
implementation of this policy, including promoting consistent interpretation; assessing Agency-
wide progress; developing recommendations for revisions of the policy as necessary; and issuing
the Peer Review Handbook, which provides additional information and procedures on
implementing this policy.
9.9.9 Training for Peer Review Requirements
In 2012, under the direction of ORD's Office of the Science Advisor, activities are currently
underway to develop a national peer review training program for EPA managers and staff. Once
developed by the PRAG, the peer review training will be made available to appropriate Region 8
managers and staff in Region 8. Region 8 specific training for staff and managers will establish
guidelines for identifying and reporting projects as emerging HIS A and ISI. This will be
developed in FY13.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
95
10 RESPONSE & QUALITY IMPROVEMENT (ACT)
W Act J
\ Plan\
kcheck
Do J
Action taken to improve quality and or in response to assessment findings or observations
(corrective action) taken describe the Act element of the PDCA quality model.
10.1 Quality Improvement
The Region 8 QA Branch is committed to continual improvement of the Region 8 Quality
System. Quality improvement is a process that proactively addresses vulnerabilities and
enhances efficiency. The Region 8 QA Branch meets regularly to discuss cross-cutting issues
and to look at ways of improving the organizational implementation of QA. Quality
improvement is incorporated as a core organizational element of the Region's quality culture and
philosophy and looks to correct systemic problems, improve consistency, and streamline QA
processes, as necessary. The Region 8 QA staff also provides a perspective on the continuous
improvement process and provides advice and recommendations to program office managers on
ways to improve the quality processes within the Region. If identified, the QA Branch will
address areas where a general policy needs to be established or changed.
The QA Branch supports continuous training for staff in quality assurance, as well as technical
subjects related to their area of expertise and to new areas of interest or of emerging importance
to the Agency and to Region 8 Programs.
Region 8 QA Branch staff and management develop an annual list of Internal Assessments to be
conducted that year, based on their experience during the past year, external reports, etc. This is
further discussed in EPA Guidance on Technical Audits and Related Assessments for
Environmental Data Operations, QA/G-7.
It is the Region's policy that the QA staff and all others associated with environmental data
generation, use and reporting strive for continuing improvement in the products and services they
provide, and in their performance in doing so. Managers should encourage innovation and
improvement and should include it as tangible elements in staff performance reviews and
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
96
awards. The RQAM and the other managers and supervisors should seek and implement
processes designed to foster an atmosphere of improvement in the Region's QA community.
10.2 Corrective Actions
Region 8's Quality Improvement Policy (Section 10.1\ encourages corrective actions to be made
at any time and by anyone; these actions may be formal or informal. Formal corrective actions,
in response to assessment findings, must be taken promptly by the designated responsible
organization, Program and/or Office representative. This person is also responsible for tracking
the response, confirming the implementation, assessing the effectiveness of any corrective
action, and reporting the response status to the RQAM.
Region 8 managers are responsible for including formal corrective actions and/or corrective
action plans into their fiscal work planning activitiesfor EPA activities as well as those of our
partners (e.g., States, Tribes, etc.) This approach ensures application of adequate resources and
establishment of a timely response/conclusion schedule. Region 8 managers are responsible for
reporting corrective action plans and completion status to the RQAM on a quarterly basis.
Documentation includes reporting the original finding, the corrective action to be taken, the
status of that action and the schedule for completion.
For findings that rise to a higher level of concern, Region 8 applies the obligations outlined in the
Federal Manager's Financial Integrity Act (FMFIA) as a tool to identify and elevate emerging
vulnerabilities to senior managers. Region 8 managers are responsible for identifying and
communicating findings that may emerge as FMFIA vulnerabilities.
10.3 Dispute Resolution
When issues regarding quality assurance are in dispute, resolution will be sought at the lowest
management level possible. Such disputes may occur in situations involving technical issues
(e.g., audits, data quality assessments) and management issues (e.g., QMP reviews, QAPP
reviews, and quality system assessments).
All parties will make every effort to resolve disputes through discussion and negotiation.
Disagreements will be resolved at the lowest administrative level possible. Should agreement not
be reached at this level, the issue will be resolved by senior management. The Regional
Administrator and the Deputy Regional Administrator have final dispute authority on all quality
issues.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
97
11 PERSONNEL QUALIFICATIONS, TRAINING AND
COMPETENCE
The region implements procedures to assure all personnel performing work for Region 8 have
the competency to effectively accomplish their work. To achieve Regional quality goals and
objectives, management and staff performing tasks related to environmental data operations must
have the skills and knowledge to effectively accomplish their work.
It is Regional policy to provide and make available to management and staff the training, including
QA training, necessary to carry out their work successfully. Senior management and supervisors,
with assistance from the Region 8 Science Council, takes the lead in ensuring that the necessary
levels of technical proficiency and Q A knowledge are maintained.
Personnel must meet the minimum qualifications defined in the Office of Personnel
Management (OPM) Qualification Standards Handbook for their series and grade. The
application of sound QA policies and procedures requires that all personnel performing quality-
related tasks associated with environmental data operations have an appropriate level of
knowledge of QA procedures and principles.
The Human Resources Shared Service Center works with the Regional Human Resources
Branch to ensure that all EPA positions are properly classified as to job series, title, and grade
based on an analysis of the duties of the position, as defined and submitted by the supervi sor
and manager of record for the position, in compliance with OPM's position classification
system. Each classified position defines the principal duties, the knowledge required, the level
of supervision, and a variety of other factors used to determine the final grade level of the
position. The knowledge, skills, and abilities needed to perform the work of the position are
incorporated as part of the qualifications identified to fill the position. Applicants for QA
positions must demonstrate that they have the required knowledge, skills, and abilities to meet
the qualifications of the position.
Typical job classes for the Region 8 QA Branch Chief and QA personnel are scientists,
chemists, engineers, life scientists, and statisticians.
It is also essential that the supervisor and manager of record for positions with QA responsibilities ensure
that incumbents have performance plans, critical job elements, and performance standards reflecting their
QA work each year, in compliance with EPA's performance management system. In this manner,
employees with QA responsibility will have identified measurable goals and objectives for each year.
The QA Branch works with the Regional Training Officer, the Region 8 Science Council, and the
environmental programs to ensure competency of Region 8 personnel involved in environmental data
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
98
operations. QA training is based on prioritized needs and implemented as resources permit. The QA
Branch may offer the following training:
ฎ QA Core Training
Delegated QA Approving Officer Training
Existing Data Training
Data of an Unknown Quality Training
QAFAP Training
Training Documentation
The RQAM (or designee) will maintain a record of all formal training sessions (i.e., training
sign-in sheets) offered by the QA Program. Training records will include the training agenda,
name of all attendees, the date and title of each course offering. This information may also be
tracked electronically. A critical function of the mandatory Agency-wide Quality System (CIO
2105.0) is identifying, developing, and implementing QA training programs. All Regional
personnel involved with environmental data operations are required to have the appropriate QA
training as outlined in the following sections. Additionally, it is the supervisor's responsibility to
ensure that the individuals in their organizations meet the minimum QA training requirements
for their assigned activities.
At the end of each fiscal year, a summary of the QA training offered in Region 8 may be
summarized in a regional annual report. The summary will include the courses offered and the
number of attendees for both EPA and non-EPA training offerings.
11.1 Core QA Training
The Core QA Training Course provides an overview of the mandatory Agency-wide Quality
System and based upon QA requirements outlined in EPA Orders and other pertinent standards.
Specifically, EPA Order 2105 requires that the Agency provide the "appropriate training, for all
levels of management and staff, to assure that QA and QC responsibilities and requirements are
understood at every stage of project implementation." (See EPA Order 2105 6.a.(l 1).)
The course directs the class participants to the QA tools developed to implement the QA/QC
requirements in Region 8, and provides updates on continuing, new, and/or changing QA/QC
requirements. Elements of the Core QA Training Course may be expanded as necessary to
address findings from internal or external quality system audits/re views and/or to incorporate
new QA regulations, requirements and standards.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
99
To meet the requirements of EPA Order 2105, all Region 8 management and staff involved with
environmental data operations (i.e., funding, managing, creating, handling, processing,
interpreting, and/or reporting environmental data) may be required to take the Core QA Training
course. EPA personnel may be required to attend the Core QA Training every two years.
The QA Branch will maintain a tracking system of attendees to its training sessions. Employees
must ensure his/her attendance is recorded in the QA Branch Tracking System.
11.2 Delegated QA Approving Officer Training
The Delegated QA Approving Officer (DAO) is a valued and important part of Region 8's
Quality System, who has been delegated project-level QA document approval authority by the
RQAM. Delegation depends, in part, on completion of special training. The training
requirements established for DAOs are designed to provide a foundation for new Approving
Officers and to keep existing Approving Officers current and abreast of new innovations. In
addition to formal training courses, candidate Approving Officers agree to other reporting and
oversight requirements outlined Sections 9.9.6 and 13.0.
Training requirements
To be eligible for delegation, a candidate Approving Officers may be required to complete the
Delegated QA Approving Officer training and any other training specified by the RQAM.
Periodic updates from the RQAM
Throughout the year, DAOs may receive updates from the RQAM. These updates, which are
pertinent to their project-level QA document approval duties, may include notice of any new
EPA Orders, guidance or review requirements; approval status updates for Contract-level QA
documents (e.g., QMPs, QAPPs, etc.); or similar information. Dissemination of this information
will typically be via email. DAOs are expected to immediately incorporate the information
and/or new practices into their reviews/approvals. DAOs should contact the RQAM (or
designee) with any questions about implementation of the update.
QA Branch Consultation
At any time, DAOs may contact the QA Branch to obtain consultation on how to conduct
project-level QA document reviews or clarity regarding other QA concerns. Generally,
consultation with the QA Branch is an optional benefit that is available to Approving Officers.
However, if an assessment reveals that an Approving Officer's supporting documentation is
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
100
deficient, then the RQAM will request the DAO to meet with the QA Branch for coaching or
consultation.
Training Documentation
The RQAM (or designee) will maintain a record of all formal training sessions {i.e., training
sign-in sheets) offered by the QA Branch for DAOs. Training records will include the training
agenda, name of all attendees, the date and title of each course offering. This information may
also be tracked electronically. Any RQAM updates to the Approving Officers shall also be
retained on file in the QA Branch.
Approving Officers are encouraged to be aware of their status, maintain a record of their required
training, and take required training at the appropriate frequency.
11.3 Existing Data Training
Region 8, staff whose functions involve the oversight and approval of projects that generate, use,
or report environmental data may be required to complete the QA for Projects using Existing
Data Training Course. Attendance is required every two years.
Training Documentation
The RQAM (or designee) will maintain a record of all formal training sessions {i.e., training
sign-in sheets) offered by the QA Branch for DAOs. Training records will include the training
agenda, name of all attendees, the date and title of each course offering. This information may
also be tracked electronically.
12 PROCUREMENT AND FINANCIAL ASSISTANCE
Region 8 has a responsibility to ensure that resources expended through grants or cooperative
agreements, Interagency agreements or acquisitions for environmental data operations are
implemented in a way that result in environmental data and decisions that are scientifically
sound, defensible and of known quality. Procedures established to oversee these activities are
described below for each funding mechanism.
EPA uses the following primary types of instruments to engage in extramural activities:
Acquisitions (contracts), and
Assistance Agreements (i.e., grants, cooperative agreements, and interagency
agreements (IAs)
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
101
12.1 Procurement of Items & ServicesAcquisitions
In order to ensure that contractually procured environmental data operations are scientifically
valid, defensible and of known quality, Contracting Officers (COs), Contract Project Officers,
and Contracting Officer's Representatives (CORs) are responsible for adhering to QA
requirements set forth in the EPA Contracts Management Manual (CMM) and Federal
Acquisition Regulation (FAR). The FAR and the CMM provide the basis for the incorporation of
quality requirements in acquisition documents; therefore, all acquisition agreements, including
simplified acquisitions, which include environmental data operations are subject to established
QA requirements contained therein.
The QA Branch developed required QA Standard Work documents for use in all Region 8
contract solicitations. Contract Officers (COs) and Contract Officer Representatives (CORs)
should contact the QA Branch for the most up-to-date Standard Work documents.
12.1.1 Statements of Work (SOWs)
Work activities directed to the contractor, including QA requirements, are summarized in a
Statement of Work (SOW), sometimes also referred as Task Orders, Delivery Orders, Work
Assignments, etc.13 SOWs come in two basic forms: Contract-level SOW and Individual or
Task-level SOW.
Contract-level SOW
The Contract-level SOW specifies the contract-level deliverables and general standards which
the Agency will use to determine that requirements have been met. During solicitation of
potential contractors, a Request for Proposal (RFP) package, which will usually include a draft
SOW that allows potential contractors to develop their proposals based on the RFP requirements.
Individual/Task-level SOW
After contract award, Individual/Task-level SOWs are developed as the COR identifies
individual work activities (sites, projects, etc.) The Individual/Task-level SOW specifies the
details of the technical requirements and related QA requirements for contractor tasks.
13 Recognizing that an alternate term may be appropriate for a particular contract mechanism, the term SOW will be
used throughout this QMP for the purposes of describing the QA requirements associated with work activities
defined for the contractor.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
102
12.1.2 QA Review Form QARF
The QARF is used to ensure that quality requirements of FAR 46.202 and FAR clause 52.246-11
are communicated to the CO, and to ensure that EPA-specific requirements (defined in CIO
Policy 2105.0 and CIO Procedure 2105-P-01-0) are met (see EPAAG 46.2.1.5.1). Regional
procurement of contracts may occur through one of two processes: procurement through HQ
managed funds or procurement processed by the Region 8 Acquisitions Branch. In either
process, when the Region is responsible for oversight of QA for that contract mechanism, all
work direction (SOWs) must be accompanied by a QARF for the RQAM signature. The SOW
(for solicitation or project level) and the accompanying QARF is submitted to the RQAM via the
CO prior to initiating any contract action (i.e., either solicitation/pre-award or project level/post-
award).
The EPAAG specifies that:
A QARF be completed for all acquisition vehicles including simplified acquisitions,
solicitations, contracts, statements of work, task orders, work assignments, delivery
orders and modifications to existing task orders, work assignments or delivery orders that
involve a significant change to the SOW.
The QARF documents the assessment of whether environmental data operations
(environmental data and/or environmental technology) are included in the scope of the
acquisition document.
If environmental data operations are included in the scope of the acquisition, the specific
type of quality requirements must be defined in the QARF.
If the QARF indicates that QA requirements are applicable to the acquisition and the
estimated value of the acquisition exceeds $500,000, the applicable Regional QA
Manager (or designee) shall be a member of the Technical Evaluation Panel (TEP).
The COR/PO shall complete a QARF for each SOW. The COR/PO's signature indicates that the
QARF clearly describes the item or service needed, it is aligned with the direction outlined in the
corresponding SOW and that the associated quality requirements are fully defined. The
completed QARF and companion SOW are submitted to the RQAM for review.
The RQAM (or designee) reviews the QARF to ensure that all environmental data operations,
contractually funded by EPA, are in compliance with QA requirements listed in 48 CFR Part 46,
EPA CIO 2105.0 and ASQ/ANSIE4-2014. Approval of the QARF by the RQAM represents an
Agency action; therefore, the QARF may not be changed or altered after the RQAM had signed
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
103
it. The QARF in the EPAAG Appendix 46.2.1-D (or reference to other program-specific QA
Review Forms), must be completed for all solicitations and/or project-level SOW(s). In addition,
any modifications to the companion SOW must be re-evaluated with a new QARF that are
marked to clearly indicate the modifications, include any up-dated QA requirements. If
approved by the RQAM the modified SOW and QARF will supersede the original. The QARF
verifies to the CO that the any task involving environmental data have the applicable QA
requirements.
A blank QARF is available in electronic (MS Word) format from the Region 8 QA Branch. The
COR is responsible for coordinating with the Contract Project Officer and Contracting Officer
assigned to that contract to ensure adequacy, consistency, and inclusion of the appropriate QA
requirements throughout the life of the work activities. The COR is expected to be
knowledgeable in the contract's QMP status and if approved, the QA requirements in the QMP
need to be incorporated into the QARF and SOW.
The CO must review the QARF and any quality assurance requirements and information
provided by the COR for each project-level SOW to be performed under the contract to ensure
that these requirements are consistent with the quality requirements of the contract (i.e.,
consistent with the QA requirement in contractor's QMP as well as the contract SOW).
Refer to Figures 5 and 6 for the flow process for QARF development and review/approval.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
104
Figure 5. QARF Approval Flow Diagram for Pre-Award (Solicitation) Contract Actions
Prepare Procurement Initiation (PIN) or Purchase
Request (PR) Package
Contract-level SOW & QARF
Prepared by COR/PO
Contract-level SOW & QARF
Reviewed by RQAM
Does work involve the generation, collection,
use or reporting of Environmental Data or
design , construction, or operation of
Environmental Technologies?
Yes
No
QARF
signed by RQAM
l_
Document QA Requirements
for the PIN or PR Package
Prepared by PO/COR & Concurred by RQAM
T
QARF
signed by RQAM
j
Finalize Contract-level SOW & QARF,
Submit with PIN or PR Package
Prepared by PO/COR & Delivered to AMU
LEGEND
Tracking Required
(QA Documents & QARF)
ABBREVIATIONS:
PIN
Procurement Initiation
PR
Purchase Request
SOW
Statement of Work
QA
Quality Assurance
QARF
QA Review Form
RQAM
Regional QA Manager
COR
Contracting Officer's
Representative
PO
Project Officer
AMU
Acquisitions Management Unit
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
105
Figure 6. QARF Approval Flow Diagram for Post-Award Contract Actions
Contract-level Tasks/WAs/Deliveries
Contract-level SOW & QARF
and/or
Individual TOs/WAs/DOs
Individual TOs/WAs/DOs SOW & QARF
Prepared by COR/PO
Yes
I
Document QA Requirements for:
Individual TO/WA/DO SOW
Prepared by COR/PO & Concurred by RQAM
t
QARF
signed by RQAM
]
COR Meets with RQAM to Discuss which:
Individual TOs/WAs/DOs SOW
Require QA Documents
Does work involve the generation, collection,
use or reporting of Environmental Data or
design , construction, or operation of
Environmental Technologies?
No
t
QARF
signed by RQAM
Finalize & Submit Contract-level &
TO/WA/DO SOWs & QARFs
Prepared by COR/PO & Delivered to AMU
Tracking Required
(QA Documents & QARF)
ABBREVIATIONS:
TO Task Order
Work Assignment
Delivery Order
Statement of Work
Quality Assurance
QA Review Form
RQAM Regional QA Manager
COR Contracting Officer's
Representative
Project Officer
Acquisitions Management Unit
WA
DO
SOW
QA
QARF
PO
AMU
12.1.3 QA Requirements for Solicitations
The QA Branch developed required QA Standard Work documents for use in all Region 8
contract solicitations. Contract Officers (COs) and Contract Officer Representatives (CORs)
should contact the QA Branch for the most up-to-date Standard Work documents.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
106
Requests for Proposal (RFPs) with Higher-level Contract Quality Requirements requiring
Technical Evaluation are those that meet the definitional requirements for Higher-level Contract
Quality Requirement AND:
the potential value of the procurement exceeds $650,000; or
the estimate of the percentage of costs or level-of-effort allocated to activities requiring
quality requirements exceeds 15%; or
procedures defined in the Agency-approved Quality Management Plan of the
organization sponsoring the work apply.
Higher-level Contract Quality Requirements apply to all acquisitions that include collection,
generation, use or reporting of environmental data and the design, construction or operation of
environmental technologies.
The quality assurance (QA) requirements outlined herein apply to any organization or individual
under direct contract to EPA to furnish services or items or perform work (i.e., a contractor)
under the authority of 48 CFR Part 46 (including applicable work assignments, delivery orders,
and task orders). Therefore, any and all RFPs request that the organization and/or individual
proposed to conduct and/or oversee the work activities has the ability to:
comply with 48 CFR Part 46; and
develop, implement, maintain, and document a quality system that demonstrates
conformance to the minimum specifications of EPA CIO 2105 and ANSI/ASQ E4-1994.
A successful response/proposal must demonstrate the offeror's ability to develop, implement,
maintain, and document a quality system that demonstrates conformance to the minimum
specifications of EPA CIO 2105 and ASQ/ANSIE4-2014.
In response to the PR, the Offeror must submit a QMP (and possibly a QAPP depending on the
contract); see the Table of QA Requirements (minimum elements to include in the QMP.
Depending on the contact, additional requirements may be specified. The Offeror must also
complete the Region 8 Quality Management Plan Crosswalk. The Offeror's QMP will be
reviewed by the RQAM (or designee) and its review documented on the Crosswalk
(accompanying the Offeror's QMP).
When the awardee(s) is/are selected, the CO will issue a conditional approval with the QMP
Crosswalk comments. The awardee(s) will have 30 days and one reiteration to submit a response
to all the QA Program comments as well as an approvable QMP for all the environmental
activities outlined in the Contract-level SOW. The contractor shall complete and return the
Region 8 QMP Crosswalk along with the QMP in both hard copy and electronic formats. The
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
107
QMP and Crosswalk shall be reviewed and approved by the EPA Contracting Officer, the COR,
and the DPM. These documents shall then be submitted to the RQAM for review and final
approval. In cases where the Contract-level QARF requires a QAPP instead of a QMP, the
review process for a QAPP applies (i.e., the use of the R8 Document Review Crosswalk) and the
process is the same as described for the QMP.
Technical Evaluation Criteria for QA Requirements are outlined and are suggested to be an
addendum to the Contract-level SOW. These requirements are available from the Region QA
Branch.
A successful response/proposal must demonstrate:
an understanding of the required Technical Evaluation Criteria (TEC) components;
experience in successful implementation of each QA requirement and TEC component by
describing examples from previous experiences conducted by the proposing organization
and individual(s) proposed to conduct and/or oversee the work activities;
that the organization and individual(s) proposed to conduct and/or oversee the work
activities have an understanding of and capacity to implement each QA Requirement and
TEC component as it relates to the work activities outlined in the RFP; and
the offeror's ability to develop, implement, maintain, and document a quality system that
demonstrates conformance to the minimum specifications of EPA CIO 2105
(http://www.epa.gov/irmpoli8/policies/21050.pdf) and ANSI/ASQ E4-2014.
12.1.4 QA Requirements for Contract Actions After Award
Immediately upon award of the contract, the contractor shall submit a Contract-level QMP that
specifically addresses how the contractor will provide the products and services outlined in the
Contract-level SOW consistent with EPA QA requirements (i.e., the QMP shall not be a generic
QA plan for the contract's corporation, unless it also meets the Contract-level SOW
requirements). The contractor shall complete and submit the Region 8 QMP Review Crosswalk
along with the QMP in both hard copy and electronic formats. The QMP and Crosswalk shall be
reviewed and approved by the EPA Contracting Officer, the COR, and the DPM. These
documents shall then be submitted to the RQAM for review and final approval. In cases where
the Contract-level QARF requires a QAPP instead of a QMP, the review process for a QAPP
prevails.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
108
Annual Reviews
Annual reviews of QA documents (e.g., QMP, QAPPs, SAPs, FSPs) are required to be
conducted in accord with Section 9.6 of this QMP. Annual reviews of the contractor's quality
system are also required. The contractor's annual QA report will summarize the previous year's
QA and QC activities; document the QMP annual review and attach the corresponding Region 8
QMP Crosswalk; document all QAPP reviews and attach the corresponding Region 8 QA
Document Review Crosswalk; outline any corrective actions taken in response to the QA/QC
activities and/or reason no corrective action was taken; and recommend the QA/QC activities
planned for the next year of the contract.
These annual QA reports shall be submitted to the Contracting Officer (CO) and the Contract
Project Officer; the CO/PO are responsible for forwarding the annual report to the RQAM.
12.2 Grants and Cooperative Agreements
EPA's QA requirements for grants and cooperative agreements are contained in the EPA Order
2105 and the 2 CFR 1500.11 Quality Assurance for state, tribes, local governments, universities,
other non-profit organizations, etc.14
Assistance agreements (grants) that require Quality Management Plans include States where
multiple environmental programs are managed through Performance Partnership Agreements
(PPAs). Refer to Section 2.4.2 for more requirements for extending delegated authority to States.
Currently (2012).
12.2.1 Procedures in IGMS to Document QA Requirements
Compliance with QA requirements for grants is tracked in the Integrated Grants Management
System (IGMS) where the Funding Recommendation package must include a confirmation that
the grantee is compliant with all QA requirements.
Procedures in IGMS documenting QA Requirements
The Grants Project Officer and Grant Specialist shall conduct the following procedures in IGMS
for grants, including PPGs:
Also refer to the process flow diagram for QA Requirements in the Grants Process, Figure 7.
14 In June 2010, the Grants & Audits Procurement Program Director approved the grants process shown in Figure 7
with the understanding that the process would be modified as necessary to ensure compliance with the EPA Order
and Federal regulations.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
109
1. The Grants Project Officer must submit to the Grants Office the Statement of Work
(SOW) or Workplan (WP) clearly stating the project performance period {i.e., October 1,
20xx to September. 30, 20xx).
2. If B5 in the Funding Recommendation identifies generation, use or reporting of
environmental data, a grant term and condition requiring a QAPP and or QMP within 90
days of grant issuance must be included. The grant condition must also state that
initiation of environmental data activity cannot commence until an approved QAPP or
equivalent QA document is in place.
3. If B5 in the Funding Recommendation identifies generation, use or reporting of
environmental data, as per 2 CFR 1500.11, the grantee must submit an QAPP (or QMP if
indicated in the term and condition). Additionally, a completed Region 8 QA Document
Review QAPP crosswalk (or QMP crosswalk) must be submitted as a companion
document. The performance period and the project activities described in the QAPP must
be consistent with those outlined in the corresponding SOW or Work Plan.
The Program Project Officer is responsible for informing the grantee of the QA requirements
associated with each grant (not PPG). They are:
complete the Region 8 QA Document Review Crosswalk and provide with each QAPP
submittal;
conduct a review of the QAPP annually and document the review using the Region 8 QA
Document Review Crosswalk; and
submit the annual review (QAPP and companion QA Document Review Crosswalk) to
the Grant Project Officer.
-------
US EPA Region 8
Quality Management Plan
July 30, 2020
110
Figure 7. QA Requirements in the Grants Process Flow
Grant Application Received
And
FR Form Initiated
>
FR (question B.5)
states Data/Info
Collection in
Workplan?
\
FR (question
B.5.a) states
QAPP
Available and
QAPP Approved
demonstration
provided?
No
'
Grant Issued.
No Further QA
Review
Grants
Places
Condition
on Grant
(i.e., 60 to
90 day
Review
Period)
Notify PO
of Grant
Condition
Notify QA
for
Technical
Assistance
PO
acknowledges
grant
condition
Project /Tribal
Manager
work with
grantee on
QMP/QAPP
r
\
QAPP
Approved
(QA and
Program
Signatures
obtained)
\
/
*1
Program/Tribal
Manager provides
demonstration
Grants Program
Condition met
Notify Grants Program
condition not met
Grant Process Path Flow
Coordination with QA
Communication Requirement for PO and PM/TM
Grants Program
Notify Applicant
application Not
Approved
Work with Applicant
to submit
approvable QAPP
12.3 Interagency AGREEMENTS (IAs)
Region 8 works with a number of Federal agencies, including but not limited to the U.S.
Geological Survey, U.S. Fish and Wildlife Service, U.S. Forest Service, Indian Health Service,
Army Corps of Engineers, and Bureau of Reclamation. When Region 8 provides funds to another
Federal organization, the organization receiving the funds is responsible for preparing a QMP
and/or QAPP or equivalent QA document which then must be approved by the RQAM for
review and approval before any environmental data operations can be conducted. It is the DPM's
responsibility to ensure that the recipient of the IA is in compliance with the following QA
requirements: CIO 2105.0, 48 CFR 17.502-1 (b)(l)(i), and CIO 2133.
DPMs for the Interagency Agreement shall review and evaluate the use of these plans, assess the
data quality of the planned activities, and monitor the schedule for the expected outcomes. If
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
111
quality issues arise with the collected data, the DPM shall communicate these issues to the
RQAM for resolution.
13 DOCUMENTATION AND RECORDS MANAGEMENT
This process serves as a vehicle for identifying quality-related documents and records requiring
management control. Moreover, this process serves to assure that QA documents and records are
accessible, protected in storage from damage and deterioration, and managed according to
appropriate records retention policies. Finally, the process ensures compliance with all statutory,
contractual, and assistance agreement requirements for records from environmental programs,
while providing adequate preservation of key records necessary to support the mission of the
Region.
Per the Federal Records Act 44 U. S. Code, Chapter 33, Section 3301, records are defined as:
"all books, papers, maps, photographs, machine readable materials, or other
documentary materials, regardless ofphysical form or characteristics, made or
received by an agency of the United States Government under federal law or in
connection with the transaction ofpublic business and preserved or appropriate
for preservation by that agency or its legitimate successor as evidence of the
organization, functions, policies, decisions, procedures, operations, or other
activities of the Government or because of the informational value of the data in
them."
Region 8 adheres to the most recent version of the following legislation, regulations, guidance
and policies as they pertain to program requirements:
44 U.S.C. Chapter 31, Records directed by the Office of Administration and Management
by Federal Agencies;
44 U.S.C. Chapter 33, Disposal of Records;
18 U.S.C. Chapter 101, Records and Reports;
Paperwork Reduction Act of 1995;
OMB Circular A-130, Management of Federal Information Resources; 36 CFR Chapter
XII, Subchapter B
CIO Policy 2155.0, Records Management, U.S. Environmental Protection Agency;
Managing Cartographic and Architectural Records (Instructional Guide Series),
National Archives and Records Administration (NARA);
Managing Electronic Records (Instructional Guide Series), NARA;
Federal Records Management Laws and Regulations, NARA;
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
112
Disposition of Federal Records: A Records Management Handbook, NARA Personal
Papers of Executive Branch Officials: A Management Guide (Management Guide
Series); and
Records Disposition Schedules, U.S. Environmental Protection Agency.
The creation and maintenance of records, and the application of basic recordkeeping principles
will enable Region 8 to support the Agency's mission and ensure that we, as an Agency, are
meeting our obligations and accountability to the Administration, Congress, and the American
people. Region 8 employees, SEEs, and contractors for EPA have three basic obligations
regarding Federal records management:
1. Create records needed to conduct the business of the Agency, record decisions and
actions taken, and document governmental activities;
2. Take care of records so that information can be easily located when needed. This
means setting up (and following) organized and logical file plans, and filing materials
(in whatever format) regularly and carefully that allows the records to be safely stored
and efficiently retrieved when necessary; and
3. Carry out the disposition of records in accordance with Agency records schedules and
Federal regulation.
The DPM is responsible for managing all documents and records (paper and electronic),
including transmittal, distribution, retention, access, preservation (including protection from
damage, loss, and deterioration), version control, retrieval, removal of obsolete documents, and
disposition in accordance with appropriate records retention schedules as well as the policies and
guidance listed above. Inherent in this responsibility is management of all QA documentation
and records that are generated for or by the DPM and/or the project the DPM oversees. The DPM
is also responsible for ensuring that records and documents accurately reflect completed work.
Similarly, each Program or Office is responsible for managing the custody and confidentiality of
evidentiary documents and records (including all QA documentation and records) in accordance
with applicable regulations. Regional Records Center staff and resources are available to assist in
carrying out these responsibilities.
Further information on records management can be found on the EPA National Records
Management Program website, http://www.epa.gov/records. Region 8 staff may access records
management information on the Region 8 intranet and may also contact the Region 8 Records
Liaison Officer with requests for technical assistance and/or training.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
113
14 COMPUTER HARDWARE AND SOFTWARE
The Region's ability to fulfill its mission is dependent upon a strong information technology
infrastructure that is capable of supporting environmental information and dynamic
communication among EPA offices. The hardware, software, and communications components
that are encompassed by information technology form the foundation for environmental
information and EPA-wide communication. The management of information technology,
therefore, is critical to the success of the EPA.
Roles and Responsibilities
OMS is responsible for managing the EPA's information technology infrastructure and
components. In that role, OMS has established information technology standards to manage and
ensure that information technology components integrate properly into the infrastructure.
Region 8's Information Management Branch (1MB) is responsible for the development and
implementation of region-wide support in the areas of information and data systems management
and computer services. 1MB also participates in overall information management for the Region.
Region 8 Information Management Systems
It is Region 8 policy to work closely with OMS on all phases of system development,
improvements, and updates. All information management system development, improvements,
and updates will comply with Information Resources Management Policy, CIO Policy 2100.0
and CIO Policy 2121.0, System life Cycle Management Policy to include a systematic and
comprehensive dialogue among the data providers, data and system users, and system developers
prior to the design of the system.
The goal of this process is to have a uniform approach and review of applications under
consideration by Region 8. The process will determine if an application has management
support, information resource management (IRM) support, is doable in the time frame needed,
and is within the resource constraints identified. Compliance with the applicable IRM standards
will ensure that all hardware and software configurations are tested prior to use, to guarantee
they perform as expected and meet user requirements.
For information technology contracts that involve applications development, the performance
work statement will include, but not be limited to, requirements for system specification reviews;
system development plans; data validation and transfer; acceptance testing, and report
generation.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
114
Data Standards
All Federal agencies are required to adhere to Federally mandated data standards and regulations.
It is the policy of Region 8 to comply with all applicable regulations, guidance, executive orders,
and internal policy documents concerning data standards.
These include, but are not limited to:
The National Institute of Standards and Technology develops standards and guidelines to
achieve the most effective use of Federal information;
The Federal Information Processing Standards, which are the Federal data standards for
all data exchange among agencies; and
The EPA Data Standards Program established and documented in the Information
Resources Management Policy, CIO Policy 2100.0 and Data Standards, CIO Policy
2133.0.
EPA's data-related policies apply to all EPA organizations and personnel, including contractors,
and other personnel assigned to EPA who design, implement, and maintain information
management systems for Region 8 and EPA.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
115
15 REFERENCES
Administrator, United States Environmental Protection Agency. (2006, January 21). Peer
Review and Peer Involvement at the Environmental Protection Agency. Memorandum.
Washington, DC: Author.
American Indian Environmental Office, Environmental Protection Agency. (2011, May 4). EPA
Policy on Consultation and Coordination with Indian Tribes. Washington, DC: Author.
Burkhardt, Mark, and William Batschelet. U.S. EPA Region 8 Environmental Laboratory Quality
Assurance Manual. (2011, June 9). Golden, CO: Authors.
Chief Information Officer, United States Environmental Protection Agency. (2005, August 24).
National Geospatial Data Policy. Policy. Washington, DC: Author.
Chief Information Officer, United States Environmental Protection Agency. (2007, October 25).
National Geospatial Data Policy. Procedure for Geospatial Metadata Management. Policy.
Washington, DC: Author.
Chief Information Officer, United States Environmental Protection Agency. (2009, October 20).
Procedure for Quality Policy. Policy. Washington, DC: Author.
Chief Information Officer, United States Environmental Protection Agency. (2000, May 5).
Policy and Program Requirements for the Mandatory Agency-Wide Quality System.
Washington, DC: Author.
Chief Information Officer, United States Environmental Protection Agency. (2009, June 8).
Records Management. Policy. Washington, DC: Author.
Chief Information Officer, United States Environmental Protection Agency. (2007, June 28).
Requesting Data Standards Conformance Waiver. Policy. Washington, DC: Author.
Current Federal Information Processing Standards (FIPS). (2010, February 24). Gaithersburg,
MD: Information Technology Laboratory, National Institute of Standards and Technology.
Retrieved May 17, 2012 from http://www.nist.gov/itl/fipscurrent.cfm.
Deputy Assistant Administrator, United States Environmental Protection Agency. (2005, June
7). OSWER DIRECTIVE 92 72.0-17: Implementation of the Uniform Federal Policy for Quality
Assurance Project Plans (UFP-QAPP) at Federal Facility Hazardous Waste Sites.
Memorandum. Washington, DC: Office of Solid Waste and Emergency Response, United States
Environmental Protection Agency.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
116
Director, Office of Management and Budget. (2004, December 16). Issuance of OMB's "Final
Information Quality Bulletin for Peer Review". Memorandum. Washington, DC: Office of
Management and Budget.
Director, Science and Ecosystem Support Division, United States Protection Agency. (2010,
May 18). Letter. Athens, GA: Author.
Environmental Data Standards Council. (2006, January 6). Latitude/Longitude Data Standard
(Standard No.: EX000017.2).
EPA Quality Manual for Environmental Programs. (2000, May 5). Washington, DC: United
States Environmental Protection Agency.
EPA Requirements for Quality Assurance Project Plans. (2001, March). Washington, DC:
United States Environmental Protection Agency.
Executive Director, Federal Geographic Data Committee. (2010, November 10). OMB issues
Circular A-16 Supplemental Guidance. Retrieved June 7, 2012, from http://www.fgdc.gov/fgdc-
news/omb-endorses-al6-supplemental-guidance.
Federal Chief Information Officer. (2010, November 10). Issuance of OMB Circular A-16
Supplemental Guidance. Memorandum. Washington, DC: Office of Management and Budget.
Field Operations Group. Operational Guidelines for Field Activities, (n.d.). [Washington,
DC?]: United States Environmental Protection Agency.
Frequently Asked Questions: Policy to Assure Competency of Organizations Generating
Environmental Measurement Data under Agency Funded Acquisitions. (2011, February 21).
Washington, DC: United States Environmental Protection Agency.
Guidance for Geospatial Quality Assurance Project Plans. (2003, March). Washington, DC:
United States Environmental Protection Agency.
Guidance for Quality Assurance Project Plans. (2002, December). Washington, DC: United
States Environmental Protection Agency.
Guidance on Assessing Quality Systems. (2003, March). Washington, DC: United States
Environmental Protection Agency.
Guidance on Choosing a Sampling Design for Environmental Data Collection. (2002,
December). Washington, DC: United States Environmental Protection Agency.
Guidance on Environmental Data Verification and Data Validation. (2002, November).
Washington, DC: United States Environmental Protection Agency.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
117
Guidance on Systematic Planning Using the Data Quality Objectives Process. (2006).
Washington, DC: United States Environmental Protection Agency.
Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of
Information Disseminated by the Environmental Protection Agency. (2002). Washington, DC:
United States Environmental Protection Agency.
Information Resources Management Policy Manual. (1992). Washington, DC: Office of
Information Resource Management, Environmental Protection Agency.
NARA Basic Laws and Authorities. (2008, February). Washington, DC: National Archives and
Records Administration. Retrieved May 17, 2012 from http://www.archives.gov/about/laws.
National Exposure Research Laboratory. (2008, May). Scientific and Ethical Approaches for
Observational Exposure Studies. Washington, DC: United States Environmental Protection
Agency.
Office of Environmental Information, United States Environmental Protection Agency. (2006,
September 6). Pre-Dissemination Review Guidelines. Washington, DC: Author.
Office of Human Resources, Environmental Protection Agency (2000, January 28). Mandatory
Quality Assurance Program [Delegations Manual 1-41], Washington, DC: Author.
Office of Management and Budget. (2000, November 28). Management of Federal Information
Resources (OMB Circular A-130-revised). Washington, DC: Author.
Office of Management and Budget. (2002, August 19). Coordination of Geographic Information
and Related Spatial Data Activities (OMB Circular A-16-revised). Washington, DC: Author.
Office of Records Services, National Archives and Records Administration. (2000), Disposition
of Federal Records: A Records Management Handbook. Washington, DC: Author.
Paperwork Reduction Act of 1995, 44 U.S.C. งง 3501-3520, P.L. 104-13.
Policies and Procedures on Protection of Human Research Subjects in EPA Conducted or
Supported Research (EPA Oder 1000.17 Al). (2011, May 27). Washington, DC: United States
Environmental Protection Agency.
Records Management Self Evaluation Guide. Section 5. Records Disposition. (2012). United
States Environmental Protection Agency. Retrieved May 17, 2012, from
http://www.epa.gov/records/tools/toolkits/evaluat/sec5.htm.
-------
US EPA Region 0
Quality Management Plan
July 30, 2020
118
Science Advisor to the Agency, United States Environmental Protection Agency. (2004, March
10). New Policy Directive on Assuring and Documenting the Competency of Agency
Laboratories. Memorandum. Washington, DC: Author.
Science Policy Council, United States Environmental Protection Agency. U.S. Environmental
Protection Agency Peer Review Handbook (4th ed.). Washington, DC: Author.
Uniform Federal Policy for Quality Assurance Project Plans (2005, March). Washington, DC:
United States Environmental Protection Agency. Guidance, Workbook and Revised Worksheets
retrieved July 31, 2012, from http://www.epa.gov/swerffrr/documents/qualitvassurance.htm.
United States Code. (2012). Title 44, Chapter 31, Sees. 3101- 3107. Washington, DC: US
Government Printing Office.
United States Code. (2012). Title 44, Chapter 33, Sees. 3301-324. Washington, DC: US
Government Printing Office.
United States Code. (2012). Title 18, Chapter 101, Sees. 2071-2076. Washington, DC: US
Government Printing Office.
U.S. Environmental Protection Agency Scientific Integrity Policy. (2012, January 15).
Washington, DC: United States Environmental Protection Agency.
------- |