Quality Management Plan
for the Federal Facilities
Enforcement Office
-------
Quality Management Plan
for
Federal Facilities Enforcement Office
(Document Control No. QA-FFEO-Ol)
Federal Facilities Enforcement Office
U.S. Environmental Protection Agency
July 12,1996
-------
Table of Contents
Section
Ease
Date Last
Revised
Glossary of Acronyms
iii
July 12, 1996
1.
Introduction
1-1
July 12. 1996
2.
Quality Management Organization
2-1
July 12, 1996
3.
Quality System Description
3-1
July 12, 1996
4.
Personnel Qualifications and Training
4-1
July 12, 1996
5.
Procurement
5-1
July 12. 1996
6.
Records Management
6-1
July 12, 1996
7.
Information Resources Management "
7-1
July 12, 1996
8.
Quality Planning
8-1
July 12, 1996
9.
Quality Implementation
9-1
July 12. 1996
10.
Quality Assessment and Response
10-1
July 12. 1996
11.
Quality Improvement
11-1
July 12, 1996
References . . . : ^
. R-l
July 12, 1996
Federal Facilities Enforcement Office ' ii Quality Management Pku
-------
Glossary of Acronyms
Acronym Definition
CFA
Civilian Federal Agency
DQA
Data Quality Assessment
DQO
Data Quality Objective
EMMC
Environmental Monitoring Management Council
EFAAR
EPA Acquisition Regulations
FAR
Federal Acquisition Regulations
FEDPLAN-PC
Federal Agency Environmental Program Planning process
FFC
Federal Facility "Coordinator
FFEO
Federal Facilities Enforcement Office
FFTS
Federal Facilities Tracking System
FRC
Federal Records Center
LAG
Interagency Agreement
IRM
Information Resource Management
MSR
Management System Review
NDPD
National Data Processing Division
OARM
Office of Administration and Resources Management
OECA
Office of Enforcement and Compliance Assurance
OIRM
Office of Information Resources Management
PM
Project Manager
PRCS
Planning, Prevention and Compliance Staff
PR
Procurement Request
QAARW
Quality Assurance Annual Report and Workplan
Q AD
Quality Assurance Division
QA/QC
Quality Assurance/Quality Control
QAPP
Quality Assurance Project Plan
QMP
Quality Management Plan
RQAO
Regional Quality Assurance Officer
SIRMO *
Senior Information Resources Management Official
SOP
Standard Operating Procedure
SRES
Site Remediation and Enforcement Staff
TQM
Total Quality Management
WAM
Work Assignment Manager
Quality Management Plan
iii
Federal Facilities Enforcement Offia
-------
1. Introduction
1.1 Purpose of the Quality Management Plan
This Quality Management Plan (QMP) describes how the Federal Facilities Enforcement
Office (FFEO) will comply with EPA's quality system requirements set forth in EPA Order
5360,1, The purpose of the QMP is to provide a blueprint for planning, implementing, and
assessing the quality system for the environmental programs administered by FFEO.
Environmental data are an essential component of the decision-making processes that
support FFEO programs. The quality of available data can affect the soundness of these
decisions. In addition, because FFEO coordinates extensively with other Federal agencies. States,
and other organizations, it is essential that data generated through FFEO programs are
appropriate, consistent, of known quality, defensible, documented, and generated in the most
cost-effective manner.
This QMP documents the FFEO quality assurance (QA) process for all activities that
generate, validate, or prepare environmental data for FFEO decision-making. The QA policies,
roles, responsibilities, and management systems to implement the office-wide QA program are
also described. The goal of the FFEO QA program is to ensure that the types and quality of data
collected and/or used in FFEO programs are appropriate for the activity being undertaken.
For the purposes of FFEO programs, environmentally related data gathering activities may
include the following:
Use of data generated by Federal facilities to support programmatic
decisions, facilitate evaluations, or establish criteria and standards, or
* Preparation of guidance or promulgation of regulations requiring Federal
agencies to collect data.
The QMP is the first step in the process of implementing a comprehensive FFEO quality
management system. Due to its broad scope, the resource and scheduling implications of the
QMP are significant, thus, FFEO will phase in any requirements outlined in the QMP that are not
currently in pjace. By October 1. 1996, FFEO will establish a schedule for phasing in these
requirements.
1.2 Organization of the FFEO Quality Management Plan
This QMP discusses quality system components applicable to all FFEO programs
generating, using, or requiring others to collect environmentally related data. Quality system
components range from the QA roles and responsibilities of FFEO management to the specific
quality assurance/quality control (QA/QC) activities of FFEO staff This QMP has been prepared
in accordance with the Quality Assurance Division (QAD) guidance document. EPA
Quality Management Plan l-l Federal Facilities Enforcement Offict
-------
r:
Requirements for Quality Management Plans (interim FinalJ, EPA QA/R-2, August 1994, The
FFEO QMP establishes policies for the following:
Quality Management Organization (Chapter 2);
Quality System Description (Chapter 3);
Personnel Qualifications and Training (Chapter 4);
Procurement (Chapter 5);
Records Management (Chapter 6);
Information Resources Management (Chapter 7);
Quality Planning (Chapter 8);
Quality Implementation (Chapter 9);
Quality Assessment and Response (Chapter 10); and
Quality Improvement (Chapter 11).
Federal Facilities Enforcement Office
1-2
Quality Management Plat
-------
¦CIS"
ran
2. Quality Management Organization
2.1 FFEO Quality Assurance Policy
The Director of FFEO is committed to ensuring that all data generated by or on behalf of
FFEO are suitable for their intended use. The systems and practices presented in this document
provide a framework for assuring the quality of all environmentally related data. Without data of
known and documented quality, FFEO cannot fulfill its mission to ensure that Federal facilities
take all the necessary actions to prevent, control, and abate environmental pollution.
The FFEO QA Program is part of EPA's overall QA structure (see Exhibit 1). In general.
FFEO QA policy states that:
FFEO programs generating, using, or requiring the collection of
environmental data will follow the policies outlined in the FFEO QMP,
EPA Order 5360.1, and other appropriate Federal or specific EPA
Headquarters or Regional QA requirements. This includes other Federal
agencies, States, contractors, and other organizations collecting data for
FFEO.
FFEO management and staff will establish the intended use(s) for
environmentally related data and level of data quality necessary to support
decisions prior to initiation of data collection efforts.
Environmentally related data generated by FFEO will be of known and
documented quality, as defined by pre-established data quality objectives
(DQO). The process of defining DQOs will be consistent with that
outlined in the guidance document. Guidance for the Data Quality
Objectives Process (Final), EPA QA/G-4, September 1994.
FFEO PMs/WAMs will evaluate environmentally related data used but not
generated by FFEO staff to determine whether QA/QC procedures used by
- the organization generating the data support the DQOs for the specific
EPA project. This usually entails evaluating whether adequate
*" documentation exists.
FFEO PMs/WAMs will develop acceptable and cost-effective Quality
Assurance Project Plans (QAPPs) and submit them to the QAM for
approval prior to any new environmental data collection activity. All
approved QAPPs will contain readily identifiable DQOs.
Quality Management Plan 2-1 Federal Facilities Enforcement Office
-------
Exhibit 1
FFEO QA Program and Organizational Structure
Office of Administration
and Resource Management
Assistant Administrator
Office of Enforcement and
Compliance Assurance
Other Federal Agencies
Office of Information
Resource Management
SIRMO
Director. FFEO
Director. PPCS
Director. SRES
Quality Assurance
Manager
PMs/WAMs and
Project Staff
PMs/WAMs and
Project Stalf
Regional FFCs I Regional QAMs
FFEO QA staff will conduct regular assessments of FFEO programs and
projects involving environmental data collection to ensure they comply
.. with.jQA requirements. In addition, QAD will conduct periodic
* management systems reviews (MSRs). FFEO will address deficiencies
highlighted in these assessments in a timely manner.
FFEO will provide adequate'resources to support QA program efforts and
to accomplish QA objectives for all environmentally related data collection
programs, projects, and tasks. This includes both intramural and
extramural activities.
"H!" ¦
Federal Facilities Enforcement Office 2 2 Quality Management Plan
-------
w 9J or ur
esna
vi
FFEO will provide QA training through QAD and external sources to staff
at all levels to ensure that QA requirements and responsibilities are
understood and implemented at ail stages of projects.
The requirements of this QMP may be waived under exceptional circumstances. Each
waiver will be considered on its own merits upon the receipt of a formal waiver request. The
waiver must be approved by the QAM and no activity will proceed unless the waiver has been
approved. The waiver request shall be sent to the QAM and must address the requirement to be
waived, the rationale and the consequences of non-approval. When a waiver is granted, the
project WA\1 must assure that the actual procedures for the data management activities are
documented and available for review and assessment during the decision-making associated with
the project.
2.2 FFEO Mission and Organization
2.2.1 FFEO Mission
Federal facilities are required to comply with numerous environmental requirements
administered by EPA. On behaJf of the Office of Enforcement and Compliance Assurance
(OECA), FFEO coordinates Federal facility enforcement, compliance assurance, and compliance
assistance efforts and has the lead role in communicating with Congress, other Federal agencies,
States, and other stakeholders on environmental matters dealing with Federal facilities. FFEO
manages the resolution of enforcement disputes with other agencies; tracks Federal facility
compliance; coordinates and participates in the negotiation and issuance of compliance orders,
compliance agreements, and interagency agreements; coordinates and oversees Regional
enforcement efforts; coordinates OECA's involvement in various interagency task forces; and
develops national Federal facility enforcement and compliance policy and guidance.
2.2.2 FFEO Organization Structure
As shown earlier in Exhibit 1, FFEO is located within the immediate Office of the
Assistant Administrator (AA) for OECA. The Director of FFEO oversees programs conducted
by the Site Remediation and Enforcement Staff (SRES) and the Planning, Prevention, and
Compliance Staff (PPCS). Both SRES and PPCS are under the supervision of a Staff Director.
The FFEO QAM, who coordinates QA policy and activities across FFEO. is located within PPCS.
In addition, the OE£A Senior Information Resources Management Official (SIRMO) and the
Records Offieer (see discussion below), assist the QAM in carrying out his/her duties.
2.3 FFEO Quality Assurance Organization and Responsibilities of Quality
Assurance Staff
2.3.1 FFEO Quality Assurance Organization
All FFEO personnel involved in environmentally related data collection are committed to
the QA principles established within the QMP. FFEO senior management is responsible for
ensuring that FFEO staff adhere to the QMP
Quality Management Plan 2 - 3 Federal Facilities Enforcement Offic<
-------
It is important to note that FFEO is a secondary user of environmental data generated by
other Federal agencies, EPA Regions, States, and other entities. Therefore, most of the
environmental data used to support FFEO activities will be will be governed by the QMPs or
equivalent policies of these entities. It is the responsibility of FFEO senior management to ensure
that these policies are consistent with the principles outlined in the FFEO QMP.
Genera] oversight of the FFEO QA program is provided by the FFEO QAM. The QAM
reports to the Director of FFEO on all QA matters, and assists Staff Directors and PMs in
interpreting EPA QA policy and in developing the QA policy and procedures for FFEO. The
QAM is also responsible for communicating QA policy throughout the organization through
meetings and other forms of communication.
2.3.2 FFEO QA Manager
The FFEO QAM is responsible for the following QA activities:
Develops and revises the FFEO QMP, develops QA policies and
procedures, and develops the Quality Assurance Annual Report and
Workplan (QAARW);
Oversees the annual review process for all project-specific FFEO QA
programs;
Approves QAPPs developed by FFEO PMsAVAMs;
Serves as the liaison between FFEO and the QAD - participates in QA
conference calls and the Annual National QA Meeting and coordinates with
QAD concerning completion, review, and approval of the FFEO QMP;
Participates in agency-wide and interagency QA functions (e.g.. Agency
Environmental Monitoring Management Council (EMMC); and
« Represents FFEO QA interests on major interagency task forces.
FFEO anticipates that the annual level of staffing resources necessary to fulfill the responsibilities
of the QAM equates to approximately one-quarter of a full-time equivalent (FTE).
2.3.3 Senior Information Resources Management Official
The SIRMO is responsible for improving management of information resources within
OECA. The SIRMO works with the Director of FFEO and serves as his/her principal senior
advisor on Information Resources Management (IRM) matters, including all aspects of QA for
data software and hardware. The SIRMO coordinates closely with the FFEO QAM and performs
the following duties:
Federal Facilities Enforcement Office 2 - 4 Quality Management Pla^
-------
Raises organizational awareness of the importance of IRM QA as an
integral component of meeting OECA/FFEO mission needs;
Serves as the organization's key representative to the Office of Information
Resources Management (OIRM);
Coordinates with, and actively participates in IRM councils, committees,
and task forces, including QA workgroups pertaining to QA for data and
information systems;
Develops and implements IRM policies and guidance to ensure that
information management resources arc used in support of OECA/FFEO
missions;
Ensures conformance with the Agency's DRM acquisitions and services for
conformity with established IRM plans, compliance with policy directives,
and completeness of specifications for the work requested;
Coordinates and monitors key IRM programs across the organization,
including records management, information security, and information
dissemination;
Reviews IRM data requirements included in OECA/FFEO methods,
standards, and guidance to ensure consistency with IRM requirements and
conformity across programs;
Oversees and reviews OECA/FFEO IRM activities and assists the FFEO
QAM with IRM aspects of the QMP and QAARW; and
Provides management and oversight to the contracts, grants,
cooperative/interagency agreements in support of OECA/FFEO IRM
activities.
2.3.4 Records Officer
The Records Officer implements the Agency and Federal Records Management
regulations and policies in FFEO. It is the Records Officer's responsibility to develop records
management activities across the office. In addition, the Records Officer serves as a liaison to
OIRM and related Agency workgroups to ensure that FFEO's records management plan and
procedures are compliant with all Agency-approved records management policies and guidance
developed by OIRM.
Quality Management Plan 2-5 Federal Facilities Enforcement Offtci
I
-------
2.4 Responsibilities of Office Director, Staff Directors, and Project Managers/Work
Assignment Managers
2.4.1 Office Director
The Office Director has ultimate responsibility for the FFEO QA Program. The Office
Director is responsible for ensuring that QA is an identifiable activity within the office and for
providing adequate resources to support QA program efforts. In addition, the Office Director is
responsible for ensuring that standards pertaining to the implementation of the office's QA
program appear in the performance agreement of the QAM and that the percentage of time
allotted for this activity is commensurate with the responsibilities. The Office Director also has
responsibility for conducting periodic management assessments of his/her QA program.
2.4.2 Staff Directors
The PPCS and SRES Directors have overall responsibility for the QA programs within
their respective staff areas. Staff Directors must ensure that QA is an identifiable activity within
planning and work plan documents, and provide adequate resources to accomplish the Agency's
QA goals. Staff Directors are responsible for conducting periodic management assessments of
their QA programs, and for ensuring that all outputs generated meet the quality requirements of
the customer, the office, and the Agency in a cost-effective manner. More specifically, Staff
Directors arc responsible for ensuring that:
DQOs arc developed for all environmentally related data-collection
programs, projects, or tasks that are supported by FFEO;
All environmental projects and tasks are covered by an approved QAPP
before environmental data are collected, and that the plan is implemented;
Audits are conducted at a sufficient frequency to determine compliance
with QA requirements;
Deficiencies that are highlighted during audits are corrected in a timely
fashion; and
. Program-specific QA training needs arc identified and addressed.
2.4.3 Project Managers/Work Assignment Managers
PM/WAMs are responsible for ensuring that the QA requirements in 40 CFR 30, 31, 48
CFR Chapter 15, and this QMP are met. It is recognized that the PM/WAM may have a limited
breadth of experience in QA Therefore, it is critical that they work closely with the QAM to
assure QA issues are appropriately addressed. PM/WAMs have primary responsibility for
coordinating the following QA and QC activities for their assigned projects:
Federal Facilities Enforcement Office
2 ¦ 6
Quality Management Plat
-------
1
m
Completing the Quality Assurance Review Form (EPA Form 1900) for any
extramural projects involving environmental measurements to indicate the
QA requirements to include in the development of a Request for Proposal
(RFP);
Ensuring that work assignments, work plans, and contract deliverables
include any QA documentation deemed necessary by the QAM or specified
in agency-wide procurement/contractual guidelines;
Developing DQOs and the related data acceptance criteria, with the
assistance of senior program staff and technical personnel, for data
produced for the project;
Preparing and implementing a QAPP appropriate for the project;
Coordinating with the QAM on the selection and design of audits
appropriate for the project;
Identifying, resolving, and implementing project-specific QA and QC
issues;
Identifying QA resource needs, including QA training needs, for a project,
and making a request for funds; and
Preparing project-related sections of all QA reports to management.
2.5 Responsibilities Outside of the EPA Headquarters Structure
2.5.1 EPA Regional Office
FFEO staff works closely with the EPA Regional staffs, especially the Regional Federal
Facility Coordinators (FFCs). In addition. Regional Quality Assurance Officers (RQAO) develop
and oversee implementation of Regional QMPs. These plans set Regional priorities and policies
regarding QA practices. For programs that are directly implemented by the Region, the Regional
QMP takes precedence over the FFEO QMP in areas where Regional policies and/or procedures
are more comprehensive and stringent.
It is FFEO policy to include the RQAO or their designated representative in the
development of all documents and policies related to Regional implementation of Federal facility
QA programs. It is the responsibility of the appropriate PM/WAM to work with the QAM to
ensure that adequate coordination is occurring with the Regions. It is particularly important for
the QAM to advise the RQAO of QA requirements being imposed on States and other
stakeholders, especially when the Region is responsible for oversight
Specific Regional QA protocols. QA SOPs, and QA responsibilities are defined in each
Regional QMP. The QAM will integrate mutual QA responsibilities and ensure consistency
Quality Management Plan
2-7
Federal Facilities Enforcement Offtci
-------
between Headquarters and Regional QA guidance. Joint review of documents will occuc as
appropriate.
FFEO is ultimately responsible for the quality of the data generated by the EPA Regional
offices for FFEO-supported programs if that data resides in a national information management
system. To ensure that all data are of acceptable and documented quality, the FFEO QAM. at the
request of FFEO senior management, periodically reviews implementation of Regional QA
activities or participates in QAD MSRs. (Further discussion of MSRs can be found in Section 10)
The QAM is also responsible for incorporating the discussion of QA issues in any Headquarters
review of Regional office programs.
2.6 Responsibilities for Assuring that Quality Assurance System is Communicated,
Understood, and Implemented
This QMP is the principle means of communication and information for the FFEO quality
management system. Lines of communication and reporting for QA program status will be
maintained to ensure that effective QA programs are implemented within FFEO and the Regions.
To help ensure effective communication, regular meetings will be held by the QAM with the Staff
Directors and PM/WAMs to discuss current QA issues and procedures. In addition, the QAM
will periodically conduct conference calls with the FFCs, RQAOs, and other appropriate Regional
staff on issues of joint interest.
Federal Facilities Enforcement Office 2 - 8 Quality Management Plan
-------
3. Quality System and Description
3,1 Definition of Environmentally Related Data
For purposes of the FFEO QMP, environmentally related data enables FFEO staff to:
Identify existing and potential environmental compliance problems (e.g.,
those generated by new requirements);
Develop and implement environmental management;
Evaluate environmental program effectiveness and compliance; and
Determine the costs and benefits associated with recommended or
implemented environmental policies.
This definition encompasses compliance, financial, locationa], and administrative data.
Environmentally related data may be quantitative or qualitative.
3.2 Types of Data Collection Activities and FFEO Programs Covered by the
Quality Management Plan
The QMP encompasses data directly generated by FFEO programs or their contractors, as
well as data obtained for FFEO programs from other sources. The QMP also covers
environmental data that FFEO programs require Federal facilities and other stakeholders to
collect. As stated earlier, FFEO is a secondary user of environmental data generated by other
Federal agencies. EPA Regions, States, and other entities. This data will be governed by the
QMPs or equivalent policies of these entities. It is the responsibility of FFEO senior management
to ensure that these policies are consistent with the principles outlined in the FFEO QMP.
In addition, the QMP covers FFEO QA responsibilities in the Information Collection
Request (ICR) process as outlined under ths-authonties of the Paperwork Reduction Act of 1980
(Public Law-96-5l 1); OMB Regulation 5 CFR 1320, Controlling Paperwork and Burden on the
Public; and OMB Circular A-130, Management of Federal Information Resources. The process
for developing ICRs as described in the Information Resources Management Policy Manual
(Number 2100, July 1987) includes a cost effective evaluation focusing on the least costly
alternative of information collection that will satisfy the decision-making needs. The process is
analogous to the DQO process and the assistance of the QAM in the process will materially
expedite effective development.
3.3 FFEO Programs Covered by the Quality Management Plan
The FFEO QMP is applicable to all FFEO programs that implement environmentally
related data-gathering activities. This includes collection or use of data generated to develop,
Quality Management Plan
3-1
Federal Facilities Enforcement Offta
-------
coordinate, track, and oversee Regional Federal facility enforcement and compliance
assurance/assistance efforts. Programs covered in subsequent revisions of this QMP may vary as
a result of statutory and administrative changes. It is FFEO policy to review the QMP annually
and to update the programs covered as necessary. j
3.4 Functional Areas within FFEO Programs Covered by the Quality Management Plan j
It is FFEO policy that QA issues be addressed during all appropriate planning, design. j
implementation, and assessment phases of a program or project. This QMP identifies the j
following five functional areas within FFEO to which QA is applicable: |
Development of policies, standards, criteria, guidance, or enforcement
regulations affecting Federal facilities;
Information management;
Compliance and enforcement;
Procurement of supplies and services; and
Oversight.
Exhibit 2 summarizes the FFEO programs covered by this QMP and their corresponding
functional areas involving QA activities. The remainder of this section defines these functional
areas and provides general guidance on appropriate QA activities applicable to each area.
Exhibit 2
FFEO Programs and their Functions Involving QA Activities
Functional fytm
FFEO Programs
Regional^
Prognun
Support
LAC Negotiation
Support
Enforconcttt
Support
Technical
Assistance
Program
Management
Development of Policies, etc.
Affecting Federal Facilities
~
~
~
~
~
Information Management
~
~
~
Compliance A Enforcement
~
~
~
~
~
Procurement of Supplies A
Services
~
Oversight
~
~
~
~
Federal Facilities Enforcement Office 3-2 Quality Management Plat
-------
osa
3.4.1 Development of Policies, Standards, Criteria, Guidance or Regulations Affecting
Federal Facilities
Policies, standards, criteria, guidance, and regulations affecting Federal facilities may be
developed based on environmental data or they may require the collection and reporting of
environmental data by these facilities. When FFEO or a designated contractor collects data
necessary to develop policies, standards, criteria, guidance, or regulations, the QA activities must:
Ensure consistency with the approved QMP (see EPA Requirements for
Quality Management Plans (Interim Final), EPA QA/R-2, August 1994);
Identify project or program objectives and associated DQOs for relevant
project activities (see EPA Requirements for Data Quality Objectives
¦ Process. EPA QA/G-4, September 1994);
Develop and implement approved QAPPs for data collection activities that
specify data collection methods. QC requirements, data documentation,
and reporting format and requirements (see EPA Requirements for Quality-
Assurance Project Plans (Interim Final), EPA QA/R-5. August 1994);
Develop guidance to perform the data collection activities detailed in
QAPPs; and
Develop a procedure to assess data quality based on the QAPPs to evaluate
whether the data are of sufficient quality to meet the DQOs of the program
or project.
When FFEO organizations use data collected by others (e.g., other Federal agencies), the
QA activities must:
Document all sources of data;
Develop criteria and protocols for evaluating the quality and use of data
collected by others;
.. Evaluate the protocols for data collection and analysis and QA/QC
procedures used by the organization that generated the data to determine
whether the data are of sufficient quality to meet the DQOs of a program
or project; and
Validate the data generated by other organizations to ensure that they meet
the established QC specifications.
All regulations, standards, criteria, and guidance developed with environmentally related
data will undergo internal and/or external peer review in accordance with Agency procedure, as
described in the Administrator's Peer Review memorandum of June 7, 1994.
«
Quality Management Plan
3-3
Federal Facilities Enforcement Office
-------
3.4.2 Information Management
Information management encompasses the administration of data and information systems.
EPA's OIRM is presently developing standards applicable to information management activities
that will be published and implemented following Green Border Review. FFEO is committed to
implementing the information management/data administration standards when they arc published.
The systems management component of the OIRM standards encompasses all aspects of
system design, maintenance, documentation, and data entry, for both non-automated and
automated systems. For purposes of this QMP, the scope of systems management activities
includes paper file systems, single-user, project-specific databases, or multi-user national
databases (such as FEDPLAN-PC or FFTS). Quality assurance activities for information
management must:
Ensure consistency with.The approved QMP (see EPA Requirements for
Quality Management Plans (Interim Final), EPA QA/R-2, August 1994;
Identify project or program objectives and associated DQOs for
information management programs establishing criteria for the timeliness,
completeness, precision, and consistency of data entered into the
information management system (see EPA Requirements for Data Quality
Objectives Process, EPA QA/G-4, September 1994);
Develop and implement approved QAPPs or study plans for information
management. QAPPs must specify compliance with Federal. EPA. and
FFEO data standards, standards for description of data sets, and the
minimum number of data sets that must be presented before the data can be
included in an official information management system and data dictionaries
(see EPA Requirements for Quality Assurance Project Plans (Interim
Final), EPA QA/R-5, August 1994);
Adhere to EPA Order 2100 on Information Resources Management (IRM)
and its subsequent implementation order. System development must
comply with EPA's Operations and Maintenance Manual, EPA Directive
" 2181, April 30, 1993;
Develop a system security plan consistent with the requirements in the
OIRM, June 3, 1994 draft of the EPA Information Security Manual;
Develop users guides and training that provides detailed instructions for
entering individual data sets, forms, and data submissions;
Develop guidance on QC of data entry, storage, access, transfer and
retrieval processes This includes edit and update error correction
processes, internal reviews, and database QCs;
Federal Facilities Enforcement Office'
3-4
Quality Management Plat
-------
Validate software;
Document system design specifications to ensure the quality of the system
development and enhancement efforts (see Office of Administration and
Resources Management [OARMJ, System Design and Development
Guidance. EPA Directive 2182, dated April 30. 1993);
Document systems and conformance to document scope, purpose, and
requirements throughout the Itfe-cycle; and
Develop data validation, data quality assessment, and audit protocols based
on the QAPPs to evaluate whether the data are of sufficient quality to meet
the DQOs of the program or project.
3.4.3 Enforcement and Compliance"
FFEO enforcement programs employ environmental data to demonstrate the compliance
status of specific Federal facilities and to document compliance trends at both Regional and
national levels. Environmental data may be used as evidence in support of judicial enforcement
actions and in setting Regional and national enforcement priorities (e.g., multi-media initiatives).
QA components for compliance and enforcement programs must:
Ensure consistency with the approved QMP (see EPA Requirements for
Quality Management Plans (Interim Final), EPA QA/R-2, August 1994);
Identify project or program objectives and associated DQOs, where appropriate,
for enforcement and compliance programs to support case development and
enforcement actions (see EPA Requirements for Data Quality Objectives Process.
EPA QA/G-4, September 1994);
Develop and implement approved QAPPs or study plans for enforcement
and compliance data collection activities. QAPPs must specify data
collection methods, QC requirements, data documentation, and reporting
format and requirements (setTPA Requirements for Quality Assurance
' Project Plans (Interim Final). EPA QA/R-5, August 1994);
* Develop guidance for the regulated community addressing program DQOs.
collection and reporting requirements;
Inspect and audit facilities i©-determine compliance with monitonng and
record-keeping requirements;
Audit to ensure that necessary evidence for State enforcement actions are
on file;
«
Quality Management Plan
3-5
Federal Facilities Enforcement Offtct
-------
Develop guidance, procedures, or protocols for inspections and data
assessments based on the QAPPs to evaluate whether the data are of
sufficient quality to meet the DQOs of the program or project.
3.4.4 Procurement of Supplies and Services
The procurement of supplies in FFEO primarily involves the purchase of computer
hardware and software through OECA support staff activities. These procurements must meet
the established administrative and QA requirements of the Purchasing Office and of OIRM. All
FFEO procurement for equipment and supplies must follow the Federal Acquisition Regulations
(FAR) and EPA Acquisition Regulations (EPAAR). In addition, purchased supplies must be
evaluated upon receipt to ensure that all purchasing specifications have been met.
Procurement of services involves QA procedures tn the planning, acquisition, evaluation,
and operation phases.
* In the planning phase, QA activities involve the use of a standard checklist.
Quality Assurance Review Form (EPA Form 1900). to identify QA
requirements for projects that involve environmental data collection, and
the inclusion of QA procedures and performance requirements in the RFP
or grant.
In the acquisition phase, QA activities involve assurance of conformance
with Agency QA requirements in the contract proposal or grant which may
include a QMP and/or a QAPP.
In the evaluation process, QA activities involve the numerical evaluation
and subsequent ranking of performance for the contract proposal or grant
which may include a QMP and/or a QAPP.
In the operation phase, QA activities require QAPPs for environmental
data collection and appropriate QA audits of the collection process or the
data itself. Time lines are needed to ensure plans are received well in
advance of data collection. Acquisition of administrative data processing
services will comply with EPA acquisition requirements for IRM in
. EPAAR 1552.210-279 and EPA's Operation and Maintenance Manual for
Modifying and Documenting Information Systems.
3.4.5 Oversight
While oversight is generally a pan of all functional areas, this QMP also classifies
oversight as a separate functional area to ensure that FFEO programs having only oversight
responsibilities adequately consider QA requirements. The implementation of such programs is
frequently the responsibility of the Regions; however, FFEO may have final responsibilities for the
use of the data collected. It is critical therefore, that FFEO' s strategies, plans, and guidance
address QA issues because subsets of the data are usually stored in national information
Federal Facilities Enforcement Office
3 6
Quality Management Plat
-------
management systems used to support national programs. Thus, at a minimum, guidance which
involves environmental measurements needs must:
Ensure consistency with the approved QMP (sec EPA Requirements for
Quality Management Plans (Interim Final), EPA QA/R-2. August 1994);
Develop and implement approved QAPPs or study plans for data collection
oversight activities (see EPA Requirements for Quality Assurance Project
Plans (Interim Final), EPA QA/R-5, August 1994);
Require audits and Regional reviews to verify conformance with guidance,
Cite QA guidance and other sources to ensure uniform procedures and
utility of results; and
Develop a protocol for conducting data assessments based on the QAPPs
to evaluate whether the data are of sufficient quality to meet the DQOs of
the program or project.
3.5 Quality Assurance Practices and Tools
3.5.1 Data Quality Objectives
DQOs are qualitative and quantitative statements developed to ensure that data of known
and appropriate quality are obtained to support specific decisions or actions. In FFEO, DQOs are
developed at both programmatic and project levels. The DQO process involves establishing the
allowable uncertainty of a data set that can lead to two kinds of error: false positive (assuming
that a problem exists when it does not) and false negatives (assuming that a problem does not
exist when it does). The acceptance probabilities of those errors, as established by decision
makers, are the DQOs. The quality of a particular data set is a measure of the types and amount
of error associated with the data. Data quality is described by qualitative and quantitative
parameters, including precision, accuracy, representativeness, completeness, comparability, and
sensitivity. The steps in the DQO process, as defined by QAD, are shown in Exhibit 3.
- « «
Quality Management Plan 3 - 7 Federal Facilities Enforcement Offict
-------
CTCfl
Exhibit 3
DQO Development Process
Step I
State the problem
Step 2
Identify the decision
Step 3
Identify decision
inputs
Step 4
Define study
boundaries
Step 5
Develop decision
rule
"Step 6
Specify acceptable
decision errortimits
Step 7
Optimize design
Identify members ot planning team
Identify primary decision-maker
Describe problem
Specify deadlines and available resources
State the decision
Categorize multiple decisions
State acuons or outcomes that could result
Identify information required to make a decision
Determine information sources
Identify information needed to establish action
level for the study
Define spacial boundaries of the decision
Define temporal boundaries of the decision
Identify practical constraints on data collection
Develop an "if...Then" statement linking data
results with possible actions
Determine possible range of parameter of interest
Define false negative and false positive ercors and their
consequences
Specify regions of error where consequences are minor
Assign probability values to region or error
Develop environmental data collection plans
Evaluate each plan's efficiency at meeting DQOs
Identify the most resource-effective plan
3
Federal Facilities Enforcement Office
3-S
Quality Management Pla,
-------
vr'AlllLJiJlilMCTgMBHWWWCW I WHHUIBtWgBWHawBgaWB
All FFEO environmental data collection or generation projects require the development of
DQOs. Use of the DQO development process will result in a statement of quality standards or
goals for environmental data to be used for decision-making Communication between the data
providers and the data users is critical to the success of the DQO development process.
Proposed DQOs arc reviewed for approval by the decision-maker, or his/her designee(s),
and for conformance with EPA requirements by the QAM. The decision-maker is the individual
having final approval authority for the study or program for which the data will be collected.
The structure for developing DQOs is the same as the line organization in FFEO. For data
collection activities to be conducted extramurally, the PM/WAM is responsible for ensuring that
DQOs are developed where appropriate. For implementation activities at the Regional level,
DQOs must be consistent and in conformance with the overall programmatic DQOs. Guidance
and technical-support in developing DQOs is provided by the QAM. Existing development,
review, and approval mechanisms for FFEO programs and projects are utilized to incorporate the
DQO process into standard procedures.
3.5.2 Quality Assurance Project Plans
EPA QA policy (Order 5360.1) stipulates that all projects and tasks involving
environmental measurements be covered by an approved QAPP A QAPP is a technical planning
document that defines the objectives of a project or continuing operation, as well as the methods,
organization, and QA and QC activities necessary to meet the goals of that project or operation.
QAPPs are required for all FFEO environmental data collection projects or continuing
data collection operations conducted by FFEO staff, Regional personnel, or other organizations.
A continuing data collection operation is one whose procedures are not modified significantly
from year-to-year.
The QAPP serves as the blueprint for implementing the data collection activity, to ensure
that the technical and quality goals of the operation arc met. It also provides the necessary link
between the required data quality constraints and the analysis to be conducted.
For internal FFEO projects, the QAPP is reviewed and approved by the QAM. QAPPs
shall be distributed to the personnel performing the assigned work, and shall be implemented as
written, unless modified to improve the QA design of projects as described above. An overview
of the process for preparing and implementing QAPPs is shown in Exhibit 4.
Only under exceptional circumstances will the requirements for preparation of DQOs and
QAPPs be waived. The person responsible for the project must prepare a formal waiver request
in accordance with the waiver process that will be developed by FFEO by FY 97. Waivers will be
based on consideration of the product size or duration, the nature of the work to be performed,
the time frame within which the data are needed, and cost. Waiver rationale must be reviewed
and approved by the QAM or his/her designee. When a waiver is granted, the project WAM must
assure that the actual procedures for the data management activities are documented and available
for review and assessment during decision-making associated with the project.
Quality Management Plan 3-9 Federal FacUities Enforcement Office
-------
Exhibit 4
Preparing and Implementing QAPPs
FFEO Quality Management Plan
Establishes policyon applicability of QAPPs
Establishes program-level DQOs' policies for preparation of QAPPs
Establishes policies, procedures, and schedules for oversight or QAPPs
Is QaPP needed?
Yes
Is waiver needed''
Yes PM/WAP applies in
writing to QAM
No
No
No QAPP prepared
Project Staff Prepares Draft QAPP
Develop project-specific DQOs
Design in accordance with QMP and comply with Draft-Fmal EPA
Requirements for QAPPs (EPA QA/R-5) or Combined Work/QA Project
Plans for Environmental Monitoring (OWRS QA-I)
No (revise)
Yes
QAM approves draft
Senior management
approvesi
Ituiugvtliwilk .
*es draft
QAM provides
implementation
guidance on
approved QAPP
Project staff
implements
approved QAPP
Integrate results into
project report
Periodic QAPP
implementation
review
Federal Facilities Enforcement Office
3-10
Quality Management Plan
I
-------
i jKuavm
Review and control mechanisms are established for each project in the QAPP. These will
vary in complexity and scope depending on the particular project. Large scale, national projects
will have QA task groups formed to provide the lead in preparing DQOs (described above) and
QAPPs. These same groups will review the QA data on an ongoing basis, conduct audits, and
recommend remedial action.
For continuing data collection operations, the QAPP is reviewed annually and revised
whenever significant changes are made in procedures or organizational responsibilities. A quality
assurance project plan must be approved by the QAM and senior management prior to the
initiation of data collection activities.
3.5.3 Standard Operating Procedures
SOPs,- as defined in the FFEO QMP, refer to documents that describe the detailed
procedures to be followed for routine or-repetitive efforts to ensure consistent results when
collecting and analyzing environmental data. SOPs should be clearly written and contain enough
detail to all FFEO users with appropriate experience and training to follow the procedure. SOPs
should be consistent with current EPA guidelines and regulations. SOPs may apply to following
activities;
Development of policies, standards, criteria, guidance, or enforcement
regulations affecting Federal facilities;
Information management;
Compliance and enforcement;
Procurement of supplies and services; and
Oversight.
Well-documented processes and procedures are critical to the defense of actions taken in
these areas and will help facilitate reviews under the Administrator's Peer Review Process. It is
recognized that it will not be possible to have one standard format for each major category. The
goal is to develop a standard procedure for documenting the process by which the activity is
developed and implemented. When developing SOPs for all FFEO functional areas, FFEO will
consult EPA Guidance for the Preparation of Standard Operating Procedures for Quality-
Related Documents, EPA QA/G-6, November 1995.
SOPs are documents that describe procedures to be followed to ensure the integrity and
quality of data. These documents serve a threefold purpose:
SOPs reduce the introduction of errors and variables, by ensuring that
appropriate procedures are used consistently;
SOPs provide a historical record of the work performed; and
Quality Management Plan 3 - It Federal Facilities Enforcement Offici
-------
I"
I « SOPs are useful training tools that serve to clearly communicate the
specific method of performing a procedure and to reduce
misunderstandings.
It is FFEO policy to utilize established SOPs whenever possible. On occasion, FFEO may
develop specific SOPs for a particular project. The need for new SOPs may be identified by the
PM/WAM prior to project initiation, or by the QAM either through audits or through the QAPP
review process. The person implementing the procedure is generally responsible for developing
the SOP with assistance from peers and supervisory personnel.
Established SOPs will be referenced in the QAPP, with any modifications or nonstandard
applications clearly defined. SOPs should be readily available to all personnel who are required to
use them and must be adhered to rigorously. Deviations from SOPs should be justified,
documented, and submitted for approval to the appropriate program manager.
Exhibit 5 illustrates the development and use of SOPs. The review and approval process
for SOPs generated within FFEO follows the existing line-management approval structure.
Additionally, all FFEO SOPs which contain QA/QC procedures must be approved by the QAM.
SOPs should be reviewed annually to ensure that they reflect current procedures. Revised SOPs
are subject to the same review/approval process as the original SOPs.
Federal Facilities Enforcement Office
3-12
Quality Management Plan
-------
smm
Exhibit 5
Preparing and Implementing SOPs
Is SOP needed?
* Yes
Answer "Yes" to one or more
Is the procedure routine'.' Would
documentation improve the procedure0
Does the QMP or applicable QAPP
require an SOP be prepared?
Acquire existing
SOP
Tailor SOP to
project requirements
b
- Is existing SOP
Yes available0
intisur a
No
Project staff
develops draft SOP
Management &
QAM review and
approve SOP
Project staff
develops final SOP
Implement SOP
Review/evaluate
SOP effectiveness
Project staff
revises SOP
Yes No
Changes needed^
F
s -
Potential Sources to Consider
Other FFEO projects/programs
Other EPA projects/programs
Regions
ORD/QAD
Observe and document process
Identify variables and potential sources of
error
Define minimum required steps/actjvities
Identify necessary documentation
Distribute to affected staff and managers
Train staff and managers per QAPP
Measure/monitor performance
Conduct MSRs as appropriate
Implement continuous improvement
Continue
implementation
Quality Management Plan
3-13
Federal Facilities Enforcement Offict
-------
4. Personnel Qualifications and Training
The application of sound QA policies and procedures requires that all staff who perform
QA tasks possess an appropriate level of knowledge of QA procedures and principles. It is FFEO
policy to provide the QA and QC training necessary to ensure that all personnel understand
FFEO's QA/QC program.
First-line supervisors are responsible for ensuring that each employee with QA-related
assignments has the necessary qualifications and proficiency for the work assigned. All FFEO
personnel involved in QA/QC will, at a minimum, attend the EPA Institute Training Courses
Orientation To Quality Assurance Management, and Data Quality Objectives. It is the
responsibility^ first-line management to discuss QA training needs with personnel during the
mid-year and annual performance evaluation process.
A quality assurance training requirement should appear within the standards of the QAM
and the Individual Development Plans of other staff as appropriate. For example, PMs and
WAMs are responsible for ensuring that all contract personnel have the necessary QA training for
their tasks and functions. Training priorities should be assigned and training should be scheduled
with management approval.
First-line management is ultimately responsible for quality. Therefore, it is critical that
these managers receive the necessary training to ensure they understand the importance of QA.
their responsibilities as line managers, and specific FFEO QA policies and procedures. Training is
accomplished by attending seminars developed by QAD for first-line managers or through
equivalent in-house training. Training categories associated with QA in environmental programs
include, but are not limited to:
Fundamental principles of quality;
Organizational policies and procedures;
Technical skills (e.g., statistics); and
Management.
PM/WAMs are responsible for ascertaining that all personnel involved with gathering
environmental data have the necessary QA training for their tasks and functions. QA
requirements should be delineated in both the RFP as well as each SOW. The adequacy of
background and training for each work assignment can best be evaluated when a detailed QAPP is
submitted.
Organizational training needs will be addressed annually in the QAARW, The QAM is
responsible for identifying training needs, disseminating information regarding available training
opportunities for FFEO staff and management, and arranging office-wide QA training.
i
.. - i --1 - 1 ¦jjw. jjriT
-------
5. Procurement
It is FFEO policy to state the designated QA and QC requirements when purchasing items
and/or services. Within FFEO, procurement functions are conducted in accordance with the FAR
and generally accepted business practices for the acquisition process. j
5.1 Procurement of Items
Procurement of items in FFEO must meet established administrative and QA requirements,
including those set out in the following:
FAR;
I
Federal Information Resources Management Regulations (FIRMR); j
Delegation 1-84 (1200 TN310);
OIRM Delegation 1-10A (September 27, 1991);
OARM's EPA Acquisition Regulations,
Chapters Four and Six of OIRM's Information Resources Management Policy
Manual (July 1987). ,
The purchase of hardware must also conform with Executive Order 12845, titled Requiring the
Agency to Purchase Energy Efficient Computer Equipment (April 1, 1993).
5.2 Contracts
Procurement of services through contracting constitutes the largest procurement activity
of FFEO. All procurement of services involving Federal Information Processing Standards (FIPS)
resources must meet the requirements the applicable administrative and QA requirements set out
in the documents listed in Section 5.1. In addition, all procedures used in procurement of services '
will comply with EPA's Contracts Management Manual and the EPA Acquisition Regulations. '
The procurement process for FFEO is detailed in The Cookbook -- How to Get Contracts \
Awarded in EPA (EPA 202-B-93-002, November 1993). Procurement of services through
contracting involves specific QA checks and the development or use of QMPs and/or QAPPs.
Exhibit 6 shows the QA activities involved in the procurement process.
i
As part of the RFP process, the QAM will determine the QA requirements and specify j
these on the Quality Assurance Review Form (EPA Form 1900) for extramural projects. This j
form is reviewed and signed by the QAM.
i *
Quality Management Plan 5 - I Federal Facilities Enforcement Office
-------
Exhibit 6
QA Elements of the Procurement Process
Contract Development Task
Develop SOW
QA Activity
Develop acquisition
I
Ian
Develop RFP
Evaluate RFP
Contract award
Issue work assignments I
QAM reviews SOW
and provides QA tasks
where required
QAM and PM define
QA oversight role in
acquisition process
QAM incorporates QA
into evaluation including
QA in sample assignment.
QMP, and QAPP
QAM serves on
selection board
QAM participates in
debriefing
QAM monitors work
assignments that
require QAPP and
monitors QA activities
Receive work products I
Federal Facilities Enforcement Office
5-2
Quality Management Plat
-------
As noted in EPA 1900 -- Contracts Management Manual, the QAM shall be a member of
the Technical Evaluation Panel (TEP) for procurements over S500.000 in cases where QA
requirements are applicable to the procurement. It is FFEO policy that contractors provide a
QAPP as a contractual deliverable for each work assignment that involves environmental
measurements. The contractor is responsible for preparing a QAPP in accordance with the
guidance document EPA Requirements for Quality Assurance Project Plans (Interim Final) EPA
QA/R-5. Preparation and initial review of the QAPP is the responsibility of the WAM. The
QAPP will be reviewed and approved by the QAM before environmental data arc collected.
Inherently governmental functions shall not be performed by contractors. Inherently
governmental functions are defined in EPA 1900 -- Contracts Management Manual, Order
1900.2: "Contracting at EPA."
5.3 Interagency Agreements
Policies and administrative procedures governing interagency agreements (LAGs) are
defined in Chapter 5 of Managing Your Financial Assistance Agreement. EPA QA requirements
related to environmental data apply to all activities funded by EPA through LAGs. Cooperative
agreements that will produce environmental data must adhere to the QA documentation
requirements in 40 CFR 30.503. These standards must be included explicitly in all cooperative
funding agreements.
All financial assistance IAGs requiring environmental data collection activities on behalf of
FFEO will include a QAPP. Where EPA is providing funds to another organization, that
organization is responsible for preparing the QAPP. The QAPP will be prepared in accordance
with EPA Requirements for Quality Assurance Project Plans (Interim Final), EPA QA/R-5.
August 1994. If the other organization has equivalent requirements for QAPPs, that guidance
may be substituted for EPA QA/R-5. If there are not comparable QA procedures, the QA
procedures agreeable to both parties must be negotiated prior to initiation of the program or
effort. All proposed cooperative funding agreements shall be reviewed to determine the
applicability of QA requirements as defined in EPA Order 5360.1. This determination shall be
documented with an approved QA review form.
Quality Management Plan
S 3
Federal Facilities Enforcement Office
-------
an
6. Records Management
6.1 Records Management Program
FFEO plans to adopt and implement all Agency-approved records management policies
and guidance developed by OIRM. The FFEO Records Management Program adheres to the
following OIRM and Federal guidance policies:
Records Management Manual (2160), U.S. Environmental Protection
Agency, OIRM. 1984;
_ IRM Policy Manual (21 00), Chapter 10, Records Management, U.S.
Environmental Protection Agency, OIRM, November 1987;
Managing Electronic Records (Instructional Guide Scries), NARA, 1990;
Federal Records Management Laws and Regulations, NARA, 1991;
Disposition of Federal Records: A Records Management Handbook,
NARA. 1992;
Using the Federal Records Center: A Guide for Headquarters Staff
(EPA/IMSD/91-004);
Practical Guide to Personal Papers: Records Management Technical
Leaflet #3 (EPA 220-F-92-019); and
Records Disposition Schedules Cross Reference Table (EPA 220-B-95-
005).
In addition. FFEO developed a Records Control Procedures report that summarizes many
of the preceding guidances and includes stefvby-step instructions for categorizing records and
other documents and for archiving records to the Federal Records Center (FRC).
Each FFECStaff (i.e., PPCS, SRES) will be responsible for maintaining its records and
files in accordance with EPA records management policies and guidelines and the Agency file
plan. Roles and responsibilities for maintenance of records will include requirements for record
transmittal, distribution, retention, protection, preservation, traceability, disposition, and
retnevableness. It should be noted that QA/QC documentation is only one type of documentation
covered by the records program.
L ' «
Quality Management Plan
6- I
Federal Facilities Enforcement Offict
-------
6.2 Quality Documentation
While data from a project may be technically sound, lack of proper documentation can
make data suspect and the defense of a project difficult, if not impossible. Documentation must
address all phases of the project involving environmentally related data collection activities. At a
minimum, these projects should include a QAPP and a final report. The final report should
provide information on whether the project objectives were satisfied. In addition, the final report
must address how the QA requirements m the QAPP were met, deficiencies in meeting the QAPP
requirements, and the impact on the project results and conclusions.
Federal Facilities Enforcement Office
6-2
Quality Management Plai t
-------
iisi.mii
7. Information Resources Management
FFEO will comply with all Federal and EPA standards and regulations pertaining to
hardware, software, system development, and data. It is an FFEO goal to achieve consistency in
the way data are generated, compiled, stored, and disseminated across all FFEO programs This
is to ensure more complete and adequate data with which to make management decisions.
7.1 Hardware and Software Standards
The OIRM and the National Data Processing Division (NDPD) are responsible for
managing the hardware, software, and communications components that are the foundation of the
Agency's information technology. As part of their role, NDPD, in conjunction with OIRM, has
established hardware and software standards with which Agency program offices must conform.
FFEO is responsible for coordinating an official review/comment on proposed hardware/software
standards and subsequent compliance and implementation with the official Agency standard.
FFEO managers and staff will comply with ail hardware and software standards delineated
in the National Data Processing Division Guidance on Hardware and Software Standards
(DraftI U.S. EPA Office of information and Resources Management, dated November 8, 1993.
This document is applicable for the personal computer platform, local area network and server
platforms, open systems platforms, Agency Electronic Mail Service (E-MAIL), IBM Compatible
Mainframe Platform, and Supercomputer Platforms. These standards address:
Compatibility;
Hardware;
Operating system;
Communication;
Database management;
.. User interface/printer interface,
a>
Application development; and
Applications.
FFEO will procure Agency-approved hardware and software that conforms with Agency-
wide information management architecture Significant changes in Agency hardware and software
policy will be assessed to determine the impact of the change. If changes are required, FFEO
System Managers will work with the SIRMO to plan and implement the most efficient approach
Quality Management Plan 7 -1 Federal Facilities Enforcement Office
-------
In some cases an FFEO program office will buy or develop application software that is not
on the Agency contract. All such software will be evaluated prior to purchase. Only vendors that
comply with Agency standards in the National Data Processing Division Guidance on Hardware
and Software Standards will be solicited. FFEO PM/WAMs will evaluate the software to
determine us performance capabilities and documentation Software will be selected based on
minimum performance standards and cost.
7.2 Information Management System Development and Operational Standards
All information management system development, enhancement, and modernization efforts
will comply with OARM's System Design and Development Guidance, EPA Directive 2182, April
30. 1993. This compliance should include a dialogue between the data providers, data/system
users, and system developers, prior to the design of the system.
It is FFEO policy to work closely with the OIRM on all phases of system development,
enhancement, or modernization. The appropriate staff from the Program Systems Division in
OIRM, as well as the appropriate FFEO procurement official and the QA Manager will review
and approve procurement requests (PRs) associated with system development, enhancement, and
modernization efforts. During the operational phases of information management systems, FFEO
will comply with requirements within OARM's Operations and Maintenance Manual, EPA
Directive 2181, April 30, 1993. Compliance with the applicable IRM standards will ensure that
all hardware and software configurations are tested prior to use and that they meet user
requirements.
7.3 Data Standards
All Federal agencies are required to adhere to Federally-mandated data standards and
regulations. The Agency's information data standards and regulations appear in the Catalog of
Data Policies and Standards, U.S. EPA Office of Information and Resources Management, July
1991. However, it is the responsibility of the FFEO QAM to be aware of the most current
standards and regulations even if they do not appear in the 1991 version of this catalog. The
National Institute of Standards and Technology develops standards and guidelines to achieve
more effective use of Federal information. .The Federal Information Processing Standards (FIPS)
are the Federal data standards for all data exchange between agencies. It is FFEO policy to
comply with-all applicable FIPS. Applicable FIPS are listed tn the draft EPA document "Agency
Catalog of Data Policies and Standards," 2IM-1019, July 1991.
EPA's OIRM is responsible for the data standards programs. The EPA Data Standards
Program is established and documented in the information Resources Management Policy
Manual (Chapter 5, Data Standards). In general, EPA's data-related policies apply to all EPA
organizations and personnel, including contractors and grantees who design, implement, and
maintain information management systems for the EPA.
FFEO will incorporate by reference all Agency data standards into data collection activities
and into new or modernized information management systems. All applicable Agency data
standards arc listed in the Agency Catalogue of Data Policies and Standards (Draft) 2IM-1019.
Federal Facilities Enforcement Office
7-2
Quality Management Pi
-------
July 1991. In addition. FFEO will implement any future Agency-wide data standards promulgated
by OIRM.
FFEO also requires that sufficient data documentation be provided with a data set to assist
secondary data users when evaluating the utility of the data set for their purposes. This data
documentation includes the original information on DQOs associated with the data as well as any
supplementary information on the direct application of the original data, known restrictions, or
cautions which will facilitate secondary data use.
Quality Management Plan 7 J Federal Facilities Enforcement Offict
-------
8. Quality Planning
It is FFEO policy thai all its activities be planned both appropriately and effectively. To
ensure that FFEO programmatic and quality goals are achieved, quality planning must occur at
two levels:
i
i Program-specific; and
Project level.
Yearly action plans, tied to the annual budget process, and appropriate strategic plans
specify the types of activities that occur. In addition, FFEO is developing an integrated
Information Resources Management Strategic Plan, which will reflect the office-wide goals
outlined in the FFEO Strategic Plan. The IRM Strategic Plan will identify the linkages between
FFEO and other EPA and Federal agency information management systems that are necessary to
meet office-wide goals. FFEO will identity annual priorities to be implemented office-wide and
will also make recommendations to OECA on office IRM resource needs during each budget
cycle that will be reflected in the FFEO budget request.
FFEO must increasingly coordinate collection and use of environmentally related data
activities across numerous organizations. FFEO staff are active members on several intra- and
interagency committees that serve as forums for these cross-organizational planning activities.
Key planning committees include: the Federal Facility Leadership Council, EPA Regional Federal
Facility Coordinators conferences, the Federal Facility Environmental Restoration Dialogue
Committee, the Civilian Federal Agency (CFA) Task Force, the Federal Agency Environmental
Roundtable. and numerous multi-media enforcement initiatives. In addition, the Federal Agency
Environmental Management Program Planning process (FEDPLAN) and the Federal Facilities
Tracking System (FFTS) are environmental information management systems designed to
facilitate interagency environmental planning.
8.2 Program-Specific Planning
FFEO programs covered by this QMP include all those that implement environmentally
related data-gathering activities. This includes collection or use of data generated to develop,
coordinate, track, and oversee Regional Federal facility enforcement and compliance
assurance/assistance efforts. Programs covered in subsequent revisions of this QMP may vary as
a result of statutory and administrative changes. It is FFEO policy to review the QMP annually
and to update the programs covered as necessary.
It is critical to include this QMP as a part of the planning process when modifying existing
programs or designing new programs. Although this QMP outlines the minimum QA
requirements for FFEO programs, it is possible that some of the programs covered by this QMP
may need more QA specificity and detail for implementing their programs. In that case,
supplemental QA components should be developed as an addendum to this QMP. These addenda
Quality Management Plan 8 ¦ 1 Federal Facilities Enforcement Offict
-------
will be included as appendices in future revisions to this QMP. All programs covered by this
QMP should review their program needs to determine if this QMP adequately covers their QA
needs and develop addenda as necessary.
The workplan component of the QAARW outlines QA activities for the up-coming year,
including budget information. QA workpians within FFEO are developed through analysis of the
FFEO budget for the coming work year, and knowledge of continuing programs. In preparing the
QAARW, activities from the previous year are reviewed to determine improvements that can be
incorporated into the annual workplan.
8.3 Project-Level Planning
A project is an organized set of activities within a program. The QAPP is the primary
vehicle for ensuring adequate data quality at the project level. Planning documentation should
identify the personnel responsible for all-components of the QAPP. PM/WAMs will be
responsible for the development of these components. As part of the project planning,
PM/WAMs will develop schedules for the development, review, and completion of required
documentation, including adherence to the Agency policy of peer review (see the Administrator's
memorandum on the Agency Peer Review Policy, June 7, 1994).
The QAM will be included in the project planning process and will review and approve
individual project QAPPs. It is also the QAM responsibility to interview PM/WAMs responsible
for environmentally related data activities for the coming fiscal year and to determine the
necessary QA activities and budget. QA should be reflected as a well defined activity within any
project plan or a separate line item of the project budget for any project involving the collection
or use of environmental data.
Project level planning utilizes Total Quality Management (TQM) and DQO approaches to
address concepts of customer satisfaction and acceptable decision uncertainty, respectively.
These techniques are an integrated part of all FFEO QA tools. TQM and DQO techniques are
used in FFEO projects to answer project planning questions, such as:
What is the problem and how.does it relate to FFEO's mission?
" Once the questions are defined, what are the variables that answer the
questions?
What is the allowable level of uncertainty permitted that enables the
question to be answered?
Who are the customers and what are their expectations? and
Who are the suppliers and what are their responsibilities?
Federal Facilities Enforcement Office"
8-2
Quality Management Plar
-------
aammnm
9. Quality Implementation
9.1 Office-Wide Policy
The FFEO QMP is applicable office-wide and in related programs within the Regions.
Any changes to the QMP will be documented in revisions and will receive the same level of
review and approval as the original. FFEO management will provide technical oversight of
environmentally related data operations. Independent oversight of the implementation of the
FFEO Quality Management System in all operations will be performed by the QAM and the Staff
Directors.
9.2 Program Implementation
All programs that generate, use. or require others to collect environmentally related data
will document their QA procedures and develop appropriate SOPs for their program. QA
procedures should be documented for all the functional components of the program outlined in
Chapter 3 of this QMP.
SOPs will be documented in writing and made accessible to all persons involved in the
implementation of the program. If SOPs do not exist for a particular program, it is the
responsibility of the management of that program to ensure development of the SOPs.
PM/WAMs, with the assistance of the QAD, will develop a plan/schedule for completion of the
SOPs.
Where the program uses data generated by others, it must develop criteria and a process
with which to evaluate the acceptability of the data supplied. The criteria and process must be
incorporated into the QMP before data collection begins. This ensures that the data fit within the
margin of error constraints, as established by EPA program management.
9. 3 Project Level Implementation
The workplan and quality products outlined in the QAPP will be implemented as
approved. Any changes to the QAPP will be documented and the QAPP amended. Any
amendments to the QAPP will need to be reviewed and approved by the QAM or Staff Directors
as appropriate. The project timeline should include specific target dates for QA/QC products
(e.g., QAPP development, auditing time-lines) so that progress and completion of the QA/QC
activities can be tracked.
PM/WAMs are responsible for ensuring that any work assignment and task delivery order
includes specific requirements for reports on the QA of products involving environmentally
related data collection, either directly (data sets) or indirectly (e.g., data used in a research
report).
Quality Management Plan 9 - I Federal Facilities Enforcement Offict
-------
10. Quality Assessment and Response
It is FFEO policy to formally evaluate the FFEO QA program on a regular basis. These
regular evaluations of QA and QC processes will occur for the FFEO quality management system,
as well as the Regions. The mechanisms to be used for this evaluation are summarized below
10.1 Annual Review of the Quality Management Plan
The office-wide QA procedures described in the QMP will be assessed annually and
updated as necessary. The QAM will be responsible for coordinating this effort and ensuring that
appropriate changes are incorporated into the QMP. The QAM is also responsible for consulting
with the Regional QAMs to ensure the QMP, where necessary, reflects any modifications to
Regional QA programs. Similarly, any modifications potentially affecting Regional QA activities
will be communicated to the Regional QAMs by the FFEO QAM. Program-specific changes will
be provided to the QAM for incorporation into the QMP. The Director of FFEO will review and
approve changes to the QMP prior to their submittal to QAD. The annual review of the QMP
will be undertaken at the same time as the development of the FFEO QAARW.
10, 2 Quality Assurance Annual Report and Workplan
In conjunction with the annual QMP review, the Staff Directors will summarize the past
year's activities for the QAARW. An evaluation of each program's implementation of QA
procedures will be part of this assessment. Strategies to improve the QA management system
should also be included. The QAARW will be approved by the Director of FFEO.
10.3 Assessments/Reviews
Internal and external assessments/reviews will be the principal means for determining
compliance with and effectiveness of the QA control systems defined in the FFEO QMP. Internal
assessments/reviews should be conducted by teams of FFEO QA and technical staff at a frequency
sufficient to ensure that appropriate QA measures are being implemented. External
assessments/reviews are conducted by an outside organization at the request of FFEO
management. All assessments/reviews shall follow standard QAD guidance. If resources arc
limited, environmental data collection programs or activities that are highly visible or that produce
results used to support enforcement actions will be given priority.
Regional QA personnel arc responsible for conducting/overseeing both internal and
external assessments and reviews of their own QA programs. FFEO staff may participate on
these assessments/reviews or provide technical assistance at the Regions request.
Staff Directors, with assistance from the QAM, are responsible for establishing
assessment/review procedures to meet the specialized needs of their programs.
Assessments/reviews of FFEO programs and activities are to be conducted in accordance with
Quality Management Plan
10- I
Federal Facilities Enforcement Office
-------
pre-established QAD protocols. FFEO will use two general protocols, MSRs and Data Quality
Assessments (DQAs).
10.3.1 Management Systems Reviews
MSRs evaluate a specific QA program associated with environmental data-collection
activities to either affirm the correctness and appropriateness of the QA approach or to identify
areas where additional attention would bring significant benefits. Both internal and external
MSRs will be conducted. Internal MSRs are conducted by FFEO QA representatives. External
MSRs are conducted by QAD auditing against the QMP. FFEO MSRs consist of two major
types, organizational and program reviews.
Organizational MSRs assess FFEO's QA management structure, the QMP and other
office-wide QA documents, to determine whether FFEO is implementing a satisfactory QA
program. During the organizational MS&, the audit team will evaluate the effectiveness of the
FFEO-approved QMP. as well as the adequacy of resources and personnel provided to implement
the QA Program. It is anticipated that an organizational evaluation will be conducted by QAD
every three to four years. The FFEO QAM will assist the review team in determining the scope of
the review, planning, scheduling and implementing the review. QAD will present their findings to
the Director of FFEO. Information on the results of the MSR will also be included in the
QAARW. Action items on any deficiencies will be developed and discussed in this report.
Review of progress on action items will be discussed at management meetings, QAM
performance reviews, and in subsequent QAARWs.
Program MSRs focus on the implementation of a single program. Program MSRs are
conducted as often as necessary based on program need or as allowed by the constraints of
available resources. These MSRs are conducted by either an FFEO team or by an external team
from QAD.
The following issues are examined during a programmatic MSR:
Adherence to the QMP;
Procedures for developing DQOs;
Procedures for developing and approving QAPPs;
Quality of existing QAPP guidance and QAPPs;
Procedures for developing and approving SOPs;
Procedures, criteria, and schedules for designing and conducting audits;
Tracking systems for ensuring that the QA Program is operating and that
corrective actions disclosed by audits have been taken;
Federal Facilities Enforcement Office
10-2
Quality Management Plan
!
-------
Degree of management support;
Responsibilities and authorities of the various line managers and the QAM
for carrying out the QA Program; and
The level of financial resources and personnel devoted to implementing the
QA Program.
At the organizational level, MSRs include most of the elements stated above, but, as
discussed previously, they focus on QA structure rather than on QA implementation.
Management systems review guidance. Guidance for the Management Systems Review
Process, EPA QA/G-3, will be used for these reviews. The QAM, with assistance from the Staff
Directors. wilL select at least one FFEO program annually for an MSR. The assessment team will
be selected by QAM and approved by the Director of FFEO. FFEO will initiate the MSR process
one year after approval of the new FFEO QMP. Internal FFEO MSRs will be performed under
the guidance of the FFEO QAM, who may also request external MSRs to be performed by QAD.
The following subsections address the various stages of the auditing process, including
developing an audit plan, audit implementation, audit reporting, and audit response actions.
10.3.2 MSR Plans
Plans for conducting MSRs will be developed by the MSR team. The purpose of the plan
is to provide a blueprint to carry out the audit, as well as the criteria for evaluating the results.
These plans will include the following sections:
MSR title
MSR number
Date of the MSR
scope
Identifies the groups and activities to be evaluated
Breadth can vary from general overview, total system, or part of a system
purpose
standards
Criteria against which the performance is evaluated
team
Team lead
Team members
MSR
*
MSR
MSR
MSR
Quality Management Plan
10-3
Federal Facilities Enforcement Offict
-------
Necessary documents
Timeline
10.3.3 MSR Reporting
A closeout meeting will occur at the end of the MSR. Positive and negative aspects of the
data activity will be discussed by the team, the management of the area assessed, and, as
necessary, the technical staff performing the environmentally related data collection activity.
Copies of the draft MSR summary and recommendations will be provided to those
attending in advance of the meeting. The report will contain:
- MSR title and number and any other identifying information;
MSR team leaders;
Background information about the program or project;
Summary and conclusions of the MSR and corrective action requirements;
and
Attachments or appendices that include all audit evaluation forms and audit
finding forms.
An MSR report should be completed within two weeks after an MSR is completed. AJ1
MSR reports will be reviewed and approved by the appropriate QAM, PMs, and senior
management within a month after completion of the MSR report. It is the responsibility of the
review team leader to assure receipt of MSR reports by the appropriate project participants.
10.3.4 Response Action
The MSR reports will be discussed with management in the organization being assessed
during a close-out meeting. A draft of the MSR report may be given to the organization to give
them an opportunity to provide additional input prior to finalization. Subsequent to the meeting,
the line management, with the assistance of the QAM, will determine necessary actions and develop
a plan and a schedule to rectify any problems. Milestones will be developed so that progress on
corrective actions can be measured. The PM/WAMs are responsible for ensuring compliance with
the recommended corrective actions. Progress is to be reported to the Staff Directors and the
Director of FFEO.
10.3.5 Data Quality Assessments
DQA's focus on the collected data. Their primary purpose is to determine whether there is
sufficient information on a given data set to determine whether the data meet the DQOs established
for the project. The result of this assessment is a quantitative measure or quantitative statements of
Federal Facilities Enforcement Office,
10-4
Quality Management Pku
-------
the limitations on the quality and potential uses of the data. The data quality assessment process is
detailed in the Guidance for Data Quality Assessment (Final), EPA QAJG-9, July 1996. This
process provides the necessary steps for the statistical analysis of data to determine whether or not
the data meet the DQOs and with what level of confidence these data may be used. The five steps
of the DQA process include reviewing the initial planning documentation (i.e., DQOs, QAPPs,
sampling and analysis plans), conducting a preliminary review of the data, selecting a statistical test,
verifying the assumptions of the statistical test, and performing the statistical test to reach a
conclusion to the question posed during the planning phase of the study. If QC deficiencies are
found, potential technical and managerial causes are examined, and follow-up measures identified.
During the DQA, an evaluation is completed to determine if the organization(s) that coilect(s) or
use(s) the data have performed their own DQA and followed through on the results of this
assessment.
Data collection activities conducted by Headquarters are audited by the QAM to ensure that
QAPPs are being implemented and that data of known and adequate quality are being collected.
DQAs are conducted during and/or at the end of the data collection activity and included in the QA
report. DQAs may be performed during the project at the request of the PM/WAM if they have
identified concerns about data quality. DQAs during the project afford an opportunity for on-going
corrective action. DQAs at the end of a project provide a means of verifying the utility of the data,
the need for a new project effort, or determination of the feasibility of a long term program. The
PM/WAM, with assistance from the QAM, is responsible for determining the need for a DQA.
Once an assessment is performed, the QAM issues a report to the PM/WAM describing any
necessary corrective action. It is the responsibility of the project staff to report any corrective
action to the QAM.
Quality Management Plan
10 -5
Federal Facilities Enforcement Offict
-------
3E* ¦- 'J!WH
11. Quality Improvement
As indicated in Chapter 3 of this QMP, it is FFEO policy that QA is a critical component of
the work functions within all programs. The intent of this QMP is to provide the basis for
integrating appropriate QA activities into the full cycle of FFEO programs from the planning phases
through the evaluation phases. If the principles outlined in the QMP are followed, problems can be
detected in a timely manner, before programmatic and financial issues become critical and hinder
program implementation and decision-making.
FFEO has been an active participant in EPA's TQM Program. TQM training has been
provided at all levels to ensure the application of TQM philosophy and tools to all FFEO programs.
These principles, including work process flow analysis, statistical analysis, and process control
methods assure that all FFEO processes-are monitored and continuously improved by the staff
responsible for them.
Within the FFEO. there are three levels of review that will help uncover problems with the
QA management system. These are: office-wide reviews, programmatic reviews, and project
reviews.
11.1 Office-Wide Reviews
Each year the QMP will be reviewed by FFEO staff and management to ensure that the
document is still relevant to the FFEO mission. The QAM is responsible for coordinating the
review. The QMP will be modified to reflect changing needs or additional guidance. Revisions will
be assigned unique, consecutive revision numbers and the date of each revision will be included in
the header information of the document and its table of contents.
The QAM meets at least quarterly with the Staff Directors. A key purpose of these
meetings is to identify QA issues of concern. Based on the recommendations of the Staff Directors
and subsequent consultation with senior management, the QAM will initiate MSRs or special
projects to address and correct QA problems identified by the Staff Directors. Actions developed
to correct any major QA deficiencies will be "documented in the QAARW and reviewed and
approved by the Director of FFEO.
11.2 Program Review
Each year all FFEO programs involved in the collection of environmentally related data will
review their QA/QC SOPs to determine if they remain relevant to the mission of the program and
are working properly to ensure data of known and sufficient quality to support programmatic
decisions. Ensuring that this review occurs is the responsibility of Staff Directors. Line
management and the QAM will meet at least annually to discuss adherence to the QA/QC SOPs
outlined in the FFEO QMP and identify areas where improvements can be made to these SOPs.
Actions will be developed to correct any major QA deficiencies. Actions to be taken will be
Quality Management Plan
Federal Facilities Enforcement Of/ici
-------
rvmasmBm
outlined in the program's QAARW submitted to the FFEO QAM. Each program's QAARW will be
incorporated into the FFEO QAARW submitted to QAD.
11.3 Project Reviews
It is FFEO policy that the PM/WAM, with assistance from the QAM, and project
participants review project implementation at regular intervals to identify where improvements in
data quality can occur. Project reviews can consist of:
Data quality assessments;
« Peer reviews:
. Conference calls; and
Meetings.
Generally there should be a meeting at the end of the data collection phase of a project.
Preliminary DQAs should be available for this meeting so participants can determine whether the
QAPP was followed and data quality was controlled to an acceptable level. If a meeting is held at
the end of a project, then the QAPP should be revised afterward, as necessary, to document the
actual process that was followed dunng the project. The SOPs should be revised to reflect changes
and improvements in procedures that were developed during the program. Problems and
recommended corrective actions for future programs should be documented in the QA section of
the final project report.
Federal Facilities Enforcement Office
11 -2
Quality Management Plan
-------
References
1) Disposition of Federal Records: A Records Management Handbook, NARA, 1992.
2) EPA Guidance for the Data Quality Objectives Process (Final), EPA QA/G-4,
September 1994.
3) EPA's Operations and Maintenance Manual, EPA Directive 2181. April 30, 1993.
4) EPA Requirements for Quality Assurance Project Plans (Interim Final), EPA
QA/R-5, August 1994.
5) EPA Requirements for Quality Management Plans (Interim Final), EPA QA/R-2,
August 1994.
6) Guidance for Data Quality Assessments! Final), EPA QA/G-9, July 1996.
7) Guidance for the Preparation of Standard Operating Procedures (SOPs)for Quality-
Related Documents, EPA QA/G-6, November 1995.
8) How to Get Contracts Awarded in EPA (EPA 202-B-93-002, November 1993).
9) IRM Policy Manual (21 00), Chapter 10, Records Management, OIRM, November
1987.
10) Managing Electronic Records (Instructional Guide Series), NARA, 1990.
11) National Data Processing Division Guidance on Hardware and Software Standards
(Draft), OIRM, November 8, 1993.
12) Practical Guide to Personal Papers: Records Management Technical Leaflet #3
(EPA 220-F-92-019).
13) Records Disposition Schedules Cross Reference Table (EPA 220-B-95-005).
14) Records Management Manual (2160), OIRM, 1984.
15) System Design and Development Guidance, EPA Directive 2182. April 30, 1993.
16) Using the Federal Records Center: A Guide for Headquarters Staff
(EPA/IMSD/91-004).
Quality Management Plan
R- 1
Federal Facilities Enforcement Office
------- |