United States
Environmental Protection
Agency
Office of Environmental
Information
Washington, DC 20460
EPA/240/R-02/008
November 2002
Guidance for Developing
Quality Systems for
Environmental Programs
EPA QA/G-1
-------
-------
FOREWORD
The U.S. Environmental Protection Agency (EPA) has developed an Agency-wide program of
quality systems for environmental data. EPA's Quality System requires documentation of both
management and technical activities. This document, Guidance for Developing EPA Quality Systems
for Environmental Programs., provides methods and tools for developing and documenting the
elements of a functional quality system. It is pertinent to organizations that carry out environmental data
operations within or for EPA.
This document helps organizations design a structured management system for ensuring quality
in its work processes, products, and services. The EPA's Quality System has been built to ensure that
environmental programs are supported by the type and quality of data needed for their appropriate use.
As required by the EPA Quality Manual for Environmental Programs, Order 5360 A1 (EPA,
2000a), this document is valid for up to five years from the official date of publication. After five years,
this document will be reissued without change, revised, or withdrawn from the U.S. Environmental
Protection Agency Quality System Series documents.
This document provides guidance to EPA employees and other organizations involved in quality
system development. It does not impose legally binding requirements on EPA or the public and may
not apply to a particular situation based on the circumstances. EPA retains the discretion to adopt
approaches on a case-by-case basis that differ from this guidance where appropriate. Interested
parties are free to raise questions about the recommendations in this document and the appropriateness
of using them in a particular situation, and EPA and other parties should consider whether the
recommendations in the document are appropriate for the particular situation. EPA may periodically
revise this guidance without public notice.
This document is one of the EPA Quality System Series documents, which describe policies
and procedures for planning, implementing, and assessing the effectiveness of a quality system.
Questions regarding this document or other EPA Quality System Series documents can be directed to:
U.S. EPA
Quality Staff (2811R)
1200 Pennsylvania Ave., N.W.
Washington, D.C. 20460
Phone: (202)564-6830
Fax: (202)565-2441
e-mail: quality@epa.gov
Copies of EPA Quality System Series documents may be obtained from the Quality Staff directly or
by downloading them from its Home Page, www.epa.gov/quality.
Final
EPAQA/G-1 i November 2002
-------
Final
EPAQA/G-1 ii November 2002
-------
TABLE OF CONTENTS
Page
1.0 INTRODUCTION 1
1.1 PURPOSE AND SCOPE 1
1.2 SUPERSESSION 3
1.3 PERIOD OF APPLICABILITY 4
1.4 HOW TO USE THIS GUIDANCE DOCUMENT 4
2.0 ELEMENTS OF A QUALITY SYSTEM 5
2.1 THE EPA QUALITY SYSTEM MODEL 5
2.1.1 Specifications for EPA Organizations 7
2.1.2 Specifications forNon-EPA Organizations 8
2.2 QUALITY SYSTEM STANDARDS AND MODELS 10
2.2.1 Elements of the ANSI/ASQC E4-1994 Quality System Standard 10
2.2.2 Other Quality System Standards and Models 10
3.0 HOW TO DEVELOP A QUALITY SYSTEM 13
3.1 OVERVIEW OF THE QUALITY SYSTEM DEVELOPMENT PROCESS 13
3.1.1 Quality Systems for Small Organizations 14
3.2 INITIATION ACTIVITIES 16
3.2.1 Senior Management Commitment 16
3.2.2 Quality System Development Team 17
3.2.3 Establish Quality Policy 20
3.2.4 Analyze Requirements 21
3.2.5 Inventory Existing Quality Procedures and Practices 22
3.2.6 Obtain Management Approvals, Resources for Development 23
3.3 DEVELOPMENT ACTIVITIES 25
3.3.1 Assign Tasks and Develop Detailed Schedule 26
3.3.2 Analyze Work Processes Using a Quality System Model 27
3.3.3 Write Standard Operating Procedures 29
3.3.4 Write a Quality Management Plan 31
3.3.5 Develop Supporting Systems and Tools 33
3.3.6 Conduct Detailed Compliance Audit 33
3.3.7 Obtain Management Approvals and Resources for Implementation 34
3.4 IMPLEMENTATION AND START-UP ACTIVITIES 35
3.4.1 Determine the Implementation Strategy and Schedule 36
3.4.2 Publish and Distribute Quality Management Plan and
Quality Procedures 38
3.4.3 Training 38
Final
EPAQA/G-1 iii November 2002
-------
Page
3.4.4 Communications 40
3.4.5 Establish a Program for Recognizing Quality Performance 41
3.4.6 Initiate Quality Procedures and Maintain Records 41
3.4.7 Conduct Periodic Internal Quality System Audits 42
3.4.8 Corrective Actions 43
3.4.9 Identify Improvements 44
3.5 ONGOING MAINTENANCE AND IMPROVEMENT 45
3.5.1 Allocate Resources per the Budget Cycle 45
3.5.2 Conduct Routine Quality Assurance 46
3.5.3 Implement Improvements 46
3.5.4 Conduct Periodic Self-Assessment of this Quality System 47
3.5.5 Maintain Quality Training, Communications, and Recognition and Rewards
Programs 47
4.0 CASE STUDIES 51
4.1 QUALITY SYSTEM DEVELOPMENT IN AN ENVIRONMENTAL
CONSULTING FIRM 51
4.1.1 Background 51
4.1.2 Initiation 52
4.1.3 Development 56
4.1.4 Implementation 59
4.1.5 On-going Maintenance and Improvement 60
4.2 QUALITY SYSTEM DEVELOPMENT IN AN EPA REGULATORY
PROGRAM: SIX-YEAR REVIEW OF NATIONAL PRIMARY DRINKING
WATER STANDARDS 61
4.2.1 Background 61
4.2.2 Initiation 62
4.2.3 Development 62
4.2.4 Implementation 63
4.2.5 On-going Maintenance and Improvement 64
5.0 TOOLS AND METHODS USED IN DEVELOPING QUALITY SYSTEMS 67
5.1 INTRODUCTION 67
5.2 STANDARD OPERATING PROCEDURES TEMPLATES 67
5.3 QUALITY MANAGEMENT PLANS 68
5.4 QUALITY MANUALS 69
5.5 DOCUMENT CONTROL AND RECORDS MANAGEMENT SYSTEMS .... 71
5.6 QUALITY SYSTEM COMPLIANCE CHECKLISTS 72
5.7 PROCESS ANALYSIS AND IMPROVEMENT TOOLS 72
Final
EPAQA/G-1 iv November 2002
-------
Page
5.7.1 Process Flow Charts 72
5.7.2 Control Charts 73
5.7.3 Cause-and-Effect (Fishbone) Diagrams 74
5.7.4 Pareto Charts 74
5.7.5 Benchmarking 76
5.7.6 Scheduling Tools 79
APPENDIX A: QUALITY SYSTEM CHECKLIST A-l
APPENDIX B: REFERENCES B-l
APPENDIX C: QUALITY ASSURANCE CONCEPTS C-l
APPENDIX D: GLOSSARY OF TERMS D-l
Final
EPAQA/G-1 V November 2002
-------
LIST OF FIGURES
Page
Figure 1. The EPA Quality System 6
Figure 2. A Four-Phase Process for Developing a Quality System 13
Figure 3. Initiating the Quality System (Phase 1) 16
Figure 4. Developing the Quality System (Phase 2) 26
Figure 5. Standard Diagram of a Work Process 28
Figure 6. Implementing the Quality System (Phase 3) 36
Figure 7. Ongoing Maintenance and Improvement (Phase 4) 46
Figure 8. Quality System for the Six-Year Review 63
Figure 9. Data Sources for Occurrence and Exposure Assessments 66
Figure 10. Example of a Process Flow Chart 73
Figure 11. Control Chart Showing a Process Under Statistical Control 74
Figure 12. Example of a Cause-and-Effect (Fishbone) Diagram 75
Figure 13. Example of a Pareto Chart 75
Figure 14. Project Gantt Chart 80
Figure 15. PERT Chart 80
LIST OF TABLES
Page
Table 1. Quality-Related Regulations and Agreements by Organization 8
Table 2. Agency-wide Internal and External Quality Policies 9
Table 3. Elements of the ANSI/ASQC E4-1994 Quality System Standard 11
Table 4. Standards Relating to Quality Systems 11
Table 5. Example General Schedule for Development of a Quality System 19
Table 6. Example Detailed Schedule for Development of a Quality Management Plan 26
Table 7. Quality System Elements Required for the MRI Environmental Monitoring Group . . 53
Table 8. Assessment of Department Practices 57
Table 9. Sample Quality System Development Activities for the 2000 Six-Year Review .... 65
Table 10. Reasons for Benchmarking 79
Final
EPAQA/G-1 vi November 2002
-------
LIST OF ABBREVIATIONS
ANSI/ASQ American National Standards Institute/American Society for Quality
ASTM American Society for Testing and Materials
CFR Code of Federal Regulations
EPA Environmental Protection Agency
GPO Government Printing Office
ISO International Organization for Standardization
MRI Marine Research Institute
MTE Measurement and Testing Equipment
NPDWRs National Primary Drinking Water Regulations
O AQP S Office of Air Quality Planning and Standards
OCW Office of Clean Water
PERT Program Evaluation and Review Technique
QA Quality Assurance
QC Quality Control
QSDT Quality System Development Team
SDEP State Department of Environmental Programs
SOP Standard Operating Procedures
EPA QA/G-1
VU
Final
November 2002
-------
Final
EPAQA/G-1 8 November 2002
-------
CHAPTER 1
INTRODUCTION
1.1 PURPOSE AND SCOPE
What does this guidance cover?
This guidance document provides methods and tools to help organizations develop a quality
system that meets its internal organizational needs and complies with U.S. Environmental Protection
Agency (EPA) requirements for quality systems. EPA's policies for quality systems are outlined in
EPA's Policy and Program Requirements for the Mandatory Agency-wide Quality System, Order
5360.1 A2 (EPA, 2000a); EPA Quality Manual for Environmental Programs, Manual 5360 Al
(EPA, 2000b); and the Federal Regulations (48 CFR 46; 40 CFR 30, 31, and 35).
Other general environmental quality system specifications applicable to EPA's programs are
described in the American National Standards Institute/American Society for Quality (ANSI/ASQ) E4-
1994, Specifications and Guidelines for Quality Systems for Environmental Data Collection and
Environmental Technology Programs (ANSI/ASQ, 1995). This document outlines the basic
guidelines for planning, implementing, and assessing a quality system for environmental data collection
and environmental technology. ANSI/ASQ's standards apply to quality systems for air and water
quality monitoring, pollution control technology development, sampling and analysis for environmental
impact studies, hazardous waste investigations, and a variety of other activities.
What is a quality system?
A quality system is the means by which an organization manages its quality aspects in a
systematic, organized manner. It provides a framework for planning, implementing, and assessing work
performed by an organization and for carrying out required quality assurance (QA) and quality control
(QC) activities. It encompasses a variety of technical and administrative elements, including:
policies and objectives,
organizational authority,
responsibilities,
accountability, and
procedures and practices.
Final
EPAQA/G-1 1 November 2002
-------
What are the purpose and scope of the EPA Quality System?
The EPA Quality System is a management system that provides the necessary elements to plan,
implement, document, and assess the effectiveness of QA and QC activities applied to environmental
programs conducted by or for EPA. The EPA Quality System encompasses the collection, evaluation,
and use of environmental data by or for EPA, and the design, construction, and operation of
environmental technology by or for EPA. This includes environmental programs such as the
demonstration of environmental technology; investigation of chemical, biological, physical, or
radioactive constituents; development, evaluation, and use of computer or mathematical models; use of
data collected for other purposes or from other sources (also called secondary data); and the collection
and use of data pertaining to the occupational health and safety of personnel in EPA facilities. Example
programs are listed in Section 1.3 of the EPA Manual 5360 Al (EPA, 2001b).
What are the benefits of EPA's quality system?
Successful implementation of the EPA Quality System leads to the benefits outlined below.
Scientific Data Integrity—EPA will be better able to produce data of known and
documented quality based on sound scientific principles.
Reduced or Justifiable Resource Expenditures—Resource expenditures can be reduced
if EPA's information needs are more closely matched to the information collection.
Through proper planning, only the correct type, amount, and quality of data will be
collected for EPA use.
Proper Evaluation of Internal and External Activities—The EPA Quality System
provides documentation of activities and improved oversight for evaluation purposes.
This reduces the potential for waste and abuse.
Reliable and Defensible Decisions—When the quality of data are better known,
determining whether the data can be used for a specific decision is facilitated. This
reduces embarrassing surprises and challenges to regulations, permit appeals, etc.
Burden Reduction—As EPA better defines the data needed for a specific application,
the burden on other organizations who are required to collect and/or report data to
EPA may be reduced.
Overall, implementation of the EPA Quality System will reduce the Agency's vulnerabilities and increase
EPA's ability to make reliable, cost-effective, and defensible decisions.
Final
EPAQA/G-1 2 November 2002
-------
Who should read this guidance?
This document is intended for managers and quality professionals within EPA organizations
(program offices, laboratories, and regional offices); states, tribal governments, recipients of external
assistance agreements; and EPA contractors.
Is my organization required to follow this guidance?
No. This is non-mandatory guidance intended to help organizations apply EPA policies and
apply quality management concepts and best practices to their own circumstances. The process
described in this guidance for developing a quality system is only one of many possible approaches for
developing a quality system. While the general process described here should work for most
organizations, the specific details will need to be tailored to each organization.
How does this guidance document relate to other the EPA Quality System documents?
This guidance describes a process for developing a quality system that follows EPA Quality
System policies set forth in EPA Order 5360.1 A2 while also meeting the internal needs of an
organization. Consequently, this guidance addresses the organizational development processes and
related tools that can be employed to establish a working quality system. Most of the other EPA
Quality System Series documents focus on a specific managerial or technical element of the EPA
Quality System, at either the organizational or project level. This guidance describes how an
organization assembles these elements into a coherent quality system. EPA QA/R-2 provides the
specifications for documenting the quality system.
Where can I find more information?
Additional documents, tools, training, and technical assistance are available through the EPA
Quality Staff (see the Foreword for contact information). Many additional resources are available for
downloading at the Quality Staff website: www.epa.gov/quality.
1.2 SUPERSESSION
No previous guidance on developing quality systems has been issued by the EPA Quality
System staff. Therefore, this document does not supersede any other documents.
Final
EPAQA/G-1 3 November 2002
-------
1.3 PERIOD OF APPLICABILITY
As described in the EPA Quality Manual (5360 Al), this document will be valid for 5 years
from the official date of publication. After 5 years, this document will either be reissued without change,
revised, or withdrawn from the EPA Quality System.
1.4 HOW TO USE THIS GUIDANCE DOCUMENT
If you are interested in learning about...
then read
Principles underlying quality management and the core elements of a
quality system
Chapter 2
Activities necessary to develop a quality system that meets your
organization's needs
Chapter 3
An example of quality system development in a narrative case study
format
References and supplemental reading
Chapter 4
Tools and methodologies that may help in the development of your quality
system Chapter 5
References
Definitions of important quality management terms
Glossary
EPA QA/G-1
Final
November 2002
-------
CHAPTER 2
ELEMENTS OF A QUALITY SYSTEM
This chapter will answer the following questions:
What are the components of the EPA Quality System?
What are the requirements of the Quality System for EPA organizations?
What are other Quality System standards that government agencies and private
organizations need to consider?
2.1 THE EPA QUALITY SYSTEM MODEL
The EPA Quality System integrates management and technical activities for the planning,
implementation, and assessment of environmental programs within the Agency's mission and scope.
The EPA's Quality System has been designed to ensure that environmental programs are supported by
the type, quality, and quantity of data needed for their intended use. The EPA Quality System
integrates policy and procedures, organizational responsibilities, and individual accountability.
The ANSI/ASQ E4-1994, Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs (ANSI/ASQ, 1995)
specifically defines a quality system as:
...a structured and documented system describing the policies, objectives,
principles, organizational authority, responsibilities, accountability, and
implementation plan of an organization for ensuring the quality in its work
processes, products, items, and services. The quality system provides the
framework for planning, implementing, and assessing work performed by the
organization and for carrying out required QA and QC.
The EPA Quality System is depicted graphically in Figure 1. At the policy level, EPA's
primary internal policy directives are EPA Policy and Program Requirements for the Mandatory
Agency-wide Quality System, Order 5360.1 A2 (EPA, 2000b) and EPA Quality Manual for
Environmental Programs, Manual 5360 Al (EPA, 2000a), which are derived in part from national
and international consensus standards on quality systems, as further explained below. EPA
requirements affecting external organizations are reflected in regulations addressing contracts and
assistance agreements. EPA program and regional offices establish policies that are consistent
with Agency-wide policies. At the organizational level, the quality system features documentation
[written in a Quality Management Plan (QMP)], systems assessments (such as Management Systems
Final
EPAQA/G-1 5 November 2002
-------
Consensus Standards
ANSI/ASQC E4
ISO 9000 Series
External Regulations
Contracts - 48 CFR 46
Assistance Agreements
Internal Policies
EPA Order 5360.1
EPA Manual 5360
•
Quality System
(e.g., QMP)
'
Supporting System Elements
(e.g., Procurements,
Computer Hardware/Software)
Training/Communication
(e.g., Training Plan,
Conferences)
ANN I
IMPLEMENTATION
' ASSESSMENT
Defensible Products and Decisions
Figure 1. The EPA Quality System
Reviews or Quality System Audits), annual reviews and planning documents (written in the QA Annual
Report and Work Plan), training and communications, and supporting system elements (such as
purchasing systems, information management systems, etc.). Implementation of each component or
tool reflects EPA's needs with respect to its environmental data operations and Agency policies. At the
project level, planning, implementation, and assessment activities are applied to EPA's data generation,
acquisition, and use. In the planning phases of project management, EPA data users apply a systematic
EPA QA/G-1
Final
November 2002
-------
planning process (such as the data quality objectives process) to specify performance criteria for data
operations, which are documented in a QA Project Plan. In the implementation phase, technical audits
and assessments are used to ensure that data are being acquired as stated in the EPA QA Project Plan.
In the assessment phase, data are formally verified and validated to ensure that they are free of major
errors and are then analyzed to determine if performance criteria have been met.
2.1.1 Specifications for EPA Organizations
EPA Order 5360.1 A2 (EPA, 2000b) defines basic quality management specifications for all
EPA organizations covered by the EPA Quality System. These specifications are summarized below.
1. Conform to the minimum specifications of ANSI/ASQ E4-1994.
2. Identify a QA Manager who reports on quality issues to senior executive leadership and
ensure that this QA Manager functions independently of direct environmental data
generation, model development, or technology development responsibility.
3. Develop a Quality Management Plan and implement this plan following Agency
approval.
4. Provide sufficient resources to implement the quality system.
5. Perform assessments of the effectiveness of the quality system at least annually and
implement corrective actions based on assessment results in a timely manner.
6. Submit a QA Annual Report and Work Plan for the organization that summarizes the
previous year's activities and outlines the work proposed for the current year.
7. Use a systematic planning approach to develop acceptance or performance criteria for
all work covered by the EPA Quality System.
8. Have approved QA Project Plans, or equivalent documents, for all applicable EPA
projects and tasks involving environmental data.
9. Assess existing data when used to support Agency decisions or other secondary
purposes to verify that they are of sufficient quantity and adequate quality for their
intended use.
10. Implement Agency-wide Quality System requirements in all applicable EPA-funded
extramural agreements.
Final
EPAQA/G-1 7 November 2002
-------
11. Implement corrective actions based on assessment results.
12. Provide appropriate training for all management and staff to assure that QA and QC
responsibilities and requirements are understood at every stage of implementation.
2.1.2 Specifications for Non-EPA Organizations
Agency-wide Quality System specifications may also apply to non-EPA organizations that
collect or analyze data for or in association with EPA. These specifications are defined in the
applicable regulations governing extramural agreements, as shown in Tables 1 and 2. Agency-wide
Quality System specifications may also be invoked as part of negotiated agreements, such as a
memorandum of understanding. Non-EPA organizations that may be subject to quality system
specifications include:
• any organization or individual under a direct contract to EPA to furnish services or
items or perform work (i.e., a contractor) under the authority of 48 CFR 46, (including
applicable work assignments, delivery orders, and task orders);
Table 1. Quality-Related Regulations and Agreements by Organization
EPA
Contractor
Federal
Agency
Hospital
University
Local
Government
Non-profit
Organization
Regulated
Entity
State
Government
Tribal
Government
Contract
N/A
48 CFR 46
N/A
48 CFR 46
48 CFR 46
48 CFR 46
48 CFR 46
N/A
48 CFR 46
48 CFR 46
Cooperative
Agreement
N/A
N/A
N/A
40 CFR 30
40 CFR 30
40 CFR 31,
40 CFR 35
40 CFR 30
N/A
40 CFR 31,
40 CFR 35
40 CFR 31,
40 CFR 35
Grant
N/A
N/A
N/A
40 CFR 30
40 CFR 30
40 CFR 31,
40 CFR 35
40 CFR 30
N/A
40 CFR 31,
40 CFR 35
40 CFR 31,
40 CFR 35
Inter-Agency
Agreement
N/A
N/A
Negotiated into
each agreement
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Other Specifications
EPA Order 5360. 1A2,
ANSI/ASQE4-1994
N/A
Contained in specific Federal
Regulation that requires data
Contained in specific Federal
Regulation that requires data
Contained in specific Federal
Regulation that requires data
Contained in specific Federal
Regulation that requires data
Contained in specific Federal
Regulation that requires data
Contained in specific Federal
Regulation that requires data
Contained in specific Federal
Regulation that requires data
Contained in specific Federal
Regulation that requires data
' Grants include Performance Partnerships Grants and Performance Agreements.
EPA QA/G-1
Final
November 2002
-------
institutions of higher education, hospitals, and other nonprofit recipients of financial
assistance (e.g., grants and cooperative agreements) under the authority of
40CFR30;
commercial business enterprises;
state, local, and tribal governments receiving financial assistance under the authority of
40CFR31 and 35; and
other government agencies receiving assistance from EPA through extramural
agreements.
Table 2. Agency-wide Internal and External Quality Policies
Agency-wide Policy
EPA Order 5360. 1A2
EPA Manual 5360 Al
48 CFR 46
40 CFR 30
40 CFR 31
40 CFR 35
Requirements
Policy and Program Requirements for the Mandatory Agency-wide Quality System
establishes minimum requirements for EPA organizations, policy for developing QMPs,
responsibilities for EPA management, staff, and Quality personnel.
EPA Quality Manual for Environmental Programs addresses ways to implement quality
management activities, reporting requirements for environmental programs, Quality Staff
guidance documents, user-friendly QA and QC guidance, etc.
Quality Assurance in the Federal Acquisition Requirements contains requirements for
contracts, work assignments, and task orders. EPA and its contractors use ANSI/ASQ
E4 as the standard for quality requirements and are required to submit a QMP or QA
Project Plan.*
Grants and Agreements -with Institutions of Higher Education, Hospitals and Other
'Non-Profit Organizations contains requirements for referenced organizations and
requires grantees to comply with ANSI/ASQ E4. Grantees must submit a QMP or QA
Project Plan.*
Uniform Administrative Requirements for Grants and Cooperative Agreement to State
and Local Governments contains requirements for grants and cooperative agreements to
State, local, and Tribal governments, requires grantees to develop QA practices to
produce data of adequate quality for project objectives.*
State and Local Assistance contains requirements for any financial assistance to State
and local governments. *
*Requirements are involved when work is within the scope of the regulations and the program.
In general, EPA requires compliance with the ANSI/ASQ E4-1994 standard for all recipients of funds
for projects involving environmental data collection. Required documentation can include:
• Documentation of the organization's quality system (usually provided in a QMP).
EPA QA/G-1
Final
November 2002
-------
Documentation of the application of QA and QC activities to an activity-specific effort
(usually provided in a QA Project Plan).
Use of existing quality system documentation, such as ISO 9001 registration, may serve as an
acceptable alternative.
2.2 QUALITY SYSTEM STANDARDS AND MODELS
The EPA Quality System is based on the national consensus standard ANSI/ASQ E4-1994
and is consistent with other consensus management system standards. This section briefly reviews this
and other relevant quality system standards and models.
2.2.1 Elements of the ANSI/ASQ E4-1994 Quality System Standard
The American National Standard, Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs (ANSI/ASQ E4-1994)
(ANSI/ASQ, 1995), was developed to promote consistency among the many quality systems
requirements for environmental programs throughout the Federal government and the environmental
industry. The standard describes the elements that, at minimum, should be in place to ensure that a
functional quality system exists for organizations that perform environmental data operations or design
or operate environmental control technology. The ANSI/ASQ E4 standard is designed to provide the
basis for an auditable agreement between two parties. For example, the government may invoke the
standard in a procurement, such that the successful bidder is required to comply with the ANSI/ASQ
E4 standard and submit to an audit by the government to assess compliance. The standard is organized
according to general elements covering organization-wide quality management (Part A), project-
oriented elements covering environmental data operations (Part B), and project-oriented design,
construction, and operations of environmental technology (Part C). Table 3 lists the specific elements
within each category. Refer to the standard for more information.
2.2.2 Other Quality System Standards and Models
There are several other quality system standards and models that government agencies and
commercial organizations may consider, depending on the nature of their work. Considerable
commonality exists among various quality standards and models, regardless of application or industry.
One such standard is ISO 9001:2000, Quality Management Systems - Requirements (ISO, 2000),
an international consensus standard that defines general requirements for quality management systems
for any organization that delivers products or services. Table 4 identifies some major quality system
standards, their methods of administration, and examples of situations in which they may apply. In
some cases, there may be a two-party agreement like that discussed above for ANSI/ASQ E4-1994.
Final
EPAQA/G-1 10 November 2002
-------
Table 3. Elements of the ANSI/ASQ E4-1994 Quality System Standard
Part A.
Management Systems
1 . Management and
Organization
2. Quality System and
Description
3. Personnel Qualifications
and Training
4. Procurement
5. Documents and Records
6. Computer Hardware and
Software
7. Planning
8. Implement Work Processes
9. Assessment and Response
10. Quality Improvements
Part B.
Environmental Data
1 1 . Planning and Scoping
12. Design of Data Collection
13 Implementation
14. Assessment and Response
Activities
1 5 . Assessment and Data Usability
Part C.
Design, Construction, and Operation
of Environmental Technology
16. Planning
17. Design of Systems
1 8 . Construction/Fabrication
19. Operations
20. Assessment and Response
2 1 . Verification and Acceptance
In other cases, third-party registrations or certification processes establish the compliance status of an
organization, such as in the case of the ISO 9001:2000. An increasing number of federal and
commercial organizations are requiring contractors and suppliers to become registered with ISO
9001:2000, and countries in the European Union and Asia have embraced the standard in many areas
of commerce. Government regulations and guidelines, such as Good Laboratory Practices (40 CFR
160 and 792), Good Clinical Practices (21 CFR 50, 56, 312), and Good Manufacturing Practices
(21 CFR 210, 211, 290), include many quality management requirements that apply to health and
pharmaceutical research and medical device manufacturing—activities in which failures could place
humans at risk. ISO 17025 (ISO, 1999) addresses competency requirements for testing and
calibration laboratories and also includes quality system requirements. The National Environmental
Laboratory Accreditation Program is an approach designed to promote coordination and efficiency in
auditing for compliance with various laboratory QA requirements across states and programs.
Table 4. Standards Relating to Quality Systems
Standard Name
ANSI/ASQ E4, Specifications and Guidelines
for Quality Systems for Environmental Data
Collection and Environmental Technology
Programs
ISO 9001:2000, Quality Management
Systems - Requirements
Method of
Administration
2-party agreement
3rd-party registration
Examples of Application
State conducting EPA-funded studies;
contractor whose primary clients are
within EPA
EPA contractor who also provides
services to ISO 9001 -registered firms
EPA QA/G-1
11
Final
November 2002
-------
Table 4. Standards Relating to Quality Systems
Standard Name
Good Laboratory Practices, Good
Automated Laboratory Practices, Good
Clinical Practices, Good Management
Practices
ISO 1 7025, General Requirements for the
Competence of Calibration and Testing
Laboratories
National Environmental Laboratory
Accreditation Program
Method of
Administration
FDA and EPA
regulations and
guidelines
Accreditation/
registration
Accreditation by
approved authority
(3rd party)
Examples of Application
Research organization conducting
toxicology testing for new drug or
chemical
Commercial metrology lab that serves
EPA contractors as well as ISO 9001-
registered commercial clients
An environmental services laboratory
based in New York, with operations
also in Florida, whose primary
accreditation in New York is
recognized by Florida
Summary
The EPA Quality System integrates both management and technical activities for the
planning and assessment of Agency environmental programs.
The authority and requirements related to EPA's quality system are described in several
federal regulations and EPA policy orders.
EPA quality systems should conform to the requirements of ANSI/ ASQ E4, which
outlines elements for management systems, environmental data collection, and the
design and operation of environmental technologies.
There are a number of other quality system standards and models that government
agencies and commercial organizations should consider depending on the nature of their
work, including the ISO 9001:2000 international consensus standard, the Good
Practices regulations and guidelines, National Environmental Laboratory Accreditation
Program, etc.
EPA QA/G-1
12
Final
November 2002
-------
CHAPTER 3
HOW TO DEVELOP A QUALITY SYSTEM
This chapter will address the following questions:
• What are the four main phases of the quality system development process?
• What is management's role in the development of a quality system?
How can organization-appropriate quality procedures be developed?
How are staff encouraged to implement a quality system?
• What do quality system experts say about developing a quality system?
3.1 OVERVIEW OF THE QUALITY SYSTEM DEVELOPMENT PROCESS
A quality system can be seen as
developing in four phases (Figure 2). In
the initiation phase, the strategic direction
is set and organizational resources are
assigned to begin the process. In the
development phase, the procedures,
documents, and tools that form the
quality system are brought together. In
the implementation phase, the system is
made operational through training and
execution of the quality system
procedures throughout the organization.
In the ongoing maintenance and
improvement phase, the system is
monitored and evaluated to ensure that it
continues to satisfy the organization's
needs and to identify and implement
opportunities for improvement.
Phase 1: Initiation
•
Phase 2: Development
•
Phase 3: Implementation
•
Phase 4: Ongoing Maintenance and Improvement
Figure 2. A Four-Phase Process for Developing a
Quality System
Embedded in each phase are activities that follow the Shewhart cycle (Deming, 2000):
Plan—Analyze the situation, develop solutions;
Do—Implement the planned solutions;
Check—Assess the results of the implementation; and
Act—Take corrective action after assessment.
EPA QA/G-1
13
Final
November 2002
-------
This cycle is based on the assumption that a quality system is not something that is performed once and
for all. Rather, a quality system evolves in four-cyclical steps, as an organization's management plans
and implements activities, evaluates the effectiveness of these activities, and then selects revised goals to
guide the next phase of the cycle. The concept of continual improvement is an implicit assumption of
the cycle.
It is EPA policy (EPA, 2000b) that environmental data operations be planned using a
systematic planning process based on the scientific method. The Shewhart cycle incorporates both a
systematic planning process in its iterative approach and the scientific method in its checking of the
results of the planned solution.
The development process involves preparing documents that describe the planned quality
system and then implementing these plans. Quality systems that operate under EPA Order 5360.1 A2
are self-certifying. No external registration body exists for these quality systems, as exists for quality
systems that comply with ISO 9001. However, assessments of a quality system (EPA, 200 Ic) that
comply with the Order determine if it is implemented and operating in the manner prescribed by its
approved QMP, and if it is consistent with current EPA policies.
3.1.1 Quality Systems for Small Organizations
The EPA quality system is characterized by the principle of the graded approach, under which
organizations base the level of QA and QC applied to an organizational area or project on the intended
use of the work product and on the confidence needed and expected in the quality of the work product.
The graded approach is also used in developing a quality system that is appropriate for the mission,
objectives, and resources of the organization that is developing the quality system. This approach starts
with the initiation phase and continues through the remaining phases of the quality system development
process. Overview of the EPA Quality System (EPA, 2002) states:
The development and implementation of a quality system should be based on a
"graded approach. " This means that the components and tools of a quality
system (Figure 2) are applied according to the scope and nature of an
organization, program, or project and the intended use of its products or services.
This approach recognizes that a "one size fits all" approach to quality
management is not appropriate and that the quality system of different
organizations and program should (and will) vary according to the specific needs
of the organization. For example, the quality expectations of a fundamental
research program are different from that of a regulatory compliance program
because the intended use of the products differs.
Final
EPAQA/G-1 14 November 2002
-------
Oakland (1993) notes that requirements for fully developed quality systems are likely to affect
those organizations employing less than 100 people. Small organizations may be unaware of how to
meet the requirements and the repercussions of establishing the quality system on its current work
processes. A small organization may perceive the cost of establishing the quality system as
disproportionate to the value of work that results from the quality system.
One solution to help a small organization develop and implement a quality system under these
conditions is to seek outside help from a professional consultant or from the governmental entity that
has oversight responsibility for the organization's quality system. During the development phase, this
help can take the form of technical assistance in preparing a QMP and standard operating procedures
(SOPs) that are appropriate for the organization. During the implementation phase, an external
assessment of the developing quality system can provide objective feedback about quality activities that
remain to be implemented. The assessors can provide technical assistance to promote the
implementation of the quality system.
The schedule for developing and implementing the organization's quality system should be
appropriate for the resources allocated and the availability of personnel to do the work. Resource and
personnel limitations do not eliminate the obligation to comply with external quality requirements, but
they may be factors in determining the rate that the quality system is developed and implemented. The
organization's senior management should consult with the government entity with oversight responsibility
about any limitations that may exist.
An organization may not have enough personnel to assign someone to implement the quality
system and conduct routine QA activities on a full-time basis. Someone from the staff may need to
function as the organization's quality manager on a part-time basis while performing other duties. A
part-time quality manager should be careful to remain objective regarding the quality system,
particularly during internal assessments of the quality system.
An example demonstrating the development of a quality system is provided in the remainder of
this chapter. The example describes a hypothetical quality system for a state environmental monitoring
program. It will be used to illustrate some of the concepts discussed in this chapter.
Quality System Development Example
Background: Quality is monitored by the State Office of Quality, but quality systems are
decentralized in individual State agencies. Each agency's quality system is designed to be
appropriate to its objectives and organizational structure.
With the promulgation of new federal environmental monitoring standards, the State
Department of Environmental Programs (SDEP) was required to establish a new monitoring
Final
EPAQA/G-1 15 November 2002
-------
program with a self-contained quality system that complies with ANSI/ASQ E4-1994. The
program would involve the gradual deployment of a large number of newly purchased
environmental sampling equipment by SDEP district offices and development of complex
analytical and data management systems in the central laboratory. Full deployment of the
monitoring network would occur over one year. The slowly increasing volume of samples
coming from the district offices to the central laboratory would allow the staff to run shakedown
tests of their systems and to uncover problems. This shakedown period also would allow the
equipment manufacturer and EPA to modify the sampling equipment and the analytical
procedures to correct these problems.
3.2 INITIATION ACTIVITIES
The initiation phase of quality system development is critical. Choices made early usually set
the tone for what follows. The organization's senior management should have a clear idea of its
purpose and goals in developing the quality system. Development of a quality system inevitably
involves some changes in the way an organization does its work, and change can be challenging.
Identifying a set of successful strategies that
will achieve the organization's goals is
important. Figure 3 shows the initiation
phase activities. These activities are
discussed further in the sections below.
3.2.1 Senior Management
Commitment
One of the basic assumptions
about a quality system is that it is primarily
a management function and secondarily a
technical function. It would then follow that
the senior management of an organization
should play a leading role in the
development of a quality system. If senior
management recognizes that the quality
system will provide real benefit to the
organization and to management itself, and
communicates this message to the staff,
there is a greater likelihood that the quality
system will be successful.
Phase 1: Initiation
Obtain Senior Management Commitment
Establish Quality System Development Team
Establish Quality Policy
Analyze Requirements
Inventory Existing Quality Procedures and
Practices
Obtain Management Approvals, Resources for
Development
Phase 2: Development
Phase 3: Implementation
Phase 4: Ongoing Maintenance and Improvement
Figure 3. Initiating the Quality System (Phase 1)
EPA QA/G-1
16
Final
November 2002
-------
Note that in this discussion the term "senior management" refers to the highest level of
management in the organization that is developing the quality system. If this organization is a part of a
larger organization, higher levels of management may exist.
If the quality system is to be useful to senior management, it should be understood by
management and address those issues that are important to management. The quality system should be
explained to management in terms of the organization's mission and goals, rather than only in terms of
external quality requirements. The anticipated benefits of achieving organizational goals and improving
the organization's performance is more significant than the existence of the quality system.
Although the impetus to develop a
quality system may originate from outside the
organization, management should see the
quality system as a necessary component of
its overall approach to running the
organization. Consequently, one of the first
milestones in developing a quality system is
to obtain senior management's commitment
to develop and implement the quality system.
This commitment should involve changes in
the basic policies of the organization and
allocation of the resources, in both funds and
staff time, to develop and implement the
quality system.
"Improvement of quality and productivity, to
be successful in any company, must be a
learning process, year by year, senior
management leading the whole
company.... Support of senior management is
not sufficient. It is not enough that senior
management commit themselves for life to
quality and productivity. They must know what
it is they are committed to—that is, what they
must do. These obligations can not be
delegated. Support is not enough: action is
required."
- Deming, 2000
Senior management will need
reasonable estimates of the resources and the
time needed to develop and implement the
quality system. These estimates should factor in the extent to which existing activities will be disrupted
and existing procedures will be changed. The preparation of these estimates should precede
management's formal commitment to develop and implement the quality system.
3.2.2 Quality System Development Team
Developing a quality system cannot be viewed as an isolated process. The quality system
affects functions and practices throughout the organization. Thus, it is important that the quality system
reflects the needs and actual practices of the organization. In some cases, the task of developing a
quality system may be delegated to one person. This is not ideal since it is not likely that a single
individual would understand all of the work processes at each level of the organization. The role of
developing a quality system is more readily filled by a quality system development team.
EPA QA/G-1
17
Final
November 2002
-------
A quality system development team is a temporary group that would exist only during the
development and implementation phases. The team should include individuals with expertise in quality
systems and individuals with expertise in the core functions of the organization. Because the entire
organization will be involved in the implementation of the quality system, key individuals in the
organization should be involved in its development. These individuals can help ensure that the structure
and components of the quality system are appropriate for the organization. They can be champions for
the system during implementation.
Careful thought should be given to the structure and composition of the team before it is
assembled. Several different and equally valid models for team structure exist and should be
considered (Brown et al, 1994; Oakland, 1993). Successful teams are composed of individuals
fulfilling different roles and having different personality types. The team may be assembled with all or
some of the following considerations:
• members are appointed by and supported by senior management;
• time spent by members in team activities is approved and encouraged by management;
membership is considered a high-priority assignment by members and management;
the team operates under a specific charge to develop the quality system;
• the team is led by one individual, who is supported by the rest of the team;
• the number of members is small enough to allow effective action;
the team includes middle managers, operational supervisors, trainers, and technical,
administrative, and quality staff from diverse core functions of the organization to
provide a thorough understanding of how work is done in all parts of the organization
and to act as quality system champions during the implementation phase;
the members are familiar with the mission, structure, and needs of the organization and
its components;
• the members are familiar, collectively, with all managerial, administrative, and technical
procedures of the organization and with basic quality concepts;
• the members have good oral and written communications skills; and
the members are even-tempered and can work with tact and reason in potentially
difficult situations.
One of the first goals for the team is to develop a general schedule for the development and
implementation of the quality system. The schedule should address all specified components of the
quality system and the organizational modifications needed to accommodate the quality system. It
should allocate time for preparation of quality documents, for internal and external assessment, and for
staff training. Deadlines for intermediate tasks in the development effort should reflect a logical
sequence of events. Table 5 shows an example of a general schedule for developing a quality system.
Final
EPAQA/G-1 18 November 2002
-------
The quality development team should assign responsibility for specific tasks to designated team
members. It also should have procedures to track development progress. Project management tools
Table 5. Example of a General Schedule for the Development of a Quality System
Major Task Name
Conduct initial assessment of existing quality procedures
Develop new quality procedures
Develop QMP
Conduct detailed compliance audit
Obtain management approval for implementation
Distribute plan to entire organization and post quality system documents on intranet
site
Conduct quality system training for staff
Implement quality system
Conduct internal assessment of quality system
External assessment of quality system
Start
Date
10/07/02
11/04/02
11/04/02
04/21/03
04/21/03
05/05/03
06/02/03
06/02/03
10/06/03
05/03/04
Stop
Date
11/04/02
01/13/03
04/21/03
05/19/03
05/05/03
05/19/03
09/29/03
09/29/03
11/03/03
06/07/04
(for example the Gantt charts described in Chapter 5), provide a way for the team to monitor how well
specific tasks are moving toward completion.
The team should maintain communications among its members. Because the team is drawn
from different subunits of the organization, individual members may not be in contact with each other
day-to-day. Some formal team communications arrangements should be developed. Periodic
face-to-face meetings or conference calls are useful for group discussions regarding development
issues. An e-mail list server can help ensure that all members are kept informed about the status of the
quality system development.
The team should also report to senior and middle management regarding the status of the
development on a regular basis. Because the quality system should be integrated into the management
system, and because development of the quality system is likely to produce changes in the
organization's policies and procedures, management will need to be kept informed about proposed
development steps and should authorize these changes. Otherwise, the team's progress could outpace
the organization's process to change procedures.
Management's participation in the development will encourage their ownership of quality, which
is important for the successful implementation of the quality system. Maintaining management's
commitment to the development process by keeping them informed is important.
EPA QA/G-1
19
Final
November 2002
-------
3.2.3 Establish Quality Policy ^^TT
1917 Huntington Monument at Newport News
A tangible accomplishment of the
team would be a documented statement of ,,TTr ,,,,,, , , ,
... , ,. ,. ^f we shall build good ships here—
mission, objectives, and quality policy. Many ., . .
. . .„, ... at a profit—if we can—
organizations will have a written mission
, , .,,, . at a loss—if we must—
statement, and others will have written 7 , , ,. „
, . . , , . but always mod ships
objectives that cover some relevant time
horizon. However, many organizations do ^ . , _ ,,1nv, ,,™ix
'. f. ,, -Dobyns and Crawford-Mason (1991)
not have a written quality policy, and the _ _ _
team may need to focus in that area.
Documentation of an organization's quality
policy is important because the process of developing such a statement may reveal those areas the
organization feels are important, and will help the team focus the quality system on those areas.
A mission statement is a short and clear expression of an organization's core principles. It
should address three key questions:
1. What does this organization exist to do? (purpose statement);
2. What activities does the organization perform to accomplish this purpose? (business
statement); and
3. What are the basic principles or beliefs that the organization shares and that guide our
activities? (values statement).
A quality policy is a written expression of senior management's overall intentions and direction for an
organization regarding quality, including objectives for quality and commitment to quality. It should
become part of the organization's formal policies and should be communicated to all members of the
organization. A quality policy informs the organization of senior management's plans to:
establish a quality system;
• identify the customer's needs and perception of needs;
• assess the ability of the organization to meet those needs economically;
ensure that procured materials and services reliably meet the required standards of
performance and efficiency;
• concentrate on a philosophy of prevention, rather than detection, of problems;
• educate and train for quality improvement; and
review the quality system to maintain progress (Oakland, 1993).
The difference between the mission and quality policy statements is in their orientation. The mission
statement addresses goals for the entire organization; the quality policy addresses senior management's
Final
EPAQA/G-1 20 November 2002
-------
specific commitments toward the quality of the organization's activities. Quality policies are
implemented by the organization's quality procedures documented in its QMP. The effort required to
develop a meaningful quality policy aligned with the organization's mission statement should help the
team understand this distinction.
"What is the quality policy? It is the state of
mind held by the company personnel
concerning how well they must do their jobs. It
is this policy, whether it has been stated or not,
that determines in advance how successfully
the next job will be done. "
-Crosby, 1979
The quality system development
team should work closely with senior
management and middle management
because management should perceive the
quality policy statement as an expression of
its own goals, objectives, and procedures.
This process should help the organization
recognize that quality is a core value and
that it is important to management. Several
revisions may be needed to produce a
statement that is based on sound quality principles and expressed in management's terms.
The team also should develop objectives for the quality system development process. The
objectives will vary with the organization and the status of its existing quality procedures. The following
are examples of tasks that help in the development of the quality system objectives:
review quality system concepts and develop an organization-specific strategy for
developing the quality system;
• identify the organization's customers (internal or external) and their quality requirements
and expectations;
conduct customer surveys to determine the degree that the organization's current work
conforms to these requirements;
• define benchmarks (i.e., a standard of excellence or achievement against which other
similar objectives should be measured or judged);
create teams to address specific operating problems;
• define a unique quality system problem-solving process; and
• create a quality system improvement plan (Hunt, 1993).
3.2.4 Analyze Requirements
The quality system development team should analyze and interpret the applicable requirements
in the quality system. The team should be familiar with the elements of a quality system defined by EPA
Order 5360.1 A2 and ANSI/ASQ E4-1994. Team members should determine which elements are
applicable to the organization. For example, the elements associated with the design, construction, and
operation of environmental technologies may not be applicable to an organization developing and using
Final
EPAQA/G-1 21 November 2002
-------
computer software. The elements that go into a specific quality system should be designed to match the
specific organization; there are no generic QMPs that are applicable to all organizations. The QMP
should reflect the mission and objectives of the organization and the existing policies and procedures for
attaining these objectives. In addition to the EPA requirements, non-EPA organizations should also
determine whether their quality system should comply with other quality standards, such as ISO 9001.
3.2.5 Inventory Existing Quality Procedures and Practices
Existing quality procedures and practices can be inventoried by conducting a self evaluation of
the organization's existing quality practices. This includes identifying existing written procedures that
address quality system objectives, identifying current undocumented quality procedures that address
these objectives, and identifying gaps where applicable objectives are not being addressed. Using
existing procedures as much as possible will help ensure that the quality system will be well integrated
into routine operations.
The purpose of this self evaluation is to obtain general knowledge of the organization's current
quality practices; a more detailed compliance audit will be conducted during the development phase.
The team needs to know if the organization already has a functioning quality system and to what extent
the quality system is documented. At this point, the team does not need to know the extent to which
procedures are actually being followed.
There are four aspects of the organization that should be investigated in the self-evaluation
(Hunt, 1993):
Climate—people's perceptions of their organization;
Processes—the organization's policies and procedures;
• Management tools—the specific techniques used to promote quality management
improvements throughout the organization; and
• Outcomes—mission accomplishment.
The steps outlined below can be taken to identify existing quality procedures during the
self-evaluation (Oakland, 1993).
• Gather existing information: locate relevant information sources from verbal inputs,
existing files, control charts, quality records, etc.; collect this information, speak to the
staff, and investigate additional sources.
Organize the collected information, which may not be in the appropriate format.
Final
EPAQA/G-1 22 November 2002
-------
Define the gaps in the collected information by asking the following questions: Is
enough information available? What further information is needed? What work
processes are undocumented? What parts of the organization are involved in these
work processes? Do any problems exist with these work processes?
Plan further information collection by: creating a list of additional information that needs
to be developed; identifying any team members who can develop this additional
information; consulting with others in the organization who are knowledgeable of the
work processes and could develop this information; and developing relationships with
other staff in the organization who can help in implementing the quality system.
The results of this self-evaluation and inventory of existing procedures and practices will
provide a basis for developing a plan for further quality system development. Details about the
technical approach, schedule, and budget should be included in the plan, as described in the next
section.
3.2.6 Obtain Management Approvals, Resources for Development
The initiation phase will be completed when senior management authorizes development of the
quality system. The quality system development team should be prepared to make a formal oral or
written presentation to management. This presentation should include:
• the specifications for the quality system;
• the results of the initial evaluation of the current status of the quality system;
a plan for the development of the quality system;
those aspects of the organization's procedures that need to be modified to conform to
EPA quality system specifications;
• documentation and procedures that need to be developed;
an estimated general schedule for the development and implementation of the quality
system; and
• an estimate of the approximate cost associated with this effort.
The quality development team should take positive steps to help senior management understand
the merits of the quality system, the need for management participation, and the precise nature of the
needed participation. Management is more likely to commit resources to the development process if
they foresee tangible benefits for the organization arising from the quality system than if they foresee
only conformance with external quality requirements. If possible, the team's presentation should
estimate the current cost of poor quality, the estimated cost reduction if the quality system is successful,
and the cost of implementing the quality system.
Final
EPAQA/G-1 23 November 2002
-------
Arriving at a rigorous estimate of the cost of poor quality for the entire organization may be
difficult. Estimates of the cost associated with specific projects may provide anecdotal evidence for the
presentation. If the team can present case histories of projects where the absence of quality planning
resulted in errors and additional costs, the benefits of the quality system may be more meaningful to
management.
An example outline for a presentation to senior management is given below.
Background
What is being proposed?
- What is a quality system?
- Why is this important now?
What are the external quality requirements?
Why invest resources? (e.g., cost of poor quality vs. cost of quality system)
• Strategic objectives
• Benefits to the organization
Technical approach
Implementation strategy;
- Task overview and responsibilities of key personnel;
- Quality system deliverables (e.g., the QMP);
Task breakdown;
Planned schedule of tasks, deliverables, and meetings;
- Staff commitment to team activities;
- Labor commitments breakdown by task; and
Budget requirements.
Quality System Development Example
Phase I: Initiation
As part of developing its monitoring program, the department assembled a team that was
charged with developing and implementing a comprehensive, yet realistic quality system that
would fit into the larger departmental quality system. The team included:
• the head of the monitoring division;
• a State Purchasing Department buyer who handles equipment purchases for
SDEP;
• a senior sampling technician from the field section in a district office;
• a chemist from the analytical services section in the central laboratory;
• a database manager from the information technology section;
• a quality assurance specialist from the quality section; and
Final
EPAQA/G-1 24 November 2002
-------
• a training specialist from the director's staff.
Once established, the team reviewed the requirements for the quality system using EPA 's
Requirements for OMP (OA/R-2) (EPA, 200la) andANSI/ASQ E4-1994. They summarized the
requirements regarding resource allocations for the program, training, assessments, reports to
management, and other topics. The team also reviewed the method description in the Code of
Federal Regulations and in supporting technical documents. They summarized the quality
requirements for procurement of acceptable sampling devices and analytical instrumentation;
for collection and transport of environmental samples; for handling, analysis, and storage of the
samples; for data management; and for assessments. Subsequently, the team met with EPA
Regional Office monitoring and quality representatives to learn how the program's quality
system would be integrated into the EPA Quality System.
The analytical chemist on the team reviewed the Analytical Services Section's written
SOPs and investigated the section's unwritten practices in light of the method's quality
requirements. This gap analysis revealed that sample handling procedures needed to be
standardized and documented. It also revealed that QC procedures needed to be improved for
some analytical techniques and that periodic performance evaluations were needed. Some SOPs
had not been updated recently and they referred to analytical instrumentation that was no
longer used in the central laboratory. After learning of the problems, the head of the analytical
services section directed the analytical chemist to update all SOPs older than 3 years. The team
met to discuss the outcome of the initiation phase activities. They prepared a brief written
report to senior management in the department regarding the quality requirements for the new
monitoring program.
3.3 DEVELOPMENT ACTIVITIES
After management has committed the organization to establishing a quality system, the
development phase can begin. This phase involves assembling and development of the various
elements of the quality system and the documentation that describes the quality system. Figure 4 shows
the activities involved in the development phase. Each activity is described in more detail below.
It is as important to maintain regular communications with senior and middle management during
the development phase as during the initiation phase. The presence of management representatives on
the team is not sufficient to ensure that all management remains informed about the development
process and that senior management remains committed to the process. The process of implementing
Final
EPAQA/G-1 25 November 2002
-------
the quality system will take time and the changes
may cause some disruption in the organization's work
processes. Management should be given enough
time to prepare for these changes.
3.3.1 Assign Tasks and Develop a Detailed
Schedule
Phase 3: Implementation
The major tasks in the development
schedule should be divided into a set of detailed
assignments. Typically, the team leader is
responsible for coordinating the team's efforts and
for meeting the deadlines of major tasks in the
development effort. Individual team members
should be given responsibility for handling specific
assignments. A detailed schedule should be
developed that shows the general schedule was
approved by senior management (see Table 6 for
an example of a detailed schedule for one
component of the quality system).
The development schedule should be
flexible enough to accommodate delays. It may be
wise to design the schedule so that individual
components of the quality system can be
developed independently and then harmonized at a
later date. Delays may occur because processes are more complex than anticipated. It is usually more
important for the team to focus on developing a quality system that functions well, rather than on
meeting an arbitrary deadline.
Phase 1: Initiation
Phase 2: Development
Assign Tasks and Develop a Detailed
Schedule
Analyze Work Processes Using a
Quality System Model
Write Standard Operating Procedures
Write Quality Management Plan
Develop Supporting Systems and Tools
Conduct a Detailed Compliance Audit
Obtain Management Approvals and
Resources for Implementation
Phase 4: Ongoing Maintenance and
Improvement
Figure 4. Developing the Quality System
(Phase 2)
Table 6. Example Detailed Schedule for Development of a Quality Management Plan
Detailed Assignment
3.
3.1
3.2
3.3
3.4
Develop QMP
Draft mission statement and quality policy
Develop outline for plan
Develop work process diagrams
Identify linkages to existing policies
Responsibility
Andrews
Andrews
Bemholt
Carson
Dayton
Start Date
11/04/02
11/04/02
11/04/02
11/04/02
11/04/02
Stop Date
04/21/03
12/02/02
12/02/02
01/13/03
12/02/02
EPA QA/G-1
26
Final
November 2002
-------
Table 6. Example Detailed Schedule for Development of a Quality Management Plan
Detailed Assignment
3.5
3.6
3.7
3.8
3.9
3.10
Distribute QMP outline for review
Review and revise QMP outline
Draft detailed QMP sections
First draft QMP submitted for review
Review of QMP by senior management
Review and revise QMP
Responsibility
Andrews
Bemholt
Entire team
Andrews
Management
Entire team
Start Date
12/02/02
12/16/02
01/13/03
02/10/03
02/24/03
03/24/03
Stop Date
12/16/02
01/13/03
02/10/03
02/24/03
03/24/03
04/21/03
Quality systems can be developed and implemented in a standard approach or a fast-track
approach (Brown et al, 1994). In the standard approach, each phase of the quality system
development process is done in sequence. By some estimates, the standard approach takes one year
for each layer of management in large organizations. Smaller organizations have fewer layers and the
quality system can be developed and implemented more quickly.
In the fast-track approach, there is considerable overlap in the phases. This approach is more
risky than the standard approach, requires a greater allocation of resources and more intensive
coordination efforts. More stress is associated with this approach because it does not allow staff to
adjust to the quality system on a gradual basis.
3.3.2 Analyze Work Processes Using a Quality System Model
One goal of a successful quality system is to improve the organization's work quality and
reduce the effort needed to do this work. If this goal is to be attained, the quality system development
team should understand how work is accomplished in the organization. Because an organization is
complex, the work should be understood as individual processes that are linked into a chain of
relationships.
For example, a chain of processes might be those involved in the review and approval of an
environmental permit. The initial process in the chain could be the handling of a permit application and
supplemental data to assemble a complete permit review package. The second process could be the
engineering review of the package to determine if the industrial facility conforms to regulatory
requirements. The third process could be the entry of data into the organization's database concerning
the approved permit. The final process in the chain could be using the information in the database to
generate a notification letter to the applicant. In each process, an input is transformed into an output
and value was added.
EPA QA/G-1
27
Final
November 2002
-------
Controls and resources also have an impact on work processes. Controls are policies and
other external requirements that specify the conditions required for the process to generate the correct
output. Resources are the people performing the transformation and the equipment they use.
Policy and Strategy
Data
Controls
External Constraints
Inputs
Material
Process
or
Activity
Data Information and Knowledge
Outputs
Equipment
and
Technology
Mechanisms
or
Resources
Products or Services
People
Each process
is represented by a
single box with
surrounding arrows
representing inputs,
controls, outputs, and
resources(see
Figure 5). The arrows
denote data or objects
related to the process.
Individual processes
can be linked together
to form process flow
diagrams (as discussed
in Chapter 5). These
diagrams are used to
represent the entire
system and the
interrelationships
among its processes to
improve understanding
of the system. Feedback loops can exist so that the output of a downstream process is an input to an
upstream process. Using such diagrams, a complex system can be broken down into a set of
interrelated processes.
The system becomes more complicated when different individuals within the organization handle
the different processes. In a system that involves multiple individuals, a single individual may not
understand the entire system and how the different processes interrelate. An individual who
understands only a single process may subtract value from the product because the overall objective of
the system is not understood.
After the system has been described in a process flow diagram, the quality system development
team can critically analyze the system to identify the specific areas where errors and poor quality may
occur. This can also determine specific points where quantitative parameters may be measured and
tracked using statistical control techniques. The team can then focus its quality improvement and
measurement efforts on the identified areas. The analysis should follow the work as it flows through the
Figure 5. Standard Diagram of a Work Process (National Institutes of
Standards and Technology 1993)
EPA QA/G-1
28
Final
November 2002
-------
system because errors that occur in the early stages of the system may not be manifested until much
later. The analysis can save resources by minimizing development efforts in those areas of the system
that are already performing well.
As the system is being analyzed, attention should be paid to the following points:
• processes should be described as objectively as possible;
• preconceived ideas about the processes should be questioned or suspended until the
analysis is complete—hasty judgements should be avoided;
all components of the system should be tested and verified;
• small details should be recorded at the appropriate level of analysis because they may
be more important to understanding the system than are the major items; and
new processes should not be considered until the undesirable or problem-causing
attributes of the existing process have been exposed.
The analysis of an organization in terms of its work processes should produce the following
results (ISO, 2000a):
• definition of the organization's activities in a manner that identifies the organization's
work products and their required quality;
identification of management and staff who are responsible for specific work processes;
identification of the carrying capacity of work processes;
• identification of critical linkages and bottlenecks between work processes;
• identification of the resources needed for the work processes; and
identification of individuals and organizations who affect and who are affected by the
work processes.
3.3.3 Write Standard Operating Procedures
During the initiation phase, the team will have identified applicable specifications of the EPA
quality system and the existing quality procedures of the organization. Team members should then
compare the procedures with the EPA specifications to determine which procedures may be used
without modification and which ones need to be revised. The modifications may be as simple as
reformatting the documents to a consistent style. However, they may be complex enough to require
changes in how work is performed in the organization. Team members should work with managers in
making the more complex revisions so that the revised documents reflect quality principles and the
existing management procedures of the organization.
Final
EPAQA/G-1 29 November 2002
-------
Write Procedures for Undocumented Work Processes
A functioning organization is likely to
"Standard operating procedures are certainly
central to organizational functioning, but staff
should be encouraged to think about them
critically. If you educate your staff so that they
are encouraged to think about the work
processes they are performing, instead of
blindly following SOPs, you may unleash all
sorts of creative energy. "
- Cohen and Brand, 1993
have developed procedures for
accomplishing most of the activities that it
performs. These procedures may be
informal and undocumented. In developing
the quality system, the team should
document any existing informal procedures
that meet the organization's quality
requirements. Team members should
interview the organization's management and
staff to investigate what procedures are
being followed. These interviews will also
help the team understand how the organization operates and may identify members of the management
and the staff who can assist in the implementation of the quality system.
Informal procedures may not be uniform across the organization. One goal of the development
effort is to harmonize the procedures to the greatest extent possible across the organization. The team
needs to be sensitive to valid reasons for procedures being different in different components of the
organization and to incorporate the valid differences in the documentation.
Guidance for Preparing Standard Operating Procedures (SOPs) (QA/G-6) (EPA, 200 Ib)
provides detailed information on the preparation and use of SOPs within a quality system. This
document describes SOPs for technical and administrative operational elements of an organization that
would be operating under a QMP and/or a QA Project Plan. Technical SOPs may document an
organization's technical activities, such as field sampling, laboratory analysis, data processing and
evaluation, modeling, risk assessment, and auditing of technical systems. Administrative SOPs may
document its administrative activities, such as contract management, document review, inspection,
training, record maintenance, data validation, and official correspondence.
Wieringa et al. (1998) provides detailed information about the principles and practice of
procedure writing. This process consists of the following seven steps:
1. Plan—What are they writing? Why are they writing it? What resources will they need?
2. Investigate—The writers do preliminary research to collect information and develop
ideas about what they will be writing.
3. Organize thoughts—Outlining is a useful organizational tool; a flow chart can help the
writer analyze the process in a systematic fashion.
4. Draft—Write a draft of the SOP.
Final
EPAQA/G-1 30 November 2002
-------
5. Review—The draft SOP should be reviewed by the writers, technical experts, users,
and management and revised to improve accuracy and readability.
6. Test—The SOP should be tested by a typical user who will perform the process as
specified in the SOP while the writer observes which SOP sections causes problems.
7. Maintenance—After the SOP has been successfully implemented, it should be revised
to incorporate any changes that have been made in the process.
The overall process should also include routine periodic review of SOPs to ensure that revisions are
incorporated in a timely manner.
Design and Document New Procedures that Fill in Current Gaps
"One of the best-documented quality systems
the author has ever seen was a small hand-tool
company. It possessed an excellent quality
manual, beautifully laid out in sections
covering each paragraph heading of ISO 9002.
Each procedure described exactly how
compliance with the standard was achieved,
and identified the responsibilities and
authorities of the individuals concerned—but it
was a work of fiction! It did not bear any
relation to what actually happened. "
-Oakland, 1993
The most difficult procedures to
develop are those that do not exist in the
organization in any form. Management and
staff of an organization may not perceive the
need for procedures that address previously
undocumented and unregulated aspects of
their work processes. They may not see the
benefits to be obtained from the new
procedures. At a more fundamental level,
they may not have critically analyzed the
work processes from a Quality Systems
needs perspective and may not be able to
articulate any recommendations for the new
procedures. The team members may wish
to develop several alternative procedures for
their consideration. The team should consider
procedures that are being developed.
3.3.4 Write Quality Management Plan
After an organization's mission, quality policy, and quality procedures have been developed by
the team, they should be documented in a QMP for the organization. The elements of a QMP are
specified in EPA Requirements for Quality Management Plans (QA/R-2) (EPA, 2001 a). See
Section 5.3 for more information about the elements of a QMP. Organizations external to EPA may
refer to this document as a "quality manual" or a "quality plan."
The value of preparing a QMP is not only in the document prepared, but also in the systematic
planning that should be involved in preparing the document. The formal process of developing the
how to apply the graded approach to the new
EPA QA/G-1
Final
November 2002
-------
mission statement and quality policy, and of preparing SOPs and the QMP, requires that management
and the team perform a critical assessment of the organization's goals and the methods used to attain
those goals. The fact that this process occurs will help ensure the successful implementation of the
quality system. The plan can be viewed as the organization's documentation that systematic planning
has occurred. Conversely, a plan developed without systematic planning and consultation may be
difficult to implement.
As the plan is being prepared, the team should continue to work closely with middle and senior
management to ensure that the quality procedures accurately represent work processes in the
organization and that the work processes meet the quality requirements. Any team-initiated
modifications of work processes should be approved by management before they are documented in
the plan. The plan should describe either the quality procedures as they currently exist in the
organization or the quality procedures that senior management has committed to have implemented in
the organization.
Although there is no required format for the plan, a plan organized to follow the structure of the
EPA Requirements for QMP (R-2) (EPA, 2001a) is recommended. The plan is more likely to
address all the necessary quality system elements, and reviewing it will be easier for external readers if
the elements of a QMP are presented in a standardized format.
Deciding how much detail is needed in a QMP will depend on the level of detail in the
administrative and technical SOPs. If the SOPs document the organization's work processes in a
comprehensive fashion, the plan can be a relatively brief document (e.g., 25 to 30 pages). Together,
the plan and the SOPs should provide the management and staff with a complete set of instructions to
implement the organization's quality policy.
The plan and the SOPs should also provide enough information about an organization's quality
procedures to serve as the written criteria for an external assessment of the organization's quality
system. Guidance on Assessing Quality Systems (QA/G-3) (EPA, 2001c) provides information
about the criteria for assessments of quality systems.
Oakland (1993) describes questions to ask about each element of the plan:
"What specific activity should be done to meet the requirement?
Why does it have to be done? (or why is it done that specific way?)
Where is the requirement met?
When is the requirement met?
Who is responsible for doing it?
How is it done? (or "What SOPs are used? " or Can it be done in another way?)"
Final
EPAQA/G-1 32 November 2002
-------
EPA's Quality Staff developed a checklist for reviewing QMPs (see www.epa.gov/quality/qs-
docs/qmp-checklist.pdf). This checklist can be used by the team to help ensure that the plan contains
all the necessary elements.
3.3.5 Develop Supporting Systems and Tools
The team will need to develop or acquire any new tools or systems needed to support
implementation of the quality system (e.g., document control and a records management information
system). Chapter 5 describes some tools and methods that can be useful when developing quality
systems.
The EPA Quality Manual (EPA, 2000) defines policies for quality-related documents and
records. The team should establish procedures for control and management of these documents and
records as it develops the quality system. It should also investigate the document and records
management system that is appropriate for the quality system and the organizational structure. The
team will have to address issues such as the following:
the extent that the document and records management system will be centralized and
standardized;
whether the system will operate in paper or electronic format or some combination;
software that needs to be obtained to support the system;
provisions to give management and staff access to quality documents and records;
provision for long-term storage of quality documents and records;
the scheme for organizing quality documents and records in the system; and
resources and responsibilities for operating the system.
3.3.6 Conduct a Detailed Compliance Audit
After the QMP has been completed, it should be checked against what is actually being
performed in the organization. This assessment should be more detailed than the evaluation conducted
during the initiation phase. Although the team prepared the QMP, it should be objective when it
assesses how well the organization conforms to the plan and to the requirements that underlie the plan.
The team should determine if the organization follows the existing procedures. Techniques used in
assessments of a quality system (EPA, 2001c) should be used for this internal assessment.
A detailed self-assessment checklist should be developed, based on the QMP. A checklist
allows the team to compare current procedures with the organization's requirements in a systematic
manner. It will also help the team identify existing documents that address specific requirements of the
organization.
Final
EPAQA/G-1 33 November 2002
-------
The assessment should be more than
just a review of documents relating to quality
procedures in the organization because the
documents may not represent how procedures
are actually performed in the organization. The
team should speak with management and staff
to determine if the existing procedures are
actually followed in routine activities. The
team should interview multiple individuals
having diverse job functions in different parts
of the organization.
3.3.7 Obtain Management Approvals
and Resources for Implementation
"A typical example was that of a quality
manager who proposed to introduce a
comprehensive formalized quality control
program into the company... He had gotten a
weak reception from the line manager but was
thoroughly sold on his own proposals. His
senior managers had then told him, in effect,
to secure the support of the line managers
first. Again I played the role of critic, and
asked him to explain to me just what would be
the tangible effects of his proposals... He was
unable to be specific enough to convince me
that the present system (or lack of system) had
enough deficiencies to warrant taking a major
step into the unknown. I later confirmed that
his senior managers had much the same
judgment."
-Juran, 1999
The plan should be presented to senior
management for approval and subsequent
distribution as the formal policy of the
organization. The team should demonstrate to
senior management that the plan is reasonable,
that it will be accepted by the middle
management and staff, and that it will produce benefits. If the plan passes these hurdles, senior
management should review and approve it after any necessary modifications have been made.
Management should make it clear to the organization that the plan is to be implemented and should
manage the organization in accordance with the plan.
Quality System Development Example
Phase 2: Development
The quality system development team reviewed its progress and developed a detailed
schedule for the remaining tasks. Each team member agreed to review their assigned work
processes and develop written procedures, if necessary.
The database manager reviewed the data management process, from sample media
preparation to the reporting of final analytical results. Commercial software was used to
generate process flow charts showing the flow of data through the system and the time
requirements associated with each data-handling component of the system. This analysis
identified three areas where bottlenecks or errors were most likely to occur in the data
management system:
EPA QA/G-1
34
Final
November 2002
-------
the requirement for "just-in-time " shipment and tracking of limited shelf-life
sampling media;
the integration of sample data, field data, and analytical data which were in
different formats; and
documentation of laboratory holding times between the receipt and the analysis
of limited shelf-life samples.
The database manager also determined which data quality problems from the field and
laboratory components of the method needed to be linked with the final analytical results for
specific samples. A database structure that included these indicators was developed.
After all team members had developed written quality procedures for their work
processes, the QA specialist began work on assembling the QMP based on these procedures. The
group's analysis of all the work processes had revealed that the manual entry of data on
hard-copy forms was a significant bottleneck in the timely handling and analysis of the limited
shelf-life samples in all components of the program. The head of the monitoring division decided
to purchase computers and bar-code scanners to track the samples from the preparation of the
sample media through to the entry of the final analytical results in the database. The automated
sample tracking system was addressed in the QMP's elements associated with management and
organization, computer hardware and software, planning, and assessment and response.
The SDEP QA specialist conducted a compliance audit of the district offices to compare
their actual quality procedures against those listed in the draft quality manual. One district
office had not yet prepared its chain-of-custody procedures for samples and had not yet
identified staff members who would be operating the sampling sites in the district. It appeared
that this district office was under staffed and was having difficulties accomplishing its existing
duties. Upon receiving the report of the compliance audit, the head of this district office
reassigned staff responsibilities to allow time for a senior technician to develop the district-level
quality procedures for the new monitoring program and to operate the sampling sites.
The team held an interim progress meeting and prepared a brief report for management
on the details of the development phase activities as well as the remaining tasks.
3.4 IMPLEMENTATION AND START-UP ACTIVITIES
After developing the elements of the quality system, the system is put into practice during the
implementation phase. The method of implementation can have a major influence on its success.
Figure 6 shows the main activities involved in implementing a quality system. The following sections
describe these activities in more detail.
Final
EPAQA/G-1 35 November 2002
-------
During the implementation phase,
management and staff should develop a clear
understanding of the quality system procedures
and policies of the organization. They should
understand EPA's quality requirements in
general terms and the specific quality procedures
for their own work processes. An understanding
of when, and under what circumstances, specific
quality documents are to be prepared and to
whom these documents should be sent for
review and approval is also needed.
Management should also have an understanding
of how to implement and document the quality
procedures and how to report the results of
these procedures. They should be aware of the
need to look for ways to improve the quality of
the work processes. Finally, they should be
prepared for internal and external assessments of
the work processes and for corrective actions
arising from the assessments.
Middle managers may face unique
problems during implementation. Even with
support and guidance from senior management,
work processes under their supervision will be
changed in the midst of maintaining normal work
flow. These managers should receive the active
support of the team to implement the quality
system for their processes successfully.
Phase 1: Initiation
Phase 2: Development
Phase 3: Implementation
Determine the Implementation Strategy
and Schedule
Publish and Distribute Quality
Management Plan and Quality
Procedures
Training
Communications
Initiate the Quality Procedures and
Maintain Records
Conduct Periodic Internal Assessments
of the Quality System
Corrective Actions
Identify Improvements
Phase 4: Ongoing Maintenance and
Improvement
Figure 6. Implementing the Quality System
(Phase 3)
3.4.1 Determine the Implementation Strategy and Schedule
The team should prepare a detailed schedule for the implementation of the quality system based
on a coherent strategy. This strategy should take into account the circumstances and unique
characteristics of the organization and should be consistent with the overall quality system development
effort.
The third step in the Shewhart cycle, "Check," involves evaluating the validity and worth of
proposed solutions. The implementation should be designed to provide opportunities for early
feedback and corrective action loops so that newly developed quality procedures can be checked for
EPA QA/G-1
36
Final
November 2002
-------
improved performance results. If appropriate and feasible, the team may want to consider conducting a
pilot study in a target program or division. Complications or oversights may be identified and corrected
before investing in the full implementation of the quality system by testing some of the implementation
activities, such as training and quality procedures. This approach gives senior management a way to
assess the validity of the procedures before they are implemented throughout the organization and can
also provide some initial estimates of the costs and benefits of the procedures. Whether or not a pilot
study approach is taken, the strategy should have the potential for significant improvements over a short
period or have a large positive impact across the entire organization.
"While you are attempting to get your staff to
adopt this new way of working, your
organization will still need to complete its
assigned tasks. Expect a long transition
period, lasting several years, where you
gradually replace one way of working with
another."
-Hunt, 1993
Although it is quite possible that the
actual implementation process will deviate
from the original schedule, developing the
schedule is still important. The schedule is a
tool for the team to track the process and
determine if unforseen problems are
occurring. Management will need tangible
milestones to justify the expenditure of
resources needed to implement the quality
system and to monitor the team's progress.
The process of developing a detailed schedule encourages the team to address the
implementation process in a rigorous fashion. The steps in this schedule should involve tangible
measurable activities that can be verified by an external observer. The implementation process should
be broken down into steps that are small enough to allow reasonable tracking of the process. Separate
schedules can be used for different subgroups in the organization.
The implementation schedule should be realistic. Full implementation may require several years.
It is important that the team not expect the process to occur swiftly and that they do not become
disillusioned because of unrealistic expectations. They should be prepared and be committed to all
efforts needed to achieve implementation. Management should be aware of the schedule and willing to
support the team during the entire implementation process.
During the initiation phase, the team may have identified individuals who are potential
champions of the quality system because of their attitudes or the strategic positions that they occupy. A
champion has the following characteristics:
power, influence, and resources to get things rolling and maintain momentum;
respect of the management and staff; and
interpersonal skills needed to persuade people to take action (Carr and Liftman, 1990).
EPA QA/G-1
37
Final
November 2002
-------
These individuals are logical choices to check the quality procedures during the early phases of
implementation or during a pilot study. They are likely to be objective about the procedures, and their
assessment of the validity of the procedures likely will be credible to the rest of the organization.
The team should be careful to involve the managers of the champion in the process (Carr and
Liftman, 1990). These individuals are possibly being asked to participate in the quality system before
they fully understand it. The team should inform them about the quality system and involve them in the
early assessment. Information gained from the early assessment or pilot study will allow the team to
modify the procedures during the later stages of quality system implementation.
3.4.2 Publish and Distribute Quality Management Plan and Quality Procedures
The QMP and the quality procedures should be known and accessible to all staff if the quality
system is to be successfully implemented. The initial publishing and distribution of the QMP and quality
procedures should be coordinated with the initial training and communications from management.
A document control system should ensure that the staff has the most current versions of the
QMP and quality procedures. However, the document control system should not become a paper
bureaucracy that expends excessive effort on managing the quality system documents. The documents
should be readily accessible to staff in a format useful to them.
If the organization has an internal website, the team should consider posting and maintaining
quality system documents online. This approach to document distribution allows these documents to be
accessible to everyone in the organization. The documents can be updated without the need to
distribute paper copies throughout the organization.
3.4.3 Training
The introduction of the quality
system may require staff to change the work
processes that are familiar to them. They
may be asked to do new things using new
techniques and new terminology. Training
will help staff obtain the conceptual tools
needed to implement the quality system.
Training should be a normal work
process in an organization and should be
integrated into other work processes as part
of the overall effort to develop, implement,
"It is the author's belief that training is the
single most important factor in actually
improving quality, once there has been
commitment to do so. For training to be
effective, however, it must be planned in a
systematic and objective in manner. Quality
training must be continuous to meet not only
changes in technology but also changes in the
environment in which an organization operates,
its structure, and perhaps most important of
all, the people who work there. "
- Oakland, 1993
EPA QA/G-1
38
Final
November 2002
-------
and continuously improve the quality system. Staff and management need to see the quality system as
an integral part of the organization and as something that has an impact on their day-to-day activities. If
staffs only interaction with the quality system is an annual visit from an assessor, they are likely to
regard the quality system as an unnecessary diversion from their work.
Everyone in the organization should receive training on the quality system. It is unlikely that
generic quality system training courses can address the specific needs of each organization. Similarly, it
is unlikely that a single training course customized for an organization can address the specific needs of
every individual in the organization. The amount of training and the topics to be included in the training
should depend on each individual's role in the organization and on their specific training needs.
Everyone should also know the basics of the quality system as well as the underlying reasons for
implementing the quality system. Everyone should also know the quality procedures needed for their
specific jobs. More specialized quality training, such as document control procedures or statistical
techniques, should be taught to those individuals who can apply these procedures in their jobs.
Information about developing and implementing a quality training program is available in Guidance for
Developing a Training Program for Quality Systems (QA/G-10) (EPA, 2000c).
Quality training should have short-term objectives (e.g., what and how) and long-term
objectives (e.g., why). Short-term training should focus on an overview of the quality system and the
specific quality procedures that individuals need to know immediately to do their work under the quality
system. Staff will be able to apply this practical knowledge immediately and better retain the
knowledge that they apply. Long-term training should focus on the overall quality system and the
quality principles that underlie it. The staff is likely to appreciate and retain this theoretical knowledge
after they have mastered the practical aspects of the quality system. Both types of training are
necessary; long-term training should not be deferred indefinitely.
Training should be provided at all levels of the organization. Neglect of training at any level can
delay, and perhaps prevent, implementation of the quality system. Senior Management should be
shown how to define the quality policy and objectives, how to establish the organizational structures
needed to implement the quality system, how to clarify the authority of the quality system, and generally
how to create the atmosphere in which the quality system will thrive. Quality Assurance Staff should
know how to assist management and staff in implementing the specific quality procedures that each
individual needs to know and how to promote the ownership of the quality system by the management
and staff. Middle management should know the technical skills needed to plan, implement, and assess
changes in work processes under their control to implement the quality system. Supervisors should
know general quality principles, the reasons for the quality system, and their role in the quality system.
At the end of their training, supervisors should be convinced of their own senior management's
commitment to the quality system. All other staff should know the basics of the quality system and the
specific quality procedures needed to perform their duties.
Final
EPA QA/G-1 39 November 2002
-------
Common approaches to training include using designated in-house trainers, cascaded training,
and training by outside consultants (Goetsch and Davis, 1995). The most common approach is to train
designated trainers in quality principles and procedures and then have them train the rest of the
organization. This approach has the advantage of consistency of instruction and can be continued
indefinitely as on-the-job training. Cascaded training involves passing the training down from senior
management to middle management and from middle management to the rest of the organization. This
approach has several advantages:
individuals are likely to work harder at learning if they expect to teach the subject
matter to others;
preparations for teaching and the actual teaching itself reinforce the teacher's
knowledge of the subject; and
when managers teach, it shows that they are serious about the subject matter.
One disadvantage of cascaded training is that individuals differ in their abilities to be effective
teachers. The third approach is to use an outside consultant to do the training. This approach may be
most useful early in the implementation process until an in-house capability can be developed. The
disadvantage of this approach is that the outside consultant may not be familiar enough with the
organization to tailor the training for the specific structure, procedures, and culture that exists in the
organization.
Mentoring is also an effective training approach to supplement formal training. Members of the
development team and other champions of quality identified by the team can train others in quality
principles and procedures as part of their everyday activities. Mentors can transfer specific information
about quality principles and procedures needed to accomplish the task at hand. The training will be
fresh when it is applied in a real setting. This approach can reduce training costs associated with
presenting information on quality principles and procedures that have no applications for specific
individuals.
3.4.4 Communications
Communications are another way to
demonstrate senior management's
commitment to quality. The staff should hear
a clear and direct message about the
importance of the quality system,
management's support of its development,
and the staffs role in its development and
implementation. Initially, management could
prepare and disseminate the mission
"Honest, open communication is probably the
single most important factor in successfully
creating a quality management environment. It
will take time, but it can lead to trust and
mutual respect, and it can sometimes be the
only thing that keeps the effort alive. If people
keep talking to one another, they can work
through problems, overcome barriers, and find
encouragement and support from others
involved in quality management efforts. "
-Hunt, 1993
EPA QA/G-1
40
Final
November 2002
-------
statement and perhaps describe the QMP. The infrastructure for developing and implementing the
quality system could be described, and subsequent messages would describe the quality system in
greater detail, the implementation schedule, and resources that are available to the staff regarding the
quality system.
Communications and training should address both the technical and human sides of the quality
system. The staffs first question concerning the quality system is more likely to be "How will the
quality system affect me?" rather than "How will the quality system change the work process?"
It is likely that the staff has heard about the quality system before it was formally announced,
and they may have misconceptions and fears about it and its impact on their jobs. They may believe
that the quality system is being introduced because someone has decided that their performance is
deficient. Unless training and communications correct this misconception, staff may resist
implementation of the quality system. Staff should be reassured that the overall organization, rather than
individuals, is the focus of the quality system.
3.4.5 Establish a Program for Recognizing Quality Performance
Recognition by management of an employee's good work can take oral or written form. It
provides motivation and support for the employee, encourages others to do likewise, and improves
productivity. Effective recognition has the following characteristics:
3.4.6
it is specific so that the employee
knows the actions that are
recognized;
it is directed at the right person;
it is genuine;
it is given closely following the
recognized activity; and
it focuses on the quality procedures
that are being encouraged, rather than
on the specific results of following
those procedures.
Initiate the Quality Procedures and
Maintain Records
"The success of quality management is
determined, in large part, by the degree
of importance the organization places
on it. Recognition is one of the most
important ways to reinforce a proactive,
positive change in behavior as it relates
to quality improvement. Recognition is
given for the successful application of
the quality management principles and
practices"
-Hunt, 1993
After the quality plans and procedures have been distributed and staff has been trained, the
actual use of those procedures should be initiated, and quality records maintained as objective evidence
that the quality system is functioning properly. As with any other major change, the quality system will
EPA QA/G-1
41
Final
November 2002
-------
not be implemented without problems. Management should plan to prevent the foreseeable problems
and quickly address the unforseen problems that inevitably arise. Management and staff should be
ready to implement the quality procedures that have been distributed. They should be aware of the
results of any pilot studies that were conducted.
Administrative and logistical support for the quality system needs to be in place to handle the
flow of paperwork that will be generated by the quality system. For example, the authors of QA
Project Plans need to know who to send the plans to for review and approval.
The quality system policies should define what quality-related records need to be maintained.
Quality records provide objective evidence that work processes conform to the quality procedures
(i.e., what has been done). These records are different from quality procedures, which specify what
should be done. Written procedures should specify how quality records are to be prepared, reviewed,
stored, and disposed. The retention time for quality records should be specified. Examples of quality
records may include inspection reports; calibration test results; corrective action forms; review and
sign-off sheets for reports, plans, and procedures.
3.4.7 Conduct Periodic Internal Assessments of the Quality System
A periodic internal assessment is a process for assessing an organization's practices as they
relate to its quality system. The focus of the assessment is on the quality system. It does not judge the
quality of data and information to support an individual decision nor does it judge performance or
competency of personnel. These assessments are designed to assess the organization's quality system
and provide an unbiased and objective source of feedback about the quality system. The assessment
seeks to determine if a quality system is implemented and is operating within an organization in the
manner prescribed by the approved QMP and consistent with current EPA policies (EPA, 200Ic).
The assessment includes quality
system document review, file examination
and review, and interviews of managers and
staff responsible for environmental data
operations. The assessment focuses not
only on recognizing the effectiveness of a
quality system and noteworthy
accomplishments, but also on the
identification of nonconformances and
needed improvements.
The purpose of periodic assessments is to determine the adequacy and effectiveness of the
quality system being applied to environmental data operations conducted by or for EPA. Because the
"Self-assessment allows an organization clearly
to discern its strengths and areas for
improvement by focusing on the relationship
between the people, processes, and results.
Within any quality-conscious organization it
should be a regular activity. "
- Oakland, 1993
EPA QA/G-l
42
Final
November 2002
-------
Agency's decisions rely on the quality of environmental data, it is imperative that the effectiveness of
QA implementation is assessed periodically. Assessments are tools for determining the adequacy and
effectiveness of the quality system applied to environmental data operations.
An assessment can answer the following questions:
Does the organization understand what it has to do to meet requirements, particularly
those given in the Order and related federal regulations?
Does the organization do what it says in its documents, particularly in its QMP?
Does the quality system work as designed to support environmental decision making
with environmental data that are sufficient in quantity and quality appropriate for their
intended purpose?
3.4.8 Corrective Actions
Some of the findings from the internal assessments will indicate work processes in which quality
practices can be improved. The assessors may have found that the quality procedures are not being
followed. Their analysis of the situation should have addressed both the work processes and the quality
procedures. In some cases, the staff may have encountered an impractical quality procedure that
looked reasonable during planning, but an alternative procedure that works or works better. It is also
possible that the quality system development team may have had an incorrect understanding of a work
process while the quality procedure was being developed. There may be external determinants that
prevent the application of the quality procedures. In the absence of extenuating circumstances, the staff
may be unwilling or unable to implement the quality procedure. Appropriate steps need to be taken to
ensure that reasonable quality procedures are used in the quality system, that the staff can adopt the
quality practice, and that the work processes have outputs that conform to EPA policies and
adequately support environmental decisions.
Deming (2000) was careful to distinguish between variability of a process that arises from
common causes inherent within the process and variability that arises from special causes from outside
the process. The former is a normal characteristic of the process and does not require corrective
action. The latter represents an out-of-control situation that requires corrective action. This distinction
is the basis of statistical process control. Deming notes that trying to correct a work process
experiencing normal variability—called "tampering"—is counterproductive and may lead to greater
variability. Deming's distinction is as valid for processes that are not easily quantified as it is for
industrial processes for which many copies of the same item are produced. Distinguishing the two types
of variability is harder. Corrective action should not be taken unless it is necessary.
The development and implementation of corrective actions are the responsibility of the
management and staff who are involved in the work process. The quality system development team or
Final
EPAQA/G-1 43 November 2002
-------
the auditors can provide recommendations and technical assistance in developing appropriate
corrective actions, but the work process does not belong to them. Any corrective action imposed from
the outside is likely to be resisted. The implementation of the corrective action should be documented
and maintained with other quality records.
3.4.9 Identify Improvements
The fourth step in the Shewart Cycle involves adjusting the quality system based on information
obtained during the assessments. As the quality system is being implemented, the team will interact with
management and staff and will obtain some feedback about the effectiveness of the various components
of the quality system. This information should be used to revise the QMP and the quality procedures as
necessary to improve the overall process.
It is likely that the quality system as implemented will differ in some respects from the quality
system originally planned and documented. These differences are not necessarily deficiencies because
the team may not have perfect knowledge of the organization's work processes. The quality system
documentation should be modified to describe the quality procedures that work for the organization.
Below are four strategies that can be followed to improve the quality system:
1. Collect meaningful data about the work process that are free from errors of
measurement or procedure and that have a direct application to the work process.
2. Identify root causes of problems, rather than the symptoms.
3. Develop appropriate solutions based on meaningful data and applied to the root causes.
4. Make thoughtful changes after careful and deliberate planning and foresight and with
adequate resources, rather than reacting too quickly (Goetsch and Davis, 1997).
By modifying the quality system based on feedback from management and staff, the team will help
ensure that the organization will see that it has ownership of the quality system.
Quality System Development Example
Phase 3: Implementation
The Quality System development team identified several possible strategies that could be
followed to implement the quality system. The "blitz " approach would involve adoption of the
quality system across the entire division at one time.
A gradual approach would involve the adoption of the quality system by different
sections. Because federal over sight was focused on individual monitoring programs, and
because quality systems in the State were decentralized, the monitoring division head decided to
Final
EPAQA/G-1 44 November 2002
-------
implement a quality system that would involve only those individuals participating in the new
program. The sampling, analytical, and data management components of the program would all
implement the quality system at the same time.
The division's QA specialist conducted periodic internal assessments of the monitoring
program's quality and technical systems. These assessments were conducted in the central
laboratory and the district offices. Performance evaluations were conducted in the central
laboratory and at individual monitoring sites. The quality representative from the EPA regional
office conducted external assessments of the program's quality and technical systems after the
first year of operations and at biannual intervals afterwards. Performance evaluation samples
were sent to SDEP from the EPA National Monitoring Laboratory on a quarterly basis.
During the performance evaluations at individual monitoring sites, the division's QA
specialist found that the sampling apparatus' calibration drifted outside of control limits for a
significant number of the network's sites on multiple occasions. This calibration drift caused
many measurements to be invalidated. Further investigations at the central laboratory revealed
a design flaw in the sampler. The samplers' manufacturer was contacted about the problem,
and a modified part was designed that brought the calibration drift back into control.
3.5 ONGOING MAINTENANCE AND IMPROVEMENT
At this stage, the quality system development team generally transfers responsibilities to a more
permanent group within the organization. The organization should develop the infrastructure to maintain
and improve the quality system as appropriate for the organization. It could involve a centralized quality
group or a decentralized group of individuals who are responsible for quality for specific work
processes.
Phase four of the quality system development process fundamentally involves activities to
establish the Shewhart cycle for continuous improvement. Figure 7 shows the activities in Phase 4, and
are explained in more detail below.
3.5.1 Allocate Resources per the Budget Cycle
After the initial implementation of the quality system, additional resources should be allocated
for its maintenance and improvement. Resources should be allocated for nonroutine activities, such as
new programs in the organization that should be integrated into the quality system. Resources should
also be allocated for ongoing quality systems training for management and staff.
Final
EPAQA/G-1 45 November 2002
-------
3.5.2 Conduct Routine Quality
Assurance
Ongoing maintenance activities can
involve a number of quality assurance activities.
These may include reviews of quality
documents, such as the organization's QMP
and the QA Project Plans for environmental
data collection activities. Activities may also
involve quality system audits and technical
systems audits and preparation of QA Annual
Report and Work Plans.
3.5.3 Implement Improvements
The Shewart cycle is based on the
concept of continual improvement; work
processes are refined to reduce errors and
variation in the quality of the product or
service. Deming (2000) expressed this
concept as: "Improve constantly and forever
the system of production and service, to
improve quality and productivity, and thus
constantly decrease costs."
Phase 1: Initiation
Phase 2: Development
Phase 3: Implementation
Phase 4: Ongoing Maintenance and
Improvement
Allocate Resources per the Budget
Cycle
Conduct Routine Quality Assurance
Implement Improvements
Conduct Periodic Self-assessments of
the Quality System
Maintain Quality Training,
Communications, and Recognition and
Rewards Programs
Figure 7. Maintaining and Improving the
Quality System (Phase 4)
Continual improvement is focused more on improving the process, rather than on improving the
output from the process. Japanese quality experts, such as Kaoru Ishikawa, see a quality system more
as an ongoing process that results in improving quality without a specific endpoint (Beckford, 1998).
Under Ishikawa's model for quality, continual improvement consists of slow, incremental improvements
in quality. All involved with operating a process are encouraged to find ways to improve the process
and are given the tools and management support to develop and implement the improvements. Product
and service quality will improve as a consequence of these improvements.
This concept is based on the assumption that many small changes to an existing process can
produce larger cumulative quality improvements than can a few radical changes. This assumption may
not be valid if the process cannot be further improved or if innovative techniques would require a
complete redesign of the process.
If continual improvement is to become an effective technique, there must be some way to
monitor the output of the process and to detect small changes in the quality of the product or service.
EPA QA/G-1
46
Final
November 2002
-------
For example, the contamination level in a blank sample might be a parameter that could be monitored in
an analytical laboratory. In an organization involved in reviewing and approving permit applications, the
time between the receipt of the application and the notification of the applicant regarding permit
approval might be a quality parameter to monitor. This approach requires that enough data has been
collected about the process to allow small quality improvements to be detected amid the normal
variability of the parameter.
"Putting out fires is not improvement. Finding
a point out of control, finding the special cause
and removing it, is only putting the process
back to where it was in the first place. It is not
improvement of the process. You are in a hotel.
You hear someone yell fire. He runs for the fire
extinguisher and pulls the alarm to call the fire
department. We all get out. Extinguishing the
fire does not improve the hotel. That is not
improvement of quality. That is putting out
fires."
- Deming, 2000
Techniques for continual
improvement may have to be altered for
research and development organizations
because each product of the process is
unique. For these organizations, the best
approach is to look for ways to monitor the
progress of the work constantly so that
refinements can be made. Continual
improvement becomes ongoing course
corrections the first time a path is taken,
rather optimizing a path traveled previously.
For example, a researcher might reduce and
analyze data as it is being collected to
determine if data quality objectives are being
attained, rather than waiting until the end of
the research project to assess whether the
entire data set attains the objectives. Another way to improve a research project is to review the
procedures used to conduct the work continuously. A researcher may find that a data collection or
data analysis technique can be streamlined or that a previously unrecorded measurement parameter
yields important insights into the phenomenon being studied. There are opportunities to improve
research and development processes if these processes are critically monitored.
3.5.4 Conduct Periodic Self-Assessments of the Quality System
The organization should conduct periodic self-assessments of the quality system to ensure that
the organization is functioning as described in the QMP. The results of these assessments may lead to
corrective actions for nonconforming processes and also may identify opportunities for improvements in
the system.
3.5.5 Maintain Quality Training, Communications, and Recognition and Rewards Programs
Training, communications, recognition, and rewards are essential to the long-term success of
the quality system. They are ways to demonstrate senior management's continued commitment to the
EPA QA/G-1
47
Final
November 2002
-------
quality system. Regular communications, recognition, and training about the quality system help to
decrease resistance to its implementation.
Ongoing training should involve senior and middle management and staff. It should involve new
employees and current employees who were trained during the implementation phase. The training
should be tailored to each group's specific quality responsibilities in the organization.
Ongoing training should be provided because the organization and the quality system are
evolving. This evolution can be considered as a three-step process as outlined below:
1. Analyze work processes to improve them.
2. Experiment with and adopt new ways of performing tasks.
3. Implement new work processes by training the management and staff to do the work in
a new way (Cohen and Brand, 1993).
Quality System Development Example
Phase 4: Ongoing Maintenance and Improvement
As part of its routine QA activities, laboratory analysts maintained routine control
charts for QC indicators, such as instrument calibration drift, QC spike recoveries, and
laboratory and field blanks. Review of these control charts revealed consistent field blank
problems associated with a specific sampling site. The SDEP QA specialists usedPareto charts
to investigate QC problems for this site. They discovered an unusually high incidence of
temperature control problems for the sampler at the site. Senior sampling technicians
investigated the sampler and found that the temperature-control printed circuit board had
failed. This discovery prompted the sampler manufacturer to redesign the printed-circuit board
to make it less vulnerable to power line voltage spikes. The incidence of field blank problems
decreased after the redesigned board was installed in all samplers in the network.
The quality system development team analyzed the quality training needs associated with
the new method. They decided that these needs would be met by a combination of existing
departmental quality training and additional training that would be specifically tailored to the
new method. All new SDEP staff are required, as a condition of their employment, to take a
one-day basic QA course given annually by SDEP's QA specialists. A small group of senior
district field technicians would be trained by the new sampler's factory representatives in
operating and maintaining the sampler and in its QC procedures. The senior technicians would
then return to the district offices where they would travel to individual monitoring sites to deploy
the samplers, train junior technicians, and begin routine sampling. All laboratory analysts and
information technology specialists in the central laboratory would attend a multi-day training
class given by the EPA Regional Office on the method's analytical procedures.
Final
EPAQA/G-1 48 November 2002
-------
Summary
The development of a quality system can be viewed as occurring in four main phases:
1. Initiation
2. Development
3. Implementation
4. Ongoing maintenance and improvement
The initiation phase is the first step in building a quality system and requires obtaining
management commitment, establishing the quality team, writing quality policy, and obtaining resources
for developing the quality system. The development phase includes writing quality documents and
SOPs, assigning project tasks and responsibilities, developing system tools, and conducting audits. The
implementation phase requires publishing and distributing quality plans and SOPs, training staff, record
keeping, and making quality improvements to Agency work processes. The ongoing maintenance and
improvement phase requires routine QA activities, periodic system audits, continual quality
improvements, and ongoing training, communications, recognition, and rewards.
Final
EPAQA/G-1 49 November 2002
-------
Final
EPAQA/G-1 50 November 2002
-------
CHAPTER 4
CASE STUDIES
This chapter presents two case studies that illustrate how the process for developing a quality
system described in Chapter 3 would be applied to two types of organizations: one operated like an
academic organization and one that is a regulatory agency. In practice, the development of a quality
system is a highly individualized process; to be effective it should be tailored to each organization. Thus
these case studies represent examples and are not intended to prescribe the development and
implementation process.
The successful development of the quality systems described in these case studies is not
intended to minimize the difficulties in the process. Within any organization, issues such as
organizational structure, mandate, scope of responsibilities, staffing, and other resource limitations may
either define or limit the development of a quality system. These issues should be identified during the
planning stage so that realistic objectives, schedules, and budgets will be established. Implementing a
modest quality system that addresses elements considered critical to their activities is better for
organizations with limited resources than to plan an elaborate system that cannot be implemented.
Section 3.2.6 discusses the assessment of these issues during the planning stage.
4.1 QUALITY SYSTEM DEVELOPMENT IN AN ENVIRONMENTAL CONSULTING
FIRM
This case study illustrates how the quality system development process and tools may be
applied at an environmental consulting firm that has several different research departments and relatively
limited resources to conduct this process. This case study is a fictitious example that is loosely based
on an actual quality system development effort. The example is not intended to portray actual policies
or depict actual events at EPA or other organization; it is for illustrative purposes only.
4.1.1 Background
In 1999, the Marine Research Institute (MRI) was awarded a six-year EPA contract to
conduct an environmental monitoring study in Massachusetts Bay. The study's purpose was to
establish baseline data that could be used to benchmark water quality in Massachusetts Bay before the
start-up of a power plant that would use high-sulfur crude oil as a fuel source. One stipulation of the
contract award was that before beginning any technical activities, MRI had to provide EPA with a
QMP that described its quality system. EPA defined the monitoring program as an assessment of the
benthic community within a 5-mile radius of the proposed ocean outfall; the deployment of caged
mussels for the analysis of uptake of trace metals, poly chlorinated biphenyls, pesticides, and
semi-volatile compounds from the water-column; and the analysis of sediments for the same suite of
Final
EPAQA/G-1 51 November 2002
-------
chemical compounds. This monitoring program involved four separate research departments at MRI:
field sampling, ecology, inorganic analytical chemistry, and organic chemistry. The contract's principal
investigator was Dr. David Marino, a senior research scientist within MRI's ecology department. Dr.
Marino's challenge was to convince his colleagues of the need to work together to establish a quality
system within each department.
At MRI, the development of a quality system based on ANSI/ASQ E4 and in compliance with
EPA's quality system requirements defined in the contract involved four main phases: (1) initiation,
(2) development, (3) implementation, and (4) ongoing maintenance and improvement.
4.1.2 Initiation
The first step in developing the quality system at MRI was to determine if the MRI research
director would approve the effort and provide the necessary resources. Dr. Marino met with the
research director and the department heads of the field sampling, inorganic analytical chemistry, and
organic chemistry departments to introduce them to the key elements of a quality system and to
propose establishing a quality system at MRI. All participants raised significant objections during this
meeting. Most notably, department heads resisted slowing researchers with time-consuming
documentation procedures and systems when their reputations as outstanding scientists were
well-recognized. The time necessary to establish quality systems in each department was a major
concern, as was the availability of staff to work as part of a team and the lack of funding for the activity.
Dr. Marino countered their objections by stressing the importance and value of the work and the EPA
contract to the organization and by stating that the establishment of a quality system should be
considered an investment for future work. He explained that the philosophy of a quality system is
consistent with the mandate of an educational institution. In the end, the research director agreed to
limited support of an initial effort that would assess what would be required, how much it would cost,
and whether MRI could develop the quality system in time to perform the work for EPA.
Each department head identified a research assistant to work with Dr. Marino on a quality
system development team (QSDT). Each team member would be vested with responsibility to
represent the department in QSDT meetings. They would routinely report to their departments on
progress, quality system issues, and implementation procedures so that concerns and problems could
be identified and addressed.
As a first item of business, the QSDT agreed that their combined units should be identified as
the MRI Environmental Monitoring Group. This would specify the departments that would be included
in the scope of the quality system to be developed. They then drafted the following quality policy
statement:
Final
EPAQA/G-1 52 November 2002
-------
It is the policy of the MRI Environmental Monitoring Group to produce
environmental data that meet the quality needs of our clients.
Based on the types of activities to be conducted for the EPA power plant project, the QSDT
determined that the quality system should include the elements listed in Table 7. Critical elements were
identified as:
the establishment of a QA function,
the need for up-front proj ect planning,
development of a documents and records management system,
establishment of basic documentation procedures, and
preparation of SOPs.
Table 7. Quality System Elements Required for the
MRI Environmental Monitoring Group
Quality System
Element
Policy Agreement
Estimated Time and
Resources to Complete
Management Policies
Management and
organization
Quality system and
description
Personnel
qualifications and
training
Procurement
Documents and
records
Computer hardware
and software
Each department head is responsible for the quality of data
generated within the department. Department heads will
designate project managers and QA Officers.
The quality system for the EPA monitoring project will be
described in a QMP that will apply to all departments
involved in the project.
There must be a record of personnel proficiency, experience,
or training for each staff member. Staff members should not
perform work on tasks for which they have not been trained
unless they are working under the supervision of a qualified
staff member.
Supplies must be of appropriate quality for the intended
end-use of any data generated and must be inspected to
ensure that they are correct and usable.
The following records must be maintained by each
department: training, SOPs, equipment maintenance and
calibration, and preparation of analytical standards. The
following records must be maintained for each project:
sample processing, analysis, data reduction procedures.
Computer hardware must be adequate for the software
application. Software must be tested. Noncommercial
software must be validated.
80 hours over 2 months
80 hours over 1 week
(4 hours for each staff
member) (based on
20 project team members)
16 hours over 1 month
(10 hours to develop and
document procedure;
2 hours to train; 4 hours to
assess)
60 hours over 3 months
20 hours over 2 weeks
Final
EPAQA/G-1 53 November 2002
-------
Table 7. Quality System Elements Required for the
MRI Environmental Monitoring Group
Quality System
Element
Policy Agreement
Estimated Time and
Resources to Complete
Management Policies, continued
Planning
Implement work
processes
Assessment and
response
Quality improvements
A QA Project Plan must be prepared for each project. Each
Plan must follow a prescribed format unless another format is
specified for the project.
The project manager must conduct the project work
according to the requirements of the QMP, the QA Project
Plan, and the appropriate SOPs.
Some form of independent assessment should be performed
for each project. The level of assessment should be defined
in the work/QA Project Plan. Assessments may include Peer
Review, Project Manager Review, external review, and QA
review.
Each department should identify areas for quality
improvement through the assessment and audit process as
well as through initiatives within the department.
10 hours to develop,
document, and train
8 hours to develop,
document, and train
8 hours to develop,
document, and train;
2 hours per project for the
first 6 months.
2 hours per week for the
first 6 months
Environmental Data Policies
Planning and scoping
Each project must be described in a work plan that defines
the objectives, scope, responsibilities, schedule, technical
activities, data quality requirements, assessment activities,
and reporting requirements.
10 hours to develop,
document, and train
Design of data
collection
Routine data collection procedures must be described in
SOPs. The project work plan must describe procedures for
documenting nonroutine, project-specific, or research
activities so that the procedures may be reproduced.
10 hours to develop,
document, and train
Implementation
Each project must be conducted according to the work plan
unless the project manager documents and receives client
approval for modifications to that plan.
4 additional hours per
week per project
Assessment and
response activities
The assessment activities appropriate for each project will be
defined in the work plan. At a minimum, each project will be
assessed by the project manager. Additional assessments
may be conducted by peers, the QA Officer, or external
reviewers. Assessment results must be documented.
Responses must include correction of deficiencies.
8 hours to develop,
document, and train;
2 hours per project for the
first 6 months
Assessment and data
usability
For projects that include the collection of environmental
data, an assessment of data usability based on the project
data quality objectives must be conducted. Data that are
considered "unacceptable" must be qualified.
8 hours to develop,
document, and train;
2 hours per project to
implement
EPA QA/G-1
54
Final
November 2002
-------
Table 7. Quality System Elements Required for the
MRI Environmental Monitoring Group
Quality System
Element
Policy Agreement
Estimated Time and
Resources to Complete
Operation of Environmental Technology Policies
Planning
Design of systems
Construction and
fabrication
Operations
Assessment and
response
Verification and
acceptance
Each project must be described in a work plan that defines
the objectives, scope, responsibilities, schedule, technical
requirements, and the required accuracy, precision, and
sensitivity of the technology, assessment activities, and
final product.
A design plan must be prepared and approved prior to
fabrication of hardware or development of software.
The construction and fabrication of equipment or software
must be according to the design plan. The materials must
meet the quality requirements of the end-use product.
Construction, fabrication, and development activities must
follow accepted engineering procedures. Routine
procedures must be defined in SOPs.
Construction and design activities must be reviewed by the
project manager at the end of each phase to ensure that the
end product will meet the design specifications.
Both a beta test and an end-use test must be performed to
ensure that the final product meets the project objectives.
10 hours to develop,
document, and train
10 hours to develop,
document, and train
8 hours to develop,
document, and train
8 hours to develop,
document, and train;
2 hours per project for the
first 6 months
8 hours to develop,
document, and train;
2 hours per project for the
first 6 months
8 hours to develop,
document, and train;
2 hours per project for the
first 6 months
The QSDT drafted simple policy statements for each element of the quality system and
reported the statements to the department heads to ensure that there was general acceptance of a
skeletal quality system. The QSDT reviewed these policy statements with Dr. Marino. He focused the
QSDT on creating the simplest quality system that would meet the needs of his project.
The QSDT members surveyed their department to identify procedures that were currently in
place. For efficiency and to ensure that similar elements of the quality system were tallied in each
department, the QSDT re-phrased the quality system elements in general terms (e.g., training
requirements and records, written methods, and data reviews). In addition, each department identified
any operating policies (e.g., ethics) that should be incorporated into the quality system. Once the
inventory was complete, the QSDT compiled the results in a two-dimensional matrix that listed each
quality system element by department. This identified elements that existed (at least to some degree)
versus those needing development. Among the elements that existed, at least rudimentarily, were
written analytical methods, instrument calibration procedures, and data review procedures.
EPA QA/G-1
55
Final
November 2002
-------
Dr. Marino used the information compiled by the QSDT to estimate the resources, cost, and
schedule required to develop the quality system for his project. He then prepared a formal proposal for
the research director, describing the need for a quality system, its advantages, the level of effort
required to complete the tasks, and the proposed schedule. At this point, several long-term benefits for
MRI were obvious to Dr. Marino and the QSDT. They agreed that the quality system would:
improve data quality and comparability through the use of documented and
standardized technical procedures to be implemented by all researchers;
implement consistent training of graduate students within departments;
standardize and improve record-keeping procedures to allow data tracking and
reproducibility; and
improve stability within a department when graduate students leave and are replaced
mid-project.
4.1.3 Development
Using the matrix and simple policy statements developed as part of the initiation stage, a draft
QMP was prepared. The QMP addressed, in general terms, the group policy for each element. The
organizational structure identified roles (e.g., department head, project manager, chief scientist) rather
than individuals for implementing the QMP for each department. Although the QMP was specific to
the EPA power plant project, Dr. Marino recognized that it could easily be expanded to include other
projects. Preparation of the QMP identified the need for SOPs to provide detail that was not
appropriate for the QMP or for the group as a whole.
Each QSDT member analyzed his or her department's work flow process using the matrix
developed in the initiation phase. The assessment included management, environmental data, and
technological activities within each department. The results of this assessment identified similarities that
could be codified as group-wide SOPs. Conversely, some practices were obviously
discipline-specific, and department-specific SOPs would be required to address some technical issues.
Dr. Marino decided that the QMP would establish the policy for determining the need for group-wide
or department-specific SOPs. The QSDT agreed that where group-wide SOPs were impractical,
equivalent SOPs would be prepared for each department. (Table 8 summarizes examples of similarities
and differences, and the types of SOPs that were identified to document these aspects of the quality
system of the group). For example, each department had different sample custody, handling, and
storage procedures. It was agreed that a group-wide sample custody SOP was impractical but that
each department should prepare a custody SOP describing its procedures. The group-wide SOP
would define the elements required in each of the custody SOPs (e.g., sample receipt and rejection
criteria, limited access areas, and sample holding times and conditions).
Final
EPAQA/G-1 56 November 2002
-------
Table 8. Assessment of Department Practices
Practices Common to All Departments
The department head assigned project management
responsibilities based on the technical requirements of the
projects.
No written planning document was prepared prior to initiation of
work.
Each department maintained staff and equipment files. Although
staff appeared to be qualified to perform their assigned
functions, records of experience or training were not maintained.
Equipment was calibrated prior to operation, but documentation
of calibration was infrequent.
Written protocols existed for most procedures, but there was no
system of review or approval.
Documentation of environmental data collection did not allow for
sample tracking and did not afford the ability to reproduce
experimental results because documentation was incomplete.
Department-Specific Practices1
Sample custody, handling, and storage varied based on the
intended analysis and department resources.
Monitoring of laboratory equipment, such as refrigerators, varied
based on the laboratory and required sensitivity.
Data reduction methods varied based on the type of data, project
requirements, and experience of the project staff.
Standard Operating Procedure to
Establish Common Policy
Preparation of project work/QA Project Plans
Preparation of project work/QA Project Plans
Documentation of technical proficiency
Documentation of equipment calibration and
maintenance
Requirements for the preparation and issuance
of SOPs
Collection and documentation of
environmental data
Quality Management Plan Topic to
Establish Common Policy
Requirements for sample handling SOPs
Requirements for Laboratory Equipment SOPs
Documentation Requirements for Data
Reduction Methods
'Department-specific SOPs will be prepared for these activities.
The issue of QA personnel was the subject of considerable debate within the QSDT. The
group finally agreed that identifying a single person as a QA Officer for the entire group was impractical
and that there was no funding for an independent unit. Further, as research scientists, no one person
could perform a full-time QA function. Dr. Marino determined that department-specific QA Officers
would be identified for his project. These QA Officers would be qualified researchers who were not
involved in the project but who could adequately assess the conformance of project activities to the
QMP and the project-specific QA Project Plan. For the development and implementation stages, the
QSDT members agreed to perform the role of QA Officers for their respective departments.
Central storage of records was the next administrative issue confronted by the QSDT. They
recognized that some administrative records were key to the quality system but no department wanted
to maintain records for the other departments. These records included SOPs, staff training and
EPA QA/G-1
57
Final
November 2002
-------
proficiency records, and documentation of assessments. Dr. Marino determined that there was no
need to create a central quality systems office; instead, all records would be maintained by the
appropriate department. Thus, original SOPs would be kept in the department of the author. SOPs
applicable to all departments would be distributed to all departments; department-specific SOPs would
be distributed within the group.
Once the draft QMP and the administrative SOPs were complete, a compliance audit was
conducted in each department to identify areas of compliance, noncompliance, and inconsistency
concerning the plan elements. Each department was assessed by a team of two assessors: the pro-tern
department QA Officer and the QA Officer of another department. The results of the audit were
reported to Dr. Marino and the appropriate department head and were then used to create a list of
tasks to address in the implementation phase. The following items are examples of nonconformance
identified during the audit.
The field team did not have records of ship maintenance, calibration of navigation
equipment, or training records for the crew.
The ecology group did not have an established sample custody procedure. Samples
were stored on office shelves. There were no procedures for the safe handling and
disposal of methanol or formalin.
The trace metals laboratory did not document the preparation of standards used for
instrument calibration. Standards could not be tracked to stock solutions or purity
records.
The organics laboratory did not document the expiration dates of spiking solutions and
calibration standards. Sample extraction methods were not documented and dilutions
were not traceable.
Documented experience (e.g., curriculum vitae) existed for only half of the staff.
Based on the results of the compliance audit, Dr. Marino prepared a formal work plan that
identified, by department, the areas that were not in conformance with the QMP elements. The level of
effort required to implement the quality system and a schedule with milestones and due dates was
developed by the QSDT.
Final
EPAQA/G-1 58 November 2002
-------
4.1.4 Implementation
The QSDT planned an event to start implementation of the quality system. The research
director presented awards to the QSDT members. Dr. Marino described the quality system and its
benefits and emphasized the key role that a quality system had played in the recent success of another
organization in obtaining a prestigious research grant. With speeches and the unveiling of the QMP, the
effort received significant advance coverage and was the feature article in the MRI newsletter, which
included interviews and pictures.
The QSDT understood the importance of some successes early in the implementation process.
They used the compliance audit to divide action items into three categories: (a) those that were easily
completed, (b) those that were critical, and (c) those that were necessary and time-consuming but not
critical. The QSDT scheduled items in lists A and B to ensure some relatively "easy" successes, while
continuing to move forward on items that were critical to the core of the quality system. For MRI, the
early "successes" included:
Establishing records of personnel experience—Each staff member in the department
was required to bring an updated curriculum vitae to his/her annual review, scheduled
for the month after the quality system start.
Placing a cabinet in a central, locked location in which benthic ecology samples would
be stored—A shelf was designated for each project, and project managers were given
additional shelf space in their offices.
The preparation of SOPs represented a daunting task to the QSDT. In keeping with a need to
use limited resources efficiently, Dr. Marino recognized that most analytical methods are based on
established literature. Therefore, each analytical SOP consisted of a table of information required for
SOPs but not provided by the method, with the method as an attachment.
Dr. Marino found that it took a significant amount of training to get both department heads and
staff members familiar with the concepts, policies, and procedures developed during the initiation stage.
Training in the QMP requirements and in the application of any new policies was accomplished by
holding short, weekly meetings. Because of the critical nature of this phase, Dr. Marino required
attendance at these meetings for all staff who wanted to be a part of the EPA monitoring project. A
syllabus was created that defined the topics to be covered each week so that participants would
understand the time commitment and schedule their work accordingly. Department head participation
and support was essential for the success of the training. Dr. Marino was surprised by the support
provided by one department head who was chagrined by the results of the compliance audit and had
been struggling to reconstruct the research of a former staff member.
Final
EPAQA/G-1 59 November 2002
-------
Within three months, a basic quality system was in place. Staff members understood the
concept of the quality system, the draft QMP was being implemented, a system for maintaining records
had been created, and many new record-keeping procedures had been initiated.
Routine quality system audits were scheduled to review implementation of the newly established
quality system. These audits included reviews of records, maintenance, training, completion of SOPs,
and compliance with SOPs. Department heads were invited to participate in laboratory reviews and
staff members were encouraged to perform self-audits. A schedule was developed for each
department with enough detail to ensure that all critical aspects of the quality system could be assessed
within six months. The results of the audits were reported to Dr. Marino, and issues that required
attention were specifically identified to the appropriate staff member and department head. A database
of corrective action issues was created so that all members of the group could track progress,
participate in addressing issues, and be recognized for successes.
Audits through the first six months continued to identify areas for improvement. For example,
the organic chemistry staff discovered that some members used colored tape to identify expired
standards while other members used colored tape to indicate the solvent used in preparation. The staff
immediately recognized the need for uniform labeling and prepared a SOP and a wall chart to avoid
confusion.
It soon became clear that some central administration or coordination of records was needed.
Dr. Marino determined that each department would maintain department-specific records and his
secretary would act as the records manager for his project.
4.1.5 Ongoing Maintenance and Improvement
The QSDT continued to operate as an active committee for the first year after the initial start of
the quality system. After 6 months, however, several members rotated off and others rotated on. It
became evident that implementation of the quality system had to continue to be active because staff
quickly fell back into old habits. To maintain the established quality system and improve procedures, a
QA Officer and an assistant QA Officer were designated in each department. This allowed for the
conduct of routine QA audits, follow-up on old issues, and identification of areas where a procedure
was inadequate to ensure data quality. In addition, the need to identify long-term costs associated with
the quality system became apparent during the annual budget cycle. It was recognized that other
projects conducted within the group were benefitting from the quality system improvements and should
share the overhead costs associated with maintaining the quality system.
The QMP specified that quality systems audits are conducted within each department annually.
The QA Officers within each department agreed to audit each other's departments. This ensured that
inconsistencies in compliance to the QMP were identified between departments and provided an
Final
EPAQA/G-1 60 November 2002
-------
"outside look" at each department's operations. The results of these audits were reported to Dr.
Marino and the department heads and entered as action items in the corrective action database.
4.2 QUALITY SYSTEM DEVELOPMENT IN AN EPA REGULATORY PROGRAM:
SIX-YEAR REVIEW OF NATIONAL PRIMARY DRINKING WATER
STANDARDS
This section presents a briefcase study to illustrate the quality system development process and
tools. This case study is a fictitious example that is loosely based on a real quality system development
effort at EPA. The example is not intended to portray actual Agency policies or depict actual events; it
is for illustrative purposes only.
4.2.1 Background
National laws governing drinking water protection require EPA to review and revise all primary
drinking water regulations promulgated by the Agency no less than every six years. EPA's Office of
Clean Water (OCW) established a Six-Year Review Team to establish a protocol for conducting these
reviews on an ongoing basis. EPA conducted its first Six-Year Review process from 1999 to 2001,
evaluating 64 contaminants in public and private water systems. In this review, EPA analyzed Federal
drinking water regulations to determine whether changes in Maximum Contaminant Level standards,
analytical methods, treatment technologies, or occurrence monitoring might provide better protection of
human health. Once high-priority contaminants were identified for review, EPA OCW analyzed
occurrence patterns in greater detail with the goal of making its database more statistically
representative of the nation's public water systems. Some difficulties encountered in evaluating current
drinking water regulations included lack of sufficient occurrence data from certain geographic regions,
the need to incorporate information from non-detect sampling, and inconsistent data quality from
diverse database sources. Data quality assurance and quality management are essential to EPA's
decision-making process in the ongoing Six-Year Review program.
EPA had established a multidisciplinary Six-Year Review Team to coordinate the overall
review program, which spans several offices and divisions within EPA's Office of Water and Office of
Research and Development. The team recognized the need to establish a quality system for their
program, so they defined quality system development as one of their early tasks. They knew that this
would be challenging because the Six-Year Review process involved staff with very different
responsibilities, training, and experiences.
The quality system development process involved four main phases: (1) initiation,
(2) development, (3) implementation, and (4) ongoing maintenance and improvement.
Final
EPAQA/G-1 61 November 2002
-------
4.2.2 Initiation
The Six-Year Review Team initiated their quality system development by establishing a quality
system subteam, which included leaders of the other major subteams covering occurrence, health
effects, treatment technology, analytical methods, and economics. The quality system subteam
identified several early tasks. These included outlining EPA's quality policy for the program, analyzing
QA requirements for the program, evaluating OCW's procedures and practices used in previous
Six-Year Reviews, and getting management approval for the resources needed to improve the process.
Critical to the success of the Six-Year Review process was the identification of a division director to
serve as the program's quality champion, helping to bolster top management commitment to QA
objectives. This EPA manager helped the quality system subteam get management approval for the
quality system development schedule, monitored the overall progress of the project, and effectively
communicated QA issues to upper levels of management. The quality system subteam worked to help
EPA managers within OCW recognize the value to their program's mission of a quality system. With
input from the managers, the subteam established a quality policy endorsed by the office director.
Management commitment and continuing improvement were essential to the success of the quality
system for the Six-Year Review.
Members of the quality system subteam worked together to develop a diagram that shows how
the Six-Year Review quality system relates to the programs that generate new data (see Figure 8). This
helped the subteam inventory and review existing quality procedures. An initial assessment of OCW
procedures helped identify any needed documentation of practices for collecting and managing
contaminant data (analytical methods used in laboratories, databases used, etc.) and to identify gaps
where applicable quality requirements were not being addressed. For example, the EPA quality team
reviewed existing SOPs, analytical methods, and the overall peer review process to identify steps in the
Six-Year Review process that needed improvement.
4.2.3 Development
The quality system subteam carefully planned tasks required to develop its quality system and
improve Agency performance in completion of the Six-Year Review. The OCW quality manager
worked with the subteam to systematically plan and define key performance and acceptance criteria for
the Six-Year Review process so that the quality objectives were tied to the program objectives. The
subteam then wrote a QMP. The OCW QA Manager assembled existing documents that described
the QA and QC activities within each of the subteam areas. The Six-Year Review QMP then
described how the subteam areas relate to each other, and filled in some missing elements related to
training and quality system audits. All quality system documents were available on the EPA Intranet site
for staff and management review. The quality system subteam reviewed SOPs and QA methods and
worked with branch chiefs and their staff to revise two organizational procedures.
Final
EPAQA/G-1 62 November 2002
-------
Policy ( Sate \f OMaiitTpnii™ \ f Office of Water \
(Drinking Wate^fQ-^PoNcy) (^IJy Management)
1 i r
Ifl
Risk % six_Ye
Manaaement £
J
a>
1
Science and % Health
Engineering •? ^J^^
Analysis and ^
.Inrlnement ,
[ Databases
Science and
Engineering
/Supporting
Data
i
ar Review Decisions i
'
Analytical
Methods
* Assessment
nd Reporting/Documentation *
Occurrence Treatment
and Exposure Technology
If Databases! j
f f\f n*.
1 Databasesl 1 ( Databases 1 l^~
,
// Supporting /
/ Data /y
/Supporting / /Supporting /
1
Quality
Assurance in
Six-Year
Review
1
1
Quality
Assurance in
Other
Programs
1
Figure 8. Quality System for the Six-Year Review
After the main quality documents and procedures were developed, the quality system subteam
turned its attention to developing a training program to help all staff involved in the Six-Year Review
understand their roles and responsibilities in the quality system. To make the training more efficient and
useful, the subteam decided to use a combination of Web-based independent study materials and
classroom workshops. Staff would be required to use the Web-based training modules before
attending a workshop, enabling them to learn the basics at their own pace and schedule. The
workshop would be used to explain key points in more detail, answer questions, share problems and
insights with colleagues, and give staff a chance to apply the concepts to their own work through
exercises.
4.2.4 Implementation
EPA management maintained a commitment to QA through the implementation phases of the
Six-Year Review study. The quality system subteam developed an implementation schedule that
outlined the time for conducting training, identifying pilot quality improvement projects, and conducting
periodic internal quality system audits to determine how well the quality system was working in practice.
The training was conducted first and was considered a major success because of the independent
study, which prepared staff to come to the workshop with questions and issues. Maintaining quality
EPA QA/G-1
63
Final
November 2002
-------
records proved to be more challenging. The quality system subteam issued some supplemental
guidelines through e-mail about how to use existing routine reports to help satisfy quality documentation
requirements, and incorporated those guidelines into updated training. For system audits conducted by
quality staff, the QA Manager provided skills training needed to plan, organize, and direct the
evaluation of contracting laboratories and data managers that measured and analyzed contaminant
information.
An early internal assessment of Agency activities and quality standards was essential for OCW
to identify areas where it could improve work procedures or data measurement and reporting methods.
An initial quality assessment by management also helped to identify high-priority issues and problems
with the data and the reporting system used to record information on health effects and occurrence.
The National Contaminant Occurrence Database features built-in QC methods to ensure consistency in
contaminant data reporting, identification, and tabulation. The quality team noted data from analytical
labs that were suspicious, unreliable, or inaccurate. QA Managers and "process owners" met to
discuss what corrective actions were practical and what information would be available in time for
review of current drinking water regulations. QA Managers for this study were successful because they
created a thorough plan and schedule for all Agency quality activities, demonstrated the benefits of QA
and QC activities, and had the resources available to maintain QA activities for the Six-Year Review.
4.2.5 Ongoing Maintenance and Improvement
The OCW conducted routine quality management checks and QC activities over different
phases of the process to improve the reliability and consistency of results obtained in the Six-Year
Review. Periodic quality system audits were essential to maintaining and improving the quality and
efficiency of previous Six-Year Review studies, but were not used in the most recent review.
Corrective actions for the review process included changes in lab detection methods for rarely
occurring contaminants, use of precision standards for maximum contaminant levels and practical
quantitation levels, accurate record-keeping of water quality data, and documentation of current SOPs
used in laboratories and by data managers. Any improvements made to data handling, data quality
assurance, and management processes were documented on EPA's Intranet site for use in future
rounds of reviews. Table 9 illustrates some quality system actions taken for the most recent Six-Year
Review over the four stages of the quality system development process.
Final
EPAQA/G-1 64 November 2002
-------
Table 9. Sample Quality System Development Activities
for the 2000 Six-Year Review
Quality System
Development Stage
Initiation
Development
Implementation
Ongoing
maintenance and
improvement
Sample Quality System Actions for the 2000 EPA Six-Year Review Process
Established quality system and Six- Year Review quality policy
Communicated QA and QC issues and requirements to all levels of management
Set schedule for gathering and reviewing contaminant occurrence data
Inventoried and reviewed existing OCW quality procedures and practices
Reviewed all SOPs for analytical methods; reviewed the peer review process for evaluating
contaminant health effects data
Defined performance and acceptance criteria and program objectives for the Six- Year
Review
Reviewed national occurrence estimates and documentation methods used for evaluating
contaminant data
Created process flow diagrams, mapped internal and contractor work processes through
each phase of the Review Process, and maintained records
Documented major revisions to analytical lab procedures, minimum detection limits, and
processes for deriving Practical Quantitation Limits (PQLs) for each contaminant
Conducted initial quality assessment and developed a quality improvement schedule for
the Six- Year Review process
Created control charts to observe trends in contaminant detection limits and upper and
lower warning limits for Practical Quantitation Limits (PQLs)
Developed an organization chart to document the chain of command and flow of
information through the review team
Used a short quality system checklist and accurate record-keeping to monitor actions taken
to ensure data quality
Process owners met to discuss what QC actions were practical and what information is
available in time for review of current drinking water regulations
Recommended a detailed quality review of national contaminant occurrence estimates and
supporting databases
Constructed process flow diagrams and reviewed data operations that supported all
Agency occurrence estimates
Periodic quality reviews, communication, and meetings between levels of EPA management
Completed Six- Year Review QA Appendix to the 2001 Protocol
Documented quality management procedures and improvements in data quality for future
Six- Year Review Processes
Final
EPAQA/G-1 65 November 2002
-------
After conducting a Six-Year Review quality system audit, the QA Manager recommended a
detailed quality review of national occurrence estimates, with the objective of improving the quality,
reliability, and efficiency of the current approach. First, the various databases that support the
occurrence estimates were mapped out in a diagram (see Figure 9). Process flow diagrams then were
constructed for each data source to document the Six-Year Review data operations that supported the
occurrence estimates. Mapping the Agency's work processes through each phase of the review helped
avoid duplication of work, identify quality check points, and ensure accountability to EPA managers for
data collection, validation, and analysis efforts.
Occurrence and Exposure Assessments
LT2ESWTR
ICR
Databases
Figure 9. Data Sources for Occurrence and Exposure Assessments
EPA QA/G-1
66
Final
November 2002
-------
CHAPTER 5
TOOLS AND METHODS USED IN DEVELOPING QUALITY SYSTEMS
This chapter will answer the following questions:
What is included in SOP templates?
What is included in an EPA Quality Management Plan?
What are document control and records management systems?
What elements are included in quality system compliance checklists?
What are some tools and methods used to analyze and improve work processes?
5.1 INTRODUCTION
This chapter describes a variety of tools and methods that can be useful when developing
quality systems. This chapter discusses tools such as SOPs, quality manuals, document control and
records management systems, quality system compliance checklists, and work process analysis and
improvement tools. This list is not comprehensive, but it describes the tools and methods that are most
used in quality systems. Many of these tools are also used in mature quality systems and originally were
developed for QC applications, but they can also be used for evaluation and identifying problems
during the development of the quality system. References for additional information and examples are
included in the text.
5.2 STANDARD OPERATING PROCEDURES TEMPLATES
Standard operating procedures are an organization's written instructions that document a
routine or repetitive activity. They detail the work processes within an organization to facilitate
consistent conformance to technical and quality system requirements and to support data quality.
EPA's Guidance for the Preparation of Standard Operating Procedures for Quality-Related
Documents (EPA QA/G-6) (EPA, 200Ib), suggests that SOPs contain the following elements:
title page,
table of contents,
control documentation,
procedural section (see below), and
reference section.
Final
EPAQA/G-1 67 November 2002
-------
The procedural section of a technical SOP contains some or all of the following components:
Scope and applicability • Sample handling and preservation
Summary of method • Sample preparation and analysis
Definitions • Troubleshooting
• Health and safety warnings • Data acquisition, calculations, and
Cautions reduction
Interferences • Computer hardware and software
• Personnel qualifications • Data management and records
• Equipment and supplies management
Instrument or method calibration • Quality control and quality
and standardization assurance
Sample collection
For administrative SOPs, the procedural section may consist of:
Title • Definitions
• Purpose • Personnel, qualifications and
Applicability responsibilities
Summary of procedure • Procedure
Quality control and quality assurance • Records management
The Standard Guide for Documenting the Standard Operating Procedures Used for the
Analysis of Water (ASTM D 5172) (ASTM, 1999) notes that significant parts of the variability in
results generated by different laboratories using the same methods are because of differences in the way
the method is performed in each laboratory. Well-written and detailed SOPs provide increased
confidence in a laboratory's ability to reproduce analytical conditions and generate reproducible results.
Staff who use a procedure should be involved in the development of the SOP so that it becomes a
thorough and precise document. An organization should maintain a master copy of their SOPs. When
procedures are modified, the revision should be distributed to all appropriate staff, and staff should
destroy the previous version. A master copy of out-of-date versions should be maintained, but
separate from the master copy of the current SOPs. Staff should be periodically audited to monitor
compliance with the SOPs that they follow. SOPs should include an effective date, revision number,
page numbers out of the total number of pages, and the author. SOPs must be approved and signed by
the appropriate staff.
5.3 QUALITY MANAGEMENT PLANS
Quality Management Plans (QMPs) are used for EPA organizations to document their quality
systems. In EPA Requirements for Quality Management Plans (QA/R-2)(EPA, 200 la), the content
Final
EPAQA/G-1 68 November 2002
-------
for QMPs is described. A QMP documents how an organization structures its quality system, and
describes its quality policies and procedures. It outlines criteria for and areas of application, and
describes it roles, responsibilities, and authorities. While the graded approach applies to QMPs, the
following are the commonly used elements:
Management and organization,
Quality system components,
Personnel qualification and training,
Procurement of items and services,
Documents and records,
Computer hardware and software,
Planning,
Implementation of work processes,
Assessment and response, and
Quality improvement.
5.4 QUALITY MANUALS
Another approach is to develop a quality manual for documenting the quality system. Although
EPA organizations can only use this approach in conjunction with EPA Manual 5360 (EPA, 2000a), it
may be a viable approach for States, tribal governments, and contractors. According to ISO 9001, a
quality manual includes:
the scope of the quality management system, including details and justification for any
exclusions;
the documented procedures established for the quality management system or a
reference to them; and
a description of the interaction between the processes of the quality management
system.
The Guidelines for Developing Quality Manuals (ANSI/ISO/ASQ Q10013-1995) (ASQ,
1995), notes that a quality manual consists of the documented quality system procedures for the overall
planning and administration of activities that have an impact on quality within an organization. A quality
manual should accurately, completely, and concisely describe the quality policy, objectives, and
governing documented procedures of an organization. A quality manual may be a compilation of quality
system procedures or a series of procedures for specific applications, more than one document, a
standalone document, or part of another document. Elements for consideration include:
Final
EPAQA/G-1 69 November 2002
-------
Title, scope, and field of
application
Table of contents
Introductory pages about the
organization and manual
Quality policy and objectives of
the organization
Description of the organizational
structure, responsibilities, and
authorities
Below is a sample outline for a quality manual.
Description of the elements of
the quality system and any
references to documented quality
system procedures
Definitions section
Guide to the quality manual
Appendix with supporting
information
i. Approvals
ii. Revision and approval record
iii. Introduction
1. Quality management system
• General requirements
Documentation requirements
2. Management responsibility
• Management commitment
• Customer focus
Quality policy
• Planning
• Responsibility, authority, and communication
• Management review
3. Resource management
• Provision of resources
• Human resources
• Infrastructure
• Work environment
4. Product realization
• Planning
• Customer-related processes
Design and development
Purchasing
• Production and service provision
• Control of monitoring and measuring devices
5. Measurement, analysis, and improvement
General
EPA QA/G-1
70
Final
November 2002
-------
Monitoring and measurement
Control of nonconforming product
Analysis of data
Improvement
5.5 DOCUMENT CONTROL AND RECORDS MANAGEMENT SYSTEMS
As outlined in 41 CFR 101-11, "Creation, Maintenance, and Use of Records," to develop a
records management system involves the five steps outlined below.
Step 1: Assign specific responsibility for the development and implementation of the
records management program to a qualified records manager.
Step 2: Apply appropriate records management practices to all records, regardless of
the medium (e.g., paper, electronic, or other).
Step 3: Control the creation, maintenance, and use of records and the collection and
dissemination of information to ensure that only necessary records are
accumulated, that forms and reports used for collecting information are efficient
and necessary, that all forms and reports are periodically reviewed, and that
records are maintained cost effectively and in a manner that allows them to be
retrieved quickly and reliably.
Step 4: Strive to improve correspondence and design forms that are user friendly, and
are easy to read, transmit, process, and retrieve.
Step 5: Organize files so that records can be easily found to ensure that records are
complete, and to facilitate the identification and retention of permanent records
and the prompt disposal of temporary records.
For document control, ANSI/ASQ E4-1994 states that documents requiring control should be
identified. Documents should be reviewed by qualified personnel for conformance with technical
requirements and quality system requirements and approved for release by authorized personnel.
Documents used to perform work, such as SOPs, should be kept current by personnel performing the
work. Measures should be taken to ensure that users understand the documents. Obsolete documents
should not be used and should be removed.
Final
EPAQA/G-1 71 November 2002
-------
5.6 QUALITY SYSTEM COMPLIANCE CHECKLISTS
During development of the quality system, a quality system compliance checklist is a useful tool
to check compliance with ANSI/ASQ E4. Part of an example checklist, based on ANSI/ASQ E4 and
ISO 9001, can be found in Appendix A. A checklist of this type can be used to demonstrate strengths
and weaknesses in the organization's existing quality system, identify the areas where the system is not
compliant with ANSI/ASQ E4 or ISO 9001 standards, and clarify the key areas of focus as the quality
system is improved or developed. The checklist can be updated and used to report progress in the
development of the quality system.
5.7 PROCESS ANALYSIS AND IMPROVEMENT TOOLS
A variety of business process analysis and improvement tools are used when developing quality
systems (Anderson, 1999; PQ Systems, Inc., 1996; Russell, 1997).
• Process Flow Charts
Control Charts
Cause-Effect (Fishbone) Diagrams
• Pareto Diagrams
• Benchmarking
Scheduling Tools
5.7.1 Process Flow Charts
A process flow chart graphically displays the steps, decisions, and actions of any process. The
chart includes key points, activities, and roles. For a new process, such as developing a quality system,
it serves as a model or blueprint. During development of a quality system, process flow charts are
particularly useful for the development activities in Phase 2 for analyzing work processes (see Section
3.3). Processes that could be charted include QA Project Plan reviews or corrective action requests to
analyze where problems occur in existing systems. A process flow chart depicts a process sequence
succinctly. Three types of process flow charts are outlined below.
1. A process outline—which is a first-cut chart for initial consideration.
2. A material flow process chart—records an object's movements to and from the
operation, when it is inspected and tested, and when it is stored, delayed, and queued.
3. A worker process chart—records operations, inspection, transport, movement, and
delays for a worker.
Final
EPAQA/G-1 72 November 2002
-------
A process flow chart can aid in reviews and evaluations of an operation. Redundant operations and
points of inspection can be easily identified in process flow charts. An example of a process flow chart
is shown in Figure 10.
Figure 10. Example of a Process Flowchart
Deming (2000) advocated the use of process flow charts to help people identify how their
work relates to others in the organization, both upstream and downstream from their own process. By
viewing the work as a system with interdependent elements, staff are more likely to identify
opportunities for improvement and see how they can optimize the overall system for peak performance,
rather than maximizing the output of individual elements, perhaps at the expense of reduced overall
performance.
5.7.2 Control Charts
Control charts provide an easy way to identify trends or instances when a control limit is
exceeded. In the 1920s, Shewhart determined that the most economical way to identify when
management should investigate a process and take corrective action was to use control charts as a
basis for distinguishing between special causes and common causes of variation (Shewhart, 1986).
During the development of a quality system, control charts can be used to track scheduling, for
instance. Trends in processing permits or requests can also be tracked with control charts.
Control charts can be divided into means charts and range charts. The average and standard
deviation are used to construct a means chart. The upper and lower warning levels are typically set at
± 2 standard deviations, and the upper and lower control levels are typically set at ± 3 standard
EPA QA/G-1
73
Final
November 2002
-------
deviations. For a range chart, differences between two values are plotted so the base line for the chart
is zero. Figure 11 shows a control chart of a process under control, which, therefore, does not warrant
management intervention.
c
o 3
=
Week
Upper
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
t3 "L Control
< Q. Limit
Z -o
~
-------
instance, could be used to generate the statistics for creating a Pareto chart. The bars are arranged in
decreasing order. A Pareto chart can be used to decide which subset of problems should be solved
first. It may also be used to show a before and after comparison of the effect of a quality improvement
measure. More information on the use of Pareto charts can be found on the SAS web site.
Methods
How and when \
to apply \
People
Negative \
Defining attitude \^ Politics
How method applies to\imProvement Extra \
nonstandard processes \ work \
No direct \ Fear of \
feedback \ change \
Unclear roles/ / /
responsibility/ /
/ Poor /
Lack of
/ followthrough / Lack of
Obstacles to\^
Implementing ^^
a Quality s/
reward structure / Poor communication / computers
/ Lack of equipment /
Bureaucracy / management involvement '"'r '/ Availability of
/
Organization
/ training materials
Materials
Figure 12. Example of a Cause-and-Effect (Fishbone) diagram)
90
80 n
OU
% 70
k.
§ 60
O 50
1-
2 40
c
V on
0)
n on
-i n
^
^^^
— ^^^
^*+
^r> —
^^^"^
^^
^—~
**^
Inadequate Equipment Miscommunication Technical
Staffing Problems Problems
Reasons for Delays in Implementing Corrective Actions
Figure 13. Example of a Pareto Chart
EPA QA/G-1
75
Final
November 2002
-------
5.7.5 Benchmarking
Continual improvement (see Section 3.5.3) involving slow, incremental changes to a process is
one method to enhance the quality and productivity of the process. Deming and other quality experts
advocate the use of the iterative Shewart cycle of Plan-Do-Check-Act as the basis of a quality system.
This approach assumes that gradual improvement is adequate and that an organization has sufficient
commitment and innovation to continue the cycle effectively. However, stronger measures may be
needed for organizations that have significant quality problems, for those that are stagnant or for those in
which creativity is suppressed.
Benchmarking is the technique of comparing and measuring an organization's operations or its
internal processes against those of a best-in-class performer from inside or outside its industry (Goetsch
and Davis, 1997). Table 10 presents reasons for benchmarking as a function of organizational quality
objectives (Oakland, 1993). Beckford (1998) writes that benchmarking is essentially an exercise in
organizational humility. It demands that participants respect the idea that there may be other
organizations that carry out a particular process more effectively than they do, rather than being
complacent about how good they are. They then have to learn from these high-performance
organizations.
The benchmarking technique is straightforward. Variations of the technique are possible, but
the technique chosen should address the elements presented in the following 14-step sequence
(Goetsch and Davis, 1997):
Step 1: Obtain Management Commitment—Benchmarking requires a substantial
investment of time and resources. The object of benchmarking is to discover
processes to replace yours or at least to make major changes to them. Without
a mandate from top management, there is no point in attempting to benchmark.
Step 2: Document Your Organization's Processes as Baselines for Benchmarking—It is
critical that you understand your organization's processes thoroughly before
comparing them with another organization's. Most organizations think they
know their processes well, but that is rarely the case if a deliberate process
characterization has not been done. The benchmarking partner (selected in
Step 7) will expect to learn about your organization's processes.
Step 3: Identify Strong and Weak Processes—Strong processes should not be
benchmarked initially. Weak processes become candidates for radical change
through benchmarking, but other processes should remain open to change as
well. If research identifies a better process, add it to the list.
Final
EPAQA/G-1 76 November 2002
-------
Step 4: Select Processes to be Benchmarked—When you have a good understanding
of your own processes and the expectations of them, decide which ones to
benchmark. An important point to remember: never benchmark a process that
you do not wish to change.
Step 5: Form Benchmarking Teams—The teams should include people who operate
the process, those who supply the process, and those who are its customers.
These people are in the best position to recognize the differences between the
process and another organization's process. Every team should have
management representation to build the support for change.
Step 6: Research Best-In-Class Organizations—It is important to select a
benchmarking partner from the best-in-class organizations that are willing to be
a partner for the selected process. The same processes may be used by many
different types of organizations, so do not limit the research only to similar
organizations or opportunities for benchmarking may be missed.
Step 7: Select Candidate Benchmarking Partners—When the best-in-class
organizations have been identified, the team decides which ones to work with.
The best benchmarking partnerships benefit both parties. If the team can find a
way to benefit its potential partner, the linkage between the two organizations
will be easier to achieve.
Step 8: Form Agreements with Benchmarking Partners—After the team has selected a
candidate, it contacts the person with the authority in that organization to
discuss an agreement covering benchmarking activities. After contact has been
made, the team should determine the organization's willingness to be a partner.
The terms of any resulting agreement should include mutual visitation
arrangements, confidentiality, and points of contact. Care must be taken to
ensure that the benchmarking activities do not excessively disrupt the partner's
normal operations.
Step 9: Data Collection—The team visits the partner to observe, collect, and document
everything about the process being benchmarked and to determine the
underlying factors and processes that make the process best-in-class. Your
organization's process operators should talk directly with their operators.
Coming away with a good understanding of the benchmarked process is
important, its antecedent and successor processes, its support requirements,
timing, and control. The team should learn enough about the process to
implement it on return to your organization.
Final
EPAQA/G-1 77 November 2002
-------
Step 10: Analyze the Benchmarking Data; Establish the Gap Between the
Processes—The team should thoroughly compare the data for the
benchmarked process with the data taken from your organization's process.
The team should establish a quantitative value for the gap (the performance
difference) between the two processes.
Step 11: Plan Action to Close the Gap or Surpass—Implementation requires planning to
minimize disruption while the change is being made and while the operators are
getting trained and accustomed to the new process. It is important to approach
implementation deliberately and with great care. Prepare for all conceivable
contingencies. Only after thorough preparation and training should an
organization implement the change to the new process. The second objective
of benchmarking is to implement a process that is itself best-in-class. Your
organization should surpass the performance of the benchmarked process. The
planning should provide for the development work necessary to achieve this
objective.
Step 12: Implement the Change—Implementation should be easy if the planning has
been thorough and if the execution adheres to the plan. The performance may
not equal the benchmark until the new equipment, personnel, and procedures
function on a routine basis. After the initial problems are solved, the
performance should be close to the benchmark. If not, another visit to the
partner may be necessary to determine what has been overlooked.
Step 13: Monitor—After the process is running routinely, its performance should match
and then surpass the benchmark. Constant attention in the form of monitoring is
needed. Statistical process control can be used to monitor the performance of
the new process.
Step 14: Update Benchmarks; Continue the Cycle—Whether the benchmark is
surpassed or not, the important thing is to maintain the goal of achieving best-in-
class. Benchmarks must be updated periodically. Stay in touch with the
partner. Continue the benchmarking process. Let continual improvement take
over for the best processes, and concentrate the benchmarking on the ones that
remain weak.
Final
EPAQA/G-1 78 November 2002
-------
Table 10. Reasons for Benchmarking
Objectives
Becoming competitive
Industry best practices
Defining customer requirements
Establishing effective goals and
objectives
Developing true measures of
productivity
Without benchmarking
Internal focus; evolutionary change
Few solutions; frantic catch-up
activity
Based on history or gut feeling;
perception
Lacking external focus; reactive
Pursuing pet projects; strengths
and weaknesses not understood;
route of least resistance
With benchmarking
Understanding of competition;
ideas from proven practices
Many options; superior
performance
Market reality; objective evaluation
Credible, unarguable; proactive
Solving real problems;
understanding outputs; based on
industry best practices
5.7.6 Scheduling Tools
There are a variety of scheduling tools available, such as Program Evaluation and Review
Technique (PERT), the critical path method (CPM), Gantt charts, and commercially-available software.
These tools can help the team developing the quality system in keeping the development on schedule.
An example schedule is presented below.
A Gantt chart graphically displays a list of tasks along a time line. Numbered tasks are listed
sometimes with additional information, such as task duration (in days, weeks, or months), start date,
end date, or other information. Tasks are depicted graphically to the right of the task list using a
horizontal bar scaled to a calendar time line. Milestones are key events that have zero duration, and
often represent the beginning or end of a set of related tasks. The logical sequencing of tasks also can
be represented by linking the horizontal task bars using arrows, as shown in Figure 14. For example, in
Figure 14 the task "Research QMP and consult with other organizations" must be completed before the
task "Write first draft of QMP" can be started. The example also shows the schedule's critical path,
the sequence of tasks that do not have any spare time. A delay in any one task along the critical path
will delay the project completion date. Gantt charts can be very effective in quickly communicating
schedules because they are easy to understand if constructed properly. Many commercially available
software packages use Gantt charts to display schedule data. The ability to analyze the critical path
easily, explore potential impacts to changes, and update actual versus planned dates makes scheduling
software packages very useful (see Oakland, 2000; Beckford, 1998).
EPA QA/G-1
79
Final
November 2002
-------
Ipredecessors
Sep I Oct I Nov I Dec I Jan iFeb I Mar I Apr I May I Jun
Management approval of project
Research QMPs, consult with other organizations
Write first draft of QMP
Odays
Mon 10/1/01
Fri 11/9/01
Identify, document key work processes
Write first drafts of SOPs
Distribute QMP, SOPs for internal review
Odays
Review, revise QMP, SOPs
Obtain management approval of QM P, SOPs
Develop training program
Conduct, evaluate pilot training
Revise training program, get management approval
Development phase completed
Odays
3,5
8,11
Figure 14. Project Gantt Chart Showing the Durations and Relationships Among Tasks, with
the Critical Path Highlighted by Cross-Hatching
PERT is a statistical technique applied to a network schedule. A PERT chart for the early
activities of the project shown in the Gantt chart is shown in Figure 15. A PERT chart reduces the
overall project time by showing the tasks that can be performed simultaneously, and reducing delays
between items performed sequentially. A PERT chart shows project activities and their interrelations
and shows the sequence of dependencies between activities. It is used to determine the minimum time
needed to complete a project, phase, or task, such as developing a quality system. The four steps are:
1. Identify tasks,
2. Determine the proper sequence of tasks,
3. Estimate the time required to perform each task, and
4. Prepare a time-scaled chart of tasks and events to determine the critical path.
' —
Approval of Project
1
10/01/01
0 Days
10/01/01
Research QMP's
Consult
2
10/01/01
6 Weeks
1 1/09/01
Write First Draft of
QMP
3
11/12/01
8 Weeks
01/04/02
Identify, Document
Work Processes
4
10/01/01
10 Weeks
12/07/01
Write First Drafts of
SOPs
5
1 2/1 0/0 1
8 Weeks
02/01/02
Distribute QMPfor
SOPs for Review
6
02/01/02
ODays
02/01/02
Review, Revise
QMP, SOPs
02/04/02
Develop!
Program
9
02/04/02
6 Weeks
03/15/02
Obtain Man
Approval o
8
03/18/02
aining
7 Weeks
03/22/02
agement
f QMP
4 Weeks
04/12/02
Development
Phase Completed
12
04/19/02
J
Conduct, Evaluate
Pilot Training
10
03/25/02
2 Weeks
04/05/02
0 Days
04/1 9/02
Revise Traning, Obtain
M an ag em e nt Ap prov al
11
04/08/02
2 Weeks
04/19/02
Figure 15. PERT Chart Showing Early Tasks of Projects Critical Path in Bold
EPA QA/G-1
80
Final
November 2002
-------
The critical path method is a technique for project planning. It can be used to determine how
long it will take to finish a project, such as developing a quality system, and which activities are critical
to finishing the project on time. Cost information can be added to help determine the optimal plan for
speeding up the project. For each activity, the following items should be known:
the projected time required to complete the activity;
• those activities that need to be completed before another one can start;
• the cost to complete (optional);
a shorter time to complete on a crash basis (optional); and
a higher cost to complete on a crash basis (optional).
A diagram is completed showing the sequential order of activities. The critical path is the one
that takes the longest. This tells you which activities need to be done on time to finish the project in the
least time.
Summary
A variety of tools and methods can be used to develop quality systems and implement quality
improvements in EPA work processes, including business process improvement charts, control charts,
fishbone diagrams, critical path methods, PERT charts, etc.
Standard operating procedures are sets of written instructions that document a routine or
repetitive activity followed by an organization. QMPs satisfy an EPA requirement to document how an
organization structures its quality system, describe its quality policies and procedures, and explain roles
and responsibilities of the quality staff. Document control and records management systems assign
responsibilities and practices for information collection. They ensure that unnecessary records are not
created and that project data are stored efficiently and cost effectively. Quality system checklists detail
the required elements of management systems, organizational responsibilities, EPA quality policy, design
and development of quality systems, personnel training and qualifications, etc.
Final
EPAQA/G-1 81 November 2002
-------
Final
EPAQA/G-1 82 November 2002
-------
APPENDIX A
QUALITY SYSTEM CHECKLIST
Standard References
E4
ISO 9001
Description/Assessment Questions
Reference
PART 2 MANAGEMENT SYSTEMS
2.1
2.1. a, 2.1. d
2.1.a,2.1.d
2.1.a,2.1.d
2.1.a,2.1.d
2.1.a,2.1.d
2.1.a,2.1.d
2.1.b,2.1.g
2.1.b,2.1.g
2.1.b,2.1.g
4.1
4.1.1
4.1.1
4.1.1
4.1.1
4.1.1
4.1.1
4.1.2
4.1.2.1
4.1.2.1
MANAGEMENT AND ORGANIZATION
Accepted
Needs
Work
Comment
Quality Policy
Are company policies, objectives, and its commitment to
quality documented?
Are these documents controlled?
See Document and Data Control (4.5).
Are the policies understood, implemented, and maintained
at all levels of the organization?
Does the quality system define what constitutes executive
responsibility?
Is the stated quality policy relevant to internal
organization goals and customer needs/expectations?
Organization
Responsibility and Authority
Is sufficient responsibility and authority assigned to
personnel for the effective resolution of problems related
to the quality of process(es), quality system, and the
product/service?
EPA QA/G-1
A-l
Final
November 2002
-------
Standard References
E4
2.1.b,2.1.g
2.1.b,2.1.g
2.1.C
2.1.e
2.1.f
2.1.h
2.1.i
ISO 9001
4.1.2.1
4.1.2.1
4.10.4
4.1.2.3
Description/Assessment Questions
Are the organizational structure, responsibilities, and
authority of management personnel documented (e.g.,
organization chart, job descriptions, etc.)?
Is the organizational structure, responsibility and
authority of all personnel related to product, process, and
quality system activities documented?
Has management identified both internal and external
customers for the work to be performed and the suppliers
of items or services?
Do policies exist for management to negotiate acceptable
measures of quality and success when constraints of time,
cost, or other problems affect the supplier's ability to fully
satisfy the customer's prestated needs and expectations?
Does management have the appropriate authority to do
so?
Is appropriate training and outreach in place so as to
ensure that applicable elements of the E4 standard are
understood and are implemented in environmental
programs defined by the E4 standard and under their
responsibility?
Are all OSHA and other relevant safety rules and
certifications current and being followed? Is
documentation of such rules/certification accessible to
employees? Is the responsibility for stopping unsafe work
practices specified in these rules or in another central
location?
Reference
Accepted
Needs
Work
Comment
Management Representative
EPA QA/G-1
A-2
Final
November 2002
-------
Standard References
E4
2.1.i
2.1.i
2.1.)
2.1.)
2.1.)
2.1.)
2.1.k
2.1.k
2.1.k
2.1.k
2.1.k
ISO 9001
4.1.2.3
4.1.2.3
4.1.3
4.1.3
4.1.3
4.1.3
4.2
4.2.1
4.2.1
4.2.1
4.2.1
Description/Assessment Questions
Has an ISO 9000 management representative been
appointed? Underwriters Laboratory's ISO program also
requires an alternate to be appointed; therefore has an
alternate management representative also been appointed?
Are reports issued by this management representative used
for improvement of the quality system?
Reference
Accepted
Needs
Work
Comment
Management Review
Does executive management periodically review and
approve all aspects of the quality system to ensure its
continuing suitability and effectiveness in satisfying the
requirements of the selected ISO 9000 Standard and the
company quality policy objectives?
Does the documentation define an executive management
review process, including by whom reviews are
conducted?
Are records maintained of management reviews?
See Control of Quality Records (4.16).
QUALITY SYSTEM
General
Are written procedures and work instructions in place to
control all activities affecting product/service quality?
Have all processes affecting product/service quality been
identified and controlled?
Do procedures exist to identify the method of ensuring
that all aspects of the system comply with the applicable
standard?
EPA QA/G-1
A-2
Final
November 2002
-------
Standard References
E4
2.1.k
2.1.k
2.2
2.2.b
ISO 9001
4.2.1
4.2.1
4.1.2.2
4.1.2.2
Description/Assessment Questions
Does the quality manual outline the documentation
structure used in the quality system?
Are the response actions required as a result of
independent assessments or self-assessments of the quality
system documented? Is there a process in place for
documenting a schedule for implementation of corrective
actions?
Resources
Have trained personnel been assigned for management,
performance of work, verification activities, and internal
quality audits?
See Training (4.18).
QUALITY SYSTEM OVERVIEW
(a) Is there an organizational chart with designated
responsibilities available and easily accessible
by all employees?
(b) Is there a quality system diagram available and
easily accessible by all employees?
(c) Are all the policies, procedures, and guidance
documents dictating quality work and
management procedures easily accessible by all
employees?
(d) Are the resources made available for
implementation of the quality system
documented (including training) on a periodic
basis (annually) and easily accessible by relevant
staff?
Reference
Accepted
Needs
Work
Comment
EPA QA/G-1
A-4
Final
November 2002
-------
Standard References
E4
2.2.d
2.2.d
2.2.d
2.2.d
2.2.d
2.2.e
2.2.g
2.2.h
2.2.i
2.3
ISO 9001
4.4.2
4.4.2
4.4.2
4.4.2
4.4.2
4.1.2.3
4.18
Description/Assessment Questions
Reference
Accepted
Needs
Work
Comment
Design and Development Planning
Are plans drawn up to identify the responsibility of each
design and development activity?
Are these plans reviewed and updated as the design
evolves?
Are personnel involved in design verification provided
with the necessary resources?
Are responsibilities defined in design activity plans?
Has a quality manual been generated and approved for the
current period of applicability (annually)?
Does the quality system in the Project Manual description
identify in general terms those items, programs, or
activities to which it applies?
Does the quality system description identify and
document activities that directly or indirectly affect
quality including:
• general and specific responsibilities for
management and staff; and
• responsibilities and authorities for technical
activities.
Is it established that at regular intervals, and at least
annually, the quality system shall be reviewed and its
description updated to reflect physical changes in the
organization as well as changes in ETV quality policy?
PERSONNEL TRAINING AND QUALIFICATIONS
EPA QA/G-1
A-5
Final
November 2002
-------
Standard References
E4
2.3.3, 2.3.b,
2.3.d, 2.3.e
ISO 9001
4.18
Description/Assessment Questions
Have documented procedures been established to identify
training needs for all personnel performing activities
affecting quality?
Reference
Accepted
Needs
Work
Comment
EPA QA/G-1
A-6
Final
November 2002
-------
APPENDIX B
REFERENCES
21 CFR 50, 56, and 312. Code of Federal Regulations. Good Clinical Practices.
21 CFR 210, 211, and 280. Code of Federal Regulations. Good Manufacturing Practices.
40 CFR 30. Code of Federal Regulations. Grants and Agreements with Institutions of Higher
Education, Hospitals, and Other Non-profit Organizations.
40 CFR 31. Code of Federal Regulations. Uniform Administrative Requirements for Grants and
Cooperative Agreement to State and Local Governments.
40 CFR 35. Code of Federal Regulations. State and Local Assistance.
40 CFR 160 and 792. Code of Federal Regulations. Good Laboratory Practices.
41 CFR 101. Code of Federal Regulations. Public Contracts and Property Management: Federal
Property Management Regulations.
48 CFR 46. Code of Federal Regulations. Federal Acquisition Regulations System, Quality
Assurance.
American National Standards Institute/American Society for Quality (ANSI/ASQ). (1995)
Specifications and Guidelines for Quality Systems for Environmental Data Collection and
Environmental Technology Programs (E4-1994). American National Standard.
American Society for Quality (ASQ). (1995) Guidelines for Developing Quality Manuals
(ANSI/ISO/ASQ Ql0013-1995).
American Society for Testing and Materials (ASTM). (1999) Standard Guide for Documenting the
Standard Operating Procedures Used for the Analysis of Water (ASTM D 5172).
Anderson, B. (1999). Business Improvement Toolbox. Milwaukee, WI: ASQ Quality Press.
Beckford. (1998). Quality: A Critical Introduction. New York: Routledge.
Brown, M.G., Hitchcock, D.E., and Willard, M.L. (1994). Why TQM Fails and What to do about
It. New York: McGraw-Hill.
Final
EPA QA/G-1 B -1 November 2002
-------
Carr, D.K. and Liftman, ID. (1990). Excellence in Government. Total Quality Management in
the 1990s (Second Edition). Arlington, VA: Coopers & Lybrand.
Crosby, P.B. (1979). Quality is Free. New York: McGraw-Hill.
Deming, W.E. (2000). Out of the Crisis. Cambridge, MA: MIT Press.
Dobyns, L. and Crawford-Mason, C. (1991). Quality or Else. Boston: Houghton-Mifflin.
Flood, R.L. (1993). Beyond TQM. New York: John Wiley & Sons.
Goetsch, D.L. and Davis, S. (1995). Implementing Total Quality. Englewood Cliffs, NJ: Prentice-
Hall.
Goetsch, D.L. and Davis, S. (1997). Introduction to Total Quality. Upper Saddle River, NJ:
Prentice-Hall.
Goetsch, D.L. and Davis, S. (2001). Total Quality Handbook. Upper Saddle River, NJ: Prentice-
Hall.
Hunt, V.D. (1993). Quality Management for Government. Milwaukee: American Society for
Quality (ASQ) Quality Press.
International Organization for Standardization (ISO). (1999). ISO/IEC 17025:1999, General
Requirements for the Competence of Testing and Calibration Laboratories. Geneva: ISO.
International Organization for Standardization (ISO). (2000a). ISO 9001:2000, Quality
Management Systems—Requirements. Geneva: ISO.
International Organization for Standardization (ISO). (2000b). Quality Management Principles.
ISO, Geneva, Switzerland. Available: www.iso.ch/iso/en/iso9000-14000/iso9000/qmp.html.
Juran, J.N. (1977). That Uninterested Top Management. Selected Papers, No. 20, The Juran
Institute, Wilton, CT. Available: www.juran.com/research/articles/SP7720.html.
Juran, J.M. (1999). The Quality Improvement Process. In: Juran's Quality Handbook, Fifth
Edition, Juran, J.M. and Godfrey, A.B., eds. New York: McGraw-Hill.
Koehler, J.W. and Pankowski, J.M. (1996). Quality Government: Designing, Developing, and
Implementing TQM. Delray Beach, FL: St. Lucie Press.
Final
EPAQA/G-1 B-2 November 2002
-------
Larson, M. (1999) . Whatever Happened to TQM? Quality 38:7. Available: www.qualitymag.com.
Maine,!. (1994). Quality Wars. The Triumphs and Defeats of American Business. New York:
Free Press.
National Institute of Standards and Technology. (1993). Integration Definition for Function
Modeling (IDEFO), Federal Information Processing Standards Publication 183.
Oakland, J.S. (1993). Total Quality Management, (SecondEdition). Boston: Butterworth-
Heinemann.
Oakland, J.S. (2000). Total Quality Management. Text with Cases (SecondEdition). Boston:
Butterworth-Heinemann.
PQ Systems, Inc. (1996). Total Quality Tools, 1st printing, version 2. Dayton, OH: PQ Systems.
Russell, J.P. (ed.). (1997). The Quality Audit Handbook. Milwaukee, WI: ASQ Quality Press.
Schuler, R.S. and Harris, D.L. (1992). Managing Quality: The Primer for Middle Managers.
Reading, MA: Addison-Wesley Publishing Company, Inc.
Senge, P.M. (1990). The Fifth Discipline: The Art and Practice of the Learning Organization.
New York: Doubleday.
Shewhart, W.A. and Deming, W.E. (1986). Statistical Method from the Viewpoint of Quality
Control. New York: Dover.
U.S. Environmental Protection Agency. (2000a). EPA Quality Manual for Environmental
Programs (EPA 5360 Al). Washington, DC.
U.S. Environmental Protection Agency. (2000b). Policy and Program Requirements for the
Mandatory Agency-wide Quality System (EPA Order 5360.1 A2). Washington, DC.
U.S. Environmental Protection Agency. (2000c). Guidance for Developing a Training Program for
Quality Systems (QA/G-10) (EPA/240/B-00/004). Washington, DC.
U.S. Environmental Protection Agency. (2001a). EPA Requirements for Quality Management
Plans (QA/R-2) (EPA/240/B-01/002). Washington, DC.
Final
EPAQA/G-1 B-3 November 2002
-------
U.S. Environmental Protection Agency. (2001b). Guidance for Preparing Standard Operating
Procedures (EPA QA/G-6) (EPA/240/B-01/004). Washington, DC.
U.S. Environmental Protection Agency. (2001c). Guidance on Assessing Quality Systems (EPA
QA/G-3) (Peer Review Draft). Washington, DC.
U.S. Environmental Protection Agency. (2002). Overview of the EPA Quality System for
Environmental Data and Technology (EPA/240/R-02/003). Washington, DC.
Wieringa, D., Moore, C. and Barnes, V. (1998). Procedure Writing. Principles and Practices
(Second Edition). Columbus, OH: Battelle Press.
SUGGESTED READING
Bendell, T. (1998). The Quality Gurus. British Department of Trade and Industry. Available:
www.dti.gov.uk/mbp^pgt/m9ja00001/m9ja000011 .html.
—This document provides a comparative insight into the philosophies and tools of the
major quality gurus (e.g., Deming, Crosby, andJuran). It outlines each guru's history,
main areas of work, the unique aspects of his message, and what benefits might be
extracted by following the philosophy and using the tools.
Deming, W.E. (2000). Out of the Crisis. Cambridge, MA: MIT Press.
—W. Edwards Deming was one of the original quality gurus and introduced Japanese
industry to quality concepts in 1946. This book offers a theory of management based on
his 14 points for management. It explains principles of management transformation that
establish quality, productivity, and competitive position.
Mortiboys, R. and Oakland, J. (1997). Total Quality Management and Effective Leadership.
British Department of Trade and Industry. Available:
www.dti.gov.uk/mbp^pgt/m9ja91001/m9ja910011 .html.
— "This document helps senior managers focus on the nature and scale of leadership and
quality challenge. It discusses the need to shift emphasis away from controlling people and
towards developing management styles based on sound common sense and quality. It
shows how TQM can be applied to improve the effectiveness, flexibility and
competitiveness of a business as a whole" (—Quoted from the title page).
Oakland, J. and Morris, P. (1998). Pocket Guide to TQM. A Pictorial Guide for Managers.
Boston: Butterworth Neinemann.
—Discussions of quality can get pretty abstract quickly. Oakland and Morris have
presented this very complex topic in terms that most people can understand. The abundant
Final
EPAQA/G-1 B-4 November 2002
-------
graphics explain as much about quality as does the clear text. This 90-page book serves as
a good starting point for understanding how quality concepts and tools can be applied in
everyday workplaces. If necessary, one could then read other books for more detailed
information about quality systems.
U.S. Office of Personnel Management. (1993). Self-Assessment Guide for Organizational
Performance and Customer Satisfaction: Based on the Presidential Award for Quality
Criteria (GPO 006-000-01397-0). Washington, DC: Government Printing Office.
—772/5 document provides guidance and instructions for conducting an organizational self-
assessment. It is designed to help organizations evaluate their quality management efforts
and measure progress. It is appropriate for use in an organization that is well into the
implementation phase of a quality system.
U.S. Office of Personnel Management. (1991). How to Get Started: Implementing Total Quality
Management (GPO No. 006-000-01355-5). Washington, DC: Government Printing Office.
—This booklet is one of a series that offers practical assistance to executives, managers,
and employees in applying total quality management principles. It is intended primarily
for someone who has a basic understanding of the principles and has at least a tentative
commitment to them.
Final
EPAQA/G-1 B-5 November 2002
-------
Final
EPAQA/G-1 B-6 November 2002
-------
APPENDIX C
GLOSSARY OF QUALITY ASSURANCE CONCEPTS
Continuous Improvement
The organization and its environment are constantly changing and evolving. Consequently,
processes that were optimized under a particular set of conditions at one time will eventually degrade
unless an effort is made to reexamine the process and make improvements. This illustrates why
continuous improvement is an important principle that drives the quality cycle of planning,
implementation, assessment, and decision making (also called the "Shewart cycle" of plan, do, check,
act).
Customer Focus
Quality itself is largely defined by the customer of a product or service, hence an organization
developing a quality system should maintain a focus on the explicit and implicit requirements and
expectations of its customers. The organization should clearly understand who its customers are, strive
to understand the many facets of their customers' needs, and stay alert to how those needs are
changing over time. Mechanisms for collecting information about customer satisfaction may be a new
but important part of quality system development for many environmental organizations.
Graded Approach
The graded approach describes the idea that the level of intensity and rigor devoted to a quality
effort should be commensurate with the scope and risks associated with the process. In essence, if the
consequences of failure are small, then relatively little is at stake and less effort and resources should be
spent on quality assurance and quality control. On the other hand, if a core business process is under
consideration and the consequences of poor quality are great (such as loss of highly valuable work or
severe damage to the organization's reputation), then a systematic and rigorous quality system may be
needed to assure that the risk of failure is acceptably low. Adherence to the graded approach helps
ensure that the quality system is cost effective and valuable to the organization.
Leadership and Management Commitment
Development of a quality system involves organizational change. Effective organizational change
requires leadership and commitment from top management to influence positive changes in behavior,
minimize fear of and resistance to change, and invest the resources necessary to help people learn new
methods and procedures. Organizational change is a challenging endeavor that often requires shifts in
organizational culture. Such change is more likely to take hold and have lasting value if the
Final
EPA QA/G-1 C -1 November 2002
-------
organization's leaders are committed to the success of the quality system and support its implementation
through consistent words and actions.
Measurement and Management by Fact
Quality management relies on objective measurement to assess performance in relation to goals.
One of the most challenging aspects of quality system development can be the determination of
appropriate quality measures, particularly for processes that deliver services. When done correctly,
measurement provides objective information about how well the organization is performing, which
provides a more transparent and defensible basis for management decision making.
Staff participation and teamwork
Given that the quality system should be based on a deep understanding of the organization's
processes, it follows that the staff who perform those processes should be involved in quality system
development. The people who perform the process tasks are usually the ones who understand best the
process strengths, weaknesses, and opportunities for improvement. A team approach to quality system
development increases the flow of information and sharing of knowledge.
Systems Thinking
Organizations are systems that must be understood as a set of interdependent elements that make
up a whole. For example, the work of an organization involved in environmental data operations and
decision making encompasses complex processes that transform inputs such as information, energy,
and materials into outputs such as measurement data, reports, and decisions. These processes work
within a complex environment of customers, suppliers, and numerous stakeholders, and involve
feedback loops and time lags. According to systems theory, the properties and behavior of a system
cannot be understood by merely analyzing the component parts; one must observe the emergent
characteristics that arise from the complex interactions among the system's elements and its
environment. It follows that if one wants to change a complex system like an organization successfully,
one must use "systems thinking." Systems thinking is a conceptual framework and a set of tools that
help make clear the patterns that emerge in complex systems, and how to make effective changes in a
system (Senge 1990). Systems thinking focuses on how the dynamics of an organization's processes
are determined largely by the system's structure. To increase the likelihood of improving system
performance reliably over time, and to avoid unintended consequences, an organization should seek to
understand the structure and dynamics of various reinforcing feedback loops, countervailing forces,
implicit goals, and time lags at work in the organization.
Final
EPAQA/G-1 C-2 November 2002
-------
APPENDIX D
GLOSSARY OF TERMS
acceptance criteria — specific limits placed on the characteristic of an item, process, or service
defined in requirements documents.
certification — the process of testing and evaluation against specifications designed to document,
verify, and recognize the competence of a person, organization, or other entity to perform a function or
service, usually for a specified time.
configuration — the functional, physical, and procedural characteristics of an item, experiment, or
document.
conformance — an affirmative indication or judgment that a product or service has met the
requirements of the relevant specification, contract, or regulation; also, the state of meeting the
requirements.
consensus standard — a standard established by a group representing a cross section of a particular
industry or trade, or a part thereof, such as the ISO-9000 series, or the ANSI/ASQ National
Standard—Guidelines for Developing Quality Manuals..
data quality objectives — qualitative and quantitative statements derived from the outputs of the first
six steps of the DQO process that clarify the study objective, define the most appropriate type of data
to collect, determine the most appropriate conditions from which to collect the data, and specify
tolerable limits on decision errors which will be used as the basis for establishing the quantity and quality
of data needed to support the decision.
data quality objectives process — a seven-step systematic planning process developed by EPA
which provides a procedure for defining the criteria that a data collection design should satisfy, including
when to collect samples, where to collect samples, tolerable level of decision errors, and how many
samples to collect.
defensible — the ability to withstand any reasonable challenge related to the veracity or integrity of
project and laboratory documents and derived data.
deficiency — an unauthorized deviation from acceptable procedures or practices, or a defect in an
item.
Final
EPA QA/G-1 D -1 November 2002
-------
demonstrated capability — the capability to meet a procurement's technical and quality specifications
through evidence presented by the supplier to substantiate its claims and in a manner defined by the
customer.
design — the specifications, drawings, design criteria, and performance requirements. Also, the result
of deliberate planning, analysis, mathematical manipulations, and design processes.
design change — any revision or alteration of the technical requirements defined by approved and
issued design output documents and approved and issued changes thereto.
design review— a documented evaluation by a team, including personnel, such as the responsible
designers, the client for whom the work or product is being designed, and a quality assurance (QA)
representative, but excluding the original designers, to determine if a proposed design will meet the
established design criteria and perform as expected when implemented.
environmental data — any measurements or information that describe environmental processes,
location, or conditions; ecological or health effects consequences; or the performance of environmental
technologies. For EPA, environmental data include information collected directly from measurements,
produced from models, and complied from other sources such as databases or the literature.
environmental processes — manufacturing or natural processes that produce discharges to or impact
the ambient environment.
environmental programs — work or activities including, but not limited to characterization of
environmental processes and conditions; environmental monitoring; environmental research and
development; the design or construction of environmental technologies; and laboratory operations on
environmental samples.
environmental technology — an all-inclusive term to describe pollution control devices and systems,
waste treatment processes and storage facilities, and site remediation technologies and their
components that may be utilized to remove pollutants or contaminants from, or prevent them from
entering the environment. Examples include wet scrubbers (air), soil washing (soils), granulated
activated carbon (water), and filtration (air, water). Usually this term applies to hardware-based
systems; however, it can also apply to methods or techniques used for pollution prevention, reduction,
or containment to prevent further movement of the contaminants, such as capping, solidification,
vitrification, and biological treatment.
financial assistance — the process by which funds are provided by one organization (usually
governmental) to another organization for the purpose of performing work or furnishing services or
Final
EPAQA/G-1 D-2 November 2002
-------
items. Financial assistance mechanisms include grants, cooperative agreements, and governmental
interagency agreements.
finding — an assessment conclusion that identifies a condition having a significant effect on an item or
activity. An assessment finding may be positive or negative, and is normally accompanied by specific
examples of the observed condition.
grade — the category or rank given to entities having the same functional use but different requirements
for quality.
graded approach — the process of basing the level of application of managerial controls applied to an
item or work according to the intended use of the results and the degree of confidence needed in the
quality of the results.
guidance — a suggested practice that is not mandatory, intended as an aid or example in complying
with a standard or requirement.
guideline — a suggested practice that is not mandatory in programs intended to comply with a
standard.
independent assessment — an assessment performed by a qualified individual, group, or organization
that is not a part of the organization directly performing and accountable for the work being assessed.
inspection — the examination or measurement of an item or activity to verify conformance to specific
requirements.
locational data — latitude/longitude coordinates and other geographic information collected and
documented with environmental and related data. This is in addition to, and not precluding, other
critical location identification data that may be needed to satisfy individual program or project needs,
such as depth, street address, elevation, or altitude.
management system — a structured, nontechnical system describing the policies, objectives,
principles, organizational authority, responsibilities, accountability, and implementation plan of an
organization for conducting work and producing items and services.
management systems review— the qualitative assessment of a data collection operation and/or
organization(s) to establish whether the prevailing quality management structure, policies, practices, and
procedures are adequate for ensuring that the type and quality of data needed are obtained.
Final
EPAQA/G-1 D-3 November 2002
-------
manager— a member of an organization whose job involves directing work that is performed by
others. Managers can be subdivided into:
• senior manager—strategic decision makers, developers of policy (e.g., agency director,
research laboratory director, city manager)
• middle manager—tactical decision makers, implementors of policy (e.g., office and
department heads, branch chief, foreman)
• operational supervisor—on-the-spot decision makers, immediately responsible for the work
of non-management staff members (e.g., section supervisors, team leader)
• quality manager—technical staff member principally responsible for oversight and
implementation of the quality system within an organization.
measurement and testing equipment — tools, gauges, instruments, sampling devices, or systems
used to calibrate, measure, test, or inspect in order to control or acquire data to verify conformance to
specified requirements.
measurement protocol — a specified procedure for making observations or performing analyses to
determine the characteristics of interest for each sampling unit. Measurement protocols include the
procedures for collecting a physical sample, handling and preparing the physical sample, and applying
an analytical method to obtain a result.
method — a body of procedures and techniques for performing an activity (e.g., sampling, chemical
analysis, quantification), systematically presented in the order in which they are to be executed.
mission statement — a short and clear expression of an organization's core principles.
nonconformance — a deficiency in a characteristic, documentation, or procedure that renders the
quality of an item or activity unacceptable or indeterminate; nonfulfillment of a specified requirement.
objective evidence — documented statement of fact, other information, or record, either
quantitative or qualitative, pertaining to the quality of an item or activity, based on observations,
measurements, or tests that can be verified.
observation — an assessment conclusion that identifies a condition (either positive or negative) that
does not represent a significant impact on an item or activity. An observation may identify a condition
that has not yet caused a degradation of quality.
Final
EPAQA/G-1 D-4 November 2002
-------
organization structure — the responsibilities, authorities, and relationships, arranged in a pattern,
through which an organization performs its functions.
peer review— a documented critical review of work generally beyond the state of the art or
characterized by the existence of potential uncertainty, conducted by qualified individuals (or an
organization) who are independent of those who performed the work but collectively equivalent in
technical expertise (i.e., peers) to those who performed the original work. Peer reviews are conducted
to ensure that activities are technically adequate, competently performed, properly documented, and
satisfy established technical and quality requirements. Peer reviews are an in-depth assessment of the
assumptions, calculations, extrapolations, alternate interpretations, methodology, acceptance criteria,
and conclusions pertaining to specific work and of the documentation that supports them. Peer reviews
provide an evaluation of a subject where quantitative methods of analysis or measures of success are
unavailable or undefined, such as in research and development.
procedure — a specified way to perform an activity.
process — a set of interrelated resources and activities that transforms inputs into outputs. Examples of
processes include analysis, design, data collection, operation, fabrication, and calculation. The
transformation of inputs produces a value-added product or service, and the value of each step of a
process can be determined by how much it contributes to meeting customer expectations. Process
mapping, brainstorming, morphological analysis, and other examination techniques are commonly used
to analyze work processes in the pursuit of quality.
project — an organized set of activities within a program.
quality — the totality of features and characteristics of a product or service that bears on its ability to
meet the stated or implied needs and expectations of the user.
quality assurance — an integrated system of management activities involving planning,
implementation, assessment, reporting, and quality improvement to ensure that a process, item, or
service is of the type and quality needed and expected by the client.
quality assurance program description/plan — see Quality Management Plan
quality assurance project plan — a formal document describing in comprehensive detail the
necessary quality assurance, quality control, and other technical activities that must be implemented to
ensure that the results of the work performed will satisfy the stated performance criteria. The QA
Project Plan components are divided into four classes: (1) Project Management,
(2) Measurement/Data Acquisition, (3) Assessment/Oversight, and (4) Data Validation and Usability.
Final
EPAQA/G-1 D-5 November 2002
-------
Requirements for preparing QA Project Plans can be found in EPA Requirements for Quality
Assurance Project Plans (EPA QA/R-5).
quality control — the overall system of technical activities that measures the attributes and
performance of a process, item, or service against defined standards to verify that they meet the stated
requirements established by the customer; operational techniques and activities that are used to fulfill
requirements for quality. The system of activities and checks used to ensure that measurement systems
are maintained within prescribed limits, providing protection against "out of control" conditions and
ensuring the results are of acceptable quality.
quality control sample — an uncontaminated sample matrix spiked with known amounts of analytes
from a source independent of the calibration standards.
quality management plan — a document that describes a quality system in terms of the organizational
structure, policy and procedures, functional responsibilities of management and staff, lines of authority,
and required interfaces for those planning, implementing, documenting, and assessing all activities
conducted.
quality policy — a written expression of senior management's overall intentions and direction of an
organization as regards quality, including objectives for quality and commitment to quality.
readiness review— a systematic, documented review of the readiness for the start-up or continued
use of a facility, process, or activity. Readiness reviews are typically conducted before proceeding
beyond project milestones and prior to initiation of a major phase of work.
record (quality) — a document that furnishes objective evidence of the quality of items or activities
and that has been verified and authenticated as technically complete and correct. Records may include
photographs, drawings, magnetic tape, and other data recording media.
repeatability — the degree of agreement between independent test results produced by the same
analyst, using the same test method and equipment on random aliquots of the same sample within a
short time period.
requirement — a formal statement of a need and the expected manner in which it is to be met.
self-assessment — the assessments of work conducted by individuals, groups, or organizations
directly responsible for overseeing and/or performing the work.
Final
EPAQA/G-1 D-6 November 2002
-------
service — the result generated by activities at the interface between the supplier and the customer, and
the supplier internal activities to meet customer needs. Such activities in environmental programs include
design, inspection, laboratory and/or field analysis, repair, and installation.
significant condition — any state, status, incident, or situation of an environmental process or
condition, or environmental technology in which the work being performed will be adversely affected
sufficiently to require corrective action to satisfy quality objectives or specifications and safety
requirements.
specification — a document stating requirements and referring to or including drawings or other
relevant documents. Specifications should indicate the means and criteria for determining conformance.
standard operating procedure — a written document that details the method for an operation,
analysis, or action with thoroughly prescribed techniques and steps and that is officially approved as the
method for performing routine or repetitive tasks.
supplier— any individual or organization furnishing items or services or performing work according to
a procurement document or a financial assistance agreement. An all-inclusive term used in place of any
of the following: vendor, seller, contractor, subcontractor, fabricator, or consultant.
surveillance (quality) — continual or frequent monitoring and verification of the status of an entity and
the analysis of records to ensure that specified requirements are being fulfilled.
technical review— a documented critical review of work that has been performed within the state of
the art. The review is accomplished by one or more qualified reviewers who are independent of those
who performed the work but are collectively equivalent in technical expertise to those who performed
the original work. The review is an in-depth analysis and evaluation of documents, activities, material,
data, or items that require technical verification or validation for applicability, correctness, adequacy,
completeness, and assurance that established requirements have been satisfied.
technical systems audit — a thorough, systematic, on-site qualitative audit of facilities, equipment,
personnel, training, procedures, record keeping, data validation, data management, and reporting
aspects of a system.
traceability — the ability to trace the history, application, or location of an entity by means of
recorded identifications. In a calibration sense, traceability relates measuring equipment to national or
international standards, primary standards, basic physical constants or properties, or reference
materials. In a data collection sense, it relates calculations and data generated throughout the project
back to the requirements for the quality of the project.
Final
EPAQA/G-1 D-7 November 2002
-------
validation — confirmation by examination and provision of objective evidence that the particular
requirements for a specific intended use have been fulfilled. In design and development, validation
concerns the process of examining a product or result to determine conformance to user needs.
variability — observed difference attributable to heterogeneity or diversity in a population. Sources of
variability are the results of natural random processes and stem from environmental differences among
the elements of the population. Variability is not usually reducible by further measurement but can be
better estimated by increasing sampling.
variance (statistical) — a measure or dispersion of a sample or population distribution.
verification — confirmation by examination and provision of objective evidence that specified
requirements have been fulfilled. In design and development, verification concerns the process of
examining a result of a given activity to determine conformance to the stated requirements for that
activity.
Final
EPAQA/G-1 D-8 November 2002
------- |