Publication Numbers: EPA: DoD: DOE: Intergovernmental Data Quality Task Force Uniform Federal Policy for Implementing Environmental Quality Systems Evaluating, Assessing, and Documenting Environmental Data Collection/Use and Technology Programs Wk T ¦ in Interim Final Version 1 November 2000 Distribution A Approved for public release; distribution is unlimited ------- THIS PAGE INTENTIONALLY LEFT BLANK ------- Intergovernmental Data Quality Task Force Uniform Federal Policy for Implementing Environmental Quality Systems Evaluating, Assessing, and Documenting Environmental Data Collection/Use and Technology Programs Interim Final Version 1 November 2000 ------- THIS PAGE INTENTIONALLY LEFT BLANK ------- TABLE OF CONTENTS INTRODUCTION 1 1,0 MANAGEMENT AND ORGANIZATION 7 1.1 Mission and Organization 7 1.2 Quality System Roles, Responsibilities, and Authorities 10 1.3 Resources to Support the Quality System 12 1.4 Quality System Customers and Suppliers 13 1.5 Communication of the Quality System ................................ 16 1.6 Management Assessment of the Quality System 17 2.0 QUALITY SYSTEM AND DESCRIPTION 19 2.1 Documentation of the Quality System 19 2.2 Quality System Tools 21 3.0 PERSONNEL QUALIFICATION AND TRAINING 23 3.1 Establish Technical Proficiency and Quality Requirements at the Organization, Program, and Project Levels 23 3.2 Evaluate Training Needs ........................................... 23 3.3 Prioritize Training Resources 24 3.4 Provide Training 24 3.5 Provide Access to Information 24 3.6 Training Effectiveness and Retraining 25 3.7 Documentation of Training 25 4.0 PROCUREMENT OF PRODUCTS, SERVICES, AND ACTIVITIES 27 4.1 Planning and Control 27 4.2 Assessment and Verification 27 4.3 Procurement Documents and Records 28 5.0 DOCUMENTS AND RECORDS 29 5.1 Identification of Technical and Quality-Related Documents and Records Requiring Controls 29 5.2 Description of Systems for Controlling Technical and Quality-Related Documents 30 5.3 Tracking and Retrieval of Technical and Quality-Related Documents and Records 31 Interim Final, Version 1, November 2000 l ------- TABLE OF CONTENTS (Continued) 6,0 COMPUTER HARDWARE AND SOFTWARE 33 6.1 Describe the Process, Roles, and Responsibilities for Configuration Testing and Documentation 34 6.2 Describe the Process, Roles, and Responsibilities for Managing Changes to Hardware and Software Configurations 35 6.3 Describe the Process, Roles, and Responsibilities for Developing and Evaluating Software 35 6.4 Describe the Process, Roles, and Responsibilities for Verifying That the Data Being Compiled and the Quality Control and Maintenance of These Data Are Appropriate for Their Intended Use 35 6.5 Describe the Process, Roles, and Responsibilities for Developing Procedures To Ensure That Historical Files Are Documented and Can Be Recovered 36 7.0 PLANNING 37 7.1 Management of Systematic Planning ... 37 7.2 Elements of Systematic Planning Processes 40 8.0 MANAGEMENT OF WORK PROCESS IMPLEMENTATION ................. 41 8.1 Procedures for Implementation in Accordance with Plans 41 8.2 Managing Quality Implementation Through the Use of SOPs 42 8.3 Work Processes Implemented Through Contracts and Other Assistance Agreements 45 9.0 ASSESSMENT AND RESPONSE ......................................... 47 9.1 Identification and Planning of Assessments 48 9.2 Design and Implementation of Assessments 48 9.3 Determination and Evaluation of Assessor Qualifications 51 9.4 Responsibilities and Authority of Assessors 51 9.5 Roles and Responsibilities of Contractors and Others in Performing Assessments 51 9.6 Documentation, Reporting, and Review of Assessments 52 9.7 Response to and Follow-up After Assessments 53 10.0 QUALITY IMPROVEMENT ............................................. 55 10.1 Quality System Improvement Process 55 10.2 Corrective Action for Quality-Related Problems 55 10.3 Creating an Environment for Facilitating and Promoting Quality Improvement 56 Interim Final, Version 1, November 2000 li ------- TABLE OF CONTENTS (Continued) APPENDIX A ELEMENTS OF A SYSTEMATIC PLANNING PROCESS 57 APPENDIX B ELEMENTS OF COMPUTER HARDWARE AND SOFTWARE AND INFORMATION TECHNOLOGY 65 APPENDIX C THE INTERGOVERNMENTAL DATA QUALITY FRAMEWORK 73 APPENDIX D QUESTIONS AND ANSWERS REGARDING THE IDQTF UNIFORM FEDERAL POLICY 81 APPENDIX E GLOSSARY 91 APPENDIX F IDQTF MEMBERS 97 Interim Final, Version 1, November 2000 in ------- THIS PAGE INTENTIONALLY LEFT BLANK Interim Final, Version 1, November 2000 iv ------- INTRODUCTION Purpose The Uniform Federal Policy for Implementing Environmental Quality Systems (IIFP) outlines essential elements of a Quality System1 for management of environmental data collection and use and environmental technology programs. The UFP will serve as a high-level policy for documenting and implementing acceptable Quality Systems for Federal agencies.2 The UFP provides a framework to ensure that essential elements are addressed. The UFP must be used to develop anew Quality System or to evaluate the adequacy of an existing Quality System. The results of that evaluation must then be used to develop plans for correcting identified deficiencies. The policy was developed as a joint initiative between the U.S. Environmental Protection Agency (EPA), the Department of Defense (DoD), and the Department of Energy (DOE) to resolve data quality inconsistencies and/or deficiencies to ensure that: ~ Environmental data are of known and documented quality and suitable for their intended uses, and ~ Environmental data collection and technology programs meet stated requirements. Ultimately, the benefits of a consistent policy for Quality Systems across Federal agencies include: ~ Improved effectiveness of Federal env ironmental programs by focusing on results, quality of data and services, and customer satisfaction; ~ Clarification of roles and responsibilities in managing and overseeing environmental data and environmental technology programs; ~ Sufficient confidence in the systems such that duplication of oversight efforts are minimized; and ~ Enhanced accountability and public confidence in environmental decisions. 'The term "Quality System." as used in this document, is adopted from ANSI/ASQC E4, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs, ANSI/ASQC E4 states that a Quality System is "A structured and documented management system describing the policies, objectives, principles, organizational authority, responsibilities, accountability, and implementation plan of an organization for ensuring quality in its work processes, products (items), and services. The Quality System provides the framework for planning, implementing, and assessing the work performed by an organization and for carrying out required quality assurance (QA) and quality control (QC) activities 2Whenever the term "Federal agencies" is used in this document, it refers to agencies, departments, and instrumentalities of the United States. Interim Final, Version 1, November 2000 1 ------- Scope This document provides requirements and guidelines to Federal agencies for documenting and implementing Quality Systems for management of environmental data collection and use and environmental technology programs. It becomes the policy for an agency when formally adopted by that agency: each agency will determine how best to implement the policy. This document represents a voluntary consensus policy. Implementation is, therefore, not subject to oversight by another Federal agency or to a Notice of Violation if one agency fails to implement all or part of the policy. Background In 1997, an audit report from the EPA Inspector General (Audit Report No. El SKB6-09- 004107100132) examined laboratory data quality at Federal facility National Priorities List Superfiind sites. An audit report from the DoD Office of the Inspector General (OIG Report 97-098, Laboratory Support Service for Environmental Testing, February 21,1997) addressed similar issues. These reports found that real or perceived inconsistencies and deficiencies in data quality within and across governmental organizations had resulted in greater costs, time delays, and the potential for increased risk. In response to these and other audit reports, as well as to environmental data quality issues related to Federal facilities in general, EPA established the Intergovernmental Data Quality Task Force (IDQTF), chaired by the Director of the Federal Facilities Restoration and Reuse Office (FFRRO). The IDQTF operates as a partnership, reaching decisions through consensus. In addition to the chair, other consensus members of the task force3 include: ~ The Department of Defense, represented by the DoD Environmental Data Quality Workgroup (EDQW) chairperson; ~ The Department of Energy, represented by the Office of Safety, Health and Security (EM-5); ~ The EPA Quality Staff, Office of Environmental Informat ion; ~ The EPA Offices of Emergency and Remedial Response and Solid Waste in the Office of Solid Waste and Emergency Response; and 3Other EPA offices and DoD components participated in various IDQTF subgroups and meetings to provide technical expertise. In addition to the above consensus members, participation of various other Federal agencies and EPA offices was actively sought throughout the development of initial IDQTF products. These included the Departments of Agriculture, Commerce (National Oceanographic and Atmospheric Administration), and Interior (U.S. Geological Survey), who were invited to become consensus members. Interim Final, Version 1, November 2000 2 ------- ~ EPA Regional Offices, represented by quality assurance staff from EPA Regions 1, 2, and 8 and Federal Facility Program staff in Region 5.4 DoD and DOE entered into separate MOUs with EPA, These MOUs have the following goals: ~ The development of a written consensus agreement on what constitutes an adequate Quality System for environmental data collection, and ~ The development of a consensus agreement that outlines the roles and responsibilities of EPA and Federal agencies with regard to data management. The parties agreed to meet these goals first by developing them in the context of hazardous waste program management with the understanding that this will establish a framework for further work under other environmental media programs. Basis for the Intergovernmental Quality System ANSI/ASQC E4 was selected as the basis for the intergovernmental Quality System because it is a national standard that specifically addresses environmental data collection and use and environmental technology programs. The E4 standard is written in three parts: ~ Part A addresses the management elements of a Quality System, ~ Part B addresses project-specific requirements related to the collection and evaluation of environmental data, and ~ Part C addresses the Quality System issues related to "design, construction and operation of environmental technology." The UFP is designed to clarify and provide additional direction for the implementation requirements of ANSI/ASQC E4 Part A. Other documents underdevelopment by the IDQTF will address ANST/ASQC E4 Part B requirements. These include: ~ Draft Federal Consensus Guidance for the Preparation of Quality Assurance Project Plans ~ Presumptive Quality Assurance/Quality Control Measures for Superfund 4 A full list of consensus members, alternates, and other regular participants in the IDQTF is found in appendix E of this document. Interim Final, Version 1, November 2000 3 ------- The IDQTF is not developing a project-specific work product to address ANSI/ASQC E4 Part C: Design, Construction, and Operation of Environmental Technology. To satisfy the second goal of the MOU, the IDQTF is developing an additional document, Roles and Responsibilities Framework for Federal Facility Oversight. Documentation of Intergovernmental Quality Systems A Quality System is documented at an organizational level in a Quality Management Plan (QMP) and at a program or project level in a Quality Assurance Project Plan (QAPP), or their functional equivalents. The organizational QMP will detail how the Quality System is to be implemented throughout the organization for which it is written. It will include information by which the organization will manage, plan, implement, assess, conduct corrective action upon, and continually improve the products, services, and activities involved in environmental data collection or use or environmental technology management. Organization This document is organized according to the 10 elements that must be addressed in an organization's QMP in order to conform with ANSI/ASQC E4 and this document: 1. Management and Organization 2. Quality System and Description 3. Personnel Qualification and Training 4. Procurement of Products, Services, and Activities 5. Documents and Records 6. Computer Hardware and Software 7. Planning 8. Management of Work Process Implementation 9. Assessment and Response 10. Quality Improvement Each of these elements is covered in a separate section of the remainder of this document. Requirements and Guidance The requirements described in this policy are intended to facilitate implementation of ANSI/ASQC E4 with regard to environmental data collection and use and environmental technology development and implementation. ANSI/ASQC E4 covers both mandator}' specifications and nonmandatory guidelines for Quality Systems. In accordance with E4, this document addresses both requirements and guidance. Requirements are presented in the text that makes up the bulk of this document, while guidance is presented in text boxes. The text boxes attempt to illuminate the requirements of this policy and enhance the reader's understanding. The text Interim Final, Version 1, November 2000 4 ------- boxes are not part of the forma! policy. The requirements of this document also incorporate by reference the definitions contained in the E4 standard. Relationship to Other Standards and Policies Questions have been raised by reviewers of this document as to the relationship between this document, international standards, and EPA policy documents. The text below is provided to describe those relationships. Relationship to International Standards The relationship of this policy to international standards is as follows: ~ ANSI/ASQC E4 conforms with the ISO 9000 series of standards, "International Standards for Quality Management," addressing Quality Systems. E4 and this policy document, however, are specifically written for environmental data collection and environmental technology programs. ~ The TSO 14000 series, Environmental Management addresses what an organization does to minimize harmful effects on the environment caused by its products, services, and activities. ISO 14000 does not address the issues covered by E4 or by the ISO 9000 series. ~ ISO/1 EC Guide 25 (superceded in 1999 by ISO 17025, General Requirements for the Competence of Testing and Calibration Laboratories) establishes specifications and requirements for calibration and testing of laboratory systems. ISO 25 is the basis for standards set by the National Environmental Laboratory Accreditation Conference, Chapter 5. While E4 does not explicitly address quality systems for environmental laboratories, it references ISO/1 EC Guide 25, and other national standards as the framework for other Quality Systems. Relationship to EPA Policy ANSI/ASQC E4 sets forth the general standards for a Quality System for environmental data collection and technology. EPA recently published a number of documents designed to implement E4 throughout EPA and for EPA funded activities. These documents include: 1. Policy and Program Requirements for the Mandatory Agencywide Qualitv Svstem (EPA Order 5360.1 A2). 2. EPA Quality Manual for Environmental Programs (EPA Manual 5360 A1), and 3. EPA Requirements for Quality Management Plans (EPA QA/R2), which establishes Quality System requirements for external organizations that implement programs for Interim Final, Version 1, November 2000 5 ------- EPA through funding mechanisms (e.g., contracts, grants, and other financial agreements), or through other agreements (e.g., interagency agreements). The EPA Order and the EPA Manual apply solely to EPA. This document differs from E4 and from QA/R2 in four important ways: ~ It provides implementation policies for Federal agencies; therefore, it is more detailed than either E4 or QA/R2. ~ It is intended to apply to Federal agencies, whereas QA/R2 applies only to organizations that implement programs for EPA. ~ It represents a consensus policy reflecting the views of the 1DQTF, ~ It benefits from the experience and lessons learned from a broader set of stakeholders involved in managing quality across several Federal agencies. This document is consistent with the EPA Order and EPA Manual. It is anticipated that Quality Management Plans (QMPs) that meet the requirements of the EPA Order will require little adjustment to conform to this policy. Interim Final, Version 1, November 2000 6 ------- 1.0 MANAGEMENT AND ORGANIZATION This section of a QMP defines management responsibilities and authorities of management for ensuring the quality of work products, monitoring work processes, and identifying and correcting system nonconformance. Key elements of this section include: ~ Management commitment to and accountability for quality. ~ Independent oversight of quality, ~ Provision of resources commensurate with quality objectives, and ~ Commitment to performance assessment and continuous improvement of the Quality- System. The "Management and Organization" section of the QMP establishes the framework that, fosters implementation of the Quality System. Managers within each organization who are responsible for the mission and programs covered by the Quality System are held accountable for quality. 1.1 Mission and Organization a. Define the mission(s) of the organization(s) covered by the Quality System as the mission relates to environmental data collection and use and or environmental technology programs. Identify the programs to which the Quality System is applied (e.g., the remediation of hazardous waste as performed under the regulatory authorities of the Resource Conservation and Recovery Act [RCRA] and the Comprehensive Environmental Response, Compensation, and Liability Act [CERCLA], or compliance assurance under the Clean Water Act's National Pollutant Discharge Elimination System [NPDES]). Programs Requiring Quality Management Controls: In this section, you are asked to give careful attention to identification of the specific programs that require quality management controls. This discussion must also address where oversight is needed to ensure data quality, and where internal coordination of quality assurance and quality control (QA/QC) activities among the group's organizational units need to occur. Interim Final, Version 1, November 2000 7 ------- Using Tables to Simplify the QMP; See text box in Section 2.0 for an example of table headings that could be used to simplify writing of the mission and organization section of the QMP. Example Mission Statements: "The Office of Analytical Services and Quality Assurance (GASQA).Js one of 30 Federal laboratories operated by EPA to support its mission to protect the environment. OASQA's primary function is to ensure the appropriate quality of scientific information for environmental decisionmaking in Region III." "The Department of Energy Nevada Radioactive Waste Acceptance Program (RWAP) will facilitate management of radioactive waste in a safe and compliant manner to ensure the integrity of the Nevada test site disposal operations, while maintaining the protection of the public, workers, and die environment." "The Army will develop and implement measures to protect and sustain the environment in support of military operations, installation management, and material development." b. Provide an organizational chart that shows all organizational units associated with achieving the mission(s) covered by the Quality System. Identify the communication reporting network and organizational relationships, the functions of each organizational unit in achieving the mission, and the placement of the quality management function in each organizational unit. Definition of Organization: An organization is defined by ANSI/ASQC E4 as "...a company, corporation, firm, enterprise, or institution, or part thereof...that has its own functions and administration." When used in this document, the term "organization" refers to the entity within a Federal agency that has "its own functions and administration" and that is responsible for writing a QMP under this policy document. In some cases the term "organizational unit" is used to refer to a unit of a larger organization that may have its own Quality System responsibilities, and for which Quality System information and procedures will require documentation in the QMP. Interim Final, Version 1, November 2000 8 ------- Using References to Readily Available Documents; The use of attachments, readily available references, or Internet links for organization charts and tables with specific names, addresses, and telephone numbers allows for easy updating of this part of the QMP without having to change text that is otherwise still current. Include (such as with attachments to the QMP, references in the QMP to readily available documents, or links to the Internet in the QMP) organizational unit titles and organization manager and quality manager names. Include the street and electronic mail addresses and telephone numbers for QA managers and organizational unit managers. c. Describe the work performed by each organizational unit identified. Specify the products, services, and activities that are supported by environmental data collection and use and environmental technology programs. d. Identify other entities that the organization will work with to achieve its mission. Level of Specificity in Describing Other Entities: You may describe these entities generic-ally (e.g., States or contractor) or specifically (e.g., by State or contractor name), depending upon the specificity the program requires. For example, a large contractor with a multi-year contract might be listed by name, while many small contractors with short-term tasks might be listed generically as categories of contracts. e. State the Organization's Quality Policy. Management must establish and implement a Quality Policy. The QMP must state the organization's policy to demonstrate management commitment to: ~ Integration of the Quality System into management policies and systems; ~ A Quality System that promotes sound science, consistency, efficiency, communication, responsiveness to customers, and innovation; ~ A Systematic Planning Process that results in the generation and use of environmental data of the type, quantity, and quality needed for decision making; ~ The use of QAPPs (e.g., project-specific quality plans, however named) in documenting project-specific data collection; Interim Final, Version 1, November 2000 9 ------- ~ Design, construction, and operation of environmental technologies that meet the established specifications; and ~ Provision of funding needed to accomplish the defined quality objectives. 1.2 Quality System Roles, Responsibilities, and Authorities a. Identify the responsibilities, authority and independence, and resources associated with the management of the Quality System, 1. Identify a Quality Assurance (QA) Manager(s), or person(s) assigned equivalent duties, however named, who is responsible for overseeing the effective implementation of the Quality System within each organizational unit identified in the QMP. * The QA Manager, however named, must function independently of direct environmental data generation and use, model development, and technology development. The QA Manager is responsible for reporting on quality issues to executive level management. 2. Describe the roles, responsibilities, and areas of independence of QA personnel in assessing, taking corrective actions on, and improving the Quality System. 3. Describe the lines of communication between QA personnel and management personnel and program/project staff in the organization. 4. Describe the corrective action process(es) including criteria used for stopping work and resource expenditure when there is cause to believe that a quality control failure or Quality System nonconformance has occurred. For each organization or organizational unit covered by this QMP, identify the responsible individuals) authorized to stop work. b. Specify the QA/QC responsibilities for senior management, line managers, program and project staff, and QA personnel in planning, implementing, assessing, and improving the Quality System, and the technical activities associated with that Quality System. Describe how attention to quality requirements is incorporated into personnel performance standards. Interim Final, Version 1, November 2000 10 ------- Accountability for the Quality System: Successful development and. implementation of the organization's Quality System depend upon commitments to quality by individuals at every level of management and staff throughout the organization. The success of these commitments is measured in performance standards established at the beginning of a performance evaluation year and is assessed during and at the conclusion of the year. Different types of standards are appropriate for different responsibilities. Some examples are described below: ~ For the Executive Manager; Ensure that organizational levels are aware of Quality System responsibilities. Ensure that all management and technical staff have performance standards that hold them accountable for implementation of the Quality System. Participate in an annual assessment of the performance of the Quality System. Ensure that performance measures are established so that feedback on performance occurs and that corrective measures are taken in a timely manner. ~ For the Technical Manager: Ensure that all organizational levels are aware of Quality System responsibilities. Ensure that all management and technical staff have performance standards that hold them accountable for implementation of the Quality System. Ensure that customers and measurements of customer quality are identified for all products, services, and activities within your areas of responsibility. Ensure that a system is in place for communication with customers and suppliers. Establish a process for ensuring that customer needs are agreed to and met. Establish a process for taking corrective action when customer needs are not met. c. Describe the roles and responsibilities of government contractors as well as holders of Federal Government-funded assistance agreements (e.g., States, universities, and nonprofit institutions operating through interagency agreements, grants, or other agreements) in implementing and overseeing government Quality Systems. Identify and describe the quality oversight functions for government contracts that are inherently governmental and must be performed by government personnel, as well as those that can be delegated to contractor personnel. Identify the quality oversight functions that are inherent to the Federal Government in the oversight of its assistance agreements, and those that can be delegated to other governmental entities and assistance agreement holders. Interim Final, Version 1, November 2000 11 ------- Inherently Governmental Functions; An "inherently governmental function" is defined, in the Federal Acquisition Regulations (FAR Part 07, Subpart 7.5,7501), as well as in the Office of Management and Budget (OMB) circular A-76 and supplemental guidance. The OMB A-76 definition notes that "an inherently governmental function is a function which is so intimately related to the public interest as to mandate performance by Government employees.... These functions include those activities which require either the exercise of discretion in applying Government authority or the use of value judgement in making decisions for the Government." The OMB circular and supplementary guidance provides numerous examples of functions that are and are not inherently governmental. According to the FAR, the definition of inherently governmental function is "a policy determination, not a legal determination." 1.3 Resources to Support the Quality System a. Describe the process used by senior management to provide resources (e.g., staff, travel, training, contractor support, equipment, and supplies) to plan, implement, assess, and improve the Quality System. 1. Resource Planning: For each organization and program covered by this QMP, identify the responsibilities and procedures for annual estimation of resources required for managing quality. Identify priority funding areas. 2. Evaluation of Expenditures: Describe an annual process for estimating expenditures and evaluating the adequacy of these resources to address potential weaknesses in the management of quality. b. Resource planning and expenditure estimates must address: Government personnel, support to Government personnel (e.g.. training, travel), and costs associated with contractor responsibilities where applicable. Interim Final, Version 1, November 2000 12 ------- Understanding What's Required for Estimating Resources; Tri this section, you arc asked to develop a process for planning adequate resources and evaluating whether the budgeted resources sufficiently met targeted quality goals. Key elements of this requirement include: ~ Deliberate use of the term "estimates" for both planning and evaluation so that significant levels of detail are not added to the planning, budgeting, and program evaluation process; ~ Recognition that you may not receive all the resources you request and that you will have to target expenditures to the most critical areas; and ~ An annual evaluation of estimated expenditures tied to the assessment of quality required in other sections of this document. 1.4 Quality System Customers and Suppliers A Word About Customers and Suppliers; Each product, service, or activity of environmental data collection or environmental technology programs involves both customers and suppliers. These relationships are often complex, and customers can sometimes be suppliers and vice versa. Customers and suppliers may be both internal and external to the organization. ~ Customers: For a Federal facility collecting environmental data for a CERCLA cleanup, for example, the customer! s) may include members of the public, the regulators, or even the facility Remedial Project Manager. When a Federal facility is collecting discharge information to support its compliance with a State NPDES permit, the customer will be the State agency that will review the compliance information. ~ Suppliers: Contractors, subcontractors, private and public laboratories, and many other providers of equipment, supplies, and services are the typical suppliers of environmental data and technology programs. It is important to recognize, however, that they may also be customers seeking guidance and direction from contract management personnel and technical staff. a. Quality System Customers: This section is designed to ensure that the Quality System includes a thoughtful process for identifying the customer for environmental data collection and use and environmental technology programs. Identify customers and their specifications, and describe the process by which those specifications are built into the systematic planning for and evaluation of environmental data collection and use and environmental technology development and implementation. Interim Final, Version 1, November 2000 13 ------- A Word About Customer Needs, Expectations, and Specifications: ANSI/ASQC E4 uses the terms "needs and expectations" when referring to the constellation of specifications that are identified by customers. These specifications can be assigned different levels of priority depending upon whether they are "legal requirements" or customer expectations or preferences separate from legal requirements. For example, it will be an absolute requirement that a treatment facility meet legally mandated effluent discharge limits. A number of choices exist for how the treatment facility is built and designed to meet these requirements. While the customers (owner of the constructed treatment facility) may expect or prefer certain design and operation features, the ability to deliver these features may be dependent on a variety of resource (financial, schedule, and personnel) and contractual constraints. Identification of customer specifications will often require distinction between requirements, expectations, and preferences, and will involve setting priorities in accordance with constraints to achieving customer demands. These constraints may be financial, contractual, and technical. Obviously more flexibility will exist in negotiating changes to customer expectations and preferences than m negotiating requirements. Although this document uses the shorthand term "specifications" when describing customer requirements, expectations, and preferences, that term is meant to reference the full range of customer demands. 1. Identification of Quality System Customers and Their Specifications; Identify the customers for each product, sendee, and activity that are the principal concern of the Quality System. Describe how customers' requirements, expectations, and preferences (also called customer specifications) are identified by senior management. Specify senior management responsibilities in defining the work objectives to satisfy the customer specifications. 2. Development of Quality Measures of Customer Specifications: Describe the process for developing quality measurements associated with customer specifications for the products, services, and activities identified for each organizational unit covered by this QMP. Briefly describe the quality controls established to evaluate the conformance of specific products, services, and activities to customers' specifications. 3. Negotiation of Changes to Customer Specifications and/or Quality Measurement: Describe the process for negotiating changes in customer specifications based on resource limitations or other constraints (e.g., technical, political). Describe a process for negotiating acceptable quality measures with customers when feedback from suppliers identifies resource or other constraints that prevent achieving the customers' desired objectives. Outline appropriate dispute resolution procedures for disagreements over quality measures. Existing dispute resolution mechanisms can be cited (e.g., Interim Final, Version 1, November 2000 14 ------- Federal facility agreements that govern CERCLA cleanups often have well- defined dispute resolution procedures that meet this requirement). 4. Accessibility to Information and Complaint Management; Describe how senior management provides customers with access to customer service personnel or the equivalent to enable them to seek assistance, conduct business, or voice complaints. Describe the organization's complaint management process, including the chain of command for management of complaint resolution. Describe a management process and roles and responsibilities that ensure that complaints are resolved promptly and effectively and that complaints are compiled and analyzed for improving performance at all organizational levels. 5. Determining Customer Satisfaction; Briefly describe the process and information used to determine customer satisfaction or dissatisfaction with products and services. Describe the process that the organization uses to follow up with customers to ensure prompt corrective action and feedback to customers on performance issues. Establishing Quality Measurements: The products, services, and activities of environmental programs are quite varied. The challenge for each organizational unit is to identify the specific product, service, or activity associated with environmental data collection and use and environmental technology development and implementation, and to develop a measurable standard of quality. For example, if an organization's environmental data collection product is compliance monitoring to compare the effluent concentrations discharged against NPDES permit requirements, the customer for this activity may be the State regulatory agency. In this case, a customer quality measurement imposed on the supplier may be the percentage of samples analyzed by a contractor laboratory over a 1-month period that meet data validation and verification requirements. b. Quality System Suppliers: Communication to and feedback from suppliers are critical elements of the Quality System's ability to meet customer specifications. This section is designed to ensure that suppliers are identified and that a process for communication to and feedback from suppliers in meeting customer needs is put in place. (Section 4.0 contains more-detailed requirements related to procurement.) 1. Identification of Suppliers: Identify the suppliers and backup suppliers of the products, services, and activities for which the organization documenting the Quality System is responsible. Interim Final, Version 1, November 2000 15 ------- Identification of Suppliers: This refers to suppliers whose products, services, and activities directly affect the quality of the organization's products, services, and activities. As many of these specific suppliers are project-dependent and may not be known in advance, the identification of generic types of suppliers may be appropriate. 2, Communication with Suppliers; Describe how customers' specifications are communicated to suppliers to ensure that the product, service, or activity meets these specifications. (See Section 1.5 below.) 3. Feedbackfrom Suppliers: Describe how feedback is obtained from suppliers on constraints (e.g., resources, technology) that affect their ability to meet specifications established for the products, services, and activities for which they are responsible. 1.5 Communication of the Quality System a. Describe how the Quality System is communicated to personnel at all organizational levels by both management (senior and line) and QA personnel. b. Describe how key decisions, directions, and specifications are effectively communicated to all customers and suppliers, both internal and external. Interim Final, Version 1, November 2000 16 ------- About Communication: ~ Importance: The most well-thought-out policies, procedures, and technical guidance to manage quality are only effective if communicated to all those who supply the organization's needs. Communication of these policies to customers prior to their adoption is essential to ensure that the policies, procedures, and guidance are aligned with customer needs. If policies and procedures are not communicated to customers until after adoption, the organization and its customers may not have the same expectations as to how the Quality System is maintained and how customer needs will be met. Differing expectations are a frequent source of conflict in the implementation of environmental programs. ~ Who? As described in this section, customers and suppliers incorporate the universe of external and internal interests who are responsible for and are users of environmental data collection and environmental technology programs. Sometimes these individuals are also described as "stakeholders" and include: employees of the organizational units; support contractors, subcontractors, and their field personnel; State and Federal regulatory agencies; local government personnel; Indian Tribes; environmental organizations; reuse committees; Restoration Advisory Boards; and many others. ~ How? Effective communication requires a thoughtful commitment to thinking through the nature of the audiences and their needs for information. A communication strategy can be used to outline communication of different information to different audiences. 1.6 Management Assessment of the Quality System a. Describe the Regular Management Assessment Process: For each organizational unit covered by the organization's QMP, describe how the implementation and effectiveness of the Quality System are regularly assessed, and documented. Such assessment will take place at least annually. Routine Management Assessments Are Important Quality Management Tools: [J To evaluate whether the system produces environmental data and technologies of a quality adequate to support the organization's mission; ~ To evaluate the adequacy of resources available to support the Quality System; and ~ To identify corrective actions that can improve, modify, and/or better implement the Quality System. b. Identify Quality Measurements for the Quality System: Describe the process for assessing successful implementation of the Quality System. Interim Final, Version 1, November 2000 17 ------- Management Quality System Measurements: Assessments of management Quality System performance and effectiveness are different but directly related to measurements of customer satisfaction. While customer Quality System measurements relate directly to customer specifications, management Quality System assessments will measure: ~ The frequency with which customer satisfaction measurements are achieved, ~ Staff knowledge of the customers and their quality requirements, and ~ The existence of a feedback system that systematically conveys customer requirements to suppliers. c. Describe the management responsibilities for the assessment of the Quality System. Each organizational unit must be assessed. d. Describe the processes used to report the results of assessments to senior management, determine appropriate corrective actions, and evaluate the effectiveness of the corrective actions. Other elements of assessment procedures and requirements are contained in Section 9.0 of this document. Interim Final, Version 1, November 2000 18 ------- 2.0 QUALITY SYSTEM AND DESCRIPTION This section specifies that organizations must document a Quality System in a QMP or equivalent document. Any QMP written in conformance with this UFP will also be in conformance with ANSI/ASQC E4. While this UFP establishes specific requirements for documenting a Quality System, it is not necessary to write a lengthy narrative that sequentially addresses each requirement. Tables and other simplifying charts are useful tools for communicating this information. For example, most of the requirements found in Section 1.1 a-c can be consolidated and. addressed in a table such as the one below. Mission Covered Organizational Products, Services, and/or Program Unit Activities Example Table Headings for Section 1.1 of the QMP. Whether documented in a Quality Management Plan organized in the manner of this document or in some equivalent document, writing down the Quality System processes and procedures is the foundation for a Quality System. Documented processes are the starting place for communication of responsibilities and requirements throughout the organization. Common expectations, when documented, yield accountability. 2.1 Documentation of the Quality System a. The Quality System of an organization performing environmental data collection and implementing environmental technology programs must be formally documented in a QMP or an equivalent document. Other Documents Included in Quality System Documentation; The Quality System itself will include other documents, such as Quality Assurance Project Plans and standard operating procedures. Furthermore, documents may be in electronic format when the controls in Section 5.0 are applied to the electronic document system. b. The QMP or equivalent document for each organization must also describe the subdivisions of the organization required to provide either separate QMPs or supplemental information to be referenced in the parent organization's QMP. Responsibilities for receiving and incorporating supplemental information into the organization's QMP must be identified. Interim Final, Version 1, November 2000 19 ------- c. If documentation of the Quality System is not consistent with the headings of this document, state how the organization's Quality System is documented and provide cross-references to the UFP. d. Describe the roles and responsibilities of senior management and QA personnel for developing, reviewing, approving, and revising Quality System documentation. Describe the frequency (at least annually) with which Quality System documentation must be reviewed and updated. Updating QMPs; Many aspects of the Quality System will remain constant over a number of years: U An annual update of the entire QMP is not necessary (although an annual review must be performed), ~ Frequent updates are needed only to change certain aspects (e.g., organizational changes, designated quality management personnel). ~ Elements of the QMP that are subject to frequent change can best be handled as tables and/or appendices that can be easily revised, or they can be referenced or linked to other readily available documents. ~ The initial QMP or equivalent document should include the expected schedule for updates, such as "annually if needed." e. For each program in the organization, provide a list of all other quality-related documents, including suborganization QMPs, program management plans, quality manuals, organizational policy statements, requirements and guidance documents, and standard operating procedures. Identifying Other Quality-Related Documents: This requirement can be accomplished either by attaching a list or by identifying (1) where the list is maintained, (2) the process for maintaining the list, and (3) personnel responsible for maintaining the list. f. Provide a signature page for the QMP that demonstrates that (1) the QMP (or equivalent document) is the official policy of the organization, and (2) the QMP has been reviewed and approved for implementation by both senior management and QA staff. Interim Final, Version 1, November 2000 20 ------- Electronic Reviews and Approvals: Reviews and approvals may be "electronic" when users and stakeholders have access to electronic document control systems and the policy of the organization allows electronic approvals. 2.2 Quality System Tools The QMP must discuss the principal tools utilized in the Quality System and describe how they are used to implement the Quality System. These tools include, but are not limited to, QMPs, management assessments (self and independent), Systematic Planning Processes, QAPPs, requirements and guidance documents, standard operating procedures, technical assessments (self and independent), data quality assessments, and corrective action tools. The discussion must also identify how and when these tools of the Quality System are to be applied to individual products, services, and activities. Table Format Ties Quality System Tools to Function: A table may be a useful way of tying products, services, and activities to specific quality tools, A few examples are provided below; Product, Service, or Activity Quality System Management Tool Requirements Planning for collection of environmental data Data Quali ty Objectives Systematic Planning Process Quality Assurance Project Plan (QAPP) For each project requiring the collection of environmental data Analysis of environmental samples Aud it of laboratory quality system Conducted annually or verified that a recent audit to an equivalent quality standard has occurred within that time period Interim Final, Version 1, November 2000 21 ------- THIS PAGE INTENTIONALLY LEFT BLANK Interim Final, Version 1, November 2000 ------- 3.0 PERSONNEL QUALIFICATION AND TRAINING Qualified and properly trained personnel are key elements of a Quality System. The requirements of this section are designed to address how the organization will educate and train personnel (management and technical) to support both the implementation of the Quality System and the quality requirements of individual projects. 3.1 Establish Technical Proficiency and Quality Requirements at the Organization, Program, and Project Levels a. Training to Implement the Quality System: Describe how the organization's management and technical training support the Quality System. Define roles and responsibilities for implementing the training activities. Include basic training requirements for: ~ Senior managers, ~ Line managers, ~ Program and project staff, ~ Quality assurance personnel, and ~ Other staff responsible for quality-related functions as appropriate. Personnel must be provided the following training: 1. T raining that provides a working understanding of the Quality System, along with tools and techniques (e.g., technical, managerial, communication, and interpersonal skills ) to enable them to participate in planning, implementing, and assessing the Quality System. 2. Training based on project-specific requirements. Personnel must be trained and qualified, for the specific projects prior to the start of work. b. Certification and Qualification Requirements for Special Activities: Describe the process, roles, and responsibilities for establishing when formal certification or qualification are required for personnel performing certain specialized activities. Describe how personnel qualifications, experience, and education for specialized activities are evaluated to determine these requirements. 3.2 Evaluate Training Needs Describe how employee training needs are determined relative to the requirements in Section 3.1. The process of identifying training needs relates evaluation of personnel job proficiency to corrective action through training. Include the management process, roles, and responsibilities for: Interim Final, Version 1, November 2000 23 ------- ~ Tracking and evaluating personnel job proficiency for required tasks, ~ Identifying skills needed to perform required tasks, ~ Seeking employee input on training needs, and ~ Periodically evaluating, updating, and improving the training program, 3.3 Prioritize Training Resources Discuss roles and responsibilities for establishing training priorities and for obtaining and allocating funds. 3.4 Provide Training Describe how Quality System and pro]ect-specific training needs are met. Describe the roles and responsibilities for providing training. Address the full range of formal and informal training opportunities provided, including, for example: ~ Technical and management training, ~ Internal and external training courses and seminars, ~ On-the-job training, and ~ Training specifically targeted to enhance the understanding of and contribution of personnel to the Quality System. 3.5 Provide Access to Information Describe how individuals involved in implementing the Quality System and site-specific projects are kept informed of updated quality policies and technical and quality requirements and procedures. Continuing access to information on new requirements is an important aspect of ongoing training. The Information on Training Must Indicate: ~ How personnel arc notified of relevant changes to the Quali ty System and technical requirements; ~ How personnel can obtain direct access to changed requirements and guidance; and ~ How personnel are kept informed of the status of current and obsolete requirements. (See also Section 5.0, "Documents and Records.") Interim Final, Version 1, November 2000 24 ------- 3.6 Training Effectiveness and Retraining Describe the process, roles, and responsibilities for determining whether and when retraining is needed (e.g., when job requirements change or when job performance is deficient). 3.7 Documentation of Training Describe the process, roles, and responsibilities for developing and maintaining formal training records. Interim Final, Version 1, November 2000 25 ------- THIS PAGE INTENTIONALLY LEFT BLANK Interim Final, Version 1, November 2000 26 ------- 4.0 PROCUREMENT OF PRODUCTS, SERVICES, AND ACTIVITIES This section addresses procurement as an essential element of a complete Quality System. Suppliers must be held accountable for supplying products, services, and activities that meet specifications. The purchasing organization must be responsible for establishing specifications and acceptance criteria and for verifying that the supplier is capable of meeting them. The organization must have a formal process for assessing and documenting a supplier's Quality System. Remember, the adequacy of an organization's Quality System depends on the adequacy of its supplier's system. Refer to Section 8.3 for additional information on work processes implemented through contracts and other assistance agreements. 4.1 Planning and Control Describe the processes, roles, and responsibilities for: ~ Identifying purchased products, services, and activities (including items purchased through contracts and subcontracts) that directly affect environmental data and data- related decision quality and environmental technology programs; ~ Defining the specifications and acceptance criteria for products, services, and activities (see Section 1.4); and ~ Identifying which suppliers must have a documented Quality System that complies with these requirements. Every supplier of products, services, and activities that affect environmental data quality and environmental technology programs must have a documented Quality System. Government Specifications for Contract Quality Requirements: Recent revisions to the Federal Acquisition Regulations (FAR), 48 CFR, Part 46, Section 46.202-4, replaced references to government specifications for higher-level contract quality requirements with references to commercial specifications. They also require that the government contracting officer indicate in the solicitation which higher-level quality standards will satisfy the Government's requirements. Examples of higher-level quality standards are ISO 9001, ISO 9002, or ISO 9003; ANSI/ASQC Q9001, Q9002. or Q9003; QS-9000; AS-9000; ANSI/ASQC E4; and ANSI/ASME NQA-1. 4.2 Assessment and Verification a. Describe the process, roles, and responsibilities for evaluating the capability of a supplier to meet the specifications and acceptance criteria for the products, services, and activities to be provided. This process must be consistent with contracting requirements and should include the following activities: Interim Final, Version 1, November 2000 27 ------- 1. Reviewing the supplier's documented Quality System to ensure that it is consistent with these consensus requirements or with other appropriate standards; 2. Performing an on-site audit of the supplier's Quality System; 3. Inspecting products, services, and activities and evaluating objective evidence furnished by the supplier; and 4. Obtaining feedback from the supplier on constraints to meeting the requirements. b. Develop procedures for resolving disputes over quality with suppliers consistent with existing dispute resolution procedures and appropriate regulations, such as Federal Acquisition Regulations (FAR). If preexisting procedures are used, provide appropriate reference to them, c. Develop procedures that specify how changes are made, tracked, and communicated to the organization. 4,3 Procurement Documents and Records Describe the process, roles, and responsibilities for developing and maintaining procurement documents and records consistent with FAR and contracting requirements. Procurement Documents Should: U Contain information clearly describing the specifications and acceptance criteria for the products, services, and activities to be procured; ~ Explain how the supplier's conformance to specifications will be verified; ~ Be reviewed for accuracy and completeness before being provided to the supplier [Note: Both the procurement specialist and a person who is technically qualified to ensure that specifications and acceptance criteria are properly developed should perform this review]; and ~ Be reviewed at the same level of review arid approval when changes occur as was done for the original documents. Interim Final, Version 1, November 2000 28 ------- 5.0 DOCUMENTS AND RECORDS This section addresses the management of effective identification, preparation, control, and storage of technical and quality-related documents and records. The requirements of this section are designed to: ~ Identify required documents and ensure that their preparation meets the needs of customers and suppliers; ~ Ensure that the latest version of policies, requirements, and guidances are known and made available to users; and ~ Maintain a historical "record" that gives Quality System users access to the actual body of information that was critical to decision making at the time the decision was made. Defining Documents and Records: ~ ANSFASQC E4 defines a document as any written or pictorial information describing, defining, specifying, reporting, or certifying activities, requirements, procedures, or results pertaining to environmental operations. ~ A record is a document that furnishes objective evidence of the quality of products, services, or activities and that has been verified and authenticated as technically complete and correct. ~ For the purposes of this section, a document or record can be available in either print or electronic media. 5.1 Identification of Technical and Quality-Related Documents and Records Requiring Controls Describe the process, roles, and responsibilities for identifying technical and quality-related documents and records requiring control. Indicate which documents and records specify quality goals and objectives and demonstrate attainment of those goals and objectives. Interim Final, Version 1, November 2000 29 ------- Examples of Documents To Be Considered for Control Examples of Records Where Controls Are Considered Blueprints * Assessment Results and Findings * Design Documents * Calculations Quality Assurance Project Plans (QAPPs) Calibration Data Field Sampling and Analysis Plans « Data Usability Results ~ Quality Management Plans (QMPs) * Field Logbooks * Health and Safety Plans Inspection Results Policy and Guidance Documents * Instrument Test Data Technical Manuals Materials Testing Results ~ Workplans ~ Personnel Qualifications Other Types of Instructions, Procedures, and Sampling and Analytical QC Data Operating Guides * Sampling and Analytical Data Standard Operating Procedures Technical and Readiness Review Results 5.2 Description of Systems for Controlling Technical and Quality-Related Documents Describe the systems for managing technical and quality-related documents and records from preparation through disposal. Include document preparation tasks such as writing, categorizing, reviewing, approving, issuing, using, and revising. Include management tasks such as collecting, indexing, filing, storing, maintaining, protecting, archiving, retrieving, authenticating, distributing, and disposing, and otherwise managing technical and quality-related documents and records. System descriptions should specifically address such topics as: 1. The process for identifying, preparing, and adopting policies, guidances, regulations, and other instructions; 2. The proper format for documents and records; 3. The process for updating and recording changes to requirements, guidance, and procedural documents in response to changes to regulations, requirements, and policies, as well as changes in the document management system; 4. The process for correcting or revising records; 5. Designation, qualification, and responsibilities of authorized personnel (e.g., writers, reviewers, approvers) for each of the above: and 6. Allowable means of records transmission and document and record security, traceability, retention, and accessibility to stakeholders. Interim Final, Version 1, November 2000 30 ------- Retention and Storage of Documents and Records: ~ Document and record retention times are often determined by contractual and statutory requirements. Tf these requirements are not applicable, senior management should specify the retention time. J Stored documents and records most be protected from damage, loss, and deterioration. Processes for the storage of documents and records must consider the uses to which the documents and records may be put in the future and the resulting need for accessibility. 5.3 Tracking and Retrieval of Technical and Quality-Related Documents and Records Describe the systems, roles, and responsibilities used to ensure documents and records are easily retrievable even when the document/record is revised, if the organization changes, or when information technologies and systems are replaced or become obsolete. a. Describe the systems, roles, and responsibilities for storing and archiving technical and quality-related documents and records, including a statement or reference indicating where they are stored. b. Describe the systems, roles, and responsibilities for controlling and maintaining technical and quality-related documents and records that require restricted access (e.g., national security, enforcement, or proprietary business information). c. Describe the control systems, roles, and responsibilities for review, approval, and revision of technical and quality-related documents and records, and for ensuring that only current/superseding versions are available and used. Document and Record Control Systems: J Control systems include techniques such as numbering consecutively and dating each new document revision. Documen ts may be revised for a variety of reasons (e.g., to add new information, to delete obsolete information, or to correct errors). J Although it is extremely important to ensure that people use the most current version of a document or record, older outdated versions should be retained to maintain the ability to reconstruct project/studies in the event they are later contested or challenged. d. Describe specific change control procedures that ensure that the user can identify those parts of previous documents that are superseded by new document versions. Interim Final, Version 1, November 2000 31 ------- e. Describe the control systems, roles, and responsibilities for establishing, implementing, and maintaining appropriate chain-oI-custody and confidentiality for evidentiary technical and quality-related documents and records. Interim Final, Version 1, November 2000 32 ------- 6.0 COMPUTER HARDWARE AND SOFTWARE This section briefly addresses how the organization will ensure that computer systems are managed such that they: ~ Meet customer needs; ~ Manage change; ~ Facilitate the development, transmission, use, and manipulation of quality data; and ~ Plan for obsolescence such that relevant information can be accessed in the future. Computer hardware and software, including applications such as electronic communications systems, are essential management tools for environmental data. Computer hardware and software are used for a variety of purposes, including making calculations and projecting the impact of information collected, storing data, and displaying visualizations. As such, the management of computer hardware and software and the data they contain is critical in meeting the quality requirements of electronic environmental data. When computer hardware and software is purchased, it has been tested by the developer prior to release. Once purchased, however, the systems are configured to meet the requirements of the user. The use and manipulation of procured hardware and software leaves room for human error and therefore unintended quality issues with data use. Finally, data is received and stored in large computer systems from a variety of sources. Controlling the quality of the data coming into these systems presents a tremendous challenge to agency managers. Attention to computer-related issues is particularly important in this era when technology is changing at a rate that makes it difficult for organizations to effectively manage such change. C ross-References to Other Sections of This Document: In addressing requirements of this computer section, you may cross-reference applicable Quality System requirements documented in your QMP. t~l Requirements regarding the procurement of computer hardware and software and the development and verification of acceptance criteria are addressed in Section 4.0, "Procurement of Products, Sendees, and Activities." ~ The documentation requirements for computer hardware/software are provided in Section 5.0, "Documents and Records." ~ Planning requirements for data collection and use are provided in Section 7.0, "Planning." Interim Final, Version 1, November 2000 33 ------- Computer Software and Computer Hardware/Software Configurations and Programs: Those processes covered by these requirements might include, but are not limited, to; ~ Design and design analysis; ~ Modeling of environmental processes and conditions; ~ Operations or process control; ~ Data transmission; U Data validation, verification, and assessment; ~ Risk assessment; ~ Data storage; and ~ Databases or document control registers (when used as the controlled source of quality information). Note: Laboratory computer systems arc addressed in the National Environmental Laboratory Accreditation Conference (NF.LAC) Standards, The QMP must include a description of the management of computer hardware, software, and electronically stored information. This description must include a discussion of the roles and responsibilities assigned to management and staff. The QMP must document how the organization manages its computer hardware and software operations that directly affect the quality of the results of environmental programs. At a minimum, the QMP must perform the functions outlined below. Additional description and detail for each of the following elements is provided in Appendix B. Cross-Reference Agency Guidance on Information Management: Many agencies have issued guidance on general information management and on specific issues such as software development methodology. For example, the EPA has issued the EPA Information Resources Management Policy Manual (EPA Directive 2100), and DoD has issued Directive 8000.1 on its Defense Information Management (IM) Program. Cross-references in the QMP to other documented processes may be appropriate to satisfy the requirements of the QMP to document the process, roles, and responsibilities for computer hardware and software and information management. 6.1 Describe the Process, Roles, and Responsibilities for Configuration Testing and Documentation ~ Describe the process, roles, and responsibilities for ensuring that systems (hardware and software) are configured to perform their intended functions, meet user needs, and preserve the integrity of the information used in the systems. ~ Describe the process, roles, and responsibilities for ensuring that computer hardware and software are installed and operated in accordance with the manufacturer's recommendations and that systems are adequately tested, inspected, and maintained on a regular basis. Interim Final, Version 1, November 2000 34 ------- About Configuration Testing and Documentation: This section does not anticipate that it is the role of the quality staff of an organization to test the hardware and software of the organization. This section does recognize that someone within the organization must have the responsibility for ensuring that the hardware and software have been configured to the manufacturer's specifications and that they operate as advertised. Responsibility for configuration management may be totally outside the environmental programs that are the subject of this document. In that case, the QMP should briefly cross-reference those procedures that are put in place by the organization to ensure appropriate configuration management. 6.2 Describe the Process, Roles, and Responsibilities for Managing Changes to Hardware and Software Configurations Include a description of how changes to an existing hardware and software configuration will be managed. 6.3 Describe the Process, Roles, and Responsibilities for Developing and Evaluating Software When purchased software is altered or programmed to perform a specific function, such alteration and/or programming must be evaluated to ensure that results are accurate. Such evaluation may range from checking the accuracy of software codes to formally peer reviewing the programming and the logic behind the programming, depending on the nature and complexity of the project. Validating Software; Commercial off-the-shelf (COTS) software that is purchased and used for its intended purpose and installed according to the manufacturer's instructions need not be validated. However, all specialized software and COTS software that is altered or programmed by the user for a particular purpose should be validated to ensure that the programming will meet user specifications. 6.4 Describe the Process, Roles, and Responsibilities for Verifying That the Data Being Compiled and the Quality Control and Maintenance of These Data Are Appropriate for Their Intended Use Evaluation of Data: A number of issues affect the evaluating of data to) ensure they are appropriate for their intended use. The Systematic Planning Process described in Section 7.0 is designed to ensure that data collected is appropriate for the use to which it will be put. When the data used are secondary data (i.e., not collected for the purpose for which they are now being used), evaluation of the appropriateness of the data application will be particularly important. Interim Final, Version 1, November 2000 35 ------- 6.5 Describe the Process, Roles, and Responsibilities for Developing Procedures To Ensure That Historical Files Are Documented and Can Be Recovered Include a description of a plan for storing and archiving relevant information so it can be recovered even as hardware and software are changed. Ensuring Information Retrieval: You should develop a plan for storing, archiving, and retrieving information in a format you will be able to use. This may involve: (a) Identifying the timeframe for which old data will be retrievable. (b) Storing old copies of software and hardware so that data can be retrieved in the future. (c) Planning for the migration of data to new software so that you will be able to access information in the I future even as hardware and software systems become obsolete. Interim Final, Version 1, November 2000 36 ------- 7.0 PLANNING This section describes how the organization or unit ensures quality in the planning of projects that involve environmental data collection and technology programs. The foundation of the planning process utilized is a Systematic Planning Process. This process: ~ Relies upon the scientific method, ~ Involves stakeholders, and either ~ Defines the quality of data appropriate for the intended uses or ~ Results in the implementation of a technology that meets customer specifications. Orienting the planning process toward developing performance criteria appropriate to the decision to be made ensures that the nature of the planning process (and the level of activity required) is varied according to the needs to be met. This is called a "graded" approach. Benefits of the Systematic Planning Process Include: ~ Encouraging comprehensive, careful planning by soliciting input from concerned customers and stakeholders; ~ Addressing costs and schedule in the design phase, the critical time to address total project constraints; ~ Communicating and documenting proposed activities and decisions to be made so that everyone has a common understanding of requirements when considering the data collection or work design, strategies, and the end use of products; ~ Addressing the concerns of customers, suppliers, and relevant technical experts for products, services, and activities, thus minimizing the possibility of repeating work because of inappropriate or inadequate project implementation; and ~ Facilitating the application of promising innovative technology by reconciling technology capabilities with site-specific considerations. 7.1 Management of Systematic Planning The QMP must include a description of the management process that each organizational unit will use to ensure the implementation of a Systematic Planning Process for environmental data collection and use and environmental technology programs. Interim Final, Version 1, November 2000 37 ------- Systematic Planning and the Data Quality Objectives Process; In 1994, EPA developed a Systematic Planning Process called the Data Quality Objectives Process and published a document called "Guidance for the Data Quality Objectives (DQO) Process" (EPA/600/R-96/055, 1996). While not mandatory, this process is the recommended planning approach for many EPA data collection activities. The processes followed by other Federal agencies do not necessarily follow the seven steps of the Systematic Planning Process. For example, using different terminology, but a similar Systematic Planning Process, the U.S. Army Corps of Engineers adopted a four-step Technical Project Planning Process to implement systematic planning for cleanup activities. The term "systematic planning" is the generic term used to describe the "data quality objectives process" (a specific type of systematic planning). Confusion is caused by the different names applied to the similar Systematic Planning Processes used by different Federal agencies and departments. Although the DQO process may not be applicable to all data collection, use, and technology implementation activities, systematic planning is applicable. Therefore, the new EPA Order, "Policy and Program Requirements for the Mandatory Quality System" (5360.1 A2 May 2000), requires that all EPA organizations use a Systematic Planning Process to develop clearly articulated decision goals from which are derived acceptance or performance criteria for the collection, evaluation, or xise of environmental data and for implementation of environmental technology programs. The term "acceptance or performance criteria" replaces the term "data quality objectives" to describe the output of the Systematic Planning Process. This document adopts the same language of the EPA Order and uses the broader term, "Systematic Planning Process," rather than the frequently misunderstood terminology currently associated with the DQO process. At a minimum, this management process must; a. Describe the Applicability of the Systematic Planning Process: Describe the products, services, and activities that require a Systematic Planning Process. b. Describe Roles and Responsibilities: ~ Describe the roles and responsibilities of managers in overseeing the implementation of the Systematic Planning Process within the organizations) addressed by the QMP; and ~ Describe the roles and responsibilities of project personnel in planning and carrying out a project. Project roles and responsibilities that must be addressed include those of the Government project manager, contractor project manager, quality assurance coordinator, technical experts, contract managers/officers, and other customers and suppliers. c. Target the Level and Type of Planning to the Significance of the Activity or Decision: Describe how a graded approach will be used to ensure that the level of Interim Final, Version 1, November 2000 38 ------- QA/QC is commensurate with the significance of the decisions and or the importance of the work and the availability of resources. d. Identify Constraints: Describe the process used to consider operational constraints (e.g., scheduling, logistics, budgetary, coordination, and communication) to adjust the Systematic Planning Process and to cost-effectively meet the operational purpose(s). e. Describe Implementation and Control: For each organizational unit, activity, or process subject to a Systematic Planning Process, describe how that planning process is implemented and controlled. This discussion should include: resolving conflict, developing project goals and acceptance criteria, and addressing the responsibility for stopping the process when issues change the direction or warrant termination of the process. f. Provide Assessment and Feedback: Describe how the management of each operational unit responsible for implementation of a Systematic Planning Process will assess its effectiveness and provide corrective actions and additional direction. g. Identify Planning Documentation: ~ Identify planning documents required by applicable standards, specifications, statutes, and other requirements of society (e.g., management plans, workplans, schedules, project plans, and operating procedures); and ~ Require that all project-specific environmental data collection and environmental technology work be documented by a project-specific QAPP containing a level of QA/QC appropriate to the decision. h. Describe Documentation Preparation, Review, and Approval: ~ Describe the process, roles, and responsibilities for preparation, review, and approval of planning documents (e.g., workplan. schedule, project plan, QAPPs, SOPs) prior to initiation of work; ~ Identify the responsible individual(s); and ~ Discuss this process as it applies to planning documents that are prepared (1) directly by the organization and (2) by the organization's contractors, subcontractors, and assistance or other type of agreement holders. Interim Final, Version 1, November 2000 39 ------- 7.2 Elements of Systematic Planning Processes The key elements of the Systematic Planning Process to be implemented for projects involving environmental data collection and use and environmental technology development and implementation must be described. The QMP should provide general guidance about the implementation of a Systematic Planning Process, while site-specific QAPPs document the individual elements. QAPPs are the primary outputs of the Systematic Planning Process and are used to document the results of planning, to implement environmental operations, and to assess project results. The National Consensus QAPP Guidance, currently under development by the IDQTF, will facilitate the development of QAPPs based on the Systematic Planning Process. The minimum elements are listed below, with additional description and detail for each element provided in Appendix A. a. Establishment of a Team-Based Approach to Planning b. Description of the Project Goal, Objectives, and Questions and Issues To Be Addressed c. Identification of Project Schedule, Resources (Including Budget) Milestones, and Any Applicable Requirements (e.g., Regulatory Requirements, Contractual Requirements) d. Matching of the Data Collection and Analysis Process to Project Objectives e. Identification of Collection and Analysis Requirements f. Description of the Generation, Evaluation, and Assessment of Collected Data Interim Final, Version 1, November 2000 40 ------- 8.0 MANAGEMENT OF WORK PROCESS IMPLEMENTATION This section describes the execution and administration of environmental data collection and technology programs. This section should also describe how work products, services, and key activities are monitored to ensure that work is performed according to the approved planning and technical documents. Who is authorized to stop work? On what grounds? The realization of appropriate quality will be achieved when the work process is correctly and deliberately implemented. Work processes can vary substantially from one activity or project to another in terms of simplicity, repetition, standardization, accuracy, and precision requirements, and the level of required supervision. The planning process will help establish these parameters and attributes at the earliest stages of the effort; the implementation structure of each project will vary based upon these factors. This section describes key QMP elements to ensure that work processes are implemented in a manner consistent with the specifications and acceptance criteria. The management process outlined in the QMP specifies how the organization makes the transition from planning an activity to doing itthe actual performance of planned tasks. It should also address the requirements for day-to-day supervision and monitoring (i.e., internal oversight) of the staff creating or performing key products, services, and activities. Relationship Between Day-to-Day Monitoring and Assessments: This day-to-day monitoring, often performed through inspection/oversight by direct supervisors, is different from the internal and external assessments performed by others in the organization (e.g., QA staff) or by | independent parties. These assessments, which are also critical to ensuring quality, are addressed in Section 9,0, "Assessment and Response." 8.1 Procedures for Implementation in Accordance with Plans The implementation section of the QMP must describe: a. The method for ensuring that work proceeds according to the approved planning and technical documents. b. The sequence of the work process and a level of organizational daily supervision and/or oversight consistent with the importance of the project and the use of project results. Interim Final, Version 1, November 2000 41 ------- For Example; If a key work process is the collection of environmental samples for effluent discharge permitting or compliance evaluation, the QMP should describe the level of regular supervision and oversight required for this data collection. The level should be based upon factors such as past performance, experience of workers, and consequences of nonconformance. This Uniform Federal Policy does not intend to add more supervision when it is not needed. c. How performance is measured against the specifications developed during the planning process (e.g., through the use of QA/QC samples or procedures). Describe the process for routinely monitoring and reporting performance against specifications, as well as the process for identifying routine corrective action. Describe the timeframes for corrective action and the personnel responsible for implementing them. (See Section 9.0 for details about the next layer of oversight, the assessment.,) d. How the organization ensures that the personnel performing the work or activity have the training and competency required to implement a specific project. Similarly, describe how the organization ensures that the supervisors) of that daily work is qualified to perform that regular monitoring. (See Section 3.0, "Personnel Qualification and Training"). 8.2 Managing Quality Implementation Through the Use of SOPs Standard operating procedures (SOPs) are valuable tools for identifying operations and the steps required to complete them. SOPs are used at the project level to facilitate consistency in the quality and integrity of the product or activity. An individual project may demand the use and/or modification of an existing SOP or the creation of a new one. The QMP must provide an overview of how this is accomplished by the organization. The QMP must: A Word About SOPs; Numerous documents guide work processes. Procedures for management of documents are outlined in Section 5.0 of this Uniform Federal Policy. These more detailed procedures recognize that SOPs are special-case documents that have an immediate impact on the way work is implemented and require special attention as to how they are prepared, revised, and managed. a. Identify Operations Requiring SOPs: The QMP must identify the operations (products, services, and activities) to which SOPs apply and for which they must be used. Completeness of this list may be ensured by cross-referencing the organizational chart, list of operational centers, and current list of SOPs. Interim Final, Version 1, November 2000 42 ------- For Example, the QMP May Specify That: SOPs (written procedures) are not required ~ When work processes are directly supervised, ~ When an initial process is in development, or ~ When the process involves problem evaluation or repair of equipment. SOPs (written procedures) are required ~ When there are multiple detailed steps, ~ When results must be within a specified accuracy and/or precision level, ~ When there is high turnover of workers, ~ When there is a need to document the steps followed for the record, such as for repeatability, ~ Where the consequences of nonconformance are significant, or ~ When the results must be comparable between different workers. Additional details about the development and use of written procedures are addressed in the next subsection. b. Describe the Process for Managing SOPs: A list of the most current, approved SOPs, with control documentation (including the revision number, date, identification number) and approval date for each, must be maintained bv the organization as part of its Quality System documentation and must be referenced in the QMP. Inventorying SOPs in a comprehensive list allows the organization to easily identify the most recent SOP versions and those that may require updates. This inventory also establishes a central database from which a document distribution and control system can be managed. (See Section 5.0, "Documents and Records"). Examples of SOPs That May Be Listed: General sampling procedures such as cleaning of sampling equipment, well purging, field measurement guidance, matrix-specific sampling techniques, and data validation procedures. Cross-checking to ensure completeness of the organization's SOPs with respect to the operations it performs is a necessary management step. The QMP must identify who is responsible for this activity and identify the process for ensuring that the activity occurs. c. Describe How the Appropriate SOP Is Identified and Selected: The QMP should describe the process and criteria used to select an appropriate SOP and the procedure for documenting the chosen SOP when an array of approved SOPs are specified for Interim Final, Version 1, November 2000 43 ------- a particular task. The QMP should also describe the method used to tailor an SOP to fit the needs of a particular project, and how this need is identified at the beginning of the process, d. Describe SOP Format and Content: The QMP should state the organization's commitment to writing SOPs that are easily understood by the user and that contain sufficient detail and clarity to ensure reproducibility in the implementation process. SOPs Should Include the Following Elements: (a) Title Page, Table of Contents, and Control Documentation (revision number, date, identification number). (b) A Procedures section addressing such items (where applicable) as scope and applicability, definitions, warnings and cautions, personnel qualifications, materials and equipment, calibration/benchmarking requirements, troubleshooting, data analysis, and data management. (c) A QA/QC section outlining the verification and/or oversight steps and materials required to check the quality of the product or activity. As an alternative to a separate QA/QC section, these steps may be integrated into the Procedures section and appropriately identified. (d) A References section listing other documents and procedures that interface with the SOP, including any applicable checklists. e. Describe the SOP Approval Process: Describe the roles and responsibilities for SOP review and approval. Describe how the organization ensures that the SOP reviewer is technically qualified to perform the review. The QMP must outline these roles and responsibilities and specify how approval of SOP changes is communicated. The QMP must also describe the timeframe; s) for these activities. f. Describe the Process for Making Allowable Changes: Describe what triggers the modification of an SOP. Describe how the organization manages the process when changes to the procedure(s) are required. For example, the QMP may specify that a field-based verbal approval to an SOP change is permissible, if immediately followed by an approved written amendment to the QAPP. The QMP should create a link between the scope of the change and the required level of approval, allowing adequate time to implement the change and generate the supporting documentation. Interim Final, Version 1, November 2000 44 ------- Building Flexibility into SOPs: Incorporating appropriate decision tree strategies and contingency plans in conjunction with preapproved, predefined SOPs during the planning process can establish a desirable level of flexibility during the implementation phase of the project, For example, if a resu lt lies outside the instalment calibration range, an SOP describing an analytical procedure might include an "If...then" analysis and contingency plan describing sample dilution processes. The QMP should describe the conditions under which this type of analysis/plan is both possible and desirable. g. Describe the Process for Periodic Review, Revisions, Distribution, and Archiving: Describe how the organization ensures that SOPs are periodically reviewed and revised. Describe what triggers the review and/or modification of an SOP. The QMP must describe the review process; the roles and responsibilities for SOP revision, approval, and distribution; and the timeframe for reviews, revision, and approval. The QMP should also address the removal and archiving of obsolete documents from work areas, methods of ensuring that outdated SOPs aren't used, the communication paths used to notify staff of procedural changes, and methods of verifying that changes are made as prescribed. Defining SOP Review Timeframes: The frequency with which SOP reviews will be conducted should be SOP-specific. The timeframe for reviewing SOPs should be appropriate to determine whether changes have occurred and to ensure that SOPs remain living documents. 8.3 Work Processes Implemented Through Contracts and Other Assistance Agreements Many organizations undertake the work processes to collect environmental data and manage environmental technology programs on behalf of Federal agencies. These organizations include contractors, grant recipients, and those entering other assistance agreements with the Government. (For the puiposes of this discussion, all of these organizations are referred to as "contractors.") The QMP must: a. Identify work processes (products, services, and activities) routinely carried out by contractors. b. Identify how the processes and procedures required by the Federal agency are communicated to contractors. Interim Final, Version 1, November 2000 45 ------- c. Identify the roles and responsibilities of managers of contracts or other assistance agreements, outside of the contractor's immediate project staff, in overseeing the implementation of work processes. Identifying Roles and Responsibilities of Contractors: Specific names do not have to be stated in the QMP, and a contractor's Quality System may be used to identify contractor roles and responsibilities. d. Describe the roles and responsibilities of Federal agency staff in overseeing the implementation of work processes by contractors and holders of other assistance agreements to ensure that work is conducted correctly and in accordance with customer specifications. e. Given the nature of contracting and other assistance agreement vehicles, describe any constraints in overseeing the work of others and steps that may be taken to address these constraints. Interim Final, Version 1, November 2000 46 ------- 9.0 ASSESSMENT AND RESPONSE This section outlines the manner in which assessments will be identified and performed. Assessments are performed so the organization can evaluate conformance with both technical and procedural requirements. Various parts of this document identify the need for assessments. Section 1.6, for example, asks that annual assessments of the implementation of the Quality System take place. Sections 7.0 and 8.0 ask that the planning process include measurements against which achievement of quality objectives can be examined, and require routine assessment monitoring and oversight of the implementation of work processes. Evaluating how work products routinely meet customer needs and expectations is discussed in several sections (e.g., Sections 1.4 and 7.0). Assessments are systematic, objective, and independent examinations of technical and management processes. They are designed to identify problems, reveal areas of strength and weakness, and allow management to evaluate the organization's processes and performance. Types of Assessments Management and Technical: Assessments differ from routine oversight in that they are objective, independent reviews conducted as part of the Quality System. Two kinds of assessments are identified. ~ Management Assessments (also known as system audits) systematically measure the adequacy and effectiveness of the organization's Quality System (as documented in the QMP) and its impact on work products, services, and activities. ~ Technical Assessments are more narrowly focused to measure the performance of the work itself, with respect to the established technical guidelines and. SOPs and project requirements as identified by the QAPP. Technical assessments will generally verify: (a) Compliance with a project's QAPP or other planning documents; (b) Effective implementation of procedures to ensure the quality of the products, services, and activities; and (c) Suitability of procedures to achieve project goals and measurement quality objectives (See Appendix A.5.). The results of routine technical assessments should feed into management assessments. Note: The process for assessing environmental data to ensure that the quality of the data supports the use for which they are intended is not addressed in this section. That assessment is an element of the Systematic Planning Process that is implemented through a project-specific QAPP or equivalent document. Interim Final, Version 1, November 2000 47 ------- 9.1 Identification and Planning of Assessments The QMP must describe: a. The types of assessments conducted (management self-assessment, management independent assessment, technical self-assessment, and technical independent assessment). b. How the organization determines the priority of the different types of assessments and the criteria for selection of assessments to be conducted. Criteria for Prioritization of Assessments: In addition to an annual assessment of the Quality System as it is implemented throughout the organization, management must identify priority technical assessments that will provide objective information on the performance of the Quality System m producing quality products, services, and activities. In selecting priority technical assessments, management should consider; ~ The importance of the product, service, or activity to the mission of the organization; ~ The cost of quality failure (in either dollar or environmental terms); ~ The expectations and concerns of customers; and ~ Other factors that the organization deems appropriate. c. How the organization determines the type, schedule, and tools for assessments. (Tools include, but are not limited to, audits, data quality assessments, management system reviews, peer reviews, performance evaluations, readiness reviews, surveillance, and technical reviews.) d. The process for determining the frequency of routine assessments (self and independent ) and any conditions and/or triggers under which additional assessments may be required. e. How the results of assessments will be evaluated to measure the effectiveness of the implemented quality system. 9.2 Design and Implementation of Assessments QMPs must describe a process for designing and implementing assessments that ensures that: a. Specific objectives are identified for each assessment. Interim Final, Version 1, November 2000 48 ------- b. Assessments are based upon examination of objective evidence (observations of current activities and examination of documentation of past performance) and, when relevant, interviews of selected participants. An Example: Examination of objective information is possible even in areas such as personnel knowledge and awareness. For example, an organizational assessment of awareness of Quality System requirements throughout the organization could be based on a combination of an objective questionnaire, distributed to personnel, and follow-up interviews with selected individuals. The objective questionnaire can seek basic information on knowledge of policies and procedures and how that knowledge is communicated. Follow-up interviews can focus 011 improvements to training and distribution of information on Quality System requirements. c. Management assessments use objective criteria to determine: ~ If the Quality System is defined (i.e., documented) and if it adheres to applicable requirements, and ~ If the Quality System has been implemented and if it is effective in meeting the needs of the organization. d. Assessments are developed to meet a specific scope of effort and to collect needed information. A Systematic Planning Process for Assessments; Assessments, like the collection of environmental information, require a Systematic Planning Process. This process starts with a decision on the scope of the assessment effort (the goal or objective) and the identification of the information needed to achieve the goals. The assessment team (which may be internal or external to the organization) will guide the effort. The assessment team can systematically identify; ~ User or "customer" specifications; ~ The source of the information and the quantity, quality, and type of data to be collected; ~ Criteria against, which the assessment will be performed; and ~ How the information collected will be evaluated to determine if it meets those criteria. Interim Final, Version 1, November 2000 49 ------- e. An assessment identifies whether it will address technical requirements, management (or Quality System) compliance, or both. Technical Versus Management: A decision as to the type of assessment (technical versus management) you are conducting will affect the choice of personnel to conduct the assessment, the amount of information collected, the information sources and, ultimately, the resources devoted to the assessment. Assessing management or "systems" requirements may often seem easier to accomplish; however, that type of assessment may or may not accomplish the objectives of the assessment. For example, if the organization wants to assess the role of QAPPs in planning for quality, it must determine the focus of the assessment. If the organization wants to know whether QAPPs are being developed and if they contain minimal specified information (e.g., data quality objectives or analytical acceptance criteria), a strategy may be to: (a) Select a percentage of projects for the assessment, (b) Select a random number of projects for which QAPPs will be reviewed, (c) Develop a checklist of what should be in the Table of Contents of the QAPP, and (d) Assign personnel to determine if a QAPP was developed and if it had the right content. Tf the purpose, however, is to assess the quality of QAPPs in documenting effective Systematic Planning Processes, a higher level of personnel will be required to technically assess the QAPPs. The development of objective qualitative measurements will be more difficult. The cost of the assessment (for a similar number of QAPPs assessed) will be more expensive. f. Assessment design will result in reliable, consistent results that can measure performance over time. Assessment Instruments: The production of consistent, reliable results often requires the production of assessment instruments. These instruments may include questionnaires, audit and/or other review checklists and procedures, or various quantitative performance measures based on results of laboratory procedures. Procedures for ensuring reliable and consistent results may include: ~ Pretesting of survey instruments, ~ Pilot testing of audit procedures, and U Analysis of like information from similar sources to establish a reasonable range of variability and uncertainty. Interim Final, Version 1, November 2000 50 ------- 9.3 Determination and Evaluation of Assessor Qualifications The QMP must describe how the organization determines qualifications for assessor personnel. These qualifications include management and technical skills, education, experience, and training requirements, and may include requirements for formal qualification (or certification). The QMP must also describe the system for ensuring that assessments are performed by qualified personnel, 9.4 Responsibilities and Authority of Assessors a. The QMP must describe the assessors' responsibilities and areas of authority. This description must ensure that auditors or assessors have: ~ Sufficient authority, ~ Objectivity and independence from assessed organizations and programs/projects, ~ Access to and support from management, and ~ Access to documents and records. b. QMPs must also demonstrate that assessors have the organizational freedom to: ~ Identify and document problems affecting quality, ~ Identify opportunities for continuous improvement, ~ Propose corrective actions, when appropriate, for problems affecting quality of work produced by assessed organizations, and ~ Independently confirm implementation and effectiveness of corrective actions. 9.5 Roles and Responsibilities of Contractors and Others in Performing Assessments The QMP must discuss the roles and responsibilities of Federal agency personnel versus contractors or holders of assistance agreements in carrying out assessment functions. Those assessment activities that are inherently governmental must be identified. In outlining the inherently governmental functions, those activities that are discretionary (i.e., may be performed by either governmental or nongovernmental personnel) must also be identified. Interim Final, Version 1, November 2000 51 ------- In outlining the inherently governmental functions that are part of the assessment process, the QMP must differentiate between internal and external assessments. Types of assessments that must be addressed include: a. External assessments performed by the Federal agency to oversee the work that the contractor (or other assistance holder) performs, b. Internal assessments conducted by the contractor or other assistance holder that support the contractor's own Quality System, and c. External assessments performed by the contractor or other assistance or agreement holder to support the Federal agency in conducting external assessments of others. For each of these, the QMP must identify: a. The roles and responsibilities of contractor personnel in independently assessing the implementation of their Quality System; b. The roles and responsibilities of Federal agency personnel in assessing the implementation of the contractor's (or other assistance holder's) Qualify System; and c. The roles and responsibilities of Federal agency personnel and contractors (or other assistance holders) in assessing the Quality Systems of other entities (e.g.. other contractors, assistance holders, or Government organizational units). 9.6 Documentation, Reporting, and Review of Assessments Documentation is a key element and output of the assessment process. The QMP must describe: a. The types of documents to be sent to those being assessed, as well as the types of documents to be requested, by assessors and the expected timeframe in which the information must be transmitted. b. The types of documents to be generated by the assessments, the recipients of the documents, the timeframe for distribution, and any opportunity for the assessed organization to provide in writing an initial response (e.g., comment on a draft to allow for changes in report findings). Interim Final, Version 1, November 2000 52 ------- Examples of Documents Sent to the Assessed: Assessment notification and checklists or identification of what will be assessed. Examples of Documents Requested by Assessors; SOPs, QMPs, QAPPs, other planning documents, and prior assessment reports. c. How conditions requiring corrective action will be communicated to those responsible for the corrective action, d. The requirements and procedures for reporting assessment findings to management. e. Procedures for maintaining and controlling assessment findings and results. 9.7 Response to and Follow-up After Assessments After an assessment is complete and the findings have been reported, the assessed organization must respond to the findings, both to the assessor and within the assessed workgroup/ organization. a. The QMP must describe the process, roles, responsibilities, and timeframe for the response. Appropriate managers, QA staff, and technical staff must be involved. b. The QMP must also discuss how follow-up is performed by the assessed organization to ensure corrective action is taken in a timely manner. c. The assessed organization must have a process in place for evaluating the effectiveness of corrective actions and for approving and monitoring the corrective actions triggered by the assessment. d. The assessed organization must describe how it measures the success of the assessment process in identifying problems (i.e., how assessments themselves are assessed for effectiveness). e. The assessed organization must document its response to and follow-up after assessments. Interim Final, Version 1, November 2000 53 ------- THIS PAGE INTENTIONALLY LEFT BLANK Interim Final, Version 1, November 2000 54 ------- 10.0 QUALITY IMPROVEMENT This section describes requirements for continual quality improvement to lead to a better and more responsive Quality System. Management must be the leader in the quality improvement process to ensure adequate resources are provided, issues are resolved, and staff are motivated and rewarded for helping the organization meet its quality improvement goals. "Quality improvement" has three purposes: ~ Improving the quality system, ~ Detecting nonconformance and initiating corrective action, and ~ Creating an environment for facilitating and promoting quality improvement and encouraging staff at all levels to participate in detecting areas for improvement and identifying solutions. Continuous Improvement: Continuous improvement is a term used to describe the incremental improvements that take place over time as the results of corrective actions are fed back into the Quality System. Continuous improvement does not suggest that major revisions constantly place the Quality System in flux. Instead, continuous improvement anticipates incremental changes that take place over time to minimize the recurrence of repetitive quality problems. 10.1 Quality System Improvement Process Describe how the Quality System is continually improved, including how regular reviews of the Quality System (described in Section 1,6) and results of assessments (described in Section 9.0) are used to improve the Quality System. Define roles and responsibilities, including who (organizationally) is responsible for quality improvement. 10.2 Corrective Action for Quality-Related Problems a. Describe the processes, roles, and responsibilities used to: ~ Detect, ~ Determine the significance of. ~ Correct, Interim Final, Version 1, November 2000 55 ------- ~ Track, and ~ Report quality-related problems, including nonconforming work items. Corrective Action: J When problems are found to be significant, the root cause must be determined before permanent preventive measures ean be implemented. ~ Corrective action must ensure that conditions adverse to quality are identified promptly and corrected as soon as practical. b. Describe the systems for planning and initiating appropriate response actions to quality-related problems (i.e., the procedures for determining the root cause before taking corrective action) and the procedures for tracking, documenting, implementing, and evaluating the effectiveness of those response actions. 10.3 Creating an Environment for Facilitating and Promoting Quality Improvement Describe how senior management ensures that the Quality System facilitates and promotes continual quality improvement. Examples of Management Involvement in Quality Improvement Efforts: U Setting and evaluating the attainment of quality improvement goals; ~ Encouraging a "no-fault." attitude; U Acknowledging successes, achievements, and contributions to quality at both the individual and group level; and ~ Recognizing and rewarding excellent performance in attaining or exceeding quality goals. Interim Final, Version 1, November 2000 56 ------- APPENDIX A ELEMENTS OF A SYSTEMATIC PLANNING PROCESS ------- THIS PAGE INTENTIONALLY LEFT BLANK ------- ELEMENTS OF A SYSTEMATIC PLANNING PROCESS The appendix that follows provides further clarification and definition of the elements of a Systemat ic Planning Process that are listed in Section 7.2, These elements are typically documented in project-specific project plans, such as a QAPP. Several guidance documents issued by EPA, DoD, and DOE describe how to write a QAPP and provide more illumination on these project-specific elements. This appendix is not meant to take the place of such documents, but rather to provide further support for the text in this document. A.1 Establishment of a Team-Based Approach to Planning Planning for environmental projects includes a wide variety of individuals and institutions, including project management and technical personnel, customers, suppliers, scientific experts, and other stakeholders, who together will determine if the project is successful. Example of a Team-Based Approach to Planning: Team-based approaches to planning are often used in the Superfund process to streamline the process of reaching cleanup decisions. By jointly determining the information (and the associated quality criteria), the "team" ensures that needs and expectations are identified up front and that rework to meet these expectations later is minimized. ~ Federal facility personnel (e.g., base personnel, contract managers, contractors) meet in a scoping meeting with their counterparts in regulatory agencies to develop the plan for environmental data collection. Technical experts in human health and ecological risk assessment, hydrogeology, chemistry, and quality assurance, along with design engineers who understand remediation choices, may participate in the process, either in team meetings or in consultations behind the scenes. ~ Community stakeholders participate in the process through routine briefings and public meetings on the proposed team approach. ~ The team identifies the decision to be made, along with known and missing information, ~ The team determines what information must be collected to make, for example, a remedy selection decision or a "no action required" decision. Team Requirements for Small Projects: A small project may not require a formal "team," but may involve one person interacting with people, as needed, one-on-one or in small group meetings. Interim Final, Version 1, November 2000 59 ------- A.2 Description of the Project Goal, Objectives, and Questions and Issues To Be Addressed In this part of the Systematic Planning Process, the project team identifies the decisions that must be made, the information that is known, and the unknowns that must be answered to make the decision. Explicit identification of uncertainties that exist, and of which of those uncertainties must be addressed in meeting project goals, is essential. A Word About Uncertainties: Meeting project objectives usually requires dealing with uncertainties. Most data collection and technology efforts cannot cost-effectively resolve all uncertainties, even if such resolution is technically practicable. The project team will, therefore, want to consider whether uncertainties are to be reduced through process design, such as the sampling strategy, mitigated through both process design and contingency planning, or ignored because they are insignificant to the final decision. In any ease, uncertainties should be fully disclosed. A.3 Identification of Project Schedule, Resources (Including Budget), Milestones, and Any Applicable Requirements (e.g., Regulatory Requirements, Contractual Requirements) This part of the planning process identifies the practical boundaries of the effortresources available, requirements that must be addressed, and the schedule and milestones that will be met. An Example of Applicable Requirements: A study project to determine the suitability of land for transfer may be subject to State standards for the cleanup of groundwater or soil, requirements from the Federal Superfund program, Occupational Safety and Health Administration (OSHA) regulations, and many other State and Federal standards. The QMP must describe the process for identifying applicable regulations and standards. This process could involve reference to, or the development of SOPs or guidance documents addressing regulations and standards for common types of activities. A.4 Match the Data Collection and Analysis Process to Project Objectives This part of the planning process results in a series of clear and concise objective statements that flow from the project goals and objectives, and it forms the framework for the design (e.g., sampling) and analysis efforts. The QMP must describe how the process of developing objectives occurs. Interim Final, Version 1, November 2000 60 ------- The QMP Might Include an Example Objective Statement; [Statement related to determining human health risk from direct contact with contaminated soil.] Determine to a percent confidence level whether hazardous chemicals are present at levels that exceed predetermined human health risk screening levels, in the top 1 foot of surface soil of the 1/4 square mile area identified on Map . A.5 Identification of Collection and Analysis Requirements This includes the process used to determine how, when, and where the data will be obtained and any constraints on data collection. Identifying the rationale used to determine how samples collected for a specific project will be analyzed (e.g., in the field or the laboratory), evaluated (e.g., QA review, validation, verification), assessed against their intended use and against the quality performance criteria documented in the project-specific QAPP (or equivalent document) typically requires several steps, which are listed below: a. Determine how, when, and where data (including existing data) will be obtained. In this step, the planning team considers whether existing data can be used or if new data collection efforts must be made. It also considers factors such as the location of samples or measurements to be taken and the time period in which they must be taken. Activities Relate Back to the Purpose of the Data Collection Effort, Examples for a Field Effort Include the Following: ~ Field sampling of soils while drilling will be different depending on the intended use of ¦ the soils data. Soil samples for risk assessment purposes may be collected using brass ring samplers (i.e., California split spoons), whereas soil samples for geotechmcal purposes may be collected using bulk sampling techniques. ~ Location of your sampling points will relate to the conceptual site model that has been developed and the areas for which you are attempting to determine the likelihood of risk. ~ Timing of your sampling effort will relate to the specific weather and geographic conditions that are important to understanding the outcome. For example, in areas with high seasonal variability in the levels and flow rate of surface water, you may wish to collect samples at two different times to understand the impact of seasonal variation on your contaminant levels. b. Determine the quantity of data needed and performance criteria for measuring quality. The quantity and quality of data needed will directly relate to the objectives identified and the desired confidence level in the data. Interim Final, Version 1, November 2000 61 ------- c. Specify QA/QC activities needed to assess the quality performance criteria. Data Quality Indicators (e.g., blanks, field duplicates, or results of sensitivity or uncertainty analysis) will be selected, and Measurement Quality Objectives (specifying acceptance criteria) will be identified that reflect the quality of data required for the decision. (See next box.) Matching Objective Statements to Requirements for Design of Work and Analysis of Results: The components of data collection and analysis, including: ~ Project objectives, ~ Data and sample collection strategies, ~ Analysis of data, and ~ Acceptance and performance criteria for sample collection and data assessment, converge in the graded approach of the Systematic Planning Process (as outlined in A.4 and A.5 above). The terminology used is sometimes confusing and therefore is clarified below. The objective statements summarize the decision goals for the project and reflect the needs and expectations of customers and stakeholders. Because these objective statements directly reflect the purpose of the data collection, they guide the development of sampling and analysis, modeling and technology design, and construction and operation activities. A sampling and analysis plan (SAP) is then designed to produce the type, quality, and quantity of data that can be used to support site decision-making. As the SAP is developed, specific data quality criteria, called Measurement Quality Objectives (MQOs), are tailored to be specific to the data generation requirements. A, 6 Describe the Process for the Generation, Evaluation, and Assessment of Collected Data Questions to be addressed include: ~ Will the samples be analyzed in the field or in a fixed laboratory? ~ What level of data review, validation, and verification will be required? Interim Final, Version 1, November 2000 62 ------- Applicability of the Systematic Planning Process to Other Programs: The examples used throughout this section generally describe typical field sampling and data collection efforts, such as those that may occur in a cleanup or compliance project, documented in a project-specific QAPP. It should be emphasized, however, that the Systematic Planning Process applies to all environmental data collection and use and environmental technology development and implementation efforts. Examples of activities to which the Systematic Planning Process will be particularly important include: (a) Development and application of models that use environmental data to estimate environmental and health affects, (b) Development and implementation of new technologies, (c) Testing of existing technologies for new applications, and (d) Demonstration of the ability of an existing technology to achieve design standards. In each case, a Systematic Planning Process that identifies an appropriate project team; describes the project goals and objectives; identifies schedules, resources, and requirements; matches project objectives to data; and identifies appropriate QC measures and how they will be assessed is important. Project-specific plans will be used to document the critical elements of the Systematic Planning Process. These may look very different from the typical QAPP that frames the process for environmental sampling and analysis efforts, but such plans will address the same issues as outlined in this section. Interim Final, Version 1, November 2000 63 ------- THIS PAGE INTENTIONALLY LEFT BLANK Interim Final, Version 1, November 2000 64 ------- APPENDIX B ELEMENTS OF COMPUTER HARDWARE AND SOFTWARE AND INFORMATION TECHNOLOGY ------- THIS PAGE INTENTIONALLY LEFT BLANK ------- ELEMENTS OF COMPUTER HARDWARE AND SOFTWARE AND INFORMATION TECHNOLOGY The appendix that follows provides further details on the elements associated with maintaining the quality of data as it is manipulated in the electronic environment. Sections 4.0 (Procurement of Products, Services, and Activities), 5.0 (Documents and Records), and 7.0 (Planning) provide further guidance on key elements of information management - hardware, software, evaluation, and use. In addition, other agency policy documents may provide specific guidance on issues related to hardware and software management. It is not intended that the user of the Uniform Federal Policy create a duplicative Quality System for computer hardware and software, but rather that the user build on what exists. The detail in this appendix is provided to ensure that the document user can verify that all of the required Quality System elements have been addressed. B.l Configuration Testing and Documentation Computer hardware and software configurations used in environmental operations have the potential to affect environmental data and data-related decision quality. This section addresses the process for ensuring that computer hardware systems used in environmental programs meet technical requirements and quality expectations. a. Describe the process, roles, and responsibilities for ensuringthat computer hardware and software systems accurately perform their intended functions and meet specific user needs. b. Describe the process, roles, and responsibilities for ensuring that purchased hardware and software systems are installed and operated in accordance with the manufacturer's recommendations. A Word on Personnel Training for Computer Applications: Ensuring that technical support personnel are regularly trained on newly developed or updated systems and programs should be a top pnority of a Quality System. c. Describe the process, roles, and responsibilities for ensuring that computer hardware/ software configurations are tested and documented prior to initial use and after any Definition of Configuration Management: Configuration management is an important aspect of systems operation and maintenance. It encompasses the management and implementation methodologies associated with increasing or correcting system capabilities, a partial system redesign, or determining software obsolescence. Source: EPA Directive 2181, Operation and Maintenance Manual Interim Final, Version 1, November 2000 67 ------- corrective actions are performed. Documentation should be readily available to personnel responsible for use and maintenance. d. Describe the process, roles, and responsibilities for ensuring that computer systems preserve the integrity, availability, and confidentiality of information used in the system. e. Describe the process, roles, and responsibilities for ensuring that a current description of hardware/software systems is maintained to assist personnel in tracking problems with the equipment and to assist in repair and replacement, and to assist users in assessing current functionality and future needs. Computer Systems Testing: Ensuring that a computer system meets the needs of users and that it performs the functions for which it is intended is a key component of a Quality System. Through testing, users can determine if hardware and software meet these objectives, as well as whether the system preserves the integrity of information generated and used on the system. f. Describe the process, roles, and responsibilities for ensuring that computer hardware/ software components are adequately tested, inspected, and maintained on a regular basis. Documentation of such operational maintenance should be kept up-to-date and easily accessible. Integrating and Updating Computer Systems: Testing configuration capabilities is a necessary step in integrating new hardware and software components into existing configurations and in updating program requirements. B.2 Change Management The QMP must address how changes to hardware are controlled to assess the impact of change on program performance and technical and quality objectives. a. Describe the process, roles, and responsibilities for assessing and documenting the need for change to hardware/software configurations and for identifying how the potential impact of the change on the performance and the technical and quality objectives of the program are evaluated. b. Describe the process, roles, and responsibilities by which the compatibility between old and new versions of software and hardware will be assessed to ensure that Interim Final, Version 1, November 2000 68 ------- archived electronic data files can be accessed and read with the new format or system configuration. Change Control Systems: A centralized change control system is recommended in which a designated person or organization is authorized to install or change program versions. The use of a designated person(s) ensures that all change requests, problems, and corrections to software errors and enhancement requests are handled in the same manner. B.3 Development and Evaluation of Computer Software for Environmental Programs a. Describe the process, roles, and responsibilities for developing, validating, and verifying computer software to meet a specific user's needs. b. Describe the process, roles, and responsibilities for ensuring that the software development methodology selected contributes to the software's accuracy and reliability in meeting user needs. c. Describe the process, roles, and responsibilities for ensuring that documentation of new software covers all phases of the software life cycle. These phases are initiation, requirements analysis, design, programming, testing and quality assurance, installation and operation, maintenance/enhancement, and retirement. Refer to Section 5.0 for additional information about documentation. d. Describe how acceptance criteria for software will be developed, documented, and evaluated to ensure that user requirements are met and to comply with applicable contractual requirements and standards. B.4 Usage, Quality Control, and Maintenance of Computerized Data The input of environmental data into computer systems, and the manipulation of that data to facilitate decision making, introduces the possibilities of additional error into the management of environmental data. This additional error can come from a number of sources: a. The quality and variability of the data entered into the data management system, b. Inappropriate use of the data, c. Programming of software that leads to erroneous conclusions when data is manipulated, or d. Mistakes in the data entry process. Interim Final, Version 1, November 2000 69 ------- Environmental information systems encompass a wide range of sizes and complexity. These include large systems used to track regulatory compliance, which require timeliness, accuracy, and consistency, as well as smaller systems, such as predictive models into which a single user or group of users input data to facilitate decision making. Automated Data Operations: The importance of quality control in every step of the information management process is heightened by the use of automated data transfer and the entry of vast quantities of data into computers for manipulation. In environmental operations, computer systems are used to gather data, store data, enter data into databases, and analyze data. Quality control at each step of the process helps to ensure that the environmental data analysis and resulting products arc accurate and reliable. Uses of Environmental Data: The uses of environmental data can vary greatly. For example, environmental data can be used to assist in making risk assessment decisions or to make decisions on appropriate technology, or as input for modeling efforts. Decisions must be made as to whether the data can be used for their intended purpose (e.g., to make risk management decisions or to support modeling efforts). A key element of QA/QC for computerized data is ensuring that the data is put to use in an appropriate manner. Project-specific QAPPs will frequently provide the source of documentation of a particular data use. a. Describe the process roles and responsibilities for verifying the appropriate quality, timeliness, and consistency of data to be gathered and stored. b. Describe the process, roles, and responsibilities for verifying that data being compiled are appropriate for their intended use. c. Describe the process, roles, and responsibilities for ensuring and documenting QA/QC of manual and electronic data input. Use of Secondary Data: "Secondary data" is defined as data that were collected for a different purpose than that for which they are now being used. In addition to a different purpose than the original data collection, the level of QA/QC provided at the time of data collection may be unknown. Issues for consideration in the use of secondary data include, but are not limited to: (a) Similarities between the purpose of the original data collection and the purpose for which data are currently used. (b) Centrality of secondary data to the current decision-making process. (c) Acceptable level of uncertainty associated with the current decision-making process to which secondary data will be applied. d. Describe the process, roles, and responsibilities for ensuring transparency and documentation of decisions in every Interim Final, Version 1, November 2000 70 ------- step of the environmental data management process, including data compilation, entry, use, analysis, and QA/QC, e. Describe the process, roles, and responsibilities for ensuring backup of all files and applications. Computer Modeling: Computerized modeling is an area of data analysis in which apparently small errors in the model structure, design, computer code, or data entry can result in inaccurate results. Modeling QC activities typically include: (a) Independent technical review of model structure and use, (b) Independent checks of model code, and (c) Independent checks of transcription and correction entry of model input parameters. B,5 Historical Files f. Describe the process, roles, and responsibilities for developing consistent procedures for managing and documenting historical files of hardware and software, hardware and software operating procedures (manuals), and hardware and software changes and version numbers, g. Describe the process, roles, and responsibilities for storing, archiving, and recovering information from obsolete systems, including archiving and recovering software and hardware necessary to retrieve stored information. Maintaining Access to Information; As hardware and software continue to evolve rapidly, it becomes increasingly important to maintain the ability to access and use information that was created using configurations that are no longer in use. Therefore, the ability to store, archive, retrieve, and use obsolete software, hardware, and the information created on these systems should be maintained over the long term. Interim Final, Version 1, November 2000 71 ------- THIS PAGE INTENTIONALLY LEFT BLANK Interim Final, Version 1, November 2000 72 ------- APPENDIX C THE INTERGOVERNMENTAL DATA QUALITY FRAMEWORK ------- THIS PAGE INTENTIONALLY LEFT BLANK ------- THE INTERGOVERNMENTAL DATA QUALITY FRAMEWORK Overview The Intergovernmental Data Quality Task Force (IDQTF) was convened to promote the exchange of environmental data quality information between Federal agencies. Since its inception in 1997, the scope of the task force has expanded to comprehensively address a myriad of problems and issues related to the management of environmental data quality at Federal facilities. The IDQTF has developed and is in the process of developing several work products to promote its goals. Mission and Goals The consensus mission of the IDQTF is "7>/ document an intergovernmental quality system beginning with the hazardous waste programs." Relying on definitions contained in the American National Standards institute/'American Society for Quality Control Standard E4 (ANS1/ASQC E4), the IDQTF agreed that although the intergovernmental Quality System should be comprehensive and cover all env ironmental programs, it will focus its initial efforts on the hazardous waste programs managed by EPA's Office of Solid Waste and Emergency Response. The IDQTF identified three initial goals to accomplish its mission: ~ To develop a written agreement on what constitutes an adequate OA program; ~ To develop a guidance/framework that outlines the roles and responsibilities of the EPA (Headquarters and Regions) and the Federal facilities with regard to QA/QC oversight; and ~ To develop guidance for implementing Federal agency wide requirements and procedures regarding data quality. Membership The IDQTF operates as a partnership and seeks consensus. The permanent chairman of the task force is the Director of EPA's Federal Facilities Restoration and Reuse Office. Other current consensus members of the IDQTF include: ~ EPA Regions 1, 2, 5, and 8 ~ Office of Emergency and Remedial Response (EPA Superfund) ~ Office of Solid Waste (EPA Resource Conservation and Recovery Act RCRA) ~ Office of Environmental Information (Quality Staff) ~ Department of Defense (DoD) ~ Department of Energy (DOE) Interim Final, Version 1, November 2000 75 ------- The IDQTF conducts much of its technical work in subgroups, which may pull in additional personnel from parts of the organization that are not involved in the formal task force. Key Work Products Using the ANSI/ASQC E4 standard, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs, as the foundation for identifying the requirements of a Quality System, the IDQTF is in the process of developing a number of major products. These products are described in detail below. Each is designed to implement a specific part of the K4 standard. Relationship of IDQTF Products to ANSI/ASQC E4 E4 Section IDQTF Document Part A, E4, Management Systems Provides guidelines and specifications for an overarching Quality System. Uniform Federal Policy for Implementing Environmental Quality Systems (UFP) Mirrors E4 in organization and provides additional step-by-stcp requirements for implementing a Quality System that meets the E4 standard. (The draft final document is out for agency review and consensus.) Part B, E4, Collection and Evaluation of Environmental Data - - Contains additional Quality System specifications and guidelines needed to plan, implement, and assess environmental data operations. Part B includes sections on planning and scoping, design of data collection operations, implementation of operations, assessment and response, and verification of data usability. Draft Federal Consensus Guidance for the Preparation of Quality Assurance Project Plans Implementation guide to Part B of E4. It provides specific guidance on the process of planning for and conducting environmental data collection operations. When adopted, it will provide the framework for writing consistent QAPPs at Federal facilities. (Draft document completed and undergoing agency review and beta-testing.) Presumptive Quality Assurance/Quality Control Measures for Superfund Based on the Superfund phase and associated data use, this product identifies those quality assurance measures that should be addressed in order to implement the QAPP guidance and Part B of E4. (Under development by IDQTF subgroup.) Roles and Responsibilities Framework for Federal Facility Oversight Designed to address ongoing sources of confusion and conflict between EPA and the Federal agencies. (Initiation of work anticipated in the fall of 2000.) Note: The IDQTF is not developing a project-specific work product to address E4 Part C: Design, Construction, and Operation ofEnvironmental Technology. The Quality System reflected in the Uniform Federal Policy, however, is designed to cover environmental technology programs as well as environmental data collection and use. Interim Final, Version 1, November 2000 76 ------- Adoption and Review of Work Products All work products of the 1DQTF are developed by subgroups, with guidance from the 1DQTF. Each work product goes through several stages of development and review. ~ The need for the product is identified and verified by the IDQTF. ~ The product is developed by the subgroup and sent to the IDQTF to review. ~ Key policy issues are raised for discussion by the IDQTF during planned quarterly meetings, ~ Once the IDQTF has indicated that the product is ready for a wider review, it is sent to EPA Regional offices, DoD components, DOE personnel, other Federal agencies, and other stakeholders for review. ~ Comments are received and documented in a database. Each comment is the subject of an individual evaluation. Responses to comments (accepted, modified, or rejected) are recorded, with explanations where appropriate. ~ The final document is reviewed by the IDQTF. The consensus members of the IDQTF determine if the document requires further review or if it is ready to be sent forward to their respective agencies for final consensus. Commitment to the work of the IDQTF is the subject of two separate memoranda of understanding (MOU). The first MOU was between the EPA and the Navy, acting as Lead Service for Environmental Data Quality for DoD. The second MOU was between the EPA and DOE. A subsequent MOU between these agencies will address implementation of adopted work products. Relationship of IDQTF Work Products to Work Conducted in Nonhazardous Waste Programs IDQTF members decided that a cooperative framework for an interagency quality assurance program should begin with hazardous waste programs and should eventually become the basis for other environmental compliance programs. The Uniform Federal Policy and the Draft Federal Consensus Guidance for the Preparation of Quality Assurance Project Plans were both developed to be suitable for adoption by all environmental programs (e.g., water, air) in their current form. The Presumptive Quality Assurance/Quality Control Measures for Supcrfund is a targeted product that could be used as a model for other programs. DoD and DOE plan to adopt the consensus documents, when approved, to be used for all environmental data collection, use, and technology programs. IDQTF members have actively reached out to other environmental Programs within EPA to encourage their participation and input into the development and implementation of the IDQTF products. Timothy Fields, Assistant Interim Final, Version 1, November 2000 77 ------- Administrator, Office of Solid Waste and Emergency Response, recently issued a memorandum to all EPA Assistant Administrators, requesting feedback on the Uniform Federal Policy and encouraging support for a joint framework for quality assurance for all EPA Programs, Relationship of IDQTF Work Products to Other Quality Documents ISO 9000 Series The IDQTF products are in conformance with the ISO 9000 series of standards addressing Quality Systems, but like ANS1/ASQC E4, the IDQTF products are specifically written for environmental data collection and use and environmental technology programs. Further, the IDQTF products contain a significantly greater level of detail for implementing Quality Systems for environmental data collection and use and for environmental technology programs than does the ISO 9000 series. ISO 14000 Series The ISO 14000 standards for Environmental Management Systems (EMS) address actions an organization takes to minimize harmful environmental impacts caused by its products, services, and activities, and do not address the quality issues covered by the IDQTF products. The IDQTF products address the design of a Quality System for the environmental data collection and environmental technology aspects of EMS. Executive Order 13148 Executive Order (EO) 13148, "Greening the Government Through Leadership in Environmental Management." mandates that Federal agencies establish Environmental Management Systems that integrate environmental accountability into basic operations and planning processes. EO 13148 focuses on achieving Government leadership in environmental management and improving the environmental compliance of Federal agencies. The Uniform Federal Policy addresses the environmental data collection and environmental technology elements of environmental management. EPA Order 5360. / A2 While both this EPA Order and the UFP are based on ANST/ASQC E4, they differ in their applicability. EPA Order 5360.1 A2 establishes the policy and program requirements for an EPA-wide Quality System, while the more detailed UFP addresses the needs and practices of Federal agencies in general and applies to those organizations that elect to implement it. Because the UFP is a consensus policy that was developed by an interagency task force, it was designed to ensure consistent implementation across agencies. The UFP is in conformance with EPA Order 5360.1 A2. Interim Final, Version 1, November 2000 78 ------- EPA Ot/R-2, QA/R-5 EPA QA/R-2, EPA Requirements for Quality Management Plans, establishes Quality System requirements for contractors, grantees, and other assistance holders. It is equivalent to Chapter 3 of EPA Order 5360.1 A2. This document is not applicable to other Federal agencies unless such an agency is a recipient of funds from EPA to conduct EPA's work, EPA QA/R-5. EPA Requirements for Quality Assurance Project Plans, similarly establishes requirements for Quality Assurance Project Plans for those receiving funds from EPA. It is equivalent to Chapter 5 of EPA Order 5360.1 A2. EPA QA/G-5 EPA QA/G-5, EPA Guidance on Quality Assurance Project Plans, was developed to assist EPA Offices and Regions in preparing QAPPs, The IDQTF Draft Federal Consensus Guidance for the Preparation of Quality Assurance Project Plans was developed for use by all Federal agencies that choose to implement it. The draft guidance is an implementation guide that integrates G-5 and the Systematic Planning Process, Tt is based on Part B of E4 and also on the Region I, EPA New England Compendium of Quality Assurance Project Plan Guidance. The IDQTF document has been carefully tailored by representatives of several agencies to be applicable and relevant to all Federal agencies, and it is more detailed than E4. Figure 1, following, describes IDQTF products and their relationship to other major documents, to each other, and to the policy, program, and project level. Interim Final, Version 1, November 2000 79 ------- Policy Level Documents International and National Products IDOTF Products ISO 9000, International Standards for Quality Management: AM Sl'ASQC £.4-1994,. Specifications and Guidelines... for Quality/Systems for. Environmental Data . Collection and Environmental Technology Programs El'A Order 5360.! A2, Policy and Program Requirements for:the Mandatory Agencv- wide Quality System Memoranda of Understanding (EPA'DoD'DOh) Roles and Responsibilities Framework for Federal Facility Oversight Program Level D ocuments National Products EPA Requirements for Quality:Management Plans (TPAQA/R-2) EPA Headquarters Program Office Q.VlPs EPA Regional Quality Management Plans (QMPs) Federal Agency QMPs IDOTF Product Uniform Federal Policy 'for implementing Environmental Quality Systems (UFP) Project-Specific I )ocuments National Products EPA Requirements..-'for Quality Assurance Project Plans (EPA QA/R-5) El \ (midance on Quality Assurance Project Plans (EPA;QA/G-5) ' IDOTF Product Draft Federal Consensus Guidance for the. . Preparation of Quality Assurance Profit Plans Presumptive Quality Assurance/Quality Control Measures for Superfund Figure 1. Quality System Components and Tools: Relationship to IDQTF Products Interim Final, Version 1, November 2000 80 ------- APPENDIX D QUESTIONS AND ANSWERS REGARDING THE IDQTF UNIFORM FEDERAL POLICY ------- THIS PAGE INTENTIONALLY LEFT BLANK ------- QUESTIONS AND ANSWERS REGARDING THE IDQTF UNIFORM FEDERAL POLICY 1. Background Q. 1 What is a Quality System? A, 1 The definition of a Quality System used in the Uniform Federal Policy (UFP) is one adopted from the national standard published by the American National Standards Institute and the American Society for Quality Control and known as ANSI/ASQC H4. "Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs." A Quality System is: A structured and documented management system describing the policies, objectives, principles, organizational authority, responsibilities, accountability, and implementation plan of an organization for assuring quality in its work processes, products (items), and services. The quality system provides the framework for planning, implementing, and assessing the work performed by an organization and for carrying out required quality assurance (QA) and quality control (QC) activities. Q.2 What is the reason for the recent interest in Quality Systems? Is the quality of existing data being questioned? By whom? A.2 Issues related to inadequate data quality oversight at Federal facilities are documented in an EPA Inspector General (IG) report (Audit Report No. E1SKB6-09-0041 -7100132) and DoD IG report (OIG Report 97-098, "Laboratory Support Service for Environmental Testing, Audit Report," February 21, 1997). In addition, recent highly publicized incidents of inappropriate laboratory practices related to work performed at some Federal facilities have cost the Government millions of dollars in unacceptable analytical work, costly resampling efforts, and delays in implementing environmental cleanup/restoration programs and site closeouts. In 1997, three Federal agencies (EPA, DoD, and DOE) joined together to "document a Quality System, starting with Hazardous Waste Programs," The Uniform Federal Policy benefits all agency participants by: 1. Improving the effectiveness and accountability to the public of environmental data and technology programs by focusing on results, quality of data and services, and customer satisfaction. 2. Promoting improved and consistent Quality Systems across EPA Regions and throughout DoD and DOE, Interim Final, Version 1, November 2000 83 ------- 3. Permitting flexibility, or graded QA approaches, that are tailored to meet the desired end uses of the data. 4. Clarifying the roles and responsibilities for managing and overseeing the collection and use of environmental data and for managing environmental technology efforts. 5. Improving confidence that the system can produce quality data and technology such that duplication of oversight efforts can be minimized. In addition to benefitting the Federal agencies involved, the implementation of the Uniform Federal Policy will also improve public confidence in environmental data and technology systems by ensuring that these activities take place in a context designed to ensure quality. Q.3 What is the purpose of the UFP? A.3 The UFP provides requirements and guidelines to Federal agencies for documenting and implementing Quality Systems for the management of environmental data collection and use and environmental technology programs. It follows the basic quality assurance principle that documentation of procedures is essential to ensuring that these procedures remain in place regardless of staffing and organizational changes. The UFP provides additional detail to ensure consistent implementation of ANSI/ASQC E4. Q.4 Why is this document needed? Doesn't the ANSI/ASQC E4 standard already define the requirements for Quality Systems? A.4 ANSI/ASQC E4 is a national voluntary consensus standard that includes standards describing the basic elements of Quality Systems; however, it lacks sufficient detail to promote the data quality needed to adequately address the DoD and EPA IG concerns. In addition, Executive Order 13148 Greening the Government Through Leadership in Environmental Managementrequires that each Federal agency implement environmental management systems to "ensure that strategies are established to support environmental leadership programs, policies, and procedures..The UFP adds additional detail to E4 based on experience and an explicit decision on the part of the Federal agencies involved that the Quality Systems adopted by each of their agencies shall be consistent. In this regard, the UFP clarifies ambiguous E4 language to facilitate uniform application of the E4 standard across DoD, DOE, and EPA. Q.5 How does the Uniform Federal Policy for Implementing Environmental Quality Systems relate to E4? Interim Final, Version 1, November 2000 84 ------- A.5 E4 addresses three components of a Quality System: Part A: Management Systems, which contains specifications and guidelines for routine quality management functions. Part B: Collection and Evaluation of Environmental Data, which contains the quality specifications and guidelines applicable to project-specific environmental activities. Part C: Design, Construction and Operation of Environmental Technology," which contains the minimum quality specifications and guidelines that apply to technology programs. This UFP addresses the elements of Part A of E4. Additional documents under development by the IDQTF will address the implementation of Part B of E4. II. Basis for the UFP Q.6 Why was E4 selected as the source document? Aren't other existing Quality Systems adequate (e.g., Total Quality Management, ISO 14000, ISO 9000)? A.6 E4 was selected as the base document for several reasons: 1. The scope of E4 (environmental data collection and use and environmental technology programs) is most appropriate for the IDQTF effort. 2. It is a national consensus quality standards document developed by an independent, unbiased, nationally recognized group of quality assurance professionals. 3. It covers all necessary elements of a Quality System and provides a good framework, but it is general enough to provide the flexibility needed by a range of agencies or organizations. ANS1/ASQC E4 is the only standard among those listed above that specifically addresses Quality Systems for environmental data collection and use and environmental technology programs. The ISO 9000 series addresses general Quality Systems but is not specific to environmental data collection and technology programs. The ISO 14000 series of standards for Environmental Management Systems (EMS) is not appropriate for application to environmental data collection and use and environmental technology programs. It addresses actions an organization takes to minimize harmful environmental impacts caused by its products, services, and activities and does not address the quality elements covered by E4 for the ISO 9000 series. Q.7 Does the UFP require the creation of a new Quality System? Interim Final, Version 1, November 2000 85 ------- A. 7 The UFP is intended to serve as a high-level policy to document the essential elements of an acceptable Quality System. The UFP does not require creation of yet another Quality System, but rather provides a set of requirements to ensure that essential elements are addressed. It can be used either to develop a new agency-specific Quality System or to evaluate the adequacy of an existing Quality System. Thus, existing Quality Systems based on E4 or other standards may be adequate for continuing use as long as they fulfill the essential elements of the UFP. If a Federal agency has a Quality System in place, then the UFP can be used as a benchmark against which to assess the existing systems, to ensure that all essential elements are addressed, and to verify that there are no gaps in the existing Quality Systems. Q.8 Why wasn't EPA Order 5360.1 A2 (Policy and Program Requirements for the Mandatory Agency-wide Quality System) used as the source document for the IDQTF Quality System? A.8 Both the EPA Order and the UFP are based on ANSFASQC E4, the national standard. However, the EPA Order is simply EPA's policy for implementing E4, while the UFP is designed for implementation by Federal agencies. The EPA Order and Manual were reviewed extensively during the process of constructing the UFP. The UFP is consistent with these EPA documents. Q.9 Why wasn't the EPA Order (5360.1 A2) and its associated manual or EPA QA/R-2 (EPA Requirements for Quality Management Plans) simply adopted by the IDQTF? A.9 The EPA Order and its associated manual reflect EPA's organization and policies. As such, many of these may not be appropriate for wholesale application to other Federal agencies. QA/R-2 applies to work performed for and funded by EPA. It is not, therefore, applicable to Federal agencies, except when they are doing work under an assistance agreement for EPA. Q.10 Doesn't use of the Contract Laboratory Program (CLP) provide sufficient quality for environmental data by "ensuring data of known and documented quality"? A. 10 No. The CLP provides a series of contract specifications, in the form of a statement of work (SOW), that cover laboratory services purchased under specific contracts for Superfund sites. While the CLP also provides guidelines for evaluating laboratory conformance to its contract specifications, it does not address project-specific data usability requirements; therefore, it does not provide assurance that collected data are appropriate for their intended uses. There are many environmental programs that are not covered by CLP and many aspects of environmental data collection and environmental technology programs that are outside its scope (e.g., the Systematic Planning Process, sampling activities, QA oversight). The CLP does not address overall Quality Systems. Interim Final, Version 1, November 2000 86 ------- III. Implementation Issues Q. 11 How will this document be used and implemented? A. 11 When formally adopted by a Federal agency, this policy document will be used to develop the Quality System for the management of environmental data collection and use and environmental technology programs. Although the UFP describes a number of requirements of a Quality System, these requirements are that the agency address certain topics. The UFP leaves it to each agency to determine how best to meet these requirements. Each individual agency will decide how many Quality Management Plans will be written, and by whom. In addition, each agency (and the organizational units responsible for writing QMPs) will decide how the mandatory requirements of the UFP are best addressed by that organization. For example, the UFP asks that the QMP describe responsibilities for the management of quality. It is up to the individual agency to describe the specific responsibilities. Q.12 How are existing programs with significant resource constraints supposed to meet the additional demands of the UFP without additional resources (e.g., resources for planning, implementation, and oversight)? A. 12 The UFP is not a new program. It is a high-level policy that outlines essential elements that must be contained in a Quality System. If a Quality System already exists, then the UFP serves as a benchmark for evaluating the completeness and effectiveness of the system. If an existing Quality System contains all the necessary elements and can be shown to be effective, then it meets the requirements of the UFP. If the existing Quality System contains only some of the essential elements, then only those areas not currently included in the Quality System need to be addressed. A Quality System based on the UFP includes provisions for regularly assessing the resources that are needed to ensure effective Quality System implementation. The UFP recognizes that senior-level EPA, DoD, and DOE decision-makers are responsible and accountable for ensuring that adequate resources are provided. The initial assessment and updating of Quality Systems are expected to require a commitment of resources; however, once the UFP is implemented, the accommodation of graded approaches and improvements in quality of work will result in streamlined project execution with fewer mistakes and rework, thereby- saving both time and money over the long term. It is important to recognize that failure to have in place a Quality System for the management of environmental data collection and use and environmental technology programs is itself costly. For example, lack of procedures to store and make retrievable previously collected environmental data often creates a need for the collection of new data because no one can trace the key quality elements of the older data. Likewise, failure to implement a systematic planning process often leads to the collection of the wrong kind of data, at a quality not Interim Final, Version 1, November 2000 87 ------- appropriate for its intended use. This, in turn, often leads to additional sample collection and analysis. Q.13 In an environmental program with limited staff, how can the QA manager "function independently of direct environmental data generation and use, model development, and technology development," as required by the UFP? A. 13 The independence of the quality assurance function (however named) is not a new concept; most existing Quality Systems contain this requirement. In practice, providing independent QA does not necessarily require either an independent organization or an independent QA staff position. The QA role in any particular project needs to be independent of the work performed on that project. This role can be performed by staff members who have data collection and environmental technology responsibilities on other projects. Q.14 What is the relationship of the UFP to other environmental management system documents? Where will it fit in the hierarchy of existing documents? A, 14 The UFP is a high-level policy document that will be used as a guide for a Quality System. It provides a benchmark for identifying the existing Quality System elements in each agency that currently oversees environmental data collection and use and environmental technology programs. When fully implemented, it will meet the requirements of the national standard known as ANST/ASQC E4. E4 is designed to implement ISO 9000 (International Standards for Quality Management) for environmental data collection and use and environmental technology programs. The Environmental Management System (EMS) required under EO 13148 may be modeled after the ISO 14000 series (Environmental Management ). The EMS is therefore focused on reducing the environmental impacts of delivering the products, services, and activities of the Federal agencies. (See Question 4 for more detail about EO 13148.) The Quality System outlined in this document is parallel to the EMS and will frame the quality process that oversees the way environmental data is collected and used and the way technology programs are managed in support of the products, services, and activities of the Federal agencies. Elements of the EMS and the Quality System developed to meet the requirements of this document may be similar and will be complementary. To avoid unnecessary duplication, agencies and organizations that adopt the UFP should review their existing management systems and associated processes to determine what elements are already addressed in other systems. The next step will be for each agency to identify gaps and use the UFP as a guide for development of a comprehensive and integrated Quality System for environmental data collection and technology programs. Interim Final, Version 1, November 2000 88 ------- Each agency must develop its own implementation strategy that is appropriate to that agency. Often, the QMP for environmental data collection and use and environmental technology programs can cross-reference elements or details in an existing document or documents that contain the appropriate requirements. Q.15 How will the UFP be implemented for suppliers and contracted work? What is the timeframe for implementation? A. 15 Federal Acquisition Regulations (FAR) already require, when appropriate, that contractors maintain high-level quality standards with a Quality System based on existing standards, such as E4. The applicability of standards will be specified in the solicitation for the contract and will be appropriate to the significance and complexity of the associated data collection efforts. Development of the UFP was conducted under two intergovernmental MOUs (one between EPA and DOE and a second between EPA and DoD). Implementation will be addressed in a future MOIJ. Since a significant portion of products, services, and activities related to environmental data collection and use and environmental technology programs are provided by contracts, the UFP will be implemented in part through contracts. Each DOE office and DoD component has unique contracting practices; therefore, each office and component will need to determine a reasonable strategy and timeframe for implementation. DOE and DoD expect that implementation will most likely occur in phases, as existing contracts expire and new contracts are put into place. Q. 16 If the Uniform Federal Policy represents requirements for Federal agencies, then what are the consequences if an agency falls to meet the requirements? Will noncompliance result in a Notice of Violation? A. 16. The simple answer to this question is, no. Nonconformance with the UFP will not result in issuance of a Notice of Violation, unless conformance is made part of a separate, enforceable agreement. The UFP contains requirements that will be implemented voluntarily by parties who decide to adopt the UFP. This voluntary commitment will be signaled by the signing of an implementation Memorandum of Understanding between EPA and the other Federal agency adopting the UFP. Since the UFP was not developed or promulgated through the rule-making process, it is not a regulation and not subject to regulatory enforcement or Notices of Violation. Each agency must determine its own procedures for assessing nonconformance and initiating corrective action. The procedures will be described in the organization's Quality Management Plan. Interim Final, Version 1, November 2000 89 ------- THIS PAGE INTENTIONALLY LEFT BLANK Interim Final, Version 1, November 2000 90 ------- APPENDIX E GLOSSARY ------- THIS PAGE INTENTIONALLY LEFT BLANK. ------- GLOSSARY5,6 Assessments Systematic, objective, and independent examinations of technical and management processes designed to identify problems, reveal areas of strength and weakness, and allow management to evaluate the organization's processes and performance. Continuous improvement Incremental improvements that take place over time as the results of corrective actions are fed back into the Quality System. Continuous improvement anticipates the incremental changes that take place over time to minimize the recurrence of repetitive quality problems. Customer specifications The constellation of specifications, including requirements, expectations, and preferences, that are identified by customers. These specifications can be assigned different levels of priority depending upon whether they are "legal requirements" or customer expectations or preferences separate from legal requirements. Although this document uses the shorthand term "specifications" when describing customer requirements, expectations, and preferences, that term is meant to reference the full range of customer demands. Data Quality Indicators The quantitative statistics and qualitative descriptors that are used to interpret the degree of acceptability or utility of data to the user. The principal data quality indicators are bias, precision, accuracy (bias is preferred), comparability, completeness, representativeness. (U.S. Environmental Protection Agency, EPA Guidance for Quality Assurance Project Plans. EPA/600/R-98/018, February 1998. EPA QA/G-5) Document Any written or pictorial information describing, defining, specifying, reporting, or certifying activities, requirements, procedures, or results pertaining to environmental operations. Inherently governmental functions "A function which is so intimately related to the public interest as to mandate performance by Government employees. ...These functions include those activities which require either the exercise of discretion in applying Government authority or the use of value judgement in making decisions for the Government." (Office of Management and Budget (OMB) circular A-76 and supplemental guidance). It is also, according to the Federal Acquisition Regulations, "a policy determination, not a legal determination."(FAR Part 07, Subpart 7.5, 7501) Measurement Quality Objectives Specified acceptance criteria used in a sampling and analysis plan and designed to produce the type, quality, and quantity of data that can be used to support site decision-making. Organization "A company, corporation, firm, enterprise, or institution, or part thereof...that has its own functions and administration." When used in this document, the term "organization" refers to the entity within a Federal agency that has "its own functions and administration" and that is 'Unless otherwise indicated, the definitions in this glossary are also found in the body of the report. 'NOTE: This glossary is included for informational purposes and is not the definitive final glossary. Interim Final, Version 1, November 2000 93 ------- responsible for writing a Quality Management Plan (QMP) under this policy document. In some cases the term "organizational unit" is used to refer to a unit of a larger organization that may have its own Quality System responsibilities, and for which Quality System information and procedures will require documentation in the QMP. Quality Assurance Project Plan A formal document describing in comprehensive detail the necessary quality assurance (QA), quality control (QC), and other technical activities that must be implemented to ensure that the results of the work performed will satisfy the stated performance criteria. The QAPP components are divided into four classes: 1) Project Management, 2) Measurement/Data Acquisition, 3) Assessment/Oversight, and 4) Data Validation and Usability. (U.S. Environmental Protection Agency, EPA Guidance for Quality Assurance Project Plans. EPA/600/R-98/018, February 1998. EPA QA/G-5) Quality Management Plan A formal document that describes the quality system in terms of the organization's structure, the functional responsibilities of management and staff, the lines of authority, and the required interfaces for those planning, implementing, and assessing all activities conducted. (U.S. Environmental Protection Agency, EPA Guidance for Quality Assurance Project Plans. EPA/600/R-98/018, February 1998. EPA QA/G-5) Quality System A structured and documented management system describing the policies, objectives, principles, organizational authority, responsibilities, accountability, and implementation plan of an organization for ensuring quality in its work processes, products (items), and services. The Quality System provides the framework for planning, implementing, and assessing the work performed by an organization and for carrying out required quality assurance (QA) and quality control (QC) activities. (ANSI, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs. American National Standard, ANS1/ASQC E4-1994.) Record A record is a document that furnishes objective evidence of the quality of products, services, or activities and that has been verified and authenticated as technically complete and correct. Secondary data Data not originally collected for the purpose for which they are now being used. In addition, the level of QA/QC provided at the time of the original data collection may be unknown. Systematic Planning Process Systematic planning is a planning process that is based on the scientific method and includes concepts such as objectivity of approach and acceptability of results. Systematic planning is based on a common sense, graded approach to ensure that the level of detail in planning is commensurate with the importance and intended use of the work and the available resources. This framework promotes communication between all organizations and individuals involved in an environmental program. Through a systematic planning process, a team can develop acceptance or performance criteria for the quality of the data collected and for the quality of the decision. (U.S. Environmental Protection Agency, EPA Guidance for the Data Quality Objectives Process. Peer Review Draft, December 1999. EPA QA/G-4) Interim Final, Version 1, November 2000 94 ------- A systematic planning process shall ensure that all organizations and/or parties who contribute to the quality of the environmental program or use the results are identified and that they participate in this process. The planning process shall also provide for direct communication between the customer and the supplier to ensure that there is a clear understanding by all participants of the needs and expectations of the customer and the product or results to be provided by the supplier. (Memorandum from Norine E. Noonan.) Interim Final, Version 1, November 2000 95 ------- THIS PAGE INTENTIONALLY LEFT BLANK Interim Final, Version 1, November 2000 96 ------- APPENDIX F IDQTF MEMBERS ------- THIS PAGE INTENTIONALLY LEFT BLANK ------- IDQTF Members, Alternates, and Participants U.S. ENVIRONMENTAL PROTECTION AGENCY Office of Solid Waste and Emergency Response (QSWER) Consensus Member: James Wool ford (IDQTF Chair) Director Federal Facilities Restoration and Reuse Office (FFRRO) 1200 Pennsylvania Avenue, NW Washington, DC 20460 (202) 260-1606 woolford.james@epa.gov Alternate to James Woolford: Mike Carter QA Manager Federal Facilities Restoration and Reuse Office (FFRRO) 1200 Pennsylvania Avenue, NW Washington, DC 20460 (202) 260-5686 carter.mike@epa.gov Consensus Member: Duane Geuder QA Officer Office of Emergency and Remedial Response (OERR) Mailcode 5202G 1200 Pennsylvania Avenue, NW Washington, DC 20460 (703) 603-8891 geuder.duane@epa.gov Consensus Member: Charles Sellers Quality Assurance Officer Office of Solid Waste (OSW) Mailcode 5307W 1200 Pennsylvania Avenue, NW Washington, DC 20460 (703) 308-0504 sellers.charles@epa.gov Alternate to Charles Sellers: Kim Kirk land Office of Solid Waste (OSW) 1200 Pennsylvania Avenue, NW Washington, DC 20460 (703) 308-0490 kirkland.kim@epa.gov Office of Environmental Information (OED Consensus Member: Nancy Wentworth Director Quality Staff (QS) Office of Environmental Information (OEI) 1200 Pennsylvania Avenue, NW Washington, DC 20460 (202) 564-6830 wentworth.nancy@epa. gov Alternate to Nancy Wentworth: Esperanza Renard Quality Staff (QS) Office of Environmental Information (OEI) 2890 Woodbridge Ave (MS 104) Edison, NJ 08837 (732)321-4355 renard. esperanza@epa. gov Interim Final, Version 1, November 2000 99 ------- IDQTF Members, Alternates, and Participants EPA Regions Consensus Member: Deborah Szaro Quality Systems Team Leader EPA New England 60 Westview Street Lexington, MA 02421 (781)860-4312 szaro.deb@epa.gov Alternate to Deborah Szaro: Moira Lataille Quality Systems Team Leader EPA New England 60 Westview Street Lexington, MA 02421 (781) 860-4635 lataille.moira@epa.gov Consensus Member: Robert Runyon Chief Hazardous Waste Support Branch, Division of Environmental Science and Assessment EPA Region 2 2890 Woodbridge Ave (Bldg 10) Edison, NJ 08837 (732)321-6645 mnyon.robert@epa.gov Alternate to Robert Runyon: Pat Sheridan Environmental Scientist Hazardous Waste Support Branch, Division of Environmental Science and Assessment EPA Region 2 2890 Woodbridge Ave (Bldg 10) Edison, NJ 08837 (732)321-6780 sheridan.patricia@epa.gov Consensus Member: Ken Tindall Chief Federal Facilities Reuse Section EPA Region 5 Mailcode SRF-5J 77 West Jackson Blvd. Chicago, IL 60604 (312) 886-9895 tindall. kcnncth@cpa.gov Consensus Member: Steve Callio Senior Chemist Technical Management Services QA EPA Region 8 999 18th Street, Suite 500 Denver, CO 80202 (303)312-7290 callio.steven@epa.gov DEPARTMENT OF DEFENSE Consensus Member: Jackie Sample Chief of Naval Operations (CNO N457L) Department of the Navy, Office of the Naval Sea Systems Command (SEA04XQ (Labs)) 1661 Redbank Road. Suite 104 Charleston, SC 29445-6511 (843) 764-7337 ext. 11 samplej h@navsea. navy, mil Alternate to Jackie Sample: Douglas Scarborough Senior Chemist U.S. Army Environmental Center (USAEC) Attn: SFIM-AEC-ERA (Mr. Scarborough) Bldg. E4480, 4480 Hoadley Road Aberdeen Proving Ground, MD 21010-5401 (410) 671-1514 douglas.scarborough@aec.apgea.army.mil Interim Final, Version 1, November 2000 100 ------- IDQTF Members, Alternates, and Participants DEPARTMENT OF ENERGY Consensus Member: David Bottrell Office of Environmental Management Office of Safety, Health and Security 19901 Germantown Road RM 1068/Cloverlcaf Building Germantown, MD 20874-1290 (301)903-7251 david.bottreIl@em.doe.gov Alternate to Dave Bottrell Bob Murray DOE, National Energy Technology Laboratory 3610 Collins Feny Road MSE06 Rm. 241 Morgantown, WV 26505 (304) 285-4133 rmurra@netl.doe.gov OTHER REGULAR IDQTF PARTICIPANTS (STAFF SUPPORT) Ray Bath DOE, Environmental Management Lab 201 Variek Street, 5th Floor New York, NY 10014 (212) 620-3637 bath@eml.doe.gov Deana Crumbling Environmental Scientist EPA, Technology Innovation Office, OSWER 1200 Pennsylvania Avenue, NW Washington, DC 20460 (703) 603-0643 crambling.deana@epa.gov Skip Darlcy Chemist NAVSEA Program Field Office (SEA 07Q) U.S. Navy 3601 Meeting Street Road Charleston, SC 29405-5733 (803) 743-8673 ext. 20 darley_skip@hq.navsea.navy.mil Khouanc Ditthavong Environmental Scientist EPA, Office of Water Mailcode 4303 1200 Pennsylvania Avenue, NW Washington, DC 20460 (202)260-6115 ditthavong.khouane@epa.gov Mark Doehnert Quality Assurance Manager EPA, Office of Air and Radiation, Office of Radiation and Indoor Air 1200 Pennsylvania Avenue, NW Washington, DC 20460 (202) 564-9386 doehnert.mark@epa.gov David Koran DoD, USAGE 20 Massachusetts Avenue Washington, DC 20314-1000 (202) 761-4989 david.koran@HQ02.usace.army.mil Interim Final, Version 1, November 2000 101 ------- IDQTF Members, Alternates, and Participants Barbara Larcom Lt. Col., USAF, BSC Deputy for Public Health, Deputy Assistant Secretary Air Force (ESOH) DoD, U.S. Air Force SAF/MTQ Air Force 1600 Pentagon Washington, DC (703)697-1019 barbara. larcom@pentagon.af.mi 1 Fred S, McLean Chemist DoD, USN, Naval Sea Systems Command (NAVSEA) NAVSEA 04XQ (Labs) 1661 Red Bank Road Goose Creek, SC 29445 (843) 764-7337 ext. 22 rnc leanfs@navsea. navy, mil Eric Reynolds Contract Lab Program, QA Coordinator Office of Emergency and Remedial Response (OERR) EPA Mailcode 52046 1200 Pennsylvania Avenue, NW Washington, DC 20460 (703) 603-9928 reynolds.eric@epa.gov George Sundstrom Environmental Scientist U.S. Department of Agriculture USDA Hazardous Materials Management Group 201 14th Street, SW Yates Bldg., 3SE Washington, DC 20250 (202) 260-6556 gsundstrom@usda.gov Daniel Welch Lt. Col., USAF, BSC Chief of Restoration HQ DLA/CAAE Environmental and Safety Policy Office (CAAE) 8725 John J. Kingman Road, Suit 2533 Ft. Bel voir, VA 22060-6221 (703) 767-6255 DanieLW elch@hq.dla.mil Contract Support Project Manager: Clem L. Rastatter Director of Strategic Planning Environmental Services Group Versar, Inc. 6850 Versar Center Springfield, VA 22151 (703)642-6776 rastacle@versar.com Task Manager; Danielle Miller Wagner Senior Policy Analyst Environmental Services Group Versar, Inc. 6850 Versar Center Springfield, VA 22151 (703) 642-6847 milledan@versar.com Interim Final, Version 1, November 2000 102 ------- |