Guidance on Satisfying EPA Quality System Requirements for STAR Grants EPA QA/G-1 STAR January 2001 United States Environmental Protection Agency Office of Environmental Information Quality Staff (2811R) 1200 Pennsylvania Avenue, NW Washington, DC 20460 ------- ------- EPA QA/G-1 STAR 11 January 2001 ------- FOREWORD This document is designed both to summarize the quality assurance requirements and to provide guidance to an academic researcher (extramural grant applicant) preparing to submit a Science To Achieve Results (STAR) research grant application. This document guides the applicant through the concept and development of a Quality System and describes how that system can be documented through the required quality statement and the Quality Management Project Plan, which may be required in some cases before an approved grant application is funded. This document is one of the EPA Quality System Series documents, which describe EPA policies and procedures for planning, implementing, and assessing the effectiveness of the quality system. Questions regarding this document or other Quality System Series documents may be directed to: U.S. Environmental Protection Agency Quality Staff (2811R) Office of Environmental Information 1200 Pennsylvania Avenue, NW Washington, DC 20460 Phone: (202) 564-6830 Fax: (202)565-2441 E-mail: quality@epa.gov Copies of the EPA Quality System Series documents may be obtained from the Quality Staff or by downloading them from the Quality Staffs home page: http://www.epa.gov/quality EPA QA/G-l STAR 1 January 2001 ------- EPA QA/G-1 STAR 11 January 2001 ------- TABLE OF CONTENTS Page Foreword i Acronyms and Abbreviations iv 1. INTRODUCTION 1 1.1 Background 1 1.2 Purpose of this Document 1 2. ENVIRONMENTAL PROTECTION AGENCY QUALITY ASSURANCE REQUIREMENTS 2 2.1 Quality Assurance and Quality System Requirements 2 2.2 Benefits of Quality Assurance to Grant Applicants 4 2.3 Required Documentation for Quality Assurance 5 3. QA STATEMENT 5 3.1 What is a QA Statement? 5 3.2 Components of the QA Statement 6 4. QUALITY MANAGEMENT PROJECT PLAN 7 4.1 What is a Quality Management Project Plan? 7 4.2 Components of the Quality Management Project Plan 7 5. QUALITY ASSURANCE REQUIREMENTS FOLLOWING GRANT AWARD 12 5.1 Possible Quality Assurance Project Plan Requirement 12 5.2 Implementation and Reporting 12 REFERENCES 13 APPENDIX A. OVERVIEW OF THE EPA QUALITY SYSTEM 15 APPENDIX B. BIBLIOGRAPHY OF USEFUL INTERNET ADDRESSES 19 APPENDIX C. BIBLIOGRAPHY OF R AND G SERIES EPA QUALITY SYSTEM DOCUMENTS 21 APPENDIX D. GLOSSARY OF TERMS AND DEFINITIONS 23 EPA QA/G-l STAR 111 January 2001 ------- ACRONYMS AND ABBREVIATIONS CFR Code of Federal Regulations QA quality assurance QC quality control RFA Request for Application STAR Science to Achieve Results (environmental research grants program) EPA QA/G-l STAR IV January 2001 ------- GUIDANCE ON SATISFYING EPA QUALITY SYSTEM REQUIREMENTS FOR STAR GRANTS 1. INTRODUCTION 1.1 Background The National Center for Environmental Research in the U.S. Environmental Protection Agency (EPA) Office of Research and Development promotes and advances environmental science in the United States by competitively awarding grants for research focusing on reduction of risks to human health and ecosystems and on reduction of uncertainty associated with risk assessment. This is done through the Science To Achieve Results (STAR) program. The preponderance of the research supported by the STAR program is funded through financial assistance agreements awarded through competitive Requests for Applications (RFAs). These RFAs are developed from the Office of Research and Development's Strategic Plan and from specific topical research plans developed by this office. The RFAs are prepared in cooperation with other parts of the Agency and focus research on areas of particular importance to the Agency and/or on areas that complement the Office of Research and Development's intramural research programs and the programs of research partners in other Agencies. From time to time, EPA will also establish larger research centers and programs competitively in specific areas of national concern requiring long-term or multi-disciplinary approaches. The most recent information about STAR programs, including grant opportunity announcements, can be found on the National Center for Environmental Research web site: http V/www.epa.gov/ncerqa 1.2 PURPOSE OF THIS DOCUMENT This document is intended to assist potential applicants and grantees in meeting the EPA quality requirements for the STAR Program. It provides information for the applicant in documenting the quality assurance approach to be used in the proposal and additional information for grantees when the award document states that more detailed information is needed. Grants for larger, often multi- university, research programs and centers will require different documentation and detail than is included here. Specific directions on the required approach will be provided in the individual solicitations and other EPA guidance documents. This document is designed to provide the grant applicant with minimum required information on the concepts of quality assurance (QA) and quality control (QC) and their appropriate application to the STAR Program. EPA hopes it will help applicants to prepare proposals that will result in data or EPA QA/G-l STAR 1 January 2001 ------- procedures that can be interpreted or applied with confidence. Following this introductory section, subsequent sections present the following information: T SECTION 2 presents the general EPA QA requirements for research involving data collection and analysis. It discusses what QA is and its importance in any type of research or production and a description of the two types of QA documentation needed for STAR research grants: the QA Statement, required of all applicants, and the Quality Management Project Plan, which will be required for certain types of projects. » SECTION 3 provides expanded information about the QA Statement. T SECTION 4 provides expanded information about the Quality Management Project Plan and its ten elements. T SECTION 5 briefly describes the QA requirements that should be implemented following grant award. • Appendix A provides an overview of the EPA Quality System. T Appendices B and C provide lists of sources of additional information. Appendix B - Internet addresses Appendix C - Bibliography of EPA Quality System Series documents. » Appendix D is a Glossary of Terms used in this document and their definitions. 2. ENVIRONMENTAL PROTECTION AGENCY QUALITY ASSURANCE REQUIREMENTS 2.1 Quality Assurance and Quality System Requirements Basic quality terms used in this guidance are defined as follows: Quality assurance (QA) is an integrated system of management activities involving planning, implementation, documentation, assessment, reporting, and quality involvement to ensure that a process, item, or service is of the type and quality needed and expected by the customer [American Society for Quality (ASQ), 1994]. Quality control (QC) is the overall system of technical activities that measures the attributes and performance of a process, item, or service against defined standards to EPA QA/G-l STAR 2 January 2001 ------- verify that they meet the stated requirements established by the customer; the operational techniques and activities used to fulfill quality requirements [American Society for Quality (ASQ), 1994]. EPA policy requires the application of QA and QC principles to all types of environmental studies. In science and engineering research studies, products may range from a clear-cut identification of a fundamental photochemical reaction mechanism (which may lead to a better understanding of ozone formation) to a realistic assessment of the efficiency and longevity of a novel catalytic converter for reducing automobile tailpipe emissions. To ensure that such products are of adequate quality for their intended use, the application of QA and QC is both prudent and necessary regardless of the level of complexity of the work undertaken. Accordingly, EPA has established quality system requirements that must be followed within EPA and by extramural contractors and financial assistance recipients for all work performed that involves environmental data production and use. EPA assistance agreement recipients must implement or have implemented a quality system that conforms to the American National Standard ANSI/ASQC E4-1994, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs (ASQ, 1994)1. The American National Standard provides the basis for planning, implementing, documenting, and assessing a quality system in support of environmental programs. It includes elements addressing requirements for management of quality systems, environmental data collection and use, and environmental technology development. For more information, see Appendix A. The quality requirement is also defined in 40 Code of Federal Regulations (CFR) Part 30, Grants and Agreements With Institutions of Higher Education, Hospitals, and Other Non-Profit Organizations, as applied to the STAR program (Parts 30.45 and 30.54). Any awards made to State, Tribal, or local governments would be covered by quality requirements defined in 40 CFR Part 31, Uniform Administrative Requirements for Grants and Cooperative Agreement to State, Tribal, and Local Governments. EPA policy is to require that the quality system be applied to all "environmental programs" within the scope of the assistance agreement in accordance with the principle of graded approach, that is, the level of detail needed to document quality practices for proposed work depends on the type of work and the intended use of the results. The simpler the project, the less detail will be needed to adequately document the quality practices for the project. However, the intended use of the results will also dictate the extensiveness of the QA and QC documentation needed to substantiate the work performed. The principle of graded approach reflects the importance of the work, not just its complexity. This policy is consistent with all environmental programs operating under the EPA quality system as defined by EPA Order 5360.1 A2, May 2000 (U.S. EPA, 2000a). 1 ANSI/ASQC E4 is available for purchase from the American Society for Quality (Phone 800-248-1946). Only in exceptional circumstances should it be necessary to consult this document. EPA QA/G-1 STAR 3 January 2001 ------- Customarily, the EPA requirements for extramural agreements include documentation of (1) the organization's quality system via a detailed Quality Management Plan, and (2) the application of quality assurance and quality control activities to specific efforts via a QA Project Plan. However, following the graded approach, and in acknowledgment that research is often conducted simply to obtain data rather than to be used in decision making, with peer-reviewed publications as a result, the STAR program allows more flexibility. The quality system documentation for the typical STAR research grant application will consist of a written "QA Statement," which accompanies the application and is described in Section 2. In some cases, EPA will require further documentation via a "Quality Management Project Plan," which is described in Section 3. 2.2 Benefits of Quality Assurance to Grant Applicants A university researcher is usually familiar with the peer review process as it applies to selection of research grants and to the publication of journal articles. EPA believes that as a potential grant applicant considers the approaches to be taken to successfully complete the research project and obtain documented quality data, the applicant should also identify and document the activities that will ensure that the product of the research is of adequate quality to be used as planned. Such planning documentation is very beneficial when a grant application is peer reviewed or a manuscript is submitted for publication. Other documents demonstrating that the data meet applicable and appropriate quality standards or criteria are critical and are positively received by many reviewers. A well-written and well-designed Quality Management Project Plan (or similar planning document) may help detect problems or incorrect assumptions before work begins and thus avoid false starts or generation of questionable or unusable data sets. Results of a well-designed experiment can be invalidated by simple things such as misunderstanding verbal directions that could have been included in standard operating procedures or research protocols2. Similarly, an uncalibrated (or improperly calibrated) sensor, such as a pH electrode, can result in data gathered at great expense of time and money being unusable by the researchers. The cost to repeat the experiment may be prohibitive. Introducing student researchers to QA and QC practices can be part of the educational process they must have to compete in today's job market. Quality is a leading force enabling businesses to compete effectively in national and international markets. People entering the business world are being increasingly expected to understand basic QA and QC principles and practices and to In this context, standard operating procedures are defined as written and officially approved documents that detail the preferred method for performing an operation such as sample collection, analysis, equipment use or other routine tasks with thoroughly prescribed techniques and steps to ensure consistency. They are typically prepared by the organization performing the operation. Research protocols are written directions that describe the plan of a scientific experiment such as the experimental design and data collection activities, and they may be adapted or modified as the experiment progresses or when an experiment is added to the project. EPA QA/G-1 STAR 4 January 2001 ------- be proficient in their use. While EPA recognizes that quality requirements may be viewed as being unnecessary burdens to the conduct of the experiment, experience has demonstrated that QA and QC practices provide prudent safeguards against the occurrence of problems and the introduction of error into the data produced which could have dramatic adverse impacts on the results and the conclusions made from them. Many research projects are so novel that the chances of failure are higher than for other types of research. However, establishing a quality system in the beginning should help lead to successful completion of the project with data of sufficient quality for their intended use. Such a quality system would provide a framework for the early detection of errors and for the documentation of the steps in the experiment to help to assure the reproducibility of the research. Furthermore, the application of a quality system will aid the researchers in defending the validity of the project results. 2.3 Required Documentation For Quality Assurance Any project involving the work listed below must be supported by sufficient QA and QC practices to assure that the results will be of the type and quality needed and expected for their intended use. This work includes: — data collection or processing, including the use of secondary data; — conducting surveys; — making environmental measurements; — describing processes or conditions; — describing ecological or health effect and consequences — creating or modifying models; and — developing environmental technology (whether hardware-based or via new techniques and methods) for pollution control and waste treatment. For STAR grants, one or both of the following QA documentation will be required: T QA Statement (mandatory) T Quality Management Project Plan (may be requested). The purpose of the QA Statement and Quality Management Project Plan is to provide information to the EPA Project Officer, QA Manager, and award official on an applicant's capabilities to provide minimum required QA and QC for the proposed work. These documents should demonstrate that QC procedures are in place to ensure that each project is successfully completed and the objective is achieved. Assistance recipients are encouraged to create written SOPs and protocols, and reference them in these documents. 3. QA STATEMENT EPA QA/G-l STAR 5 January 2001 ------- 3.1 What is a QA Statement? The QA Statement is a brief description of the quality assurance and quality control practices that will be applied during a research project to assure that the results obtained satisfy the project objectives. Typically, the QA Statement is presented as a narrative; there is no prescribed format required. Each STAR grant applicant must provide this statement. The QA Statement usually will not exceed two consecutively numbered, 8.5x11-inch pages of single-spaced standard 12-point type with 1-inch margins. EPA QA/G-l STAR 6 January 2001 ------- 3.2 Elements of the QA Statement EPA requires that, for each item listed below, the QA Statement either present the requested information, reference (by page and paragraph number) the specific relevant portion of the project description containing the information, or provide a clear logical justification as to why the item does not apply to the proposed research. (1) Discuss the activities to be performed or hypothesis to be tested and the criteria for determining acceptable data quality. (Note: Such criteria may be expressed in terms of precision, accuracy, representativeness, completeness, and comparability, or in terms of data quality objectives or acceptance criteria. Furthermore, these criteria must also be applied to determine the acceptability of existing or secondary data to be used in the project. In this context secondary data may be defined as data previously collected for other purposes or from other sources, including the literature, compilations from computerized data bases, or results from models of environmental processes and conditions.) (2) Describe the study design, including sample type and location requirements, all statistical analyses that were or will be used to estimate the types and numbers of physical samples required, or equivalent information for studies using survey and interview techniques. (3) Describe the procedures for the handling and custody of samples, including sample collection, identification, preservation, transportation, and storage. (4) Describe the procedures that will be used in the calibration and performance evaluation of all analytical instrumentation and all methods of analysis to be used during the project. Explain how the effectiveness of any new technology will be measured and how it will be benchmarked to improve existing processes, such as those used by industry. (5) Discuss the procedures for data reduction and reporting and describe all statistical methods, with reference to any statistical software to be used, to make inferences and conclusions. Discuss any computer models to be designed or utilized, with associated verification and validation techniques. (6) Describe the quantitative and/or qualitative procedures that will be used to evaluate the success of the project, including any plans for peer or other reviews of the study design or analytical methods prior to data collection. EPA QA/G-l STAR 7 January 2001 ------- If parts of the QA Statement are not applicable to the project, a brief statement of justification should be substituted for that element of the QA Statement. This is expected to occur infrequently. 4. QUALITY MANAGEMENT PROJECT PLAN 4.1 What is a Quality Management Project Plan? If the scope of the project involves significant and complex environmental data collection and analysis or environmental technology development, applicants for STAR research grants may be required to submit a more detailed explanation of the QA and QC practices to be used in the project. A Quality Management Project Plan is a document that includes elements of the QA Statement but generally contains more information and details about the QA and QC activities planned. The Quality Management Project Plan may also be applied to small data collection tasks, small assistance agreements for basic or exploratory research, and similar work of limited scope and duration, at the recommendation of the EPA Project Officer or QA Manager. The EPA Project Officer will identify when a project requires a Quality Management Project Plan. The Quality Management Project Plan should be prepared in accordance with the guidance provided in this section and should describe the QC and QA practices to be implemented by the applicant for the proposed project in sufficient detail to present a clear picture of what is to be done and when. This is basically an enhancement of the QA Statement, without a page limit. The Quality Management Project Plan must be submitted for review and approval by the EPA Project Officer and QA Manager before beginning any work. Below are several questions to consider and answer as part of planning the Quality Management Project Plan: • What is the intended use of the data and what decisions may be made as a result of this research (i.e., what is the basis for the hypothesis)? • What quantity and quality of data are needed to be able to make the decision (i.e., test the hypothesis)? » What procedures or methods will be used to sample and analyze environmental or laboratory media ? T What analytes are planned to be measured, and what are the performance criteria for any type of measurements planned? 4.2 Elements of the Quality Management Project Plan The following information should be included in the Quality Management Project Plan in appropriate sections: EPA QA/G-l STAR 8 January 2001 ------- » Project description; Proposed schedule with start and end dates and milestones for the project; T Statement of the project objectives and statistical hypotheses; T Description of the experimental design; » Description of the sampling and analytical methods; » Description of the process for sample handling and custody; T List of key project staff; T Description of how quality will be ensured during the project; and » Identification of any needed special reports on the QA and QC activities performed, as appropriate. If any of these items have been covered in detail in the text of the application, they may be included in the Quality Management Project Plan by reference to the specific page number in the project narrative of the original application, although a stand-alone document would be preferred. The content needed for each of these ten items is described below, along with brief examples of text and tables. Science and engineering topics for STAR grants are so numerous that only a few selected examples can be included; however, not all elements may apply to a particular STAR grant topic. If an element does not seem to apply, state this and indicate why in a brief discussion in the text. The specific elements of the Quality Management Project Plan are as follows: (1) Project Description A brief synopsis of the project (e.g., a few concise paragraphs) should describe the purpose of the study, including the hypothesis to be tested and the unique research aspects of the project. Also note other anticipated uses of information or data generated in the project. Particularly important is a description of how the data collection activities will be performed, including the process or environmental system that will be tested. If the project is part of a larger EPA program, describe how it fits in the program. The abstract included in the application may be adequate for this section. (2) Statement of the Project Objectives Project objectives of the research or test(s) should be summarized in this section. Clearly identify the primary goals of the project in quantitative terms, if possible, and state how the anticipated results will achieve the research objectives of the project. For example, a project objective may be stated as: The objective of this project is to demonstrate a removal efficiency of 90 percent or higher with a confidence level of 95 percent for the heavy metals identified in the top 5 centimeters ofestuarine sediment. EPA QA/G-l STAR 9 January 2001 ------- Some objectives may not be stated entirely in quantitative terms. The criteria for successful completion of the work should be described; that is, how will one know the objectives of the project have been met? EPA QA/G-1 STAR 10 January 2001 ------- (3) Proposed Schedule with Start and End Dates and Milestones for the Project The actual start and expected end dates of the project, including intermediate milestones and results, should be identified. A milestone chart or table indicating start or end dates for major milestones may be helpful. If the project involves sampling, the Quality Management Project Plan should also address the schedule for collecting samples and the laboratory analysis of the samples. (4) Description of the Experimental Design Describe the experimental design of the project with emphasis on the critical and noncritical measurement aspects of the project. Critical measurements are those necessary to achieve the project objectives; noncritical measurements are those used for process control or general background measurements. What must be accomplished or what measurements taken to achieve those objectives? Discuss the sampling design, including sample type and location requirements, how randomness was achieved in the sampling design and any statistical analyses that were used to estimate the types and numbers of samples required. The experimental design should be linked (as much as possible) to the quantitative project objectives discussed in a previous section. (5) Description of the Process for Sample Handling and Custody Describe the system for identifying (or numbering) samples, sample containers, required sample volume or mass, avoidance of contamination, preservation, transportation (if applicable), holding times, storage times and conditions (if applicable), and safe final disposal of samples. Samples may be attached as documentation for sample labels. Describe the procedure for recording sample history, sampling conditions, and any other pertinent information. Formal sample custody is usually not necessary but samples must still be tracked in some manner, especially those that are toxic or radioactive. Not all environmental data come from physical samples. For example, real-time air data are produced directly from continuous emission or ambient air monitors and do not require sample labels, preservation, or transportation associated with physical samples. It is important to note, however, how a gaseous sample will be brought from the point of sampling to the monitor. It also is important that a system be in place to accurately match data to the time and place of the air sampling. (6) Description of the Sampling and Analytical Methods, and Equipment Describe the sampling and analytical methods that will be used including any instruments and instrument details such as detection limits, range, accuracy, etc. When standard methods are used, such as those published by EPA, the American Society for Testing and Materials, the American Public Health Association, or other standards development organization, a reference to the method with any deviations or choices specifically noted is sufficient documentation. Generally, this description should document questions such as: EPA QA/G-1 STAR 11 January 2001 ------- • Who will test proposed new industrial equipment or processes and how will their accuracy and effectiveness be demonstrated? • What types of QC samples will be included in the sampling and analysis routines, including, for example, any spikes, duplicates, matrix spikes, matrix spike duplicates, and surrogates that will be used and why they were chosen. • What methods of calibration for instruments will be used and how frequently will they be calibrated? What standards will be used for calibration, how they are prepared, how are they stored, and do they provide traceability to higher accuracy standards? What are the relevant performance criteria for the sampling and analytical methods? • Have analytical blanks, instrument response check samples, interferences, and the effects of temperature or pressure on the analytical method been considered and documented? (7) List of the Key Proj ect Staff All key personnel and their assigned responsibilities should be listed in this section. The list should include geographic locations, telephone and fax numbers, and electronic mail addresses. For managerially complex projects, an organizational chart or communications plan may be helpful. Any subcontractors or consultants (if used) and their responsibilities should be included. In addition, any special skills or need for training should be determined and steps to provide those skills or training identified. Project personnel and organization may be shown in tabular form. (8) Description of How Quality Will be Ensured During the Project The process by which quality will be ensured during the project should be described. Overall, EPA is interested in the quality of information (including environmental data) that the project requires and how the necessary quality will be obtained and documented. Some of the key questions that may be asked in developing an appropriate description are: • Can quantitative objectives be established for quality? Will only qualitative objectives be possible? How will the data comparability and representativeness be determined? EPA QA/G-1 STAR 12 January 2001 ------- • What calculations will be performed and how will the correctness of calculations be ensured? • What statistical procedures will be used to analyze the data (identify the commercial source of all programs)? Can the quality of the information or data be independently verified, and how? • How will success be measured (measurement performance criteria) and how is it related to the project's objectives? (9) Description of Data Reduction Methods and Procedures Discuss how original or raw data measurements will be verified after they have been transferred from instrument data recording devices and/or floppy diskettes or laboratory notebooks and processed by computer or manually. Identify the specific descriptive statistical methods (for example, regression analyses, analysis of variance, or multivariate analyses) that will be used to present results. Discuss how raw and processed data that are used in statistical analyses will be verified after statistical analyses have been completed. Also include a complete citation of software programs that will be used for these statistical analyses and to present results. When using secondary data, include a discussion of the quality of these data and how the data will be transferred into computer files for various analyses and how they are verified through these processes. Discuss how the quality of these secondary data affect the results being reported. Provide a complete citation of these data sources so that these data can be reviewed if necessary. (10) Description of Data Evaluation, Validation, and Verification Describe how data are to be evaluated and by whom. Identify who is responsible for verifying the accuracy of the original data and whether or not the data met measurement quality objectives. The performance criteria of the project should be discussed and the process that will be used to ensure the performance criteria have been met. In addition, any performance evaluations, audits, surveillance, and other assessment procedures planned should be described, including the procedures to be used for data validation and verification. Also, there should be a discussion of how any corrective actions will be implemented and documented and their effectiveness confirmed for any audits performed. Discuss any plans for peer or other reviews of the design or analytical methods prior to data collection. If the project involves the development or use of a model, the plan should also include a description of the project configuration; management; key assumptions; quality control procedures for data requirements and acceptance/rejection criteria; how the code will be maintained; the mechanism for EPA QA/G-1 STAR 13 January 2001 ------- transferring the code to the EPA Project Officer and potential users; and the model evaluation and testing, including limitations of the model and its intended application. (Note: the award document contains requirements for further documentation of models in reports.) For projects generating or using GIS/Remote Sensing data, discuss in the Quality Management Project Plan (and reports) how these elements will be addressed: positional accuracy; attribute accuracy; logical consistency; time; lineage; resolution accuracy; and the completeness of coverage, classification, and verification. See Appendix D for definitions. 5. QUALITY ASSURANCE REQUIREMENTS FOLLOWING GRANT AWARD 5.1 Possible Quality Assurance Project Plan Requirement Although a STAR grant application may have been recommended for funding, there may be questions about the adequacy and suitability of the QA Statement or Quality Management Project Plan resulting in the recommendation including a conditional QA approval. In this case, the grant's Project Officer or QA Manager may require that a more comprehensive QA Project Plan be prepared and approved before any research activities begin (see Appendix A). These individuals make the final decision about whether to require a QA Project Plan and then determine who will provide the EPA approval. The QA Project Plan typically may be required for studies producing large volumes of data, or determined to be controversial by EPA, or of a highly complex nature that may need more extensive documentation of the planning process, and for major grants in terms of the impact of or funding for the work. If the approval is conditional, a term and condition statement will be added to the award document which specifies that more documentation is necessary, and that work involving environmental data generation may not begin until the EPA Project Officer or QA Manager provides written notification that the QA plan is approved. Often, a dialogue between the Principal Investigator and the EPA Project Officer can resolve quality assurance requirements questions. For instance, after award, sections of the Quality Management Project Plan could be expanded, or a standard operating procedure or protocol for performing a certain task could be prepared and attached to the Quality Management Project Plan as an appendix to supply additional information. This action must be approved by the EPA Project Officer or the QA Manager for National Center for Environmental Research and documented prior to the initiation of research. 5.2 IMPLEMENTATION AND REPORTS While summaries of QA and QC results may be included as an appendix to the final report, specific OA and OC text descriptions are required by the grant's terms and conditions to be addressed EPA QA/G-1 STAR 14 January 2001 ------- as part of annual and final reports. It is suggested that you review the terms and conditions of the agreement as they are legally binding. In reports, be sure to note any laboratory's QA and QC plans as well as any calibration and verification services performed outside the project research unit (for example, verification of a microbalance, calibration of a flow meter, or tuning of a spectrophotometer). Also, any secondary data used in the project must have a complete citation. Following the grant award and approval of the QA Statement, Quality Management Project Plan (if required) and QA Project Plan (if required), EPA expects that the elements of the applicable document will be implemented as part of the agreement. If the approach changes as the research progresses to improve the integrity of the work or the validity of the results, these changes should be briefly discussed in any annual and final report. REFERENCES 40 CFR 30, Code of Federal Regulations, "Grants and Agreements With Institutions of Higher Education, Hospitals, and Other Non-Profit Organizations." 40 CFR Part 31, Code of Federal Regulations, "Uniform Administrative Requirements for Grants and Cooperative Agreement to State and Local Governments." American Society for Quality. 1994. Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs. ANSI/ASQC E4-1994. Milwaukee, WI. U.S. Environmental Protection Agency, 2000. Guidance for the Data Quality Objectives Process (QA/G-4). EPA/600/G-961055, Office of Environmental Information. U.S. Environmental Protection Agency, 1999. EPA Requirements for Quality Assurance Project Plans (QA/R-5). Interim Final. Office of Environmental Information. EPA Order 5360.1 A2, (May 2000), Policy and Program Requirements for the Mandatory EPA Quality System. U.S. Environmental Protection Agency, Washington, DC. EPA QA/G-1 STAR 15 January 2001 ------- EPA QA/G-1 STAR 16 January 2001 ------- EPA QA/G-1 STAR 17 January 2001 ------- APPENDIX A OVERVIEW OF THE ENVIRONMENTAL PROTECTION AGENCY QUALITY SYSTEM The EPA Quality System has been in place since 1979. The EPA Quality System operates under the authority of Order 5360.1 A2, Policy and Program Requirements for the Mandatory EPA Quality System (May 2000). The quality system is applicable to work done by EPA employees as well as by contractors and grant recipients. The quality system provides the management and technical practices needed to ensure that environmental data are of adequate quality and usable for their intended purposes. Figure A-l illustrates the components of the EPA Quality System. This rather elaborate diagram can be broken down into three interactive structural levels: policy, organization or program, and project. A.1 QUALITY SYSTEM — POLICY LEVEL The top level of Figure A-l illustrates the policy section. It is described here for background information only. Each STAR grant applicant or grant holder need only be aware of the policies and standards; it is not necessary to review them to establish a quality system. The policy section may be thought of as the by-laws or reference library of the quality system. The internal and external EPA quality policies are described here, including 40 Code of Federal Regulations (CFR) Part 30, Grants and Agreements With Institutions of Higher Education, Hospitals, and Other Non-Profit Organizations, which applies to nonprofit organizations working under EPA grants. Each policy reflects the consensus American National Standard, ANSI/ASQC E4, Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs [American Society for Quality, 1994]. This American National Standard was authorized by the American National Standards Institute and developed by the American Society for Quality. EPA played a significant role in creating and writing the standard. A.2 QUALITY SYSTEM — ORGANIZATION/PROGRAM LEVEL The second level of the quality system diagram shows the organization and program elements - that is, the management structure. A Quality Management Plan is the focus of this section. Please consult EPA Requirements for Quality Management Plans (QA/R-2) (U.S. EPA, 1999) for more information on Quality Management Plans. The Quality Management Plan is one of the requirements for organization and research center grants, but not for STAR grants. For STAR grants, the Quality Management Project Plan is sufficient. The Quality Management Project Plan combines the organizational, program management, and training aspects of the Quality Management Plan with the technical details usually found in the QA Project Plan. EPA QA/G-1 STAR 18 January 2001 ------- Consult the following documents for further information on preparing and using the QA Project Plan. Both are available on the Quality Staffs home page (http://www.epa.gov/quality/qa_docs.html): EPA Requirements for Quality Assurance Project Plans (QA/R-5) (U.S. EPA, 1999) and EPA Guidance on Quality Assurance Project Plans (QA/G-5) (U.S. EPA, 1998). A.3 QUALITY SYSTEM — PROJECT LEVEL The third level of the quality system diagram, Figure A-l, shows QA and QC activities at the project or grant work level. This is where a systematic plan for the work is documented in a QA Project Plan (or, in the case for a STAR grant, in the QA Statement or Quality Management Project Plan), the work is implemented according to the plan and any standard operating procedures, and critical assessments of the experimental results and conclusions are made. This process leads to defensible products (data of known quality) and conclusions (decisions) that are communicated through reports, journal articles, and presentations at symposia or seminars. EPA QA/G-l STAR 19 January 2001 ------- o LU O a: a. (D o a: a. N a: o o _i o a. Consensus Standards ANSI/ASQC E4 ISO 9000 Series Internal EPA Policies EPA Order 5360.1 EPA Manual 5360 External Policies Contracts - 48 CFR 46 Assistance Agreements - 40 CFR 30, 31, and 35^ EPA Prog ram & ** Regional Policy "* Quality System Documentation Supporting System Elements Training/Communication Annual Review and Planning Systems Assessments Conduct Study/ s. Experiment s Data Verification & Validation I i Standard Operating Procedures i k Technical Assessments PLANNING 4.... -*• IMPLEMENTATION ASSESSMENT Defensible Products and Decisions EPA QA/G-l STAR Figure A-l EPA Quality System Components 20 January 2001 ------- EPA QA/G-1 STAR 21 January 2001 ------- APPENDIX B BIBLIOGRAPHY OF USEFUL INTERNET ADDRESSES 1. Air and Waste Management Association http://www.awma.org 2. American Chemical Society, Division of Environmental Chemistry http://www.acs-envchem.duq.edu 3. American Society for Quality http://www.asq.org 9. American Society for Quality, Energy and Environmental Division http ://www. envnet. org/asq/eed 10. EPA Grants Information http://www.epa.gov/ORD/grants.htm 11. EPA Peer Review Program http ://www. epa.gov/ORD/spc/2peerrev.htm 12. International Organization for Standardization (ISO 9000 and ISO 14000) http://www.iso.ch 13. National Academies of Science and Engineering, and Institute of Medicine http://www.nas.edu 14. National Center for Environmental Research (Office of Research and Development, U.S. Environmental Protection Agency) http://www.epa.gov/ncerqa 15. U.S. EPA Quality Staff http://www.epa.gov/quality EPA QA/G-l STAR 22 January 2001 ------- EPA QA/G-1 STAR 23 January 2001 ------- APPENDIX C BIBLIOGRAPHY OF R AND G SERIES EPA QUALITY SYSTEM DOCUMENTS The EPA has the following documents to assist research grant applicants and recipients in complying with quality assurance (QA) requirements: S EPA Requirements for Quality Management Plans (QA/R-2) S EPA Requirements for Quality Assurance Project Plans (QA/R-5) S Guidance for the Data Quality Objectives Process (QA/G-4) S Guidance on Quality Assurance Project Plans (QA/G-5) S Guidance for the Preparation of Standard Operating Procedures for Quality- Related Documents (QA/G-6) S Guidance on Technical Audits and Related Assessments (QA/G-7) S Guidance for Data Quality Assessment: Practical Methods for Data Analysis (QA/G-9) S Data Quality Assessment Statistical Toolbox (DataQUEST) (QA/G-9D) These documents and others may be downloaded from the EPA Quality Staff home page at: http://www.epa.gov/quality The documents are available in the Adobe Acrobat PDF format and may be printed using the free Adobe Acrobat Reader, which is available on the Internet. EPA QA/G-l STAR 24 January 2001 ------- EPAQA/G-1STAR 25 January 2001 ------- APPENDIX D GLOSSARY OF TERMS AND DEFINITIONS calibration - comparison of a measurement standard, instrument, or item with a standard or instrument of higher accuracy to detect and quantify inaccuracies and to report or eliminate those inaccuracies by adjustments. environmental data - any measurement or information that describes environmental processes, locations, or conditions; ecological or health effects and consequences; or the performance of environmental technology. For EPA, environmental data include information collected directly from measurements, produced from models, and compiled from other sources such as databases or the literature. environmental technology - an all-inclusive term used to describe pollution control devices and systems, waste treatment processes and storage facilities, and site remediation technologies and their components that may be utilized to remove environmental pollutants or contaminants from or prevent them from entering the environment. Examples include wet scrubbers (air), soil washing (soil), granulated activated carbon units (water), and filtration (air, water). Usually, this term applies to hardware-based systems; however, it also applies to methods or techniques used for pollution prevention, pollutant reduction, or containment to prevent further movement of the contaminants, such as capping, solidification or vitrification, and biological treatment. GIS definitions - Attribute Accuracy—the accuracy of the variables describing a map feature The completeness of: - Classification-assessment of how well the chosen classification is able to represent the data — Coverage—proportion of data available for the area of interest - Verification-the amount and distribution of field measurements or other independent sources of information that were used to develop the data Lineage—the history of the data set, including its sources and processing steps. Logical Consistency—the logical relations among data elements. Positional Accuracy-the deviation of a mapped object from its true ground position. Resolution Accuracy-the smallest discernible unit or object represented in the GIS Time—whether the data is up-to-date enough for its intended use graded approach - the process of basing the level of application of managerial controls applied to an item or work according to the intended use of the results and the degree of confidence needed in the quality of the results. EPA QA/G-l STAR 26 January 2001 ------- guideline - a suggested practice that is nonmandatory in programs intended to comply with a standard. QA manager- the individual designated as the principal manager within the organization having management oversight and responsibility for planning, coordinating, and assessing the effectiveness of the quality system for the organization. quality management - that aspect of the overall management system of the organization that determines and implements the quality policy. Quality management includes strategic planning, allocation of resources, and other systematic activities (e.g., planning, implementation, and assessment) pertaining to the quality system. quality system- a structured and documented management system describing the policies, objectives, principles, organizational authority, responsibilities, accountability, and implementation plan of an organization for ensuring quality in its work processes, products (items), and services. The quality system provides the framework for planning, implementing, and assessing work performed by the organization and for carrying out required QA and QC activities. secondary data - data previously collected for other purposes or from other sources, including the literature, compilations from computerized data bases, or results from models of environmental processes and conditions. EPA QA/G-l STAR 27 January 2001 ------- |