Guidance on Satisfying
EPA Quality System Requirements for STAR Grants
EPA QA/G-1 STAR
January 2001
United States Environmental Protection Agency
Office of Environmental Information
Quality Staff (2811R)
1200 Pennsylvania Avenue, NW
Washington, DC 20460
-------
-------
EPA QA/G-1 STAR 11 January 2001
-------
FOREWORD
This document is designed both to summarize the quality assurance requirements and to provide
guidance to an academic researcher (extramural grant applicant) preparing to submit a Science To
Achieve Results (STAR) research grant application. This document guides the applicant through the
concept and development of a Quality System and describes how that system can be documented
through the required quality statement and the Quality Management Project Plan, which may be
required in some cases before an approved grant application is funded.
This document is one of the EPA Quality System Series documents, which describe EPA
policies and procedures for planning, implementing, and assessing the effectiveness of the quality
system. Questions regarding this document or other Quality System Series documents may be
directed to:
U.S. Environmental Protection Agency
Quality Staff (2811R)
Office of Environmental Information
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: (202) 564-6830
Fax: (202)565-2441
E-mail: quality@epa.gov
Copies of the EPA Quality System Series documents may be obtained from the Quality Staff or by
downloading them from the Quality Staffs home page:
http://www.epa.gov/quality
EPA QA/G-l STAR 1 January 2001
-------
EPA QA/G-1 STAR 11 January 2001
-------
TABLE OF CONTENTS
Page
Foreword i
Acronyms and Abbreviations iv
1. INTRODUCTION 1
1.1 Background 1
1.2 Purpose of this Document 1
2. ENVIRONMENTAL PROTECTION AGENCY QUALITY ASSURANCE
REQUIREMENTS 2
2.1 Quality Assurance and Quality System Requirements 2
2.2 Benefits of Quality Assurance to Grant Applicants 4
2.3 Required Documentation for Quality Assurance 5
3. QA STATEMENT 5
3.1 What is a QA Statement? 5
3.2 Components of the QA Statement 6
4. QUALITY MANAGEMENT PROJECT PLAN 7
4.1 What is a Quality Management Project Plan? 7
4.2 Components of the Quality Management Project Plan 7
5. QUALITY ASSURANCE REQUIREMENTS FOLLOWING GRANT AWARD 12
5.1 Possible Quality Assurance Project Plan Requirement 12
5.2 Implementation and Reporting 12
REFERENCES 13
APPENDIX A. OVERVIEW OF THE EPA QUALITY SYSTEM 15
APPENDIX B. BIBLIOGRAPHY OF USEFUL INTERNET ADDRESSES 19
APPENDIX C. BIBLIOGRAPHY OF R AND G SERIES EPA QUALITY SYSTEM
DOCUMENTS 21
APPENDIX D. GLOSSARY OF TERMS AND DEFINITIONS 23
EPA QA/G-l STAR 111 January 2001
-------
ACRONYMS AND ABBREVIATIONS
CFR Code of Federal Regulations
QA quality assurance
QC quality control
RFA Request for Application
STAR Science to Achieve Results (environmental research grants program)
EPA QA/G-l STAR IV January 2001
-------
GUIDANCE ON SATISFYING
EPA QUALITY SYSTEM REQUIREMENTS FOR STAR GRANTS
1. INTRODUCTION
1.1 Background
The National Center for Environmental Research in the U.S. Environmental Protection Agency
(EPA) Office of Research and Development promotes and advances environmental science in the
United States by competitively awarding grants for research focusing on reduction of risks to human
health and ecosystems and on reduction of uncertainty associated with risk assessment. This is done
through the Science To Achieve Results (STAR) program. The preponderance of the research
supported by the STAR program is funded through financial assistance agreements awarded through
competitive Requests for Applications (RFAs). These RFAs are developed from the Office of
Research and Development's Strategic Plan and from specific topical research plans developed by this
office. The RFAs are prepared in cooperation with other parts of the Agency and focus research on
areas of particular importance to the Agency and/or on areas that complement the Office of Research
and Development's intramural research programs and the programs of research partners in other
Agencies. From time to time, EPA will also establish larger research centers and programs
competitively in specific areas of national concern requiring long-term or multi-disciplinary approaches.
The most recent information about STAR programs, including grant opportunity
announcements, can be found on the National Center for Environmental Research web site:
http V/www.epa.gov/ncerqa
1.2 PURPOSE OF THIS DOCUMENT
This document is intended to assist potential applicants and grantees in meeting the EPA quality
requirements for the STAR Program. It provides information for the applicant in documenting the
quality assurance approach to be used in the proposal and additional information for grantees when the
award document states that more detailed information is needed. Grants for larger, often multi-
university, research programs and centers will require different documentation and detail than is
included here. Specific directions on the required approach will be provided in the individual
solicitations and other EPA guidance documents.
This document is designed to provide the grant applicant with minimum required information on
the concepts of quality assurance (QA) and quality control (QC) and their appropriate application to
the STAR Program. EPA hopes it will help applicants to prepare proposals that will result in data or
EPA QA/G-l STAR 1 January 2001
-------
procedures that can be interpreted or applied with confidence. Following this introductory section,
subsequent sections present the following information:
T SECTION 2 presents the general EPA QA requirements for research involving data
collection and analysis. It discusses what QA is and its importance in any type of
research or production and a description of the two types of QA documentation
needed for STAR research grants: the QA Statement, required of all applicants, and the
Quality Management Project Plan, which will be required for certain types of projects.
» SECTION 3 provides expanded information about the QA Statement.
T SECTION 4 provides expanded information about the Quality Management Project
Plan and its ten elements.
T SECTION 5 briefly describes the QA requirements that should be implemented
following grant award.
• Appendix A provides an overview of the EPA Quality System.
T Appendices B and C provide lists of sources of additional information.
Appendix B - Internet addresses
Appendix C - Bibliography of EPA Quality System Series documents.
» Appendix D is a Glossary of Terms used in this document and their definitions.
2. ENVIRONMENTAL PROTECTION AGENCY QUALITY ASSURANCE
REQUIREMENTS
2.1 Quality Assurance and Quality System Requirements
Basic quality terms used in this guidance are defined as follows:
Quality assurance (QA) is an integrated system of management activities
involving planning, implementation, documentation, assessment, reporting, and
quality involvement to ensure that a process, item, or service is of the type and
quality needed and expected by the customer [American Society for Quality
(ASQ), 1994].
Quality control (QC) is the overall system of technical activities that measures the
attributes and performance of a process, item, or service against defined standards to
EPA QA/G-l STAR 2 January 2001
-------
verify that they meet the stated requirements established by the customer; the
operational techniques and activities used to fulfill quality requirements [American
Society for Quality (ASQ), 1994].
EPA policy requires the application of QA and QC principles to all types of environmental studies. In
science and engineering research studies, products may range from a clear-cut identification of a
fundamental photochemical reaction mechanism (which may lead to a better understanding of ozone
formation) to a realistic assessment of the efficiency and longevity of a novel catalytic converter for
reducing automobile tailpipe emissions. To ensure that such products are of adequate quality for their
intended use, the application of QA and QC is both prudent and necessary regardless of the level of
complexity of the work undertaken. Accordingly, EPA has established quality system requirements
that must be followed within EPA and by extramural contractors and financial assistance recipients for
all work performed that involves environmental data production and use.
EPA assistance agreement recipients must implement or have implemented a quality system that
conforms to the American National Standard ANSI/ASQC E4-1994, Specifications and Guidelines
for Quality Systems for Environmental Data Collection and Environmental Technology
Programs (ASQ, 1994)1. The American National Standard provides the basis for planning,
implementing, documenting, and assessing a quality system in support of environmental programs. It
includes elements addressing requirements for management of quality systems, environmental data
collection and use, and environmental technology development. For more information, see Appendix
A. The quality requirement is also defined in 40 Code of Federal Regulations (CFR) Part 30, Grants
and Agreements With Institutions of Higher Education, Hospitals, and Other Non-Profit
Organizations, as applied to the STAR program (Parts 30.45 and 30.54). Any awards made to
State, Tribal, or local governments would be covered by quality requirements defined in 40 CFR Part
31, Uniform Administrative Requirements for Grants and Cooperative Agreement to State,
Tribal, and Local Governments.
EPA policy is to require that the quality system be applied to all "environmental programs"
within the scope of the assistance agreement in accordance with the principle of graded approach, that
is, the level of detail needed to document quality practices for proposed work depends on the type of
work and the intended use of the results. The simpler the project, the less detail will be needed to
adequately document the quality practices for the project. However, the intended use of the results will
also dictate the extensiveness of the QA and QC documentation needed to substantiate the work
performed. The principle of graded approach reflects the importance of the work, not just its
complexity. This policy is consistent with all environmental programs operating under the EPA quality
system as defined by EPA Order 5360.1 A2, May 2000 (U.S. EPA, 2000a).
1 ANSI/ASQC E4 is available for purchase from the American Society for Quality (Phone 800-248-1946).
Only in exceptional circumstances should it be necessary to consult this document.
EPA QA/G-1 STAR 3 January 2001
-------
Customarily, the EPA requirements for extramural agreements include documentation of (1) the
organization's quality system via a detailed Quality Management Plan, and (2) the application of quality
assurance and quality control activities to specific efforts via a QA Project Plan. However, following
the graded approach, and in acknowledgment that research is often conducted simply to obtain data
rather than to be used in decision making, with peer-reviewed publications as a result, the STAR
program allows more flexibility. The quality system documentation for the typical STAR research grant
application will consist of a written "QA Statement," which accompanies the application and is
described in Section 2. In some cases, EPA will require further documentation via a "Quality
Management Project Plan," which is described in Section 3.
2.2 Benefits of Quality Assurance to Grant Applicants
A university researcher is usually familiar with the peer review process as it applies to selection
of research grants and to the publication of journal articles. EPA believes that as a potential grant
applicant considers the approaches to be taken to successfully complete the research project and
obtain documented quality data, the applicant should also identify and document the activities that will
ensure that the product of the research is of adequate quality to be used as planned. Such planning
documentation is very beneficial when a grant application is peer reviewed or a manuscript is submitted
for publication. Other documents demonstrating that the data meet applicable and appropriate quality
standards or criteria are critical and are positively received by many reviewers.
A well-written and well-designed Quality Management Project Plan (or similar planning
document) may help detect problems or incorrect assumptions before work begins and thus avoid false
starts or generation of questionable or unusable data sets. Results of a well-designed experiment can
be invalidated by simple things such as misunderstanding verbal directions that could have been
included in standard operating procedures or research protocols2. Similarly, an uncalibrated (or
improperly calibrated) sensor, such as a pH electrode, can result in data gathered at great expense of
time and money being unusable by the researchers. The cost to repeat the experiment may be
prohibitive.
Introducing student researchers to QA and QC practices can be part of the educational
process they must have to compete in today's job market. Quality is a leading force enabling
businesses to compete effectively in national and international markets. People entering the business
world are being increasingly expected to understand basic QA and QC principles and practices and to
In this context, standard operating procedures are defined as written and officially approved documents
that detail the preferred method for performing an operation such as sample collection, analysis, equipment use or
other routine tasks with thoroughly prescribed techniques and steps to ensure consistency. They are typically
prepared by the organization performing the operation. Research protocols are written directions that describe the
plan of a scientific experiment such as the experimental design and data collection activities, and they may be
adapted or modified as the experiment progresses or when an experiment is added to the project.
EPA QA/G-1 STAR 4 January 2001
-------
be proficient in their use. While EPA recognizes that quality requirements may be viewed as being
unnecessary burdens to the conduct of the experiment, experience has demonstrated that QA and QC
practices provide prudent safeguards against the occurrence of problems and the introduction of error
into the data produced which could have dramatic adverse impacts on the results and the conclusions
made from them.
Many research projects are so novel that the chances of failure are higher than for other types
of research. However, establishing a quality system in the beginning should help lead to successful
completion of the project with data of sufficient quality for their intended use. Such a quality system
would provide a framework for the early detection of errors and for the documentation of the steps in
the experiment to help to assure the reproducibility of the research. Furthermore, the application of a
quality system will aid the researchers in defending the validity of the project results.
2.3 Required Documentation For Quality Assurance
Any project involving the work listed below must be supported by sufficient QA and QC
practices to assure that the results will be of the type and quality needed and expected for their intended
use. This work includes:
— data collection or processing, including the use of secondary data;
— conducting surveys;
— making environmental measurements;
— describing processes or conditions;
— describing ecological or health effect and consequences
— creating or modifying models; and
— developing environmental technology (whether hardware-based or via new techniques
and methods) for pollution control and waste treatment.
For STAR grants, one or both of the following QA documentation will be required:
T QA Statement (mandatory)
T Quality Management Project Plan (may be requested).
The purpose of the QA Statement and Quality Management Project Plan is to provide
information to the EPA Project Officer, QA Manager, and award official on an applicant's capabilities
to provide minimum required QA and QC for the proposed work. These documents should
demonstrate that QC procedures are in place to ensure that each project is successfully completed and
the objective is achieved. Assistance recipients are encouraged to create written SOPs and protocols,
and reference them in these documents.
3. QA STATEMENT
EPA QA/G-l STAR 5 January 2001
-------
3.1 What is a QA Statement?
The QA Statement is a brief description of the quality assurance and quality control practices
that will be applied during a research project to assure that the results obtained satisfy the project
objectives. Typically, the QA Statement is presented as a narrative; there is no prescribed format
required. Each STAR grant applicant must provide this statement. The QA Statement usually will not
exceed two consecutively numbered, 8.5x11-inch pages of single-spaced standard 12-point type with
1-inch margins.
EPA QA/G-l STAR 6 January 2001
-------
3.2 Elements of the QA Statement
EPA requires that, for each item listed below, the QA Statement either present the requested
information, reference (by page and paragraph number) the specific relevant portion of the project
description containing the information, or provide a clear logical justification as to why the item does not
apply to the proposed research.
(1) Discuss the activities to be performed or hypothesis to be tested and the criteria for
determining acceptable data quality. (Note: Such criteria may be expressed in terms of
precision, accuracy, representativeness, completeness, and comparability, or in terms of
data quality objectives or acceptance criteria. Furthermore, these criteria must also be
applied to determine the acceptability of existing or secondary data to be used in the
project. In this context secondary data may be defined as data previously collected for
other purposes or from other sources, including the literature, compilations from
computerized data bases, or results from models of environmental processes and
conditions.)
(2) Describe the study design, including sample type and location requirements, all
statistical analyses that were or will be used to estimate the types and numbers of
physical samples required, or equivalent information for studies using survey and
interview techniques.
(3) Describe the procedures for the handling and custody of samples, including sample
collection, identification, preservation, transportation, and storage.
(4) Describe the procedures that will be used in the calibration and performance evaluation
of all analytical instrumentation and all methods of analysis to be used during the
project. Explain how the effectiveness of any new technology will be measured and
how it will be benchmarked to improve existing processes, such as those used by
industry.
(5) Discuss the procedures for data reduction and reporting and describe all statistical
methods, with reference to any statistical software to be used, to make inferences and
conclusions. Discuss any computer models to be designed or utilized, with associated
verification and validation techniques.
(6) Describe the quantitative and/or qualitative procedures that will be used to evaluate the
success of the project, including any plans for peer or other reviews of the study design
or analytical methods prior to data collection.
EPA QA/G-l STAR 7 January 2001
-------
If parts of the QA Statement are not applicable to the project, a brief statement of justification
should be substituted for that element of the QA Statement. This is expected to occur infrequently.
4. QUALITY MANAGEMENT PROJECT PLAN
4.1 What is a Quality Management Project Plan?
If the scope of the project involves significant and complex environmental data collection and
analysis or environmental technology development, applicants for STAR research grants may be
required to submit a more detailed explanation of the QA and QC practices to be used in the project.
A Quality Management Project Plan is a document that includes elements of the QA Statement but
generally contains more information and details about the QA and QC activities planned. The Quality
Management Project Plan may also be applied to small data collection tasks, small assistance
agreements for basic or exploratory research, and similar work of limited scope and duration, at the
recommendation of the EPA Project Officer or QA Manager. The EPA Project Officer will identify
when a project requires a Quality Management Project Plan.
The Quality Management Project Plan should be prepared in accordance with the guidance
provided in this section and should describe the QC and QA practices to be implemented by the
applicant for the proposed project in sufficient detail to present a clear picture of what is to be done and
when. This is basically an enhancement of the QA Statement, without a page limit. The Quality
Management Project Plan must be submitted for review and approval by the EPA Project Officer and
QA Manager before beginning any work.
Below are several questions to consider and answer as part of planning the Quality
Management Project Plan:
• What is the intended use of the data and what decisions may be made as a result of this
research (i.e., what is the basis for the hypothesis)?
• What quantity and quality of data are needed to be able to make the decision (i.e., test
the hypothesis)?
» What procedures or methods will be used to sample and analyze environmental or
laboratory media ?
T What analytes are planned to be measured, and what are the performance criteria for
any type of measurements planned?
4.2 Elements of the Quality Management Project Plan
The following information should be included in the Quality Management Project Plan in
appropriate sections:
EPA QA/G-l STAR 8 January 2001
-------
» Project description;
Proposed schedule with start and end dates and milestones for the project;
T Statement of the project objectives and statistical hypotheses;
T Description of the experimental design;
» Description of the sampling and analytical methods;
» Description of the process for sample handling and custody;
T List of key project staff;
T Description of how quality will be ensured during the project; and
» Identification of any needed special reports on the QA and QC activities performed, as
appropriate.
If any of these items have been covered in detail in the text of the application, they may be
included in the Quality Management Project Plan by reference to the specific page number in the
project narrative of the original application, although a stand-alone document would be preferred. The
content needed for each of these ten items is described below, along with brief examples of text and
tables. Science and engineering topics for STAR grants are so numerous that only a few selected
examples can be included; however, not all elements may apply to a particular STAR grant topic. If an
element does not seem to apply, state this and indicate why in a brief discussion in the text.
The specific elements of the Quality Management Project Plan are as follows:
(1) Project Description
A brief synopsis of the project (e.g., a few concise paragraphs) should describe the purpose
of the study, including the hypothesis to be tested and the unique research aspects of the project. Also
note other anticipated uses of information or data generated in the project. Particularly important is a
description of how the data collection activities will be performed, including the process or
environmental system that will be tested. If the project is part of a larger EPA program, describe how it
fits in the program. The abstract included in the application may be adequate for this section.
(2) Statement of the Project Objectives
Project objectives of the research or test(s) should be summarized in this section. Clearly
identify the primary goals of the project in quantitative terms, if possible, and state how the anticipated
results will achieve the research objectives of the project. For example, a project objective may be
stated as:
The objective of this project is to demonstrate a removal efficiency of 90 percent
or higher with a confidence level of 95 percent for the heavy metals identified in
the top 5 centimeters ofestuarine sediment.
EPA QA/G-l STAR 9 January 2001
-------
Some objectives may not be stated entirely in quantitative terms. The criteria for successful completion
of the work should be described; that is, how will one know the objectives of the project have been
met?
EPA QA/G-1 STAR 10 January 2001
-------
(3) Proposed Schedule with Start and End Dates and Milestones for the Project
The actual start and expected end dates of the project, including intermediate milestones and
results, should be identified. A milestone chart or table indicating start or end dates for major milestones
may be helpful. If the project involves sampling, the Quality Management Project Plan should also
address the schedule for collecting samples and the laboratory analysis of the samples.
(4) Description of the Experimental Design
Describe the experimental design of the project with emphasis on the critical and noncritical
measurement aspects of the project. Critical measurements are those necessary to achieve the project
objectives; noncritical measurements are those used for process control or general background
measurements. What must be accomplished or what measurements taken to achieve those objectives?
Discuss the sampling design, including sample type and location requirements, how randomness was
achieved in the sampling design and any statistical analyses that were used to estimate the types and
numbers of samples required. The experimental design should be linked (as much as possible) to the
quantitative project objectives discussed in a previous section.
(5) Description of the Process for Sample Handling and Custody
Describe the system for identifying (or numbering) samples, sample containers, required sample
volume or mass, avoidance of contamination, preservation, transportation (if applicable), holding times,
storage times and conditions (if applicable), and safe final disposal of samples. Samples may be
attached as documentation for sample labels. Describe the procedure for recording sample history,
sampling conditions, and any other pertinent information. Formal sample custody is usually not
necessary but samples must still be tracked in some manner, especially those that are toxic or
radioactive. Not all environmental data come from physical samples. For example, real-time air data
are produced directly from continuous emission or ambient air monitors and do not require sample
labels, preservation, or transportation associated with physical samples. It is important to note,
however, how a gaseous sample will be brought from the point of sampling to the monitor. It also is
important that a system be in place to accurately match data to the time and place of the air sampling.
(6) Description of the Sampling and Analytical Methods, and Equipment
Describe the sampling and analytical methods that will be used including any instruments and
instrument details such as detection limits, range, accuracy, etc. When standard methods are used, such
as those published by EPA, the American Society for Testing and Materials, the American Public Health
Association, or other standards development organization, a reference to the method with any deviations
or choices specifically noted is sufficient documentation. Generally, this description should document
questions such as:
EPA QA/G-1 STAR 11 January 2001
-------
• Who will test proposed new industrial equipment or processes and how will their
accuracy and effectiveness be demonstrated?
• What types of QC samples will be included in the sampling and analysis routines,
including, for example, any spikes, duplicates, matrix spikes, matrix spike duplicates, and
surrogates that will be used and why they were chosen.
• What methods of calibration for instruments will be used and how frequently will they be
calibrated?
What standards will be used for calibration, how they are prepared, how are they
stored, and do they provide traceability to higher accuracy standards?
What are the relevant performance criteria for the sampling and analytical methods?
• Have analytical blanks, instrument response check samples, interferences, and the
effects of temperature or pressure on the analytical method been considered and
documented?
(7) List of the Key Proj ect Staff
All key personnel and their assigned responsibilities should be listed in this section. The list
should include geographic locations, telephone and fax numbers, and electronic mail addresses. For
managerially complex projects, an organizational chart or communications plan may be helpful. Any
subcontractors or consultants (if used) and their responsibilities should be included. In addition, any
special skills or need for training should be determined and steps to provide those skills or training
identified. Project personnel and organization may be shown in tabular form.
(8) Description of How Quality Will be Ensured During the Project
The process by which quality will be ensured during the project should be described. Overall,
EPA is interested in the quality of information (including environmental data) that the project requires and
how the necessary quality will be obtained and documented. Some of the key questions that may be
asked in developing an appropriate description are:
• Can quantitative objectives be established for quality? Will only qualitative objectives be
possible?
How will the data comparability and representativeness be determined?
EPA QA/G-1 STAR 12 January 2001
-------
• What calculations will be performed and how will the correctness of calculations be
ensured?
• What statistical procedures will be used to analyze the data (identify the commercial
source of all programs)?
Can the quality of the information or data be independently verified, and how?
• How will success be measured (measurement performance criteria) and how is it related
to the project's objectives?
(9) Description of Data Reduction Methods and Procedures
Discuss how original or raw data measurements will be verified after they have been transferred
from instrument data recording devices and/or floppy diskettes or laboratory notebooks and processed
by computer or manually. Identify the specific descriptive statistical methods (for example, regression
analyses, analysis of variance, or multivariate analyses) that will be used to present results. Discuss how
raw and processed data that are used in statistical analyses will be verified after statistical analyses have
been completed. Also include a complete citation of software programs that will be used for these
statistical analyses and to present results.
When using secondary data, include a discussion of the quality of these data and how the data
will be transferred into computer files for various analyses and how they are verified through these
processes. Discuss how the quality of these secondary data affect the results being reported. Provide a
complete citation of these data sources so that these data can be reviewed if necessary.
(10) Description of Data Evaluation, Validation, and Verification
Describe how data are to be evaluated and by whom. Identify who is responsible for verifying
the accuracy of the original data and whether or not the data met measurement quality objectives. The
performance criteria of the project should be discussed and the process that will be used to ensure the
performance criteria have been met. In addition, any performance evaluations, audits, surveillance, and
other assessment procedures planned should be described, including the procedures to be used for data
validation and verification. Also, there should be a discussion of how any corrective actions will be
implemented and documented and their effectiveness confirmed for any audits performed.
Discuss any plans for peer or other reviews of the design or analytical methods prior to data
collection.
If the project involves the development or use of a model, the plan should also include a
description of the project configuration; management; key assumptions; quality control procedures for
data requirements and acceptance/rejection criteria; how the code will be maintained; the mechanism for
EPA QA/G-1 STAR 13 January 2001
-------
transferring the code to the EPA Project Officer and potential users; and the model evaluation and
testing, including limitations of the model and its intended application. (Note: the award document
contains requirements for further documentation of models in reports.)
For projects generating or using GIS/Remote Sensing data, discuss in the Quality Management
Project Plan (and reports) how these elements will be addressed: positional accuracy; attribute
accuracy; logical consistency; time; lineage; resolution accuracy; and the completeness of coverage,
classification, and verification. See Appendix D for definitions.
5. QUALITY ASSURANCE REQUIREMENTS FOLLOWING GRANT AWARD
5.1 Possible Quality Assurance Project Plan Requirement
Although a STAR grant application may have been recommended for funding, there may be
questions about the adequacy and suitability of the QA Statement or Quality Management Project Plan
resulting in the recommendation including a conditional QA approval. In this case, the grant's Project
Officer or QA Manager may require that a more comprehensive QA Project Plan be prepared and
approved before any research activities begin (see Appendix A). These individuals make the final
decision about whether to require a QA Project Plan and then determine who will provide the EPA
approval.
The QA Project Plan typically may be required for studies producing large volumes of data, or
determined to be controversial by EPA, or of a highly complex nature that may need more extensive
documentation of the planning process, and for major grants in terms of the impact of or funding for the
work. If the approval is conditional, a term and condition statement will be added to the award
document which specifies that more documentation is necessary, and that work involving environmental
data generation may not begin until the EPA Project Officer or QA Manager provides written
notification that the QA plan is approved.
Often, a dialogue between the Principal Investigator and the EPA Project Officer can resolve
quality assurance requirements questions. For instance, after award, sections of the Quality
Management Project Plan could be expanded, or a standard operating procedure or protocol for
performing a certain task could be prepared and attached to the Quality Management Project Plan as an
appendix to supply additional information. This action must be approved by the EPA Project Officer or
the QA Manager for National Center for Environmental Research and documented prior to the initiation
of research.
5.2 IMPLEMENTATION AND REPORTS
While summaries of QA and QC results may be included as an appendix to the final report,
specific OA and OC text descriptions are required by the grant's terms and conditions to be addressed
EPA QA/G-1 STAR 14 January 2001
-------
as part of annual and final reports. It is suggested that you review the terms and conditions of the
agreement as they are legally binding. In reports, be sure to note any laboratory's QA and QC plans as
well as any calibration and verification services performed outside the project research unit (for example,
verification of a microbalance, calibration of a flow meter, or tuning of a spectrophotometer). Also, any
secondary data used in the project must have a complete citation.
Following the grant award and approval of the QA Statement, Quality Management Project
Plan (if required) and QA Project Plan (if required), EPA expects that the elements of the applicable
document will be implemented as part of the agreement. If the approach changes as the research
progresses to improve the integrity of the work or the validity of the results, these changes should be
briefly discussed in any annual and final report.
REFERENCES
40 CFR 30, Code of Federal Regulations, "Grants and Agreements With Institutions of
Higher Education, Hospitals, and Other Non-Profit Organizations."
40 CFR Part 31, Code of Federal Regulations, "Uniform Administrative Requirements for Grants and
Cooperative Agreement to State and Local Governments."
American Society for Quality. 1994. Specifications and Guidelines for Quality Systems for
Environmental Data Collection and Environmental Technology Programs. ANSI/ASQC
E4-1994. Milwaukee, WI.
U.S. Environmental Protection Agency, 2000. Guidance for the Data Quality Objectives Process
(QA/G-4). EPA/600/G-961055, Office of Environmental Information.
U.S. Environmental Protection Agency, 1999. EPA Requirements for Quality Assurance Project
Plans (QA/R-5). Interim Final. Office of Environmental Information.
EPA Order 5360.1 A2, (May 2000), Policy and Program Requirements for the Mandatory EPA
Quality System. U.S. Environmental Protection Agency, Washington, DC.
EPA QA/G-1 STAR 15 January 2001
-------
EPA QA/G-1 STAR 16 January 2001
-------
EPA QA/G-1 STAR 17 January 2001
-------
APPENDIX A
OVERVIEW OF THE ENVIRONMENTAL PROTECTION AGENCY
QUALITY SYSTEM
The EPA Quality System has been in place since 1979. The EPA Quality System operates
under the authority of Order 5360.1 A2, Policy and Program Requirements for the Mandatory EPA
Quality System (May 2000). The quality system is applicable to work done by EPA employees as well
as by contractors and grant recipients. The quality system provides the management and technical
practices needed to ensure that environmental data are of adequate quality and usable for their intended
purposes.
Figure A-l illustrates the components of the EPA Quality System. This rather elaborate diagram
can be broken down into three interactive structural levels: policy, organization or program, and project.
A.1 QUALITY SYSTEM — POLICY LEVEL
The top level of Figure A-l illustrates the policy section. It is described here for background
information only. Each STAR grant applicant or grant holder need only be aware of the policies and
standards; it is not necessary to review them to establish a quality system. The policy section may be
thought of as the by-laws or reference library of the quality system. The internal and external EPA
quality policies are described here, including 40 Code of Federal Regulations (CFR) Part 30, Grants
and Agreements With Institutions of Higher Education, Hospitals, and Other Non-Profit
Organizations, which applies to nonprofit organizations working under EPA grants. Each policy
reflects the consensus American National Standard, ANSI/ASQC E4, Specifications and Guidelines
for Quality Systems for Environmental Data Collection and Environmental Technology Programs
[American Society for Quality, 1994]. This American National Standard was authorized by the
American National Standards Institute and developed by the American Society for Quality. EPA played
a significant role in creating and writing the standard.
A.2 QUALITY SYSTEM — ORGANIZATION/PROGRAM LEVEL
The second level of the quality system diagram shows the organization and program elements -
that is, the management structure. A Quality Management Plan is the focus of this section. Please
consult EPA Requirements for Quality Management Plans (QA/R-2) (U.S. EPA, 1999) for more
information on Quality Management Plans. The Quality Management Plan is one of the requirements for
organization and research center grants, but not for STAR grants. For STAR grants, the Quality
Management Project Plan is sufficient. The Quality Management Project Plan combines the
organizational, program management, and training aspects of the Quality Management Plan with the
technical details usually found in the QA Project Plan.
EPA QA/G-1 STAR 18 January 2001
-------
Consult the following documents for further information on preparing and using
the QA Project Plan. Both are available on the Quality Staffs home page
(http://www.epa.gov/quality/qa_docs.html): EPA Requirements for Quality
Assurance Project Plans (QA/R-5) (U.S. EPA, 1999) and EPA Guidance
on Quality Assurance Project Plans (QA/G-5) (U.S. EPA, 1998).
A.3 QUALITY SYSTEM — PROJECT LEVEL
The third level of the quality system diagram, Figure A-l, shows QA and QC activities at the
project or grant work level. This is where a systematic plan for the work is documented in a QA
Project Plan (or, in the case for a STAR grant, in the QA Statement or Quality Management Project
Plan), the work is implemented according to the plan and any standard operating procedures, and
critical assessments of the experimental results and conclusions are made. This process leads to
defensible products (data of known quality) and conclusions (decisions) that are communicated through
reports, journal articles, and presentations at symposia or seminars.
EPA QA/G-l STAR
19
January 2001
-------
o
LU
O
a:
a.
(D
o
a:
a.
N
a:
o
o
_i
o
a.
Consensus Standards
ANSI/ASQC E4
ISO 9000 Series
Internal EPA Policies
EPA Order 5360.1
EPA Manual 5360
External Policies
Contracts - 48 CFR 46
Assistance Agreements -
40 CFR 30, 31, and 35^
EPA Prog ram &
** Regional Policy "*
Quality System
Documentation
Supporting System Elements
Training/Communication
Annual Review
and Planning
Systems
Assessments
Conduct Study/
s. Experiment s
Data Verification
& Validation
I
i
Standard
Operating
Procedures
i
k
Technical
Assessments
PLANNING
4....
-*• IMPLEMENTATION
ASSESSMENT
Defensible Products and Decisions
EPA QA/G-l STAR
Figure A-l EPA Quality System Components
20
January 2001
-------
EPA QA/G-1 STAR 21 January 2001
-------
APPENDIX B
BIBLIOGRAPHY OF USEFUL INTERNET ADDRESSES
1. Air and Waste Management Association
http://www.awma.org
2. American Chemical Society, Division of Environmental Chemistry
http://www.acs-envchem.duq.edu
3. American Society for Quality
http://www.asq.org
9. American Society for Quality, Energy and Environmental Division
http ://www. envnet. org/asq/eed
10. EPA Grants Information
http://www.epa.gov/ORD/grants.htm
11. EPA Peer Review Program
http ://www. epa.gov/ORD/spc/2peerrev.htm
12. International Organization for Standardization (ISO 9000 and ISO 14000)
http://www.iso.ch
13. National Academies of Science and Engineering, and Institute of Medicine
http://www.nas.edu
14. National Center for Environmental Research (Office of Research and Development, U.S.
Environmental Protection Agency)
http://www.epa.gov/ncerqa
15. U.S. EPA Quality Staff
http://www.epa.gov/quality
EPA QA/G-l STAR 22 January 2001
-------
EPA QA/G-1 STAR 23 January 2001
-------
APPENDIX C
BIBLIOGRAPHY OF R AND G SERIES
EPA QUALITY SYSTEM DOCUMENTS
The EPA has the following documents to assist research grant applicants and recipients in complying
with quality assurance (QA) requirements:
S EPA Requirements for Quality Management Plans (QA/R-2)
S EPA Requirements for Quality Assurance Project Plans (QA/R-5)
S Guidance for the Data Quality Objectives Process (QA/G-4)
S Guidance on Quality Assurance Project Plans (QA/G-5)
S Guidance for the Preparation of Standard Operating Procedures for Quality-
Related Documents (QA/G-6)
S Guidance on Technical Audits and Related Assessments (QA/G-7)
S Guidance for Data Quality Assessment: Practical Methods for Data Analysis
(QA/G-9)
S Data Quality Assessment Statistical Toolbox (DataQUEST) (QA/G-9D)
These documents and others may be downloaded from the EPA Quality Staff home page at:
http://www.epa.gov/quality
The documents are available in the Adobe Acrobat PDF format and may be printed using the free
Adobe Acrobat Reader, which is available on the Internet.
EPA QA/G-l STAR 24 January 2001
-------
EPAQA/G-1STAR 25 January 2001
-------
APPENDIX D
GLOSSARY OF TERMS AND DEFINITIONS
calibration - comparison of a measurement standard, instrument, or item with a standard or instrument
of higher accuracy to detect and quantify inaccuracies and to report or eliminate those inaccuracies by
adjustments.
environmental data - any measurement or information that describes environmental processes,
locations, or conditions; ecological or health effects and consequences; or the performance of
environmental technology. For EPA, environmental data include information collected directly from
measurements, produced from models, and compiled from other sources such as databases or the
literature.
environmental technology - an all-inclusive term used to describe pollution control devices and
systems, waste treatment processes and storage facilities, and site remediation technologies and their
components that may be utilized to remove environmental pollutants or contaminants from or prevent
them from entering the environment. Examples include wet scrubbers (air), soil washing (soil),
granulated activated carbon units (water), and filtration (air, water). Usually, this term applies to
hardware-based systems; however, it also applies to methods or techniques used for pollution
prevention, pollutant reduction, or containment to prevent further movement of the contaminants, such as
capping, solidification or vitrification, and biological treatment.
GIS definitions -
Attribute Accuracy—the accuracy of the variables describing a map feature
The completeness of:
- Classification-assessment of how well the chosen classification is able to represent
the data
— Coverage—proportion of data available for the area of interest
- Verification-the amount and distribution of field measurements or other independent
sources of information that were used to develop the data
Lineage—the history of the data set, including its sources and processing steps.
Logical Consistency—the logical relations among data elements.
Positional Accuracy-the deviation of a mapped object from its true ground position.
Resolution Accuracy-the smallest discernible unit or object represented in the GIS
Time—whether the data is up-to-date enough for its intended use
graded approach - the process of basing the level of application of managerial controls applied to an
item or work according to the intended use of the results and the degree of confidence needed in the
quality of the results.
EPA QA/G-l STAR 26 January 2001
-------
guideline - a suggested practice that is nonmandatory in programs intended to comply with a standard.
QA manager- the individual designated as the principal manager within the organization having
management oversight and responsibility for planning, coordinating, and assessing the effectiveness of the
quality system for the organization.
quality management - that aspect of the overall management system of the organization that
determines and implements the quality policy. Quality management includes strategic planning, allocation
of resources, and other systematic activities (e.g., planning, implementation, and assessment) pertaining
to the quality system.
quality system- a structured and documented management system describing the policies, objectives,
principles, organizational authority, responsibilities, accountability, and implementation plan of an
organization for ensuring quality in its work processes, products (items), and services. The quality
system provides the framework for planning, implementing, and assessing work performed by the
organization and for carrying out required QA and QC activities.
secondary data - data previously collected for other purposes or from other sources, including the
literature, compilations from computerized data bases, or results from models of environmental
processes and conditions.
EPA QA/G-l STAR 27 January 2001
------- |