Guidance on Satisfying
EPA Quality System Requirements for STAR Grants
                   EPA QA/G-1 STAR
                     January 2001
              United States Environmental Protection Agency
                  Office of Environmental Information
                      Quality Staff (2811R)
                  1200 Pennsylvania Avenue, NW
                     Washington, DC 20460

-------

-------
EPA QA/G-1 STAR                                  11                                         January 2001

-------
                                       FOREWORD

       This document is designed both to summarize the quality assurance requirements and to provide
guidance to an academic researcher (extramural grant applicant) preparing to submit a Science To
Achieve Results (STAR) research grant application. This document guides the applicant through the
concept and development of a Quality  System and describes how that system can be documented
through the required quality statement and the Quality Management Project Plan, which may be
required in some cases before an approved grant application is funded.

       This document is one of the EPA Quality System Series documents, which describe EPA
policies and procedures for planning, implementing, and assessing the effectiveness of the quality
system. Questions regarding this document or other Quality System Series documents may be
directed to:

                     U.S. Environmental Protection Agency
                     Quality Staff (2811R)
                     Office of Environmental Information
                     1200 Pennsylvania Avenue, NW
                     Washington, DC 20460
                     Phone: (202) 564-6830
                     Fax: (202)565-2441
                     E-mail:  quality@epa.gov

Copies of the EPA Quality System Series documents may be obtained from the Quality Staff or by
downloading them from the Quality Staffs home page:

                     http://www.epa.gov/quality
EPA QA/G-l STAR                               1                                   January 2001

-------
EPA QA/G-1 STAR                                  11                                         January 2001

-------
                            TABLE OF CONTENTS

                                                                        Page

Foreword  	  i
Acronyms and Abbreviations	  iv

1.     INTRODUCTION	1
      1.1    Background	1
      1.2    Purpose of this Document	1

2.     ENVIRONMENTAL PROTECTION AGENCY QUALITY ASSURANCE
      REQUIREMENTS 	2
      2.1    Quality Assurance and Quality System Requirements  	2
      2.2    Benefits of Quality Assurance to Grant Applicants	4
      2.3    Required Documentation for Quality Assurance  	5

3.     QA STATEMENT 	5
      3.1    What is a QA Statement?	5
      3.2    Components of the QA Statement 	6

4.     QUALITY MANAGEMENT PROJECT PLAN	7
      4.1    What is a Quality Management Project Plan?  	7
      4.2    Components of the Quality Management Project Plan	7

5.     QUALITY ASSURANCE REQUIREMENTS FOLLOWING GRANT AWARD	12
      5.1    Possible Quality Assurance Project Plan Requirement	12
      5.2    Implementation and Reporting 	12

REFERENCES	13

APPENDIX A.  OVERVIEW OF THE EPA QUALITY SYSTEM  	15
APPENDIX B.  BIBLIOGRAPHY OF USEFUL INTERNET ADDRESSES	19
APPENDIX C.  BIBLIOGRAPHY OF R AND G SERIES EPA QUALITY SYSTEM
      DOCUMENTS	21
APPENDIX D. GLOSSARY OF TERMS AND DEFINITIONS  	23
EPA QA/G-l STAR                         111                             January 2001

-------
                         ACRONYMS AND ABBREVIATIONS

CFR          Code of Federal Regulations
QA           quality assurance
QC           quality control
RFA          Request for Application
STAR        Science to Achieve Results (environmental research grants program)
EPA QA/G-l STAR                             IV                                  January 2001

-------
                              GUIDANCE ON SATISFYING
           EPA QUALITY SYSTEM REQUIREMENTS FOR STAR GRANTS

1.     INTRODUCTION

1.1    Background

       The National Center for Environmental Research in the U.S. Environmental Protection Agency
(EPA) Office of Research and Development promotes and advances environmental science in the
United States by competitively awarding grants for research focusing on reduction of risks to human
health and ecosystems and on reduction of uncertainty associated with risk assessment. This is done
through the Science To Achieve Results (STAR) program. The preponderance of the research
supported by the STAR program is funded through financial assistance agreements awarded through
competitive Requests for Applications (RFAs). These RFAs are developed from the Office of
Research and Development's Strategic Plan and from specific topical research plans developed by this
office. The RFAs are prepared in cooperation with other parts of the Agency and focus research on
areas of particular importance to the Agency and/or on areas that complement the Office of Research
and Development's intramural research programs and the programs of research partners in other
Agencies.  From time to time, EPA will also establish larger research centers and programs
competitively in specific areas of national concern requiring long-term or multi-disciplinary approaches.

       The most recent information about STAR programs, including grant opportunity
announcements, can be found on the National Center for Environmental Research web site:
                        http V/www.epa.gov/ncerqa
1.2    PURPOSE OF THIS DOCUMENT

       This document is intended to assist potential applicants and grantees in meeting the EPA quality
requirements for the STAR Program. It provides information for the applicant in documenting the
quality assurance approach to be used in the proposal and additional information for grantees when the
award document states that more detailed information is needed. Grants for larger, often multi-
university, research programs and centers will require different documentation and detail than is
included here. Specific directions on the required approach will  be provided in the individual
solicitations and other EPA guidance documents.

       This document is designed to provide the grant applicant with minimum required information on
the concepts of quality assurance (QA) and quality control (QC) and their appropriate application to
the STAR Program. EPA hopes it will help applicants to prepare proposals that will result in data or

EPA QA/G-l STAR                               1                                    January 2001

-------
procedures that can be interpreted or applied with confidence.  Following this introductory section,
subsequent sections present the following information:

       T      SECTION 2 presents the general EPA QA requirements for research involving data
              collection and analysis. It discusses what QA is and its importance in any type of
              research or production and a description of the two types of QA documentation
              needed for STAR research grants: the QA Statement, required of all applicants, and the
              Quality Management Project Plan, which will be required for certain types of projects.

       »      SECTION 3 provides expanded information about the QA Statement.

       T      SECTION 4 provides expanded information about the Quality Management Project
              Plan and its ten elements.

       T      SECTION 5 briefly describes the QA requirements that should be implemented
              following grant award.

       •      Appendix A provides an overview of the EPA Quality System.

       T      Appendices B and C provide lists of sources of additional information.

                     Appendix B - Internet addresses
                     Appendix C - Bibliography of EPA Quality System Series documents.

       »      Appendix D is a Glossary of Terms used in this document and their definitions.

2.     ENVIRONMENTAL PROTECTION AGENCY QUALITY ASSURANCE
       REQUIREMENTS

2.1    Quality Assurance and Quality System Requirements

       Basic quality terms used in this guidance are defined as follows:

              Quality assurance  (QA) is an integrated system of management activities
              involving planning, implementation, documentation, assessment, reporting, and
              quality involvement to ensure that a process, item, or service is of the type and
              quality needed and expected by the customer [American Society for Quality
              (ASQ), 1994].

              Quality control (QC) is the overall system of technical activities that measures the
              attributes and performance of a process, item, or service against defined standards to

EPA QA/G-l STAR                              2                                    January 2001

-------
               verify that they meet the stated requirements established by the customer; the
               operational techniques and activities used to fulfill quality requirements [American
               Society for Quality (ASQ), 1994].

EPA policy requires the application of QA and QC principles to all types of environmental studies. In
science and engineering research studies, products may range from a clear-cut identification of a
fundamental photochemical reaction mechanism (which may lead to a better understanding of ozone
formation) to a realistic assessment of the efficiency and longevity of a novel catalytic converter for
reducing automobile tailpipe emissions.  To ensure that such products are of adequate quality for their
intended use, the application of QA and QC is both prudent and necessary regardless of the level of
complexity of the work undertaken. Accordingly, EPA has established quality system requirements
that must be followed within EPA and by extramural contractors and financial assistance recipients for
all work performed that involves environmental data production and use.

        EPA assistance agreement recipients must implement or have implemented a quality system that
conforms to the American National Standard ANSI/ASQC E4-1994, Specifications and Guidelines
for Quality Systems for Environmental Data Collection and Environmental Technology
Programs (ASQ, 1994)1. The American National Standard provides the basis for planning,
implementing, documenting, and assessing a quality system in support of environmental programs.  It
includes elements addressing requirements for management of quality systems, environmental data
collection and use, and environmental technology development. For more information, see Appendix
A.  The quality requirement is also defined in 40 Code of Federal Regulations (CFR) Part 30, Grants
and Agreements With Institutions of Higher Education, Hospitals, and Other Non-Profit
Organizations, as applied to the STAR program (Parts 30.45 and 30.54). Any awards made to
State, Tribal, or local governments would be covered by quality requirements defined in 40 CFR Part
31, Uniform Administrative Requirements for  Grants and Cooperative Agreement to State,
Tribal, and Local Governments.

        EPA policy is to require that the quality system be  applied to all "environmental programs"
within the scope of the assistance agreement in accordance with the principle of graded approach, that
is, the level of detail needed to document quality  practices for proposed work depends on the type of
work and the intended use of the results. The simpler the project, the less  detail will be needed to
adequately document the quality practices for the project.  However, the intended use of the results will
also dictate the extensiveness of the QA and QC  documentation needed to substantiate the work
performed. The principle of graded approach reflects the importance of the work, not just its
complexity. This policy is consistent with all environmental programs operating under the EPA quality
system as defined by EPA Order 5360.1 A2, May 2000 (U.S. EPA, 2000a).
        1 ANSI/ASQC E4 is available for purchase from the American Society for Quality (Phone 800-248-1946).
Only in exceptional circumstances should it be necessary to consult this document.

EPA QA/G-1 STAR                               3                                    January 2001

-------
        Customarily, the EPA requirements for extramural agreements include documentation of (1) the
organization's quality system via a detailed Quality Management Plan, and (2) the application of quality
assurance and quality control activities to specific efforts via a QA Project Plan.  However, following
the graded approach, and in acknowledgment that research is often conducted simply to obtain data
rather than to be used in decision making, with peer-reviewed publications as a result, the STAR
program allows more flexibility. The quality system documentation for the typical STAR research grant
application will consist of a written "QA Statement," which accompanies the application and is
described in Section 2.  In some cases, EPA will require further documentation via a "Quality
Management Project Plan," which is described in Section 3.

2.2     Benefits of Quality Assurance to Grant Applicants

        A university researcher is usually familiar with the peer review process as it applies to selection
of research grants and to the publication of journal articles. EPA believes that as a potential  grant
applicant considers the  approaches to be taken to successfully complete the research project and
obtain documented quality data, the applicant should also identify and document the activities that will
ensure that the product of the research is of adequate quality to be used as planned.  Such planning
documentation is very beneficial when a grant application is peer reviewed or a manuscript is submitted
for publication.  Other documents demonstrating that the data meet applicable and appropriate quality
standards or criteria are critical and are positively received by many reviewers.

        A well-written and well-designed Quality Management Project Plan (or similar planning
document) may help detect problems  or incorrect assumptions before work begins and thus avoid false
starts or generation of questionable or unusable data sets.  Results of a well-designed experiment can
be invalidated by simple things such as misunderstanding verbal directions that could have been
included in standard operating procedures or research protocols2.  Similarly, an uncalibrated (or
improperly calibrated) sensor, such as a pH electrode, can result in data gathered at great expense of
time and money being unusable by the researchers. The cost to repeat the experiment may be
prohibitive.

        Introducing student researchers to QA and QC practices can be part of the educational
process they must have  to compete in today's job market. Quality is a leading force enabling
businesses to compete effectively in national and international markets.  People entering the business
world are being increasingly expected to understand basic QA and QC principles and practices and to
        In this context, standard operating procedures are defined as written and officially approved documents
that detail the preferred method for performing an operation such as sample collection, analysis, equipment use or
other routine tasks with thoroughly prescribed techniques and steps to ensure consistency.  They are typically
prepared by the organization performing the operation. Research protocols are written directions that describe the
plan of a scientific experiment such as the experimental design and data collection activities, and they may be
adapted  or modified as the experiment progresses or when an experiment is added to the project.

EPA QA/G-1 STAR                                 4                                       January 2001

-------
be proficient in their use. While EPA recognizes that quality requirements may be viewed as being
unnecessary burdens to the conduct of the experiment, experience has demonstrated that QA and QC
practices provide prudent safeguards against the occurrence of problems and the introduction of error
into the data produced which could have dramatic adverse impacts on the results and the conclusions
made from them.

       Many research projects are so novel that the chances of failure are higher than for other types
of research. However, establishing a quality system in the beginning should help lead to successful
completion of the project with data of sufficient quality for their intended use. Such a quality system
would provide a framework for the early detection of errors and for the documentation of the steps in
the experiment to help to assure the reproducibility of the research.  Furthermore, the application of a
quality system will aid the researchers in defending the validity of the project results.

2.3    Required Documentation For Quality Assurance

       Any project involving the work listed below must be supported by sufficient QA and QC
practices to assure that the results will be of the type and quality needed and expected for their intended
use.  This work includes:

       —     data collection or processing, including the use of secondary data;
       —     conducting surveys;
       —     making environmental measurements;
       —     describing processes or conditions;
       —     describing ecological or health effect and consequences
       —     creating or modifying models;  and
       —     developing environmental technology (whether hardware-based or via new techniques
               and methods) for pollution control and waste treatment.

For STAR grants, one or both of the following QA documentation will be required:

       T       QA Statement (mandatory)
       T       Quality Management Project Plan (may be requested).

       The purpose of the QA Statement and Quality Management Project Plan is to provide
information to the EPA Project Officer, QA Manager, and award official on an applicant's capabilities
to provide minimum required QA and QC for the proposed work. These documents should
demonstrate that QC procedures are in place to ensure that each project is successfully completed and
the objective is achieved. Assistance recipients are encouraged to create written SOPs and protocols,
and reference them in these documents.

3.     QA STATEMENT

EPA QA/G-l STAR                               5                                     January 2001

-------
3.1    What is a QA Statement?

       The QA Statement is a brief description of the quality assurance and quality control practices
that will be applied during a research project to assure that the results obtained satisfy the project
objectives. Typically, the QA Statement is presented as a narrative; there is no prescribed format
required. Each STAR grant applicant must provide this statement.  The QA Statement usually will not
exceed two consecutively numbered, 8.5x11-inch pages of single-spaced standard 12-point type with
1-inch margins.
EPA QA/G-l STAR                                6                                     January 2001

-------
3.2    Elements of the QA Statement

       EPA requires that, for each item listed below, the QA Statement either present the requested
information, reference (by page and paragraph number) the specific relevant portion of the project
description containing the information, or provide a clear logical justification as to why the item does not
apply to the proposed research.

       (1)     Discuss the activities to be performed or hypothesis to be tested  and the criteria for
               determining acceptable data quality.  (Note: Such criteria may be expressed in terms of
               precision, accuracy, representativeness, completeness, and  comparability, or in terms of
               data quality objectives or acceptance criteria.  Furthermore, these criteria must also be
               applied to determine the acceptability of existing or secondary data to be used in the
               project.  In this context secondary data may be defined as data previously collected for
               other purposes or from other sources, including the literature, compilations from
               computerized data bases, or results from models of environmental processes and
               conditions.)

       (2)     Describe the study design, including sample type and location requirements, all
               statistical analyses that were or will be used to estimate the types and numbers of
               physical samples required, or equivalent information for studies using survey and
               interview techniques.

       (3)     Describe the procedures for the handling and custody of samples, including sample
               collection, identification, preservation, transportation,  and storage.

       (4)     Describe the procedures that will be used in the  calibration  and performance evaluation
               of all analytical instrumentation and all methods of analysis to be used during the
               project.  Explain how the effectiveness of any new technology will be measured and
               how it will be benchmarked to improve existing processes, such as those used by
               industry.

       (5)     Discuss the procedures for data reduction and reporting and describe all statistical
               methods, with reference to any statistical software to be used, to make inferences and
               conclusions. Discuss any computer models to be designed  or utilized, with associated
               verification and validation techniques.

       (6)     Describe the quantitative and/or qualitative procedures that  will be used to evaluate the
               success of the project, including any plans for peer or other reviews of the study design
               or analytical methods prior to data collection.
EPA QA/G-l STAR                                7                                     January 2001

-------
       If parts of the QA Statement are not applicable to the project, a brief statement of justification
should be substituted for that element of the QA Statement. This is expected to occur infrequently.

4.     QUALITY MANAGEMENT PROJECT PLAN

4.1    What is a Quality Management Project Plan?

       If the scope of the project involves significant and complex environmental data collection and
analysis or environmental technology development, applicants for STAR research grants may be
required to submit a more detailed explanation of the QA and QC practices to be used in the project.
A Quality Management Project Plan is a document that includes elements of the QA Statement but
generally contains more information and details about the QA and QC activities planned.  The Quality
Management Project Plan may also be applied to small data collection tasks, small assistance
agreements for basic or exploratory research, and similar work of limited scope and duration, at the
recommendation of the EPA Project Officer or QA Manager. The EPA Project Officer will identify
when a project requires a Quality Management Project Plan.

       The Quality Management Project Plan should be prepared in accordance with the guidance
provided in this section and should describe the QC and QA practices to be implemented by the
applicant for the proposed project in sufficient detail to present a clear picture of what is to be done and
when. This is basically an enhancement of the QA Statement, without a page limit. The Quality
Management Project Plan must be submitted for review and approval by the EPA Project Officer and
QA Manager before beginning any work.

       Below are several questions to consider and answer as part of planning the Quality
Management Project Plan:

       •      What is the intended use  of the data and what decisions may be made as a result of this
              research (i.e., what is the basis for the hypothesis)?
       •      What quantity and quality of data are needed to be able to make the decision (i.e., test
              the hypothesis)?
       »      What procedures or methods will be used to sample and analyze environmental or
              laboratory media ?
       T      What analytes are planned to be measured, and what are the performance criteria for
              any type of measurements planned?

4.2    Elements of the Quality Management Project Plan

       The following information should be included in the Quality Management Project Plan in
appropriate sections:
EPA QA/G-l STAR                               8                                    January 2001

-------
       »       Project description;
               Proposed schedule with start and end dates and milestones for the project;
       T       Statement of the project objectives and statistical hypotheses;
       T       Description of the experimental design;
       »       Description of the sampling and analytical methods;
       »       Description of the process for sample handling and custody;
       T       List of key project staff;
       T       Description of how quality will be ensured during the project; and
       »       Identification of any needed special reports on the QA and QC activities performed, as
               appropriate.

       If any of these items have been covered in detail in the text of the application, they may be
included in the Quality Management Project Plan by reference to the specific page number in the
project narrative of the original application, although a stand-alone document would be preferred. The
content needed for each of these ten items is described below, along with brief examples of text and
tables. Science and engineering topics for STAR grants are so numerous that only a few selected
examples can be included; however,  not all elements may apply to a particular STAR grant topic. If an
element does not seem to apply, state this and indicate why in a brief discussion in the text.

       The specific elements of the Quality Management Project Plan are as follows:

(1)    Project Description

        A brief synopsis of the project (e.g., a few concise paragraphs) should describe the purpose
of the study, including the hypothesis to be tested and the unique research aspects of the project.  Also
note other anticipated uses of information or data generated in the project.  Particularly important is a
description of how the data collection activities will be performed, including the process or
environmental system that will be tested. If the project is part of a larger EPA program, describe how it
fits in the program.  The abstract included in the application may be adequate for this section.

(2)    Statement of the Project Objectives

       Project objectives of the research or test(s) should be summarized in this section.  Clearly
identify the primary goals of the project in quantitative terms, if possible, and state how the anticipated
results will achieve the research objectives of the project.  For example, a project objective may be
stated as:

       The objective of this project is to demonstrate a removal efficiency of 90 percent
       or higher with a confidence level of 95 percent for the heavy metals identified in
       the top 5 centimeters ofestuarine sediment.
EPA QA/G-l STAR                                9                                     January 2001

-------
Some objectives may not be stated entirely in quantitative terms. The criteria for successful completion
of the work should be described; that is, how will one know the objectives of the project have been
met?
EPA QA/G-1 STAR                                10                                   January 2001

-------
(3)    Proposed Schedule with Start and End Dates and Milestones for the Project

       The actual start and expected end dates of the project, including intermediate milestones and
results, should be identified. A milestone chart or table indicating start or end dates for major milestones
may be helpful.  If the project involves sampling, the Quality Management Project Plan should also
address the schedule for collecting samples and the laboratory analysis of the samples.

(4)    Description of the Experimental Design

       Describe the experimental design of the project with emphasis on the critical and noncritical
measurement aspects of the project. Critical measurements are those necessary to achieve the project
objectives; noncritical measurements are those used for process control or general background
measurements. What must be accomplished or what measurements taken to achieve those objectives?
Discuss the sampling design, including sample type and location requirements, how randomness was
achieved in the sampling design and any statistical analyses that were used to estimate the types and
numbers of samples required.  The experimental design should be linked (as much as possible) to the
quantitative project objectives discussed in a previous section.

(5)    Description of the Process for Sample Handling and Custody

       Describe the system for identifying (or numbering) samples, sample containers, required sample
volume or mass, avoidance of contamination, preservation, transportation (if applicable), holding times,
storage times and conditions (if applicable), and safe final disposal of samples.  Samples may be
attached as documentation for sample labels. Describe the procedure for recording sample history,
sampling conditions, and any other pertinent information. Formal sample custody is usually not
necessary but samples must still be tracked in some manner, especially those that are toxic or
radioactive. Not all environmental data come from physical samples. For example, real-time air data
are produced directly from continuous emission or ambient air monitors and do not require sample
labels, preservation, or transportation associated with physical samples.  It is important to note,
however, how a gaseous  sample will be brought from the point of sampling to the monitor.  It also is
important that a system be in place to accurately match data to the time  and place of the air sampling.

(6)    Description of the Sampling and Analytical Methods, and Equipment

        Describe the sampling and analytical methods that will be used including any instruments and
instrument details such as detection limits, range,  accuracy,  etc.  When standard methods are used,  such
as those published by EPA, the American Society for Testing and Materials, the American Public Health
Association, or other standards development organization, a reference to the method with any deviations
or choices specifically noted is sufficient documentation.  Generally, this description should document
questions such as:
EPA QA/G-1 STAR                                11                                    January 2001

-------
       •       Who will test proposed new industrial equipment or processes and how will their
               accuracy and effectiveness be demonstrated?

       •       What types of QC samples will be included in the sampling and analysis routines,
               including, for example, any spikes, duplicates, matrix spikes, matrix spike duplicates, and
               surrogates that will be used and why they were chosen.

       •       What methods of calibration for instruments will be used and how frequently will they be
               calibrated?

               What standards will be used for calibration, how they are prepared, how are they
               stored, and  do they provide traceability to higher accuracy standards?

               What are the relevant performance criteria for the sampling and analytical methods?

       •       Have analytical blanks, instrument response check samples, interferences, and the
               effects of temperature or pressure on the analytical method been considered and
               documented?

(7)    List of the Key Proj ect Staff

       All key personnel and their assigned responsibilities should be listed in this section.  The list
should include geographic locations, telephone and fax numbers, and electronic mail addresses. For
managerially complex projects, an organizational chart or communications plan may be  helpful.  Any
subcontractors or consultants (if used) and their responsibilities should be included. In  addition, any
special skills or need for training should be determined and steps to provide those skills or training
identified. Project personnel and organization may be shown in tabular form.

(8)    Description of How  Quality Will be Ensured During the Project

       The process by which quality will be ensured during the project should be described.  Overall,
EPA is interested in the quality of information (including environmental data) that the project requires and
how the necessary  quality will be obtained and documented. Some of the key questions that may be
asked in developing an appropriate description are:

       •       Can quantitative objectives be  established for quality?  Will only qualitative objectives be
               possible?

               How will the  data comparability and representativeness be determined?
EPA QA/G-1 STAR                                12                                    January 2001

-------
       •      What calculations will be performed and how will the correctness of calculations be
              ensured?

       •      What statistical procedures will be used to analyze the data (identify the commercial
              source of all programs)?
               Can the quality of the information or data be independently verified, and how?

       •      How will success be measured (measurement performance criteria) and how is it related
              to the project's objectives?

(9)    Description of Data Reduction Methods and Procedures

       Discuss how original or raw data measurements will be verified after they have been transferred
from instrument data recording devices and/or floppy diskettes or laboratory notebooks and processed
by computer or manually. Identify the specific descriptive statistical methods (for example, regression
analyses, analysis of variance, or multivariate analyses) that will be used to present results.  Discuss how
raw and processed data that are used in statistical analyses will be verified after statistical analyses have
been completed.  Also include a complete citation of software programs that will be used for these
statistical analyses and to present results.

       When using secondary data, include a discussion of the quality of these data and how the data
will be transferred into computer files for various analyses and how they are verified through these
processes. Discuss how the quality of these secondary data affect the results being reported.  Provide  a
complete citation of these data sources so that these data can be reviewed if necessary.

(10)   Description of Data Evaluation, Validation, and Verification

       Describe how data are to be evaluated and by whom. Identify who is responsible for verifying
the accuracy of the original data and whether or not the data met measurement quality objectives. The
performance criteria of the project should be discussed and the process that will be used to ensure the
performance criteria have been met. In addition, any performance evaluations, audits,  surveillance, and
other assessment procedures planned should be described, including the procedures to be used for data
validation and verification.  Also, there should be a discussion of how any corrective actions will be
implemented and documented and their effectiveness confirmed for any audits performed.

       Discuss any plans for peer or other reviews of the design or analytical methods prior to data
collection.

       If the project involves the development or use of a model, the plan should also include a
description of the project configuration; management; key assumptions; quality control procedures  for
data requirements and acceptance/rejection criteria; how the code will be maintained; the mechanism for


EPA QA/G-1 STAR                                13                                    January 2001

-------
transferring the code to the EPA Project Officer and potential users; and the model evaluation and
testing, including limitations of the model and its intended application. (Note: the award document
contains requirements for further documentation of models in reports.)

       For projects generating or using GIS/Remote Sensing data, discuss in the Quality Management
Project Plan (and reports) how these elements will be addressed: positional accuracy; attribute
accuracy; logical consistency; time; lineage; resolution accuracy; and the completeness of coverage,
classification, and verification. See Appendix D for definitions.

5.     QUALITY ASSURANCE REQUIREMENTS FOLLOWING GRANT AWARD

5.1    Possible Quality Assurance Project Plan Requirement

       Although a STAR grant application may have been recommended for funding, there may be
questions about the adequacy and suitability of the QA Statement or Quality Management Project Plan
resulting in the recommendation including a conditional QA approval. In this case, the grant's Project
Officer or QA Manager may require that a more comprehensive QA Project Plan be prepared and
approved before any research activities begin (see Appendix A).  These individuals make the final
decision about whether to require a QA Project Plan and then determine who will provide the EPA
approval.

        The QA Project Plan typically may be required for studies producing large volumes of data, or
determined to be controversial by EPA, or of a highly complex nature that may need  more extensive
documentation of the planning process, and for major grants in terms of the impact of or funding for the
work.  If the approval is conditional, a term and condition statement will be added to the award
document which specifies that more documentation is necessary, and that work involving environmental
data generation may not begin until the EPA Project Officer or QA Manager provides written
notification that the QA plan is approved.

       Often, a dialogue between the Principal Investigator and the EPA Project Officer can resolve
quality assurance requirements questions.  For instance,  after award, sections of the Quality
Management Project Plan could be expanded, or a standard operating procedure or protocol for
performing a certain task could be prepared and attached to the Quality Management Project Plan as an
appendix to supply additional information. This action must be approved by the EPA Project Officer or
the QA Manager for National Center for Environmental Research and documented prior to the initiation
of research.

5.2    IMPLEMENTATION AND REPORTS

       While summaries of QA and QC results may be included as an appendix to the final report,
specific OA and OC text descriptions are required by the grant's terms  and conditions to be addressed


EPA QA/G-1 STAR                               14                                   January 2001

-------
as part of annual and final reports. It is suggested that you review the terms and conditions of the
agreement as they are legally binding. In reports, be sure to note any laboratory's QA and QC plans as
well as any calibration and verification services performed outside the project research unit (for example,
verification of a microbalance, calibration of a flow meter, or tuning of a spectrophotometer). Also, any
secondary data used in the project must have  a complete citation.

       Following the grant award and approval of the QA Statement, Quality Management Project
Plan (if required) and QA Project Plan (if required), EPA expects that the elements of the applicable
document will be implemented as part of the  agreement. If the approach changes as the research
progresses to improve the integrity of the work or the validity of the results, these changes should be
briefly discussed in any annual and final report.
                                      REFERENCES

40 CFR 30, Code of Federal Regulations, "Grants and Agreements With Institutions of
       Higher Education, Hospitals, and Other Non-Profit Organizations."

40 CFR Part 31, Code of Federal Regulations, "Uniform Administrative Requirements for Grants and
       Cooperative Agreement to State and Local Governments."

American Society for Quality.  1994. Specifications and Guidelines for Quality Systems for
       Environmental Data Collection and Environmental Technology Programs. ANSI/ASQC
       E4-1994. Milwaukee,  WI.

U.S. Environmental Protection Agency, 2000. Guidance for the Data Quality Objectives Process
       (QA/G-4). EPA/600/G-961055, Office of Environmental Information.

U.S. Environmental Protection Agency, 1999. EPA Requirements for Quality Assurance Project
       Plans (QA/R-5).  Interim Final. Office of Environmental Information.

EPA Order 5360.1 A2, (May 2000), Policy and Program Requirements for the Mandatory EPA
       Quality System.  U.S. Environmental Protection Agency, Washington, DC.
EPA QA/G-1 STAR                              15                                  January 2001

-------
EPA QA/G-1 STAR                                  16                                      January 2001

-------
EPA QA/G-1 STAR                                  17                                      January 2001

-------
                                       APPENDIX A

            OVERVIEW OF THE ENVIRONMENTAL PROTECTION AGENCY
                                    QUALITY SYSTEM

       The EPA Quality System has been in place since 1979. The EPA Quality System operates
under the authority of Order 5360.1 A2, Policy and Program Requirements for the Mandatory EPA
Quality System (May 2000).  The quality system is applicable to work done by EPA employees as well
as by contractors and grant recipients. The quality system provides the management and technical
practices needed to ensure that environmental data are of adequate quality and usable for their intended
purposes.

       Figure A-l illustrates the components of the EPA Quality System. This rather elaborate diagram
can be broken down into three interactive structural levels: policy, organization or program, and project.

A.1    QUALITY SYSTEM — POLICY LEVEL

       The top level of Figure A-l illustrates the policy section. It is described here for background
information only.  Each STAR grant applicant or grant holder need only be aware of the policies and
standards; it is not necessary to review them to establish a quality system.  The policy section may be
thought of as the by-laws or reference library of the quality system. The internal and external EPA
quality policies are described here, including 40 Code of Federal Regulations (CFR) Part 30, Grants
and Agreements With Institutions of Higher Education, Hospitals, and Other Non-Profit
Organizations, which applies to nonprofit organizations working under EPA grants. Each policy
reflects the consensus American National Standard, ANSI/ASQC E4, Specifications and Guidelines
for Quality Systems for Environmental Data Collection and Environmental Technology Programs
[American  Society for Quality, 1994]. This American National Standard was authorized by the
American National Standards Institute and developed by the American Society for Quality. EPA played
a significant role in creating and writing the standard.

A.2    QUALITY SYSTEM — ORGANIZATION/PROGRAM LEVEL

       The second level of the quality system diagram shows the organization and program elements -
that is, the management structure. A Quality Management Plan is the focus of this section. Please
consult EPA Requirements for Quality Management Plans (QA/R-2) (U.S. EPA, 1999) for more
information on Quality Management Plans.  The Quality Management Plan is one of the requirements for
organization and research center grants, but not for STAR grants. For STAR grants, the Quality
Management Project Plan is sufficient. The Quality Management Project Plan combines the
organizational, program management, and training aspects of the Quality Management Plan with the
technical details usually found in the QA Project Plan.
EPA QA/G-1 STAR                               18                                   January 2001

-------
          Consult the following documents for further information on preparing and using
          the QA Project Plan. Both are available on the Quality Staffs home page
          (http://www.epa.gov/quality/qa_docs.html):  EPA Requirements for Quality
          Assurance Project Plans (QA/R-5)  (U.S. EPA, 1999) and EPA Guidance
          on Quality Assurance Project Plans (QA/G-5) (U.S. EPA, 1998).
A.3    QUALITY SYSTEM — PROJECT LEVEL

       The third level of the quality system diagram, Figure A-l, shows QA and QC activities at the
project or grant work level.  This is where a systematic plan for the work is documented in a QA
Project Plan (or, in the case for a STAR grant, in the QA Statement or Quality Management Project
Plan), the work is implemented according to the plan and any standard operating procedures, and
critical assessments of the experimental results and conclusions are made.  This process leads to
defensible products (data of known quality) and conclusions (decisions) that are communicated through
reports, journal articles, and presentations at symposia or seminars.
EPA QA/G-l STAR
19
January 2001

-------
        o
        LU

        O
        a:
        a.
            (D
            o
            a:
            a.
            N
            a:
            o
                 o
                 _i
                 o
                 a.
                                        Consensus Standards

                                             ANSI/ASQC E4

                                             ISO 9000 Series
Internal EPA Policies

    EPA Order 5360.1

    EPA Manual 5360
 External Policies

 Contracts - 48 CFR 46

Assistance Agreements -

 40 CFR 30, 31, and 35^
                                             EPA Prog ram &

                                         **  Regional Policy  "*
                                               Quality System

                                               Documentation
                     Supporting System Elements
                                    Training/Communication
          Annual Review
           and Planning
   Systems

 Assessments
                                                Conduct Study/

                                               s. Experiment  s
                                                Data Verification

                                                 & Validation
I
i
Standard
Operating
Procedures

i
k
Technical
Assessments
                    PLANNING

                       4....	
                                         -*• IMPLEMENTATION
                                                                            ASSESSMENT
                            Defensible Products and Decisions
EPA QA/G-l STAR
Figure A-l EPA Quality System Components
                      20
                             January 2001

-------
EPA QA/G-1 STAR                                  21                                      January 2001

-------
                                      APPENDIX B

                 BIBLIOGRAPHY OF USEFUL INTERNET ADDRESSES

1.      Air and Waste Management Association
                     http://www.awma.org

2.      American Chemical Society, Division of Environmental Chemistry
                     http://www.acs-envchem.duq.edu

3.      American Society for Quality
                     http://www.asq.org

9.      American Society for Quality, Energy and Environmental Division
                     http ://www. envnet. org/asq/eed

10.     EPA Grants Information
                     http://www.epa.gov/ORD/grants.htm

11.     EPA Peer Review Program
                     http ://www. epa.gov/ORD/spc/2peerrev.htm

12.     International Organization for Standardization (ISO 9000 and ISO 14000)
                     http://www.iso.ch

13.     National Academies of Science and Engineering, and Institute of Medicine
                     http://www.nas.edu

14.     National Center for Environmental Research (Office of Research and Development, U.S.
       Environmental Protection Agency)
                     http://www.epa.gov/ncerqa

15.     U.S. EPA Quality Staff
                     http://www.epa.gov/quality
EPA QA/G-l STAR                              22                                  January 2001

-------
EPA QA/G-1 STAR                                  23                                      January 2001

-------
                                     APPENDIX C

                        BIBLIOGRAPHY OF R AND G SERIES
                        EPA QUALITY SYSTEM DOCUMENTS

The EPA has the following documents to assist research grant applicants and recipients in complying
with quality assurance (QA) requirements:

       S     EPA Requirements for Quality Management Plans (QA/R-2)

       S     EPA Requirements for Quality Assurance Project Plans (QA/R-5)

       S     Guidance for the Data Quality Objectives Process (QA/G-4)

       S     Guidance on Quality Assurance Project Plans (QA/G-5)

       S     Guidance for the Preparation of Standard Operating Procedures for Quality-
             Related Documents (QA/G-6)

       S     Guidance on Technical Audits and Related Assessments (QA/G-7)

       S     Guidance for Data Quality Assessment: Practical Methods for Data Analysis
              (QA/G-9)

       S     Data Quality Assessment Statistical Toolbox (DataQUEST) (QA/G-9D)

       These documents and others may be downloaded from the EPA Quality Staff home page at:

                                http://www.epa.gov/quality

The documents are available in the Adobe Acrobat PDF format and may be printed using the free
Adobe Acrobat Reader, which is available on the Internet.
EPA QA/G-l STAR                             24                                January 2001

-------
EPAQA/G-1STAR                                 25                                     January 2001

-------
                                        APPENDIX D

                       GLOSSARY OF TERMS AND DEFINITIONS

calibration - comparison of a measurement standard, instrument, or item with a standard or instrument
of higher accuracy to detect and quantify inaccuracies and to report or eliminate those inaccuracies by
adjustments.

environmental data - any measurement or information that describes environmental processes,
locations, or conditions; ecological or health effects and consequences; or the performance of
environmental technology.  For EPA, environmental data include information collected directly from
measurements, produced from models, and compiled from other sources such as databases or the
literature.

environmental technology - an all-inclusive term used to describe pollution control devices and
systems, waste treatment processes and storage facilities, and site remediation technologies and their
components that may be utilized to remove environmental pollutants or contaminants from or prevent
them from entering the environment.  Examples include wet scrubbers (air), soil washing (soil),
granulated activated carbon units (water), and filtration (air, water).  Usually, this term applies to
hardware-based systems; however, it also applies to methods or techniques used for pollution
prevention, pollutant reduction, or containment to prevent further movement of the contaminants, such as
capping, solidification or vitrification, and biological treatment.

GIS definitions -
        Attribute Accuracy—the accuracy of the variables describing a map feature
        The completeness of:
              - Classification-assessment of how well the chosen classification is able to represent
              the data
              — Coverage—proportion of data available for the area of interest
              - Verification-the amount and distribution of field measurements or other independent
              sources of information that were used to develop the data
       Lineage—the history of the data set, including its sources and processing steps.
       Logical Consistency—the logical relations among data elements.
       Positional Accuracy-the deviation of a mapped object from its true ground position.
       Resolution Accuracy-the smallest discernible unit or object represented in the GIS
       Time—whether the data is up-to-date enough for its intended use

graded approach - the process of basing the level of application of managerial controls applied to an
item or work according to the intended use of the results and the degree of confidence needed in the
quality of the results.
EPA QA/G-l STAR                                26                                   January 2001

-------
guideline - a suggested practice that is nonmandatory in programs intended to comply with a standard.

QA manager- the individual designated as the principal manager within the organization having
management oversight and responsibility for planning, coordinating, and assessing the effectiveness of the
quality system for the organization.

quality management - that aspect of the overall management system of the organization that
determines and implements the quality policy.  Quality management includes strategic planning, allocation
of resources, and other systematic activities (e.g., planning, implementation, and assessment) pertaining
to the quality system.

quality system- a structured and documented management system describing the policies, objectives,
principles, organizational authority, responsibilities, accountability, and implementation plan of an
organization for ensuring quality in its work processes, products (items), and services. The quality
system provides the framework for planning, implementing, and assessing work performed by the
organization and for carrying out required QA and QC activities.

secondary data - data previously collected for other purposes or from other sources, including the
literature, compilations from computerized data bases, or results from models of environmental
processes and conditions.
EPA QA/G-l STAR                                27                                    January 2001

-------