INTERIM DRAFT
EPA REQUIREMENTS FOR
  QUALITY ASSURANCE
    PROJECT PLANS
       EPA QA/R-5
       Region 6 U.S. EPA
     Office of Quality Assurance

       Dallas, Texas 75202
         MAY 1994

-------
                     QUALITY ASSURANCE PROJECT PLANS
       Introduction

       This document incorporates the national U.S. EPA Requirements for Quality
Assurance Project Plans (QAPP) and the Region 6 policy requirements into one document for
ease of use by our customers.

       It is both a Regulatory requirement and policy of EPA that all projects and tasks
involving environmentally related measurements  shall have a QAPP developed and approved
by the Agency prior to any such activities being  accomplished. This chapter presents
detailed instructions on what information must be contained in a Quality Assurance Project
Plan (QAPP) for environmental  data operations performed by or on behalf of EPA Region 6
and on the procedures for its review and approval. While there is diversity in the various
programs, it is Region 6 policy  that all elements  applicable to the QAPP Use Category, to be
discussed in detail further on in  this document, will be addressed in each QAPP. To do
otherwise would be  to deviate from the national standard and could dilute EPA's goal of
national consistency for Quality  Assurance requirements.

       The complexity of environmental data operations demands that a systematic process
and structure be established  to provide decision makers with the necessary confidence in the
quality of data produced for the  decisions to be made,  as well as with the means to determine
when the data are not fully usable and what to do about the situation. This process and
structure is provided by the  quality system for the organization conducting the environmental
data operations.  EPA policy requires that the collection of environmental data by and on
behalf of the  Agency be supported by a mandatory Quality System, documented in  a Quality
Management  Plan (QMP).

       An area that  must be addressed in each QMP is the systematic planning process used
for  each project or task such as  the Data Quality Objective process.  All projects involving
the  generation, acquisition and use of environmental data shall be planned using a systematic
planning process such as the Data Quality Objective process as defined by the current
revision of Guidance for Planning for Data Collection in Support of Environmental Decision
Making Using the Data Quality  Objectives Process, EPA QA/G-4, or acceptable alternate.
The use of the Data Quality Objectives process leads to definition of many of the specific
elements required in a QAPP.

       It is EPA policy that all decisions and work involving the use of environmental data
be supported  by a Quality Assurance Project Plan. The QAPP is the principal product of the
planning process inasmuch as it  integrates all technical and quality aspects for the life-cycle


                                          1

-------
 of the project, including planning, implementation, and assessment.  The QAPP provides a
 project or task-specific blueprint of how QA and QC are applied to an environmental data
 operation to assure that the results obtained are of the type and quality needed and expected.
 Effective implementation of the QAPP should provide the  best opportunity to achieve the
 technical and  quality goals of a project.

       The ultimate success of an environmental program  or project depends on the quality
 of the environmental data collected and used in decision-making, and this may depend
 significantly on the adequacy of the QAPP and its effective implementation.  Quality
 planning is an absolutely essential component of project management and the QAPP provides
 the mechanism for documenting the results of the planning process.  In the sections to
 follow, the elements of the QAPP are discussed in detail.  These elements represent the
 information that EPA believes to be necessary for data operations involving the character-
 ization of environmental  processes and conditions.
       Quality Assurance Project Plan Requirements

              QAPP Policy

       All work performed by or on behalf of EPA that involves the collection and use of
environmental data1 shall be supported by an Agency-approved Quality Assurance Project
Plan (QAPP). No work performed under contract, work assignment, technical directive,
delivery order, assistance agreement, or inter-agency agreement involving the environmental
data generated from direct measurement activities, collected from other sources, or compiled
from computerized data bases and information systems shall be implemented without an ap-
proved QAPP or without a condition stating that the actual collection of data shall not occur
until after the QAPP is approved. Region 6 has implemented this requirement by
establishing a process that requires a joint certification of the presence and approval status of
QMPs and QAPPs by the Region 6 Office of  Quality Assurance and the Region 6 Program
Office that has management responsibilities for the particular project.

             QAPP Purpose

       The QAPP is a critical planning document for any environmental data operation. The
QAPP documents how QA and QC activities will be planned, implemented, and assessed
during the life cycle of a program, project, or task.  The QAPP is the "road map"  or
blueprint for how a particular project is integrated with the QA program of the organization
performing the work and how the specific QA and QC activities will be applied during a
particular project.
       Environmental data include any information collected, produced, or derived from measurements, analyses,
or models of environmental processes and conditions, or from experimental systems representing such processes
and conditions, including results from laboratory analyses.

-------
             Applicability of QAPPs

       These QAPP requirements apply to all (intramural and extramural) environmental data
operations that acquire, generate, or compile environmentally-related data and that are per-
formed by or on behalf of EPA.  These operations include work performed  through con-
tracts,  interagency agreements, and financial assistance agreements (e.g., cooperative agree-
ments, grants).

       Environmental data operations encompass diverse and complex activities, and
represent efforts pertaining to rulemaking, compliance with regulations, and research.
Consequently, any requirements developed to define how QA/QC  should be applied to
environmental activities must contain considerable flexibility.  This may mean, for
example, that some environmental data operations, perhaps involving research projects, may
only require a qualitative discussion of the experimental process and its objectives, such as a
project narrative statement.  Others may require extensive documentation in  order to
adequately describe a complex environmental program. This means that the content and
level of detail in each QAPP will vary according to the nature of the  work being performed
and the intended use of the data. For QAPPs submitted to EPA  Region 6 the decision on
the content and  level of detail is to be consistent with national EPA requirements, as defined
by the  current revision of EPA Requirements for Quality Assurance  Project  Plans for
Environmental Data Operations, EPA QA/R-5. It is EPA policy that the type and quality of
environmental data needed for their intended use shall be defined and documented using the
EPA Data Quality Objectives  (DQO) Process,  or its equivalent, to the extent possible or
practicable.  The results  of the DQO Process provide key inputs to the QAPP and will
largely determine the level of detail required in the QAPP.  These inputs from the DQO
Process are identified later.

             QAPP Use Categories

       Because  of the diversity and variability in the mission requirements of the organiza-
tions (e.g., state program offices, EPA regions, research laboratories, municipal
organizations) comprising the environmental community, it is not always possible to define a
single checklist  of elements and details needed for all QAPPs.  To provide some of the
needed flexibility, several EPA organizations, including Region 6, have been categorizing
QAPP  requirements according to the type  of work being performed and the intended use  of
the data.  Four categories have been defined that vary the level of detail and  rigor prescribed
for particular QAPPs.

       These Use Categories are mandatory for QAPPs prepared and submitted for
approval to U.S. EPA Region 6. They are presented as an aid to determining the level of
detail that may be needed in a QAPP for a particular  type of work.  This approach
recognizes that not all environmental data  operations require QAPPs with the same level of
detail.  For example, data collected for compliance or enforcement decisions  in a Region will

-------
require a more comprehensive QAPP than an exploratory research project conducted for an
ORE) R&D laboratory.  The categories are:

       •      Category I -         Direct Support to Rulemaking, Enforcement, Regulatory,
                                  or Policy Decisions

              These projects include environmental data operations that directly support
              rulemaking, enforcement, regulatory, or  policy decisions.  They also include
              research projects of significant national interest, such as those typically
              monitored by the Administrator.  Category I projects require the most detailed
              and rigorous QA and QC for legal and scientific defensibility.  Category I
              projects are typically  stand-alone; that is, the results from such
              projects are sufficient to  make the needed decision without input
              from  other projects.

       •      Category n -        Complementary Support to Rulemaking, Regulatory, or
                                  Policy Decisions

              These projects include environmental data operations that complement other
              projects in support of rulemaking, regulatory,  or policy decisions.  Such
              projects are of sufficient  scope and substance that their results could be
              combined with those from other projects  of similar  scope to provide the
              necessary information for decisions.  Category II projects may also  include
              certain high-visibility projects as defined  by  EPA management.

       •      Category HI -        Interim Studies

              These projects include environmental data operations performed as interim
              steps  in a larger group of operations.  Such projects include testing  research
              hypotheses,  estimating effects, developing methods, and other work producing
              results that are used to evaluate and select options for interim decisions or to
             perform feasibility studies or preliminary assessments of unexplored areas for
             possible future work.

       •      Category IV -        Basic Studies

             These are projects involving environmental data operations  to study basic
             phenomena or issues,  including proof of concept and qualitative screening for
             particular analytical species.

      The final determination of a project's category is made by the EPA Region 6

-------
Program Office2 responsible for the work.  Where no category is defined, the preparer
of the QAPP shall assume that it is a Category I project.  It should be noted that projects
may contain specific tasks or subtasks that vary in the level of QA/QC requirements.  Such
conditions should be considered when deciding on the Use Category for a particular project.

             Special Requirements

       In some cases, it may be necessary for special  requirements to be added to the QAPP
that are not included in this document. When this occurs, such additional elements  must also
be addressed by the QAPP in the same manner as  the  elements listed in this document. The
EPA Region 6 program office sponsoring the work shall delineate any  specific requirements
beyond those listed in this manual.  If none are specified, the QAPP shall address all
elements required, as specified by their particular Use Category.  Attached documentation,
such as an approved Work Plan,  Standard Operating Procedures (SOPs),  etc., may be
referenced in response to a particular QAPP element.  This is, in fact,  encouraged to reduce
the time required to prepare the QAPP.   The QAPP shall also address related QA planning
documentation (e.g., Quality Management Plans, QA Project Plans) from subcontractors or
suppliers of services  critical to the technical and quality objectives of the project or  task.  In
any case, all referenced documents must  be attached to the QAPP itself or be placed on file
with the appropriate EPA office for routine referencing when needed.  Such references must
be kept current  by the submitter.

             QAPP Process Responsibilities and Approvals

       The QAPP may be prepared by different groups.  The QAPP may be  prepared by the
EPA principal investigator for an in-house project and may be reviewed by the principal
investigator's immediate supervisor before it goes to the QA manager or coordinator for final
approval.  The QAPP may be prepared by a contractor or an assistance agreement holder.
In addition, the QAPP may be prepared by another Federal agency under an interagency
agreement.  These specific situations must be addressed in the organization's  QMP to
establish how, when, and by whom review, approval,  and effective oversight of QAPPs
occurs.  Except where specifically delegated, all QAPPs prepared by non-EPA organiza-
tions must be approved by EPA for implementation. Region 6 policy  allows for  the
development of a QAPP to occur only after a QMP has been approved, and specifically
precludes approval of a QAPP until the applicable QMP has been approved.

       EPA believes that the necessary flexibility in content and level of  detail in the QAPP
may be best achieved by having the QAPP requirements reviewed and confirmed by the EPA
       Organization refers to the EPA Program Office, Region, or ORD Laboratory having an approved Quality
Management Plan that describes its quality system for planning, implementing, and assessing environmental
programs.

-------
project manager (or officer)3 with the assistance and concurrence of the EPA QA Manager.
In the case of Region 6, the authority to review and approve QAPPs has been delegated to
the Region 6 Program Offices, and is covered and defined by the Quality Management Plans
of the various divisions.

       None of the environmental work addressed by the QAPP should be started until the
initial QAPP has been approved and distributed to project personnel. It is Region 6 policy
that sixty days prior to the initiation of any environmental measurements or data generation,
the recipient (of any EPA  funding) shall submit to the EPA Project Officer, for review and
approval, a  written Quality Assurance Project Plan for the project.  Any costs for
environmental measurements or data generation incurred prior to approval of the
QAPP by the EPA Project Officer will be ineligible for reimbursement.  The Region 6
Policy does  not authorize granting conditional approval to a QAPP. There are only two
recognized statuses of a QAPP are approved, and not approved. It is the responsibility of
the organization performing the work to assure that no environmental data are acquired
before the QAPP is approved and received by project personnel.
             QAPP Implementation and Revision

       It is EPA policy that all approved QAPPs shall be implemented for the intended
work.  It is the responsibility of the group performing the work to implement the approved
QAPP and to ensure that all personnel involved in the work have copies of the approved
QAPP and all other necessary planning documents and understand their requirements prior to
the start of data generation activities.

       Because of the complex and diverse nature of environmental data operations, changes
to original plans are often needed.  When such changes occur, it is the responsibility of the
Project Manager for whom  the work is being performed to  determine if the change signifi-
cantly impacts the technical and quality objectives of the project. This determination should
be done in consultation with the QA Manager.  When substantive change is contemplated,
the originator of the QAPP  shall modify the QAPP as necessary to document the change and
submit the revision for approval by the same parties as for the original review.  Only after
the revision has been approved and receipt of the change (at least verbally  with written
follow-up) by the project personnel performing the work, shall the change  be implemented.

       It is absolutely essential that the QAPP be kept current and that all personnel involved
in the work effort have a current version of the QAPP available.  For  programs or projects
of long duration, such as multi-year monitoring programs, the QAPPs  shall be reviewed at
least annually by the Project Manager, revised if necessary  to reflect current needs, and
       3This term refers to the responsible EPA official for the project and includes such descriptors as project
officer, work assignment manager, and principal investigator.

-------
resubmitted for review and approval.  If the entire QAPP is current, valid and accurately
reflects the project goals and the organization's policy, at a minimum, each year the
organization will submit to EPA Region 6 a certification that the plan is current, to include a
copy of new, signed approval pages for the QAPP.

       OAPP Elements

              General Content

       All projects involving the generation, acquisition, and use of environmental data shall
be planned and documented. The QAPP is the logical product of the planning process and
must provide sufficient detail to demonstrate that:

       •      the project technical and quality objectives (e.g., DQOs, or equivalent) are
              identified and agreed upon;

       •      the intended measurements or data acquisition methods are appropriate for
              achieving project objectives;

       •      assessment procedures (including QA and QC) are sufficient for obtaining data
              of the type and quality needed and expected; and

       •      any limitations on the use of the data can be identified and documented.

       Environmental data operations require  the coordinated efforts of many individuals,
including managers, engineers,  scientists, chemists, statisticians, and others.  The QAPP
must integrate the contributions and requirements of everyone involved into a clear, concise
"blueprint" of what is to be accomplished, how it will be done, and by whom.  This means it
must provide understandable instructions to those who must implement all parts of the
QAPP, including the field sampling team, the analytical laboratory, and the data reviewers.
In all aspects of the QAPP, the use of national standards and practices, and inclusion of
standard operating procedures is encouraged.

       The QAPP elements that follow are presented  in an order corresponding to planning,
implementation, and assessment. They have been grouped for convenience into four types of
elements.  The individual QAPP elements are numbered sequentially within each group. The
four types of elements and their intent are summarized as  follows:

       A      Project Management - This group of QAPP elements covers the basic area of
              project management, including the project history  and objectives, roles and
              responsibilities of the participants, etc.  These elements ensure that the project
              has a goal, that the participants understand the goal and the approach to be
              used, and that project planning  is documented.

-------
       B     Measurement/Data Acquisition - This group of QAPP elements covers all of
             aspects of measurement systems design and implementation, ensuring that
             appropriate methods for sampling ,  analysis, data handling, and QC are
             employed and will be thoroughly documented.

       C     Assessment/Oversight - This group  of QAPP elements addresses the activities
             for assessing the effectiveness of the implementation of the project and associ-
             ated QA/QC.  The purpose of assessment is to ensure that the QAPP is
             implemented as prescribed.

       D     Data Validation and Usability - This group of QAPP elements covers the QA
             activities that occur after the data collection phase of the project is completed.
             They ensure that the individual data elements conform to the specified criteria,
             thus enabling reconciliation with the project objectives.

       The Use Categories and elements  are summarized in Table 4-1.  As stated earlier, the
decision to use this approach and the determination of a project's category and the actual
QAPP elements required is made by the EPA organization. Region 6 requires the Use
Category Codes be utilized in QAPPs. The term DQO after the QAPP Element title
indicates that the information needed to address this element in the QAPP is usually
identified in the DQO process.
                       TABLE 4-1  USE CATEGORY CHART
    CATEGORY
ELEMENT
DESCRIPTION
PROJECT MANAGEMENT

      I, II, III, IV        Al
      I, II, III            A2
      I, II, III, IV        A3
      I, II, III, IV        A4
      I, II, III, IV        A5
      I, II, III            A6
      I, II, III            A7
               IV       A8
      I                  A9
      I, II, III            A10
                 Title and Approval Sheet
                 Table of Contents
                 Distribution List
        (DQO)   Project/Task Organization
        (DQO)   Problem Definition/Background
        (DQO)   Project/Task Description
        (DQO)   Data Quality Objectives for Measurement Data
                 Project Narrative (ORD Only)
                 Special Training Requirements/Certification
                 Documentation and Records

-------
                      TABLE 4-1 USE CATEGORY CHART
                                 (CONTINUED)
    CATEGORY
ELEMENT
DESCRIPTION
MEASUREMENT/DATA ACQUISITION
      I, II, HI
      I, II, III
      I, II, III
      I, II, III
      I, II, III
      I, II

      I, II, III
      I

      I, II, III

      I, II
    Bl  (DQO)   Sampling Process Design (Experimental Design)
    B2          Sampling Methods Requirements
    B3          Sample Handling and Custody Requirements
    B4  (DQO)   Analytical Methods Requirements
    B5          Quality Control Requirements
    B6          Instrument/Equipment Testing, Inspection, and
                Maintenance Requirements
    B7          Instrument Calibration and Frequency
    B8          Inspection/Acceptance Requirements for Supplies
                and Consumables
    B9          Data Acquisition Requirements (Non-direct Mea-
                surements)
    BIO         Data Management
ASSESSMENT/OVERSIGHT
      I, II, III
      I, II, III
    Cl
    C2
Assessments and Response Actions
Reports to Management
DATA VALIDATION AND USABILITY
      I, II, III

      I, II
      I, II, III
    Dl

    D2
    D3
Data Review, Validation, and Verification Re-
quirements
Validation and Verification Methods
Reconciliation with Data Quality Objectives

-------
              Group A, Project Management


       Al           Title and Approval Sheet

              Include:

                            Title of the plan
                            Name of the organization(s) implementing the project
                            Names, titles, signatures of appropriate approving officials and
                            approval  dates for:

                                   Organization's Project Manager (Required)
                                   Organization's Quality Assurance  Manager (Required)
                                   Project Subordinate Supervisors Concurrence (Optional)
                                   Region 6 EPA Project Manager  (Required)
                                   Region 6 EPA Approving Official (Required)4
                                   Others, as  needed (e.g., State, other Federal Agency)

                            Title and  Region 6 QTRAK number of the approved Quality
                            Management Plan applicable to submitted QAPP

              Submission of at least two original approval pages is recommended.

       A2           Table of Contents

              List the sections, figures, tables, references, and appendices.  Document
              control format may be required at the option of the Project Manager and QA
              Manager,  and is encouraged by EPA Region 6.  When required, use document
              control format in the upper right-hand corner of each page following the Title
              and Approval Sheet.  For example:

                            Section No.
                           Revision No.
                           Date
                           Page	of
       4
       "EPA Approving Official" is the Region 6 Program Office Manager or staff person designated and authorized by Certification of the Region 6 QA
Officer to approve QAPPs. If the EPA Project Manager/Officer is unable to determine the approving official, contact the Office of Quality Assurance.

                                           10

-------
A3           Distribution List

       List the individuals and  their organizations who will receive copies of the
       approved QAPP and any subsequent revisions.  Include all managers who are
       responsible for implementing the plan, as well as the QA managers and
       representatives  of all groups involved.
A4           Project/Task Organization               (DQO)

       Identify the individuals or organizations participating in the project and discuss
       their specific roles and responsibilities. Include the principal data users and
       the decision-makers.  The project quality assurance manager shall be indepen-
       dent of the unit generating the data.  This does not include senior officials,
       such as corporate managers or agency administrators, who are nominally but
       not functionally involved in data generation, data use, or decision-making.
       The QAPP should also identify the person(s) responsible for approving and
       accepting final products and deliverables.

       Provide a concise organization chart  showing the relationships and the lines
       of communication among all project participants.  Include other data users
       who are outside of the organization generating the data, but for whom the data
       are nevertheless intended; e.g., modelers, risk assessors, design engineers,
       toxicologists, etc. Where direct contact between project managers and data
       users does not occur, such as,  between a project consultant for a Potentially -
       Responsible Party and EPA risk assessment staff, the organization chart should
       show the route by which information is exchanged.  The organization chart
       should  also identify any subcontractor  relationships relevant to environmental
       data operations.  This chart should be  realistic and practical, and should reflect
       only actual lines of authority and communication for the project  described.
       Names of current incumbent occupying a position is essential, as is the
       identification of vacant positions.
A5          Problem Definition/Background          (DQO)

       State the specific problem to be solved or decision to be made.  Include
       sufficient background information to provide a historical perspective for this
       particular project.  For example, this would include the regulatory or alleged
       toxic exposure situation that led to the need for this project.  The discussion
       must include enough information about the problem, the past history, any
       previous work or data,  and  any regulatory or legal context to allow a techni-
       cally-trained reader to make sense of the project objectives and activities.  This
       discussion also identifies the decision maker(s) and the principal customer(s)

                                    11

-------
              for the results.  (When the Data Quality Objectives [DQO] process has been
              used, this information should be readily available.)
       A6          Project/Task Description                 (DQO)

              Provide a description of the work to be performed. This discussion may not
              need to be lengthy or overly detailed, but it should give a overall
              picture of how the project will resolve the problem or question
              described in A5.  Describe in general terms the following, as needed:

              •     Measurements that are expected during the course of the project and the
                    approach that will be used.

              •     Applicable technical,  regulatory, or program-specific quality standards,
                    criteria,  or objectives.

              •     Any special personnel and equipment requirements that may indicate
                    the complexity of the project.

              •     The assessment tools needed (i.e., program technical reviews, peer
                    reviews, surveillances, and technical audits as needed and/or specified
                    by the QMP) for the project.

              •     A schedule for the work to be performed.

              •     Project and quality records  required, including the types of reports
                    needed.
       A7           Quality Objectives and Criteria for Measurement Data   (DQO)

             Any QAPP must include a statement of the project quality objectives and
             measurement performance criteria.  EPA recommends that a graded approach
             be used in planning, such as the Data Quality Objectives (DQO) Process.
             Even in those cases in which the formal DQO Process is not needed, a
             statement of the project quality objectives and measurement performance
             criteria is needed. The DQO process provides quality objectives based on
             several factors chosen by the user of the data.  For details on the DQO
             Process and when it should be used, please see the EPA guidance document
             (QA/G-4)5.
       5   EPA QA/G-4 is Guidance for Planning for Data Collection in Support of Environmental Decision Making Using flie Data Quality Objectives
Process.


                                          12

-------
The project quality objectives should be stated in quantitative terms to the
extent possible:

       Example:  UV Treatment of Contaminated Groundwater.  "The purpose
       of this project is to demonstrate whether or not the residual
       trichlorethene concentration in the treated  water is less than 0.5 /xg/L at
       a confidence level of 95 percent."

Without such quantitative goals, it is difficult to know whether the selected
analytical method is sufficiently sensitive or precise, or whether a sufficient
number of samples are being collected.  Sometimes, of course, project objec-
tives must be stated  somewhat less quantitatively, particularly in those situa-
tions when the use of the formal DQO Process is not needed.

       Example:  Oil Spill Remediation.  "One objective of this project is to
       determine whether the populations of clams, mussels, and sand fleas
       recover more rapidly in the treated than the untreated area."

       Example:  Epidemiology Study.  "The purpose of this project is to
       determine whether the concentrations of indoor NO2 and soot are
       greater in  the residences  of lung cancer patients than in the residences
       of healthy persons."

The section on project quality objectives and measurement performance criteria
should address, as appropriate,  the following:

•      the scope of the project;  that is, the domain (geographical locale and
       boundaries, environmental medium,  time period, etc.) over which
       conclusions and decisions will apply;

•      the time,  resource, or other constraints on the measurement project;

•      the intended uses  of the  data,  in order of importance and  the expected
       users of the data;

•      the specific data needed:  type, quantity, matrices involved;

•      the action  levels or standards upon which decisions will be made,
       including the detection limits and data reporting units, and the source(s)
       of this  information;

•      the population parameter(s) of interest; e.g., mean, maximum, or
       range,  which specify the form the data will be in when compared
       against action levels or standards;
                             13

-------
 •     the acceptable level of confidence in the data needed for the stated
       purposes or the acceptable amount of uncertainty;

 •     the quantitative sensitivity, precision, bias, and completeness criteria
       for each major measurement planned (including all pollutant and
       process measurements) for each sample matrix, based on the DQO
       statements;

 •     the units of expression of the precision and bias goals, which  should
       correspond to the methods selected to assess data precision and bias;
       and

 •     the goals for achieving data representativeness and comparability, and
       the planning considerations for attaining these goals (unlike precision,
       bias, and completeness, these objectives are not usually expressed or
       assessed quantitatively);
Data quality or measurement performance criteria may be typically specified in
terms of detection or quantitation limits, precision, bias, and comparability.
However, simply listing requirements for precision, bias, and completeness
without further discussion is not sufficient.  Even statements such as "bias will
be measured as percent recovery of a matrix spike sample"  are of marginal
help. In specifying data quality, it is thus essential to specify exactly how
such quality will be measured and interpreted.

       Example:  A possible statement of bias requirement. "Bias will be mea-
       sured a minimum of five times throughout this project by the analysis
       of standard reference materials No. 956B.  Recovery of TCE from this
       SRM should average 85 percent  or greater, with a relative standard
       deviation of no more than 20 percent."

Data specifications should also be distinguished from the specific QC proce-
dures that are  routinely carried out as part of each measurement. QC proce-
dures are used while carrying  out specific procedures; data specifications are
used for selecting the appropriate methods and QC criteria.

The QAPP may need to define different types of sensitivity  (i.e., qualitative,
quantitative, screening,  etc.) that may be appropriate for different parts of the
project.

The quantitative goals should reflect the total measurement,  if possible, or
address the field,  laboratory, and data handling components separately.  In the
event there is  no basis for defining data quality goals for the project, goals
                              14

-------
       may be estimated based on prior knowledge of the measurement system, and
       on method validation studies (using replicates, spikes, standards, recovery,
       studies, etc.) Explain the circumstances under which these goals were estab-
       lished.

       If defining quantitative goals is not relevant for certain measurements, indicate
       this and state the reason.

       Data representativeness is the degree to which the environmental samples truly
       reflect the population or material in the real world.  It can be affected by the
       time,  place, and manner by which the samples are collected.

       Data comparability is dependent upon consistency in sampling conditions,
                    selection of sampling procedures, sample preservation
                    methods,  analytical methods, and data reporting units,
                    throughout the project, and with previous projects with
                    which these results will be compared.

       The DQO Process for compliance and/or enforcement projects in the Regions
       may not be within the control of the EPA project manager and QA manager.
       The measurement performance criteria are specified in regulations, permits, or
       orders.  Whether or not those criteria satisfy the requirements of this docu-
       ment, they are absolutely required.  Often the results  of the DQO Process are
       not expressible in the terms stated here, such as precision, bias, or compara-
       bility.  The affected source is simply required to follow a specified method.
       In such cases, it is sufficient for the portion of the QAPP addressing the DQO
       Process to state that the testing will  satisfy the regulatory  requirements speci-
       fied.   This does not, however,  relieve the project manager or QA manager
       from their responsibility to comply with all other applicable QAPP require-
       ments in this document.
A8          Project Narrative

       Discuss in a narrative form the following issues as they pertain to the project
       or task, as needed:

             work to be performed or hypothesis to be tested,
             anticipated use of the data,
             how (quantitatively or qualitatively) the success of the project or task
             will be determined  (A7, D3),
             survey design requirements and description (Bl),
             sample type and sampling location requirements (B2),
             sample handling and custody requirements (B3),

                                    15

-------
                     selection of analytical methods (B4),
                     calibration and performance evaluation samples for sampling and
                     analytical methods used (B5),
                     sampling or analytical instrumentation requirements (B6),
                     plans for peer or readiness reviews prior to data collection (Cl), and
                     any on-going assessments during actual operation (oversight) (Cl).

              QAPP elements corresponding to the items to be addressed in the narrative are
              given in parentheses. The narrative should allow technical or QA readers  to
              relate the project or  task to the DQOs and to the Problem Definition stated
              earlier in the QAPP. (The use of the Project Narrative is consistent with the
              Use Category scheme described in Appendix A.  Since this element addresses
              in narrative form many  other QAPP elements, it is not necessary to specifical-
              ly include those elements in a Category IV QAPP.  Moreover, because the
              elements listed above would be addressed in more detail in Category I, II, and
              III QAPPs, it is not  necessary to include this element for those categories.)

       A9           Special Training Requirements/Certification

              Identify and describe any specialized training or certification requirements  for
              personnel in order to successfully complete the project or task.  Discuss how
              such training will be provided and how the necessary skills will be assured and
              documented.
       A10         Documentation and Records

              Itemize the information and records which must be included in a data report
              package for the project or task, and specify the reporting format,  if desired.
              Documentation can include raw data, filed logs, instrument printouts, and
              results of calibration and QC  checks.  Specify the laboratory data reporting
              turnaround time.  Specify whether a field sampling and/or laboratory analysis
              "case narrative"6 is required to provide a complete description of any difficul-
              ties encountered during sampling or analysis.

              Specify any requirements for  the final disposition of records and documents
              from the project, including location and length of retention period.
       6"Case Narrative" refers to an annotated summary of the analytical work performed by a laboratory that
describes in narrative form what activities were performed and identifies any problems encountered. The case
narrative provides additional information to user in interpreting the data received.

                                           16

-------
              Group B, Measurement/Data Acquisition

       The following QAPP elements describe the requirements related to the actual methods
to be used for the:

              •     collection, handling, and analysis of samples;

              •     direct measurement of parameters that do not require the actual collec-
                    tion of a sample; and

              •     the management (i.e., compiling, handling) of the data.

       The methods described in these elements may have been cited earlier in element A6;
however, the purpose here is  to include sufficient detailed information  to ensure  that the
methods are verified and documented. As long as the designated methods are well docu-
mented and are readily available to all project participants,  specific citations are  adequate.  If
these methods are not readily available, detailed copies of the methods and/or SOPs  must
accompany the QAPP as attachments.

       Bl           Sampling Process  Design (Experimental Design)   (DQO)

              Outline in general terms the experimental design of the project and the antici-
              pated project activities, including the types of samples required, sampling
              network design, sampling  frequencies,  sample matrices,  measurement parame-
              ters of interest, and the rationale for the design.  If individual sampling plans
              will be developed for discrete project phases, include their preparation sched-
              ule.

              Describe techniques or guidelines to be followed in selecting sampling points
              and frequencies, well installation design (when applicable), field decontamina-
              tion procedures and materials needed, and sampling equipment. When field
              screening techniques will be used to identify  samples for laboratory analysis,
              describe the criteria for sample selection.  Similarly, when locational data are
              to be collected, stored, and transmitted, the method(s) used must be specified
              and described (or referenced). Key elements to be addressed include how
              locations and their bias are determined.

              All measurements should be classified as critical (i.e., required  to  achieve
              project objectives) or non-critical (informational purposes only). Critical
              measurements will undergo closer scrutiny during the review and data gather-
              ing process, and will  have first-claim on limited budget resources.

              For non-standard methods or unusual sample matrices and situations, appropri-
              ate method validation study information is needed to confirm the performance


                                          17

-------
       of the method for the particular matrix.  Such validation studies may included
       round-robin studies performed by other organizations.  If previous validation
       studies are not available, they must be developed during the project and
       included as part of the project results.  It is very important for this element to
       include complete documentation and validation of both the sampling and
       analytical methodologies.  Identifying standard methods by number, date, and
       regulatory citation (as appropriate) is often sufficient.  However, many
       published (and even regulatory) methods allow the user to select from various
       options. The method citations should state exactly which options are being
       selected.

       Measurement of process conditions is often essential to a project (e.g., indus-
       trial plant or control equipment operation associated with a compliance test,
       meteorological parameters associated with impoundment volatization). In such
       cases, the experimental design must include the design and validation tech-
       niques as described above.
B2          Sampling Methods Requirements

       Describe the procedures for collecting samples. Identify the required sampling
                    methods (and/or equipment, if automated), including any
                    implementation requirements, decontamination
                    procedures  and materials needed, and any specific
                    performance requirements for the method.  For each
                    sampling method, identify any support facilities needed.
                    The discussion should also address what to do if there is
                    a failure in the sampling or measurement system and who
                    is responsible for corrective action.

       Describe the preparation and decontamination of sampling equipment, includ-
       ing disposal of decontamination by-products; the selection and preparation of
       sample containers, sample volumes, preservation methods, and maximum
       holding times to sample extraction and/or analysis. A tabular presentation
       format is strongly recommended,  particularly when two or more sample
       matrices are involved.
B3           Sample Handling and Custody Requirements

       Describe the provisions for sample handling and shipment, taking into account
       the nature of the samples and the maximum allowable sample holding times
       before extraction or analysis.  Describes the following provisions for sample
       custody, in both the field and the laboratory:


                                   18

-------
             Forms, notebooks and procedures to record the exact location and
             ambient conditions associated with sample collection, possession and
             analysis.  In the laboratory, a sample custody log should be maintained.

             Examples of sample documentation forms, such as sample labels,
             custody seals,  and chain-of-custody forms.

             Labeling procedures and information entered on the forms, including
             sample preservation, if any, and dates and times of sample transfer and
             analysis.

             Procedures for transferring and maintaining custody of samples.
B4           Analytical Methods Requirements       (DQO)

       Identify the analytical methods and/or equipment required, including any
       extraction methods needed, laboratory decontamination procedures and materi-
       als needed (such as in the case of hazardous or radioactive samples), waste
       disposal requirements (if any), and any specific performance requirements  for
       the method.  The QAPP should also address what to do if there is a failure in
       the analytical system and who is responsible for corrective action.

B5           Quality Control Requirements

       Discuss QC procedures that should be associated with each sampling, analysis,
       or measurement technique.  Such specific procedures are performed routinely
       during the measurement process, and the results are required to be evaluated
       immediately by the technician upon completion of the  test.  Results must fall
       within certain acceptance criteria, or specific corrective action is required.
       For projects at or beyond the  "proof-of-concept" stage, or for projects employ-
       ing well-characterized methods, this  section should list each required QC
       procedure, along with the associated acceptance criteria and corrective action.
       Because standard methods are often vague or incomplete in  specifying QC
       requirements, simply relying on the cited method to provide this information is
       usually insufficient.  In any case, QC procedures must frequently be modified
       on a project-specific basis in order to meet data specifications.

       QC procedures must be compatible with the data specifications discussed
       above.  This means, if a measurement must be precise within ±20 percent, the
       stability of calibration checks  must be somewhat better than +20 percent.  For
       some research-oriented  projects,  the  analytical  technique may not be available
       until well into the  project.  In such instances, detailed QC requirements may


                                   19

-------
       not need to be specified in the initial QAPP.  More appropriately, the initial
       document might specify general requirements for precision, bias, and detection
       limits, and the means of achieving these goals would be developed by the
       principal investigator during the course of the project.

       List the required QC checks, such as matrix spikes, duplicates, blanks,
       laboratory control samples, surrogates, or second column confirmation.  State
       the frequency of analysis for each type of QC check, and the spike compounds
       and levels. State or reference  the required control limits for each QC check
       and corrective action required  when control limits are exceeded.

       Describe the procedures to be  used to calculate each of the QC statistics,
       including the QC checks described in the preceding paragraph as well as
       precision and bias. Copies of  the formulas are acceptable as long as the
       accompanying  narrative or explanation specifies clearly how the calculations
       will address difficult  situations such  as missing data values and "less than" or
       "greater than"  values.
B6           Instrument/Equipment Testing, Inspection, and Maintenance Re-
              quirements

       Discuss how inspections and acceptance testing, including the use of QC
       standards and reference materials, of environmental sampling and measurement
       systems and their components must be performed and documented to assure
       their intended use as specified by the design. Identify and discuss how final
       acceptance shall be performed by independent personnel (e.g., personnel other
       than those performing  the work).  Discuss how deficiencies are to be resolved
       when acceptance criteria are not met, and how  and when re-inspection will be
       performed as necessary.

       Discuss how periodic preventive and corrective maintenance of measurement
       or test equipment shall be performed to ensure availability and satisfactory
       performance of the systems.  Identify the equipment and/or systems requiring
       periodic maintenance.  Discuss how the availability of critical spare parts,
       identified in the operating guidance and/or design specifications of the systems,
       will be assured and maintained.
B7          Instrument Calibration and Frequency

       Identify all tools, gauges, instruments, and other sampling, measuring, and test
       equipment used for data collection activities affecting quality that must be
       controlled and, at specified periods, calibrated to maintain bias within specified


                                    20

-------
       limits. Discuss how calibration shall be conducted using certified equipment
       and/or standards with known valid relationships to nationally recognized
       performance standards.  If no such nationally recognized standards exist,
       document the basis for the calibration.  Identify the certified equipment and/or
       standards used for calibration.  Indicate how documentation of calibration shall
       be maintained and be traceable to the instrument.
B8           Inspection/Acceptance Requirements for Supplies and Consumables

       Discuss how and by whom supplies and consumables shall be inspected and
       accepted for use in the project.  Supplies and consumables are those items
       necessary to support the sampling and analytical operation.  They include, but
       are not limited to: sample bottles, calibration gases,  reagents, hoses, materials
       for decontamination of sampling equipment, deionized water, and potable
       water.  Identify acceptance criteria for such supplies and consumables in order
       to  satisfy the technical and quality objectives of the project or task.
B9           Data Acquisition Requirements (Non-direct Measurements)

       Identify the type of data acquired from non-measurement sources such as
       computer data bases, spreadsheets,  and programs, and literature files.  Define
       acceptance criteria for the use of the data in this project.  Discuss any limita-
       tions on the use of the data based on uncertainty in the quality of the data and
       discuss the nature of that uncertainty.
BIO         Data Management

       Outline the project data management scheme, tracing the path of the data,
       beginning from receipt from the field or laboratory, to the use or storage of
       the final reported form.  Describe the standard  record-keeping procedures,
       document control system, and the approach used for data storage  and retrieval
       on electronic media.  Discuss the control mechanism for detecting and correct-
       ing paperwork errors and for preventing loss of data during data reduction
       (i.e., calculations), data reporting, and data entry to forms, reports, and
       databases.   Provide examples of any forms or checklists to be used.

       Identify and describe all data handling equipment and procedures  that will be
       used to process, compile, and analyze the data. This includes procedures for
       addressing data generated as part of this project as well as data from other
       sources.  The specifications should include any required computer hardware
       and software and should address any specific performance requirements for the


                                    21

-------
       hardware/software configuration used.  Describe the procedures that will be
       followed to demonstrate acceptability of the hardware/software configuration
       required.
       Group C, Assessment/Oversight

Cl          Assessments and Response Actions

       Identify the number, frequency, and type of assessment activities needed for
       this project.  Assessments include, but are not limited to, the following:

             surveillance,
             peer review,
             management systems review,
             readiness review,
             technical systems audit,
             performance evaluation,
             audit of data quality, and
             data quality assessment.

       Discuss the information expected from the assessment and success criteria
       (i.e., goals, performance objectives, acceptance criteria specifications, etc.)  for
       each assessment proposed.  For each proposed assessment, list the approximate
       schedule of activities, and discuss the information expected from the assess-
       ment and the criteria for success.  For any planned self-assessments (utilizing
       personnel from within the project groups), identify the participants and their
       exact relationship within the project organization.  For independent assess-
       ments, identify the organization and person(s) that will perform the assess-
       ments.  Discuss how and to whom the results of the assessments will be
       reported. Define the authorities of the assessors.  For example, if the asses-
       sors should order a work suspension upon finding a significant condition, this
       section delineates clearly their authority to do so.  Define explicitly the
       unsatisfactory conditions under which the assessors are authorized to act.
       Recognizing that assessments may be needed  at any time during the project,
       provide a schedule for the assessments to be performed.

       Discuss how response actions to non-conforming conditions will be addressed
       and by whom.  Identify who is responsible for implementing the response
       action.  Describe how response actions will be verified, validated, and docu-
       mented.
                                   22

-------
C2           Reports to Management

       Identify the frequency, content, and distribution of reports issued to inform
       management of the following:

       •      status of the project;
       •      results of performance evaluations and system audits;
       •      results of periodic data quality assessments; and
       •      significant quality assurance problems and recommended solutions.

       Identify the responsible organization(s) that will prepare the reports, and the
       recipients of the reports.  Identify any other status reports to management as
       well as their content and frequency.
       Group D, Data Validation and Usability

Dl           Data Review, Validation, and Verification Requirements

       State the criteria used to review and validate - that is, accept, reject, or qualify
       - data,  in an objective and consistent manner.  Provide examples of any forms
       or checklists to be used.

       Project-specific calculations or algorithms should be discussed.  Some projects
       may require special calculation during or after data generation:

              Example:  Indoor Air Pollution.  Consider a project that is meant to
              estimate the number of residences within the greater Washington,  D.
              C., area exhibiting N02 concentrations greater than 100 /xg/m3 at a
              frequency of 30 of more days per year.  Once NO2 measurements are
              available, one would attempt to extrapolate the limited information to
              the greater metropolitan area.  The QAPP  should explain the statistical
              techniques that will be employed, including how uncertainties will be
              assessed.

       For other projects, one may only need to calculate a mass balance of ade-
       struction/removal efficiency.  While these are much simpler
       requirements, the specific formulas and  data inputs should be listed.  This
       approach helps  assure that at least the minimum necessary data are collected
       for the intended interpretation (even if additional interpretation schemes are
       eventually employed).
                                    23

-------
D2           Validation and Verification Methods

       Describe the process to be used for validating and verifying data, including the
       chain of custody for data throughout the life cycle of the project or task.
       Discuss how issues shall be resolved and the authorities for resolving such
       issues.  Describe how the results are conveyed to data users.  The review of
       data can include checks of the following: transmittal errors, field and labora-
       tory QC data,  detection limits,  instrument calibration, special sampling or
       analysis conditions, performance evaluations, technical systems audits, contract
       compliance issues (e.g., holding times), and statistical data treatments, such as
       tests for identification of potential outliers.
D3           Reconciliation with Data Quality Objectives

       Describe how the results obtained from the project or task will be reconciled
       with the results of the DQO Process.  Describe how issues will be resolved.
       Discuss how limitations on the use of the data will be reported to decision
       makers.  Identify the procedures used to assess precision, bias, and complete-
       ness for the project data.

       A methodology has been developed to assist users in reconciling data results
       with the DQOs.  The Data Quality Assessment (DQA) Process*7* is used to
       assess the scientific and statistical quality of data  collected for a specific
       purpose.  In the DQA Process, the data will be analyzed scientifically to
       inspect for technical anomalies and to judge that the context of the data is
       correct.  At the same time, the data will be evaluated statistically to confirm
       that the statistical model was correct by selecting a statistical test and validat-
       ing  the test by verifying assumptions, such as for distribution and indepen-
       dence.  The outcome of the DQA process will indicate whether a decision can
       be made using the existing data or additional data must be collected.
                                    24

-------
                                     APPENDIX A

                             TERMS AND DEFINITIONS
Activity - an all-inclusive term describing a specific set of operations or related tasks to be
performed, either serially or in parallel (e.g., research  and development, field sampling,
analytical operations,  equipment fabrication), that in total result in a product or  service.

Assessment - the evaluation process used to measure the performance or effectiveness of a
system and its elements.  In this Standard, assessment is an all-inclusive term used to denote
any of the following:  audit, performance evaluation, management systems review, peer
review, inspection or  surveillance.

Audit - a planned and documented investigative evaluation of an item or process to
determine the adequacy and effectiveness as well as compliance with established procedures,
instructions, drawings, QAPPs, and other applicable documents.

Characteristic  - any property or attribute of a datum, item, process, or service  that is
distinct, describable, and measurable.

Computer Program - a sequence of instructions suitable for processing by a computer.
Processing may include the use of an assembler, a compiler, an interpreter, or a translator to
prepare the program for execution.  A computer program may be stored on magnetic media,
and be referred to as  "software", or may be stored permanently on computer chips, and  be
referred to as "firmware". Computer programs  covered by this Standard are those used for
design analysis, data acquisition, data reduction, data storage (data bases),  operation or
control, and data base or document  control registers when used as the controlled source  of
quality information.

Contractor - any organization or individual that contracts to furnish services or items or
perform work.

Corrective Action - measures taken to rectify conditions adverse to quality and, where
necessary, to preclude their recurrence.

Customer - any individual or organization for whom items or services  are furnished or work
performed in response to defined requirements and expectations.

Data  Quality Assessment (DQA) - a process of statistical and scientific evaluation that  is
used to assess the validity and performance of the data  collection design and statistical test,
and to establish whether a data set is adequate  for its intended use.


                                           25

-------
Data Quality Objectives (DQOs) - a statement of the precise data, the manner in which
such data may be combined, and the acceptable uncertainty in those data in order to resolve
an
environmental problem or condition.  This may also include the criteria or specifications
needed to design a study that resolves the question or decision addressed by the DQO
process.

Data Quality Objectives Process - a Total Quality Management (TQM) tool, based on the
Scientific Method and developed by the U.S. Environmental Protection Agency to facilitate
the planning of environmental data collection activities. The DQO process enables planners
to focus their  planning efforts  by specifying the use of the data (the decision), the decision
criteria (action level), and the  decision maker's acceptable decision error rates. The products
of the DQO process are the DQOs (See also Graded Approach).

Data Usability - the process of ensuring or determining whether the quality of the data
produced meets the intended use of the data.

Design Review - a documented evaluation by a team, including personnel other than the
original designers, the responsible designers, the customer for the work or product being
designed, and a  QA representative to determine if a proposed design will meet the
established design criteria and  perform as expected when implemented.

Engineered Environmental Systems - an all-inclusive term used to describe pollution
control devices and systems, waste treatment processes  and storage facilities,  and site
remediation technologies and their components that may be utilized to remove pollutants or
contaminants from the environment.  Examples include wet scrubbers  (air), soil washing
(soil),  granulated activated carbon unit (water), and filtration (air, water). Usually, this term
will apply to hardware-based systems; however, it will also apply to methods or techniques
used for pollutant reduction or containment of contamination to prevent further movement of
the contaminants, such  as capping, solidification or vitrification,  and biological treatment.

Environmental Conditions - the description of a physical medium (e.g., air,  water, soil,
sediment) or biological system expressed in terms of its physical, chemical, radiological, or
biological characteristics.

Environmental Data - any measurements or information that describe environmental
processes or conditions, or the performance of engineered environmental systems.

Environmental Data Operations - work performed to obtain, use, or report information
pertaining to environmental processes and conditions.

Environmental Monitoring - the process of measuring or collecting environmental data.

Environmental Processes - manufactured or natural processes that produce discharges to or

                                          26

-------
impact the ambient environment.

Environmental Programs - an all-inclusive term pertaining to any work or activities
involving the environment, including but not limited to: characterization of environmental
processes and
conditions; environmental monitoring; environmental research and development; the design,
construction, and operation of engineered environmental systems; and laboratory operations
on environmental samples.

Environmentally Related Measurements - the data collection activity or investigation
involving the assessment  of chemical, physical or biological factors in the environment which
affect human health or the quality of life.

Financial Assistance - the process by which funds are provided by one organization (usually
government) to another organization for the purpose of performing work or furnishing
services or items.  Financial assistance mechanisms include grants, cooperative agreements,
and government interagency  agreements.

Graded Approach - the process of basing the level of application of managerial controls
applied to an item or work according to the intended use of results and the degree of
confidence needed in the  quality of the results. (See Data Quality Objectives Process).

Hazardous Waste - any waste materials that satisfies the definition of "hazardous  waste" as
given in 40 CFR Part 261, "Identification and Listing of Hazardous Waste."

Independent Assessment - an assessment performed by a qualified individual, group, or
organization that is not a  part of the  organization directly performing and accountable for the
work being assessed.

Inspection - examination or  measurement of an item or activity to verify conformance to
specific requirements.

Item - an all-inclusive term used in place of the following:  appurtenance, facility,  sample
assembly, component, equipment, material, module,  part, product,  structure, subassembly,
subsystem, system, unit, documented concepts, or data.

Management - those individuals directly responsible and accountable for planning,
implementing, and assessing  work.

Management System - a structured  non-technical system describing the policies, objectives,
principles,  organizational authority, responsibilities, accountability,  and implementation plan
of an organization for conducting work and producing items and services.

Management System Review (MSR) - the qualitative assessment of a data collection

                                          27

-------
operation and/or organization (s) to establish whether the prevailing quality management
structure, policies, practices, and procedures are adequate for ensuring that the type and
quality of data needed are obtained.

May - denotes permission but not a requirement.

Method - a body of procedures and techniques for performing an activity (e.g., sampling,
chemical analysis, quantification) systematically presented in the order in which they are to
be executed.

Mixed Waste -  hazardous waste material, as defined by 40 CFR part 261 (RCRA),  mixed
with radioactive constituents.

Must - denotes a requirement  that has to be met.

Peer Review - a documented critical review of work generally beyond the state of the art or
characterized by the existence  of potential uncertainty. The peer review is conducted by
qualified individuals (or organization) who are independent of those who performed the
work,  but are collectively equivalent in technical expertise (i.e., peers) to those who
performed the original work.  The peer review is conducted to ensure that activities are
technically adequate, competently performed, properly documented, and  satisfy established
technical and quality requirements.  The peer review is an in-depth assessment of the
assumptions, calculations, extrapolations, alternate interpretations, methodology, acceptance
criteria, and conclusions pertaining to specific work and of the documentation that supports
them.  Peer reviews provide an evaluation of a subject where quantitative methods of
analysis or measures of success are unavailable or undefined, such as in  research and
development.

Performance Evaluation (PE) - a type of audit in which the quantitative data generated in a
measurement system are obtained independently and compared  with routinely obtained data to
evaluate  the proficiency of an analyst or laboratory.

Procedure - a documented set of steps or actions that systematically specifies or describes
how an activity is to be performed.

Process - an orderly system  of actions that are intended to achieve a desired end or result.
Examples of processes include analysis, design, data collection, operation, fabrication, and
calculation.

QTRAK - is a Computer Program that contains database information on Quality
Management Plans and Quality Assurance Project Plans to the Program Managers, Project
Officers, and the OQA for planning and assessment  of the status of regional Quality
Management Plans and the associated Project Plans.
                                          28

-------
Qualified Data - any data that have been modified or adjusted as part of statistical or
mathematical evaluation, data validation, or data verification operations.

Quality - the sum of features and properties/characteristics of a process, item, or service that
bears on its ability to meet the stated needs and expectations of the user.

Quality Assurance (QA) - an integrated system of management activities involving planning,
implementation,  assessment,  reporting, and quality improvement to ensure that a process,
item, or service is of the type and quality needed and expected by the customer.

Quality Assurance Management Staff (QAMS) - the U.S. EPA's headquarters staff
element that establishes and promulgates Quality Assurance Policy.

Quality Assurance Officer (QAO) - the designated Region 6 staff member that has the
delegated authority for approval of all Quality Management Plans in Region 6, Chief of the
Office of Quality Assurance.

Quality Assurance Program Description/Plan -see Quality Management Plan.

Quality Assurance Project Plan (QAPP) - a formal document describing in comprehensive
detail the necessary QA, QC, and other technical activities that must be implemented to
ensure that the results of the work performed will satisfy the stated performance criteria.

Quality Control (QC) - the overall system of technical activities that measures the attributes
and performance of a process, item, or service against defined standards to verify that they
meet the stated requirements established by the customer.

Quality Improvement - a management program for improving the quality of operations.
Such management programs generally entail a formal mechanism for encouraging worker
recommendations with timely management evaluation and feedback or implementation.

Quality Indicators - measurable attributes of the attainment of the necessary quality for a
particular environmental decision. Indicators of quality include precision, bias, completeness,
representativeness,  reproducibility, comparability, and statistical  confidence.

Quality Management - that aspect  of the overall management system of the organization that
determines and implements the quality policy. Quality management includes strategic
planning, allocation of resources, and other systematic activities  (e.g., planning,
implementation, and assessment) pertaining to the quality system.

Quality Management Plan (QMP) - a formal document that describes the quality system in
terms of the organizational structure, functional responsibilities of management and staff,
lines of authority, and required interfaces for those planning, implementing, and assessing all
activities conducted.

                                          29

-------
Quality System - a structured and documented management system describing the policies,
objectives, principles, organizational authority, responsibilities accountability, and
implementation plan of an organization for ensuring quality in its work processes, products
(items), and services.  The quality system provides the framework for planning,
implementing, and assessing work performed by the organization and for carrying out
required QA and QC.

Radioactive Waste - waste material containing radionuclides, or contaminated by
radionuclides.

Readiness Review - a  systematic, documented review of the readiness for the startup or
continued use of a facility, process, or activity.  Readiness reviews are typically conducted
before proceeding beyond project milestones and prior to initiation of a major phase of work.

Remediation - the process of reducing the concentration of a contaminant (or contaminants)
in air, water, or soil media to a level that poses an acceptable risk  to human health.

Research (Applied) - a process, the objective of which is to gain knowledge or
understanding necessary for determining the means by which a recognized and specific need
may be met.

Research (Basic) - a process, the objective of which is to gain knowledge or understanding
of the fundamental aspect of phenomena and of observable facts without specific applications
toward processes or products in mind.

Research Development/Demonstration - Systematic use of the knowledge and understanding
gained from research and directed toward the production of useful materials, devices,
systems, or methods, including prototypes and processes.

Self-Assessment - Assessments of work conducting by individuals, groups, or organizations
directly responsible for overseeing and/or performing the work.

Service - the category of economic activity that does not produce manufactured items. In
environmental data operations or engineering projects, such activities include design,
inspection, laboratory and/or field analysis, repair, and installation.

Shall - denotes a requirement that is mandatory and has to be met.

Should - denotes a guideline or recommendation.

Significant Condition - any state, status, incident, or situation of an environmental process
or condition of an engineered environmental system in which the work being performed will
be adversely affected in a manner sufficiently serious to require corrective action to satisfy
quality objectives or specifications and safety requirements.

                                          30

-------
Software Life Cycle - the period of time that starts when a software product is conceived
and ends when the software product is no longer available for routine use.  The software life
cycle
typically includes a requirements phase, a design phase, an implementation phase, a test
phase, an installation and check-out phase, an operation and maintenance phase, and
sometimes a retirement phase.

Standard Operating Procedure (SOP) - a written document that details the method for an
operation, analysis, or action  with thoroughly prescribed techniques and steps,  and that is
Officially approved as the method for performing certain routine or repetitive tasks.

Supplier - any individual or organization furnishing items or services or performing work
according to a procurement document or financial  assistance agreement.  This is an all-
inclusive term used in place of any of the following: vendor,  seller, contractor,
subcontractor, fabricator, or consultant.

Surveillance - the act of monitoring or observing a process or activity to verify conformance
to specified requirements.

Technical Review - a documented critical review of work that has been performed within the
state of the art.   The review is accomplished  by one or more  qualified reviewers who are
independent of those who performed the work, but are collectively equivalent in technical
expertise to those who performed the original work.  The reviews are an in-depth analysis
and evaluation of documents,  activities, material, data, or items that require technical
verification or validation for applicability, correctness,  adequacy, completeness, and
assurance that established requirements are satisfied.

Technical Systems Audit (TSA) - a thorough, systematic, on-site qualitative audit of
facilities, equipment, personnel, training  procedures,  record keeping, data validation, data
management, and reporting  aspects of a system.

Total Quality Management (TQM) - the process  of applying quality management to all
activities of the organization,  including technical and administrative operations.  See Quality
Management and Quality System.

Validation - an activity that demonstrates or  confirms that a process, item, data set, or
service satisfies the requirements defined by the user.

Verification - the act of authenticating or formally asserting the truth that a process, item,
data set, or service is, in fact, that which is claimed.

Work - the process of performing a defined task or activity (e.g., research and development,
field sampling, analytical operations, equipment fabrication).
                                          31

-------