Unitcii States Health Effects Research Laboratory EPA-600 1-79-013
f nviro imeni.ii Protection Office of Research and Development February 1979
As,<•! / U S Environmental Protection Agency
Research Triangle Park, NC 27711
Fie search anil Development
&EPA Guides for Quality
Assurance in
Environmental
Health Research
Health Effects
Research
Laboratory/RTP,NC
»;p eoo/i
79-013
-------
RESEARCH REPORTING SERIES
Research reports of the Office of Research and Development, U S. Environmental
Protection Agency, have been grouped into nine series These nine broad cate-
gories were established to facilitate further development and application of en-
vironmental technology Elimination of traditional grouping was consciously
planned to foster technology transfer and a maximum interface in related fields
The nine series are
1 Environmental Health Effects Research
2 Environmental Protection Technology
3. Ecological Research
4 Environmental Monitoring
5 Socioeconomic Environmental Studies
6 Scientific and Technical Assessment Reports (STAR)
7. Interagency Energy-Environment Research and Development
8 "Special" Reports
9 Miscellaneous Reports
This report has been assigned to the ENVIRONMENTAL HEALTH EFFECTS RE-
SEARCH series This series describes projects and studies relating to the toler-
ances of man for unhealthful substances or conditions This work is generally
assessed from a medical viewpoint, including physiological or psychological
studies In addition to toxicology and other medical specialities, study areas in-
clude biomedical instrumentation and health research techniques utilizing ani-
mals — but always with intended application to human health measures
This document is available to the public through the National Technical Informa-
tion Service, Springfield, Virginia 22161
-------
EPA-600/1-79-013
February 1979
GUIDES FOR QUALITY ASSURANCE
IN ENVIRONMENTAL HEALTH RESEARCH
HEALTH EFFECTS RESEARCH LABORATORY
RESEARCH TRIANGLE PARK, NORTH CAROLINA
BY
C. E. Tatsch, Ph.D.
Research Triangle Institute
Research Triangle Park, North Carolina 27709
AND
Ferris B. Benson
U. S. Environmental Protection Agency
Research Triangle Park, North Carolina 27711
CONTRACT NUMBER - 68-02-2612
PROJECT OFFICER
Ferris B. Benson
Research Advisory and Special Studies Office
Health Effects Research Laboratory
Research Triangle Park, North Carolina 27711
OFFICE OF RESEARCH AND DEVELOPMENT
U. S. ENVIRONMENTAL PROTECTION AGENCY
HEALTH EFFECTS RESEARCH LABORATORY
RESEARCH TRIANGLE PARK, NORTH CAROLINA 27711
-------
DISCLAIMER NOTICE
This report has been reviewed by the Health Effects Research
Laboratory, U. S. Environmental Protection Agency, and approved for
publication. Approval does not signify that the contents necessarily
reflect the views and policies of the U. S. Environmental Protection
Agency, nor does mention of trade names or commercial products
constitute endorsement or recommendation for use.
-------
FOREWORD
The U. S. Environmental Protection Agency's Health Effects Research
Laboratory located at Research Triangle Park, North Carolina conducts an
extensive research program to evaluate the human health implications of
environmental factors related to our industralized society. The purpose of
this research is to provide information necessary to formulate environmental
regulatory policies for the protection or improvement of public health and
welfare while at the same time enhancing the nation's productivity. To
this end, the Laboratory conducts a comprehensive environmental research
program in toxicology, epidemiology, and research on human subjects under
controlled laboratory conditions.
The quality of the data resulting from this research is an overriding
factor in determining its usefulness in EPA's regulatory activities. In
recognition of the importance of data quality assurance, our Laboratory
has instituted an active, comprehensive program to coordinate the development
and implementation of effective quality assurance planning into all research
within the Laboratory. This document represents the current statement of
our effort. I am confident that full implementation of our data quality
assurance policy, with the help of the guideline manuals and the increased
awareness of the importance of data validation and good management procedures,
will enhance the scientific merit of our research program.
F. Gordon "Hueter
Director
Health Effects Research Laboratory
ill
-------
ACKNOWLEDGEMENTS
These guidelines were completed through the auspicies of the
Quality Assurance Committee, HERL-RTP. Committee members and other
HERL research staff provided assistance and their valuable time to
review, criticize, and make many helpful suggestions.
The preparation of the manuscript was supported through contract
EPA-68-02-2612 with the Research Triangle Institute, Research Triangle
Park, North Carolina.
iv
-------
CONTENTS
Page
1.0 SUMMARY 1-1
2.0 INTRODUCTION 2-0
2.1 Laboratory Mission 2-1
2.2 Data Quality at HERL/RTP 2-1
2.3 Definitions 2-2
2.3.1 Quality 2-2
2.3.2 Quality Assurance (QA) 2-3
2.3.3 Quality Control (QC) 2-3
2.3.4 Task 2-3
2.3.5 Protocol 2-4
3.0 MANAGEMENT POLICY 3-0
3.1 Quality Goals 3-1
3.2 Quality Policies 3-2
3.2.1 Task QA Design 3-3
3.2.2 Task QA Review and Approval 3-3
3.2.3 Laboratory QA Program 3-4
3.2.4 Scope of Quality Assurance Program 3-4
3.3 Quality Assurance Program Organization 3-5
3.3.1 Organizational Structure for
Quality Assurance 3-6
3.3.2 Functional Responsibilities 3-8
-------
4.0 GUIDELINES FOR PROJECT OFFICERS 4-0
4.1 General 4-2
4.1.1 Control Charts as Applied to
Research Projects 4-4
4.2 Experimental Design 4-4
4.2.1 Statistical Experimental Design 4-5
4.2.2 Quality Control Considerations 4-13
4.2.3 Data Collection and Analysis 4-13
4.2.4 Biological Systems 4-15
4.3 Personnel 4-17
4.4 Facilities and Equipment 4-19
4.5 Recordkeeping 4-22
4.6 Supplies 4-24
4.7 Sample Collection 4-27
4.8 Sample Analysis 4-28
4.9 Internal Audits 4-29
4.10 Preventive Maintenance 4-31
4.11 Calibration 4-33
4.12 Documentation Control 4-41
4.13 Configuration Control 4-42
4.14 Data Validation 4-43
4.15 Feedback and Corrective Action 4-44
4.16 Data Processing and Analysis 4-45
4.17 Report Design 4-47
5.0 DATA QUALTIY ASSURANCE FOR RESEARCH PROJECTS 4-0
5.1 Quantitative Estimates of Data Quality 5-1
5.2 Qualitative Samples of Data Quality 5-2
6.0 GUIDELINES FOR ATMOSPHERE GENERATION AND MONITORING.. 6-0
6.1 Introduction 6-1
6.2 Atmosphere Generation 6-1
6.2.1 General Considerations 6-2
6.2.2 Particulate or Aerosol Atmospheres 6-4
VI
-------
6.3 Sample Collection and Analysis 6-6
6.3.1 Introduction 6-6
6.3.2 Sample Representation'ty 6-7
6.3.3 Physical Characterization of the
Atmosphere and Sampling Equipment 6-8
6.3.4 Sample Quantity 6-8
6.3.5 Sample Handling and Storage 6-9
6.3.6 Recommendations for Sampling and Analysis. 6-10
7.0 ANIMAL DOSING 7-0
7.1 Preparation 7-1
7.1.1 Preparation of Animal s 7-1
7.1.2 Preparation of the Test Substance 7-2
7.1.3 Preparation of the Control Substance 7-3
7.1.4 Preparation of the Vehicle 7-4
7.1.5 Mixing 7-5
7.2 Administration 7-5
References R-0
-------
FIGURES
Page
Figure 1. Functional Management Structure HERL/RTP 3-7
Figure 2. HERL Quality Assurance Organization 3-9
Figure 3. Example of Major Topics Addressed in a Research
Task Protocol 4-6
Figure 4. Proposed Protocol Contents EPA 4-7
Figure 5. Summary of EPA's Proposed GLP's for Health Effects 4-8
Figure 6. Typical Calibration Curve 4-40
Figure 7. Minimum Report Technical Content for
EPA Health Effects Tests 4-48
TABLES
Table 1. Currently Available NBS-SRMs for Environmental
Research and Control
4-36
Table 2. Summary of Measurements Methods for Selected
Pollutants
6-11
vm
-------
SECTION 1
SUMMARY
This document is the statement of the Quality Assurance (QA)
policy at the Health Effects Research Laboratory of the U.S. Environ-
mental Protection Agency at Research Triangle Park, North Carolina
(HERL/RTP). It provides guidelines for functional managers as they
implement agency policy and evaluate research protocols. It provides
guidelines for project officers as they develop and execute protocols
for intramural and extramural research tasks in support of the HERL
mission.
Since the necessity of assuring data of adequate quality pervades
the entire scope of the HERL research effort, the QA program is de-
signed to be correspondingly pervasive, including quality control and
quality assurance planning and activities. Each aspect of the research
task is analyzed from the perspective of designing, evaluating, and
executing a research protocol to ensure adequate data quality.
The project officer holds primary responsibility for selecting
specific quality assurance techniques and developing an appropriate
quality assurance program for each of his tasks. Functional management
is responsible for including evaluation of these quality assurance
programs in the regular review and approval process, in the planning
stages as well as during execution. The quality assurance organization
is structured to assist project officers and management with task-
specific problems and to evaluate and document laboratory-wide issues
of data quality.
In order to aid both project officers and management in research
task quality assurance, the research task is analyzed in detail in
Section 4 in terms of its various operational phases, from the planning
and experimental design through data quality aspects of the final
report.
Following this discussion are QA guidelines for pollutant expo-
sure and dose monitoring.
1-1
-------
Programs for research quality assurance must evolve, since the
concept of a formalized QA program and budget is relatively new to
health research and since the research to which the QA program is
applied, by nature, changes and develops. For this reason, the QA
guidelines for HERL are extended, updated, and issued on a regular
basis.
1-2
-------
INTRODUCTION
-------
.0 INTRODUCTION 2-1
2.1 Laboratory Mission 2-1
2.2 Data Quality at HERL/RTP 2-1
2.3 Definitions 2-2
2.3.1 Quality 2-2
2.3.2 Quality Assurance (QA) 2-3
2.3.3 Quality Control (QC) 2-3
2.3.4 Task 2-3
2.3.5 Protocol 2-4
2-0
-------
SECTION 2
INTRODUCTION
2.1 LABORATORY MISSION
The Health Effects Research Laboratory, Research Triangle Park,
North Carolina (HERL/RTP) conducts animal and human studies under con-
trolled conditions and performs additional studies on human populations
in order to assess the hazard to human health of exposure to environ-
mental pollutants. Laboratory scientists determine the effects of
environmental pollutants both alone and in combination; pollutant types
that are studied include air pollutants, pesticides, toxic substances,
and nonionizing radiation. Controlled laboratory studies are devoted to
determining effects of pollutants on normal biological function as
measured by clinical, chemical, biochemical, physiologic, histopatho-
logic, growth, reproduction, and other parameters. HERL/RTP develops,
evaluates, and improves analytical chemical methods and biological
screening techniques for direct and indirect measurement of exposure to
environmental toxicants. It also serves as a resource for information
on the health effects of environmental pollutants and coordinates
health-related activities with international organizations.
2.2 DATA QUALITY AT HERL/RTP
HERL/RTP has long recognized the importance of quality control in
its research activities. For example, pesticide QA programs have been
supported for several years and have been of service to over 100 labor-
atories. In addition to the pesticide analytical procedures, several
related QA manuals have been prepared under this program. Interlabora-
tory pesticide QA programs have also been maintained for several years.
However, quality control has generally been practiced on a
project-by-project basis, with the preparation and implementation of
quality control activities being the decision of individual project
2-1
-------
officers. Due to increased awareness of the deleterious effects of
pollutants on living systems, HERL/RTP management recognizes the need
for a formal, comprehensive, laboratory-wide data quality program.
A formal HERL/RTP data quality program was initiated in May 1976
with the issuance of a "Quality Assurance Plan" by HERL/RTP. Subse-
quently, a Quality Assurance Coordinator was appointed as chairman of
the Quality Assurance Committee for the express purpose of designing
and implementing a Laboratory-wide QA program appropriate to the unique
requirements of the HERL/RTP. Quality assurance guidelines have since
been developed and published for management policy [1] and for research
task planning [2]. While specific quality assurance guidelines have
been developed for environmental pollutant measurements [3,4,5], lack
of adequate and comprehensive guidelines for quality assurance in bio-
logical research has hampered the completion of an integrated quality
assurance program at HERL/RTP.
2.3 DEFINITIONS
The American Society for Quality Control has carefully defined
terms that apply to quality [6] and is currently revising these
definitions to reflect current understanding of quality terminology.
The Quality Assurance Handbook for Air Pollution Measurement Systems
[3] provides similar definitions of quality terminology applicable to
air pollution data collection systems. For the sake of clarity,
several terms related specifically to health research data quality are
defined as they are used in these guidelines.
2.3.1 Quality
Quality means the totality of characteristics of
research data that bear on their ability to satisfy pre-
viously specified criteria. For laboratory measurement
systems, accuracy, precision, and representativeness are
characteristics of major importance. Completeness is an
additional characteristic appropriately applied to larger
systems, such as air monitoring networks.
2-2
-------
This definition implies appropriate planning for,
and specifications of, the quality characteristics to be
achieved. Included in the establishing of the specified
criteria are total resource considerations (e.g., eco-
nomics, safety, maintainability).
2.3.2 Quality Assurance (QA)
Quality Assurance means planned, systematic
actions that are necessary to ensure that the specified
quality criteria are achieved. Thus, quality assurance
(QA) planning is necessary at the management level in the
development of QA policy. QA planning is also necessary
in the development of the details of task protocols by
project officers (see Section 2.3.5). QA activities
result from QA planning and consist of a variety of
activities. Quantitative measurements—such as cali-
bration, interlaboratory tests, and analysis of "blind
samples"— are used. Qualitative measures—such as site
visits by qualified professionals—are also used to
evaluate the capability of a total measurement system for
providing specified quality data. QA, in planning and
execution, is a management function independent of task
operating personnel.
2.3.3 Quality Control (QC)
Quality Control is a system of activities designed
to achieve and maintain a previously specified level of
quality in data collection, processing, and reporting.
QC is performed by the organization actually carrying out
the task or project; i.e., it is executed by task person-
nel. QC activities include control or correction for all
variables suspected of affecting data quality. These
variables are discussed in Section 4.
2.3.4 Task
A task is an in-house or contracted project or
grant, or an interagency agreement, the purpose of which
is to reproduce technical research data for the HERL/RTP
program.
2-3
-------
2.3.5 Protocol
As used in this document, the term protocol should
be understood to include all task or project planning
documents used at the HERL/RTP. Specifically included
are research protocols, support activity procedure state-
ments, contractors' work plans, and scopes-of-work, irre-
spective of the nature of the task or the organization
actually performing the task.
Protocols are specifically understood to include
plans for total task quality assurance—from the develop-
ment of appropriate experimental design through the final
report.
2-4
-------
MGMT. POLICY
-------
Page
3.0 MANAGEMENT POLICY 3-1
3.1 Quality Goals 3-1
3.2 Quality Policies 3-2
3.2.1 Task QA Design 3-3
3.2.2 Task QA Review and Approval 3-3
3.2.3 Laboratory QA Program 3-4
3.2.4 Scope of Quality Assurance Program 3-4
3.3 Quality Assurance Program Organization 3-5
3.3.1 Organizational Structure for Quality Assurance. 3-6
3.3.2 Functional Responsibilities 3-8
3.3.2.1 Program Coordinator 3-8
3.3.2.2 Task Management 3-10
3.3.2.3 Functional Management 3-11
3.3.2.4 Quality Assurance Coordinator 3-12
3.3.2.5 Quality Assurance Representative 3-13
3.3.2.6 Quality Assurance Committee 3-13
3-0
-------
SECTION 3
MANAGEMENT POLICY
The purpose of this section is to describe management policies and
goals and the organizational structure for the implementation of a
Quality Assurance program at HERL/RTP. The QA organization, consisting
of a Quality Assurance Coordinator and a Quality Assurance Committee,
serves in an advisory capacity to HERL/RTP professional and technical
staff.
Planning for the application of QA measures is the responsibility
of project officers. It is the responsibility of functional management
to ensure that any project-oriented document or plan has incorporated
appropriate QA measures. It is also the manager's responsibility to
ensure that task QA plans are implemented and that task data quality is
adequate for its intended purpose. To aid in carrying out these re-
sponsibilities, the QA organization is available to HERL technical and
management personnel for consultation and, upon request, for active
participation. EPA's commitment to QA is reflected in several ways
(see, for example, reference [9]).
The goal of the HERL/RTP Quality Assurance Program is to ensure,
assess, and document the medical and scientific reliability of
laboratory and field data used in EPA's activities and documents
relating to human health. Managerial, administrative, statistical,
investigative, preventive, and corrective techniques are employed to
maximize reliability of the data.
3.1 QUALITY GOALS
Specific goals of the HERL/RTP QA program are to:
a. Provide a vehicle, including an organizational struc-
ture, that will alert all personnel within HERL/RTP to
3-1
-------
the basic concepts of quality assurance and to the level
of quality expected within HERL/RTP.
b. Establish and maintain guidelines to assist HERL/RTP per-
sonnel in the logical development of general and specific
quality assurance plans for current and future HERL/RTP
research.
c. Provide a means for evaluating proposed and ongoing tasks
for appropriateness to the current and anticipated data
requirements of HERL/RTP.
d. Implement, as part of the management plan, a procedure to
review data quality aspects of research protocols and
data currently being collected or data collected in the
past, as deemed appropriate.
e. Encourage the use and development of methods of analysis
and data treatment that are capable of meeting the data
quality needs required by the HERL/RTP mission, as well
as the use for which the data are intended.
f. Monitor the operational performance of HERL/RTP through
appropriate intralaboratory and interlaboratory quality
evaluation programs. This may be accomplished through
cooperation with: evaluation services provided by other
laboratories of ERC/RTP; other EPA laboratories; other
governmental agencies (NIOSH, FDA, NBS); and private
contractors.
g. Ensure that EPA project officers and contractors develop
protocols with approved QA plans and procedures prior to
task initiation and adhere to them in all stages of re-
search.
h. Identify data quality problem areas and alert management
to them. Also, validate the soundness of the solution
to such problems.
3.2 QUALITY POLICIES
The HERL/RTP quality assurance program encompasses all funded
technical tasks, intramural and extramural, contract and grant. Each
research task protocol must contain a quality control plan delineating
the QC practices and procedures to be followed at each level of task
responsibility and each phase in the life of the project. Each
3-2
-------
research task protocol must contain a quality assurance plan for inde-
pendently assuring the effectiveness of that task's quality control
program.
3.2.1 Task QA Design
Responsibility for the design of a task QA plan rests with the
respective project officer. As the professional most intimately
familiar with the purpose(s) and procedures of the task, the project
officer is the logical choice to assume this responsibility. He must
choose specific QA activities appropriate to the data quality
requirements of the task and to the specific nature of the data
collection and data processing system. For instance, routine
monitoring operations require QA plans and activities significantly
different than those for measurement methods under development. Truly
unique measurement methods are used quite rarely; well-characterized
methods ("unit operations") are the norm, even for tasks that are
highly research-oriented. Thus, the task QA plan focuses on specific
unit operations and the data collection system in which they are
employed.
QA plans for tasks conducted under grant or contract are prepared
by the grantee or contractor and reviewed and approved by the EPA
project officer (with optional assistance from the QA Coordinator). QA
plans for in-house tasks are incorporated in the research protocol by
the responsible project officer (again, with optional assistance from
the QA Coordinator).
3.2.2 Task QA Review and Approval
Responsibility for the review and approval of task QA plans rests
with HERL/RTP functional management. As an integral part of research
planning, quality assurance plans (as they are implemented) provide the
means by which functional management may assess that suitable data
quality have been obtained in a cost-effective manner. Assistance from
3-3
-------
the QA organization is available to management for the evaluation of
task QA plans and of the effectiveness of their implementation.
3.2.3 Laboratory QA Program
The responsibility for the design and implementation of the
Laboratory-wide QA program rests with the QA organization headed by the
QA Coordinator.
The HERL/RTP Quality Assurance Coordinator, with assistance from
the Quality Assurance Committee, establishes and administers quality
assurance procedures for independently monitoring and assessing the
adequacy of task quality assurance programs. The QA procedures should
be applied uniformly throughout the duration of the project. However,
at any time during the task life, either the respective project officer
or the QA Coordinator, using accepted QA techniques, may assess the
project's ongoing QA program as necessary.
3.2.4 Scope of Quality Assurance Program
The quality assurance program for extramural grants or tasks (con-
tracts) provides for quality control procedures applied to the request
for proposal (RFP), the proposal, and extending through proposal evalu-
ation, work plan approval, project and quality control execution, and
final report preparation. It also provides for appropriate data
quality audits.
In the case of intramural tasks, quality control procedures begin
with the drafting of the protocol). In particular, consideration of
the hypothesis to be tested, data and data processing requirements,
data quality assurance plans and procedures, data analysis techniques,
and anticipated problem areas should be clearly addressed. Protocol
review and approval includes evaluation of QA plans.
Planning for technical tasks should include provision for an
appropriate QA program. This program will be comprised of both QC and
3-4
-------
QA activities. The following are major aspects of project data quality
that should be addressed and are individually discussed in Section 4 of
this document:
-- Experimental Design
-- Personnel
-- Facilities and Equipment
-- Recordkeeping
-- Supplies
-- Sample Collection
-- Sample Analysis
-- Internal Audits
-- Preventive Maintenance
-- Calibration
-- Documentation Control
-- Configuration Control
-- Data Validation
-- Feedback and Corrective Action
-- Data Processing and Analysis
-- Report Design
3.3 QUALITY ASSURANCE PROGRAM ORGANIZATION
In planning a QA program for a particular task, the project
officer will attempt to account for all variables that are known or
suspected to affect the data to be produced. Planning for such moni-
toring is not a simple task and performing it with the necessary care
is still more difficult. However, it is becoming increasingly neces-
sary to provide for such a QA program considering the number of reports
which appear indicating that reagent quality and identity are not what
the manufacturer claims them to be, instruments do not properly perform
the function for which they are intended, electronic circuits are
discovered to generate false signals due to mismatches, etc. Due to
these general, and some specific, data quality problems, the EPA is
currently developing comprehensive QA guidelines [7]. Federal standards
for nonclinical laboratories have been promulgated [8] and health
effects test standards have been proposed that will apply to testing
under the Toxic Substances Control Act [9a,b]. The American Public
Health Association published "Quality Assurance Practices in Health
Laboratories," [10] and guidelines relating to health and biological
3-5
-------
research have been published by EPA [1,2,11]. Current research
increasingly depends on sophisticated automated data collection
systems, whether an isolated laboratory is involved or an entire
monitoring system. The cost of this research is increasing at a
corresponding rate. Efficient, reliable operation under such
conditions requires systematically designed quality assurance plans for
research tasks.
In order to better support HERL/RTP project officers and manage-
ment in the rational design and execution of QA plans, the quality
assurance substructure is interwoven within the existing management
structure in HERL/RTP. The organization of this substructure, the
functional responsibility of QA personnel, and the lines of
communication for achievement of a cost-effective QA program are the
subjects of this section.
3.3.1 Organizational Structure for Quality Assurance
The HERL/RTP functional management structure is shown in Figure 1 .
As noted above, one purpose of the Quality Assurance program is to
encourage awareness and usage of quality assurance principles at all
levels of HERL functional and task management. The Quality Assurance
Coordinator, who reports on QA matters directly to the HERL/ RTP
Laboratory Director, is primarily responsible for the design and
implementation of the program. The Quality Assurance Committee,
chaired by the QA Coordinator, is responsible for evaluating the
effectiveness of the program throughout the laboratory and for
recommending viable improvements. The QA Committee members act as
liaison between the QA Committee and their respective Divisions or
Offices.
In the standard review and approval process of any project-
oriented document (e.g., RFP, proposal, work plan), the Quality
Assurance plan (including provisions for QC and QA activities) is
reviewed and approved along with the other technical or analytical
aspects of the work. The actual definition and incorporation of
3-6
-------
Z
iy«-
O
"p
u. BE
HEALTH EF
LABO
u
C
Z
\
j
V
h-
0
w*
§8
£
•>•
—
^^
TIC TOXICOIOOV
DIVISION
Z
o
RIMENTAL BIOLOQY
DIVISION
a.
X
i
(0
5
Q
C
0
S DIVISION
1
(0
S"
NEUROTOXICOLO
DIVISION
MENTAL TOXICOLOOY
DIVISION
s
tc
tu
TAOENESIS «. CELLULAR
OXICOLOGV BRANCH
§
|
3EVELOPMENTAL BIOLOGY
BRANCH
LyJ
Is
2*
1
RESEARCH
fcNCM
oS
O
O
R
BEHAVIORAL TOXI
BRANCH
r
NALVTICAL CHEMISTRY
•RANCH
«
I
BIOCHEMICAL ft MtCKOtUU.
GENETICS SECTION
~~I
H REPRODUCTIVE PHYSI
OLOQY SECTION
AGFMENT
WCH
i
H SERVICES
ANCH
M
i
i
NEUROPHYSIOL
•RANCH
|
1 CHEMICAL REPOSITORY
SECTION
L
MUTAGENESIS ft CYTO
GENETICS SECTION
—]~
i
li
is
s
i
rtlOlOGY
fcNCB
|S
i
£
II
X
1
II
I
i
s*
5i §
Ii 1 !e
z| ii 1"
i" M !
i i i
l!
•
w Z *•
s £ •- S i *" y*
•8 _,_ ^_ _,. 1*
i i i i *
CL
CC
_J
a:
OJ
$-
3
4->
in
c
(U
CD
as
c
(O
c
o
o
c
(U
S-
cn
3-7
-------
quality assurance procedures into individual tasks is the responsibil-
ity of each task's project officer. Beginning with the concept paper
or request for proposal, all documents that contain a description of
the technical or analytical aspects of the project must be accompanied
by an appropriate description of the quality assurance requirements and
how they will be met. The Quality Assurance Coordinator (or qualified
designee) is available to work with project officer and management to
design or refine the specific QA activities to meet laboratory QA
program requirements.
Figure 2 depicts the QA organization as described above. The
particular functions of each position are shown in the diagram and the
channels of communication are delineated in the following subsections.
3.3.2 Functional Responsibilities
The functional responsibility assignments for individuals and
organizational components are outlined in this section.
3.3.2.1 Program Coordinator—
Frequently, research quality assurance plans focus on technically
related activities, such as calibration, acceptance testing, audits,
and the like. The presumption is implicit that the data should be
collected and attention focused on characterizing and controlling their
quality. A thorough quality assurance program, however, includes
provisions for specifically evaluating the desirability of collecting
specific research data, apart from their quality.
In this context, the Program Coordinator is in an advantageous
position to evaluate research proposals and protocols. By having ready
access to interdisciplinary Decision Unit information, he can assess
the relevance of each proposal (and task) to broadly defined Labora-
tory-wide and Agency-wide goals and needs.
As the reporting focal point for general program areas, the Pro-
gram Coordinator is aware of correspondingly wide-ranging research data
needs in these program areas. In addition to identifying and instigat-
3-8
-------
HERL/RTP
DIRECTOR
QA
COMMITTEE
QA
CONTRACTOR
QA
COORDINATOR
I
PROGRAM
COORDINATOR
QA
REPRESENTATIVE
DIVISION
DIRECTOR
BRANCH
CHIEF
SECTION
CHIEF
Advisory channel
for Quality
Assurance
PROJECT
OFFICER
Line of approval
for technical and
Quality Assurance
procedures
Figure 2. HERL quality assurance organization,
3-9
-------
ing specific program-related research tasks, he evaluates related pro-
posals and their relationship to other tasks in his program area.
Thus, independent of the issue of the quality of research data, the
Program Coordinator is responsible for evaluating issues such as cost
and the need for obtaining one type of research data relative to the
need for other types of data.
Additionally, the Program Coordinator is responsible for summariz-
ing reports of work in his project areas for presentation to various
groups. In this capacity he can identify research progress and impend-
ing needs in specific areas. He may thus word his reports in such a
way as to encourage proposals for research tasks to fill these specific
needs, aligning the production of research data with the general
Laboratory and Agency needs.
3.3.2.2 Task Manager--
The HERL task manager (i.e., project officer) has the responsi-
bility of fulfilling the technical and administrative requirements of a
task or portion of a task. In order to fulfill this responsibility, he
must be able to knowledgeably and adequately assure and document the
quality of the task product; i.e., the research data and conclusions.
The project officer draws upon his professional training and expertise,
in collaboration with the HERL QA organization, to determine precisely
which QA techniques most appropriately apply to a particular task
quality assurance program.
In order to assure the technical aspects of research data quality,
the project officer must plan ahead in systematic detail. This
planning involves, among other things, anticipating events that might
threaten data quality (e.g., slowly degrading reagents), contingency
planning for unforeseeable failures and problems, and obtaining
objective, independent evaluation of task data quality as the task
progresses. These topics are sufficiently important to require their
own discussion in Section 4 of this document.
The documentation of quality assurance activities is necessary in
order to permit the communication and objective evaluation of task
3-10
-------
plans and results. HERL QA guidelines and activities are being devel-
oped in order to facilitate this requirement. In order to adequately
document QA plans and activities, the project officer will need to
become acquainted with technical and administrative aspects of HERL QA
policy, explicitly include data quality considerations in the various
task-related documents (e.g., RFP's, reports), and collaborate with the
HERL QA organization in applicable QA activities (e.g., collaborative
testing, split sampling—see Section 4.9)
3.3.2.3 Functional Manager—
With respect to the HERL QA program, functional managers are
administratively responsible for ensuring the quality of research data
that are produced under their direction. In this context, functional
managers support the QA programs of project officers under their
jurisdiction, as well as ensure that these QA programs are properly
planned and implemented.
Management support of QA programs—for individual tasks as well as
for Laboratory-wide effort—must be visible and active. Development
and support of data evaluation techniques appropriate to health re-
search data may be coordinated through the QA organization. Project
officers using similar research techniques may be informed of, and en-
couraged to participate in, interlaboratory and intralaboratory testing
programs. QA programs and techniques for many areas of health-oriented
research may be readily developed using currently available standards
and procedures. Functional managers should also actively support
development of standards and QA procedures in new measurement areas.
Coordination of effort by functional management in the development and
application of effective QA techniques is essential to the development
of the laboratory-wide QA program.
Management can ensure implementation of appropriate QA planning
and techniques through the regular review and approval process. This
can occur by requiring that all approved task documentation demonstrate
concern for data quality by including a description of how data quality
is evaluated and provided for in the ongoing task. Peer review of QA
3-11
-------
techniques, through the QA committee, may also be used effectively in
the evaluation of task planning and execution. Section 4 of this docu-
ment is designed to address critical aspects of data quality during the
various evolutionary stages of a task. In this way, functional managers
can use the contents of Section 4 to evaluate proposals, plans, pro-
gress reports, and final reports for their assessment of data quality.
These activities of functional managers serve to demonstrate to
scientific and technical personnel the actual degree of commitment of
HERL management to the quality assurance program.
3.3.2.4 Quality Assurance Coordinator—
The Quality Assurance Coordinator is responsible for the develop-
ment, evaluation, and documentation of QA policy and procedures
appropriate to the HERL mission. This includes evaluation of the cost
effectiveness of QA programs and plans, and recommendations for their
improvement. He also interacts with others involved in quality
assurance programs through his professional contacts. As advisor to
the Laboratory Director, he periodically reports on the progress and
deficiencies of the Laboratory QA program and specific needs (e.g.,
method development and problem areas). He also recommends to the
Laboratory Director specific courses of action for strengthening the
HERL quality assurance program.
As chairman of the Quality Assurance Committee, the QA Coordinator
initiates efforts to develop Laboratory-wide QA guidelines and pro-
cedures. He is the coordinator of methods development efforts for new
QA procedures for specific HERL research techniques, and assimilates
data provided by the Committee regarding evaluation of the QA program
(e.g., weaknesses or the needs for new audit techniques). He is also
responsible for the development of special audit programs for
Laboratory-wide measurement techniques.
As quality assurance consultant, he is available to consult and
recommend to the HERL professional staff (project officers, investi-
gators, etc.) appropriate and necessary quality assurance methods and
plans for ensuring the quality of the research data produced.
3-12
-------
3.3.2.5 Quality Assurance Representative--
Each Division Director designates a QA representative, and an
alternate, to serve as a member of the HERL QA Committee. The
representative serves as a Division QA coordinator in that he consults
on matters of quality assurance, serves as a source of information on
research quality assurance matters, and is available to aid in
implementing the QA program within his Division. As liaison with his
Division, the representative is the prime source of information on QA
matters.
As a committee member, the representative becomes increasingly
aware of the requirements (and defects) of HERL QA policies and proce-
dures with the aid of the QA Coordinator. He recommends and reviews
proposals for improvements in QA policies and procedures. He also
reports and evaluates (potential) data quality problem areas, as
necessary.
3.3.2.6 Quality Assurance Committee--
The Quality Assurance Committee serves as an advisory committee to
the Laboratory Director, with the objective of furthering the
continuity and applicability of the Quality Assurance Program
throughout HERL/RTP. Specifically, the committee's functions include
assisting in the evaluation and refinement of data quality objectives
of the QA program so that they meet the Laboratory needs with minimum
disruption of existing workloads and procedures, reviewing
recommendations presented to the committee, and assessing the
effectiveness of the QA guidelines.
3-13
-------
P.O. GUIDELINES
-------
Page
4.0 GUIDELINES FOR PROJECT OFFICERS 4-1
4.1 General 4-2
4.1.1 Control Charts as Applied to Research Projects.. 4-4
4.2 Experimental Design 4-4
4.2.1 Statistical Experimental Design 4-5
4.2.1.1 Objectives and Hypotheses to be Tested. 4-9
4.2.1.2 The Experimental Design 4-9
4.2.1.3 Data Processing 4-11
4.2.1.4 Data Analysis 4-12
4.2.2 Quality Control Considerations 4-13
4.2.3 Data Collection and Analysis 4-13
4.2.4 Biological Systems 4-15
4.3 Personnel 4-17
4.4 Facilities and Equipment 4-19
4.5 Recordkeeping 4-22
4.6 Suppl i es 4-24
4.7 Sample Collection 4-27
4.8 Sample Analysis 4-28
4.9 Internal Audits 4-29
4.10 Preventive Maintenance 4-31
4.11 Cal i brat i on 4-33
4.11.1 Introduction 4-33
4.11.2 A Calibration Model 4-35
4.11.2.1 The Input Phase 4-35
4.11.2.2 The Operation Phase 4-38
4.11.2.3 The Output Phase 4-39
4.12 Documentation Control 4-41
4.13 Configuration Control 4-42
4.14 Data Validation 4-43
4.15 Feedback and Corrective Action 4-44
4.16 Data Processing and Analysis 4-45
4.17 Report Design 4-47
5.0 DATA QUALITY ASSURANCE FOR RESEARCH PROJECTS 5-1
5.1 Quantitative Estimates of Data Quality 5-1
5.2 Qualitative Estimates of Data Quality 5-2
4-0
-------
SECTION 4
GUIDELINES FOR PROJECT OFFICERS
The purpose of this section is to present guidelines for the
development of quality assurance (QA) plans by project officers as they
oversee the development and implementation of plans for individual
intramural and extramural tasks at HERL/RTP. Specific purposes of this
section are to:
a. Support the project officer in systematic planning for
comprehensive quality assurance appropriate to all
areas of his research.
b. Collect, in one section, general data quality checks.
c. Document data quality checks currently in use at
HERL/RTP, for use by HERL/RTP professional and
technical staff and other interested parties.
d. provide a logical framework within which additional
research relating to HERL/RTP data quality may be
programmed.
The responsibility for the development and implementation of an
appropriate QA plan for a research task rests with the respective
project officer. Section 3.3.2.2 contains a general description of a
project officer's responsibility for QA planning. This section
includes specific discussion of quality control and quality assurance
principles that may be used by all HERL/RTP project officers. The
discussion of QA planning is organized to nominally parallel the
sequence of events in the life of a research task. It is designed to
be comprehensive and to complement the professional training and
experience of HERL/RTP investigators.
Basic to the discussion which follows is the assumption that the
research-trained project officer regularly performs various QA
4-1
-------
functions in his area of major expertise. These guidelines are intend-
ed to describe principles that complement and document these functions
for every aspect of a research task that may be performed under the
auspices of HERL/RTP; they do not provide solutions in detail. Re-
search quality control (that is, activities to be performed by task
operating personnel) is addressed in the following broad areas:
a. General approach to quality control in research.
b. Planning—experimental design, personnel, facilities
and equipment, recordkeeping, supplies.
c. Experimental — sample collection, sample analysis.
d. Data quality activities—internal audits, preventive
maintenance, calibration, documentation control, con-
figuration control, data validation, feedback and
corrective actions.
e. Results—data processing and analysis, report design.
Recommendations for quality assurance activities (i.e., independent of
task operating personnel) are then discussed.
4.1 GENERAL
As performed at HERL/RTP, health-related research is frequently
state-of-the-art, in concept as well as in technique. As such, it is
not obviously susceptible to the normally available QA techniques.
However, virtually every research or support task within HERL/RTP con-
sists of two principal areas, whether the task is laboratory research,
a monitoring program, or research support.
a. Data Collection and Processing—routine measurements
performed by skilled technical personnel using
well-character!zed techniques (e.g., pH measure-
ments).
4-2
-------
b. Data Analysis and Reporting--nonroutine data analysis
performed by the HERL investigator using physical
models, statistical techniques, and other tools in a
nonroutine, creative manner.
Each of these aspects of research is susceptible to the use of QA
techniques by the project officer. Data collection techniques general-
ly have adequately characterized quality control procedures associated
with them that are quantitative in nature. The project officer uses
professional judgment in determining the frequency, number, and specif-
ic reference materials to be used. Quality assurance of data analysis
is less straightforward. Peer interaction, from the protocol stage to
the report stage of a task, plays an important role. It is, therefore,
important that effective mechanisms for peer review be used.
The production of research data is strongly affected by the "weak
link" phenomenon. Thus, if experiment design, equipment maintenance,
and data analysis are excellent and quality of the sample analysis is
poor, the overall task data quality is lowered. Similarly, no amount
of competent technical skills, data analysis, etc., can compensate for
poor experimental design.
In addition, there are aspects of a research task which affect
data quality, but which are not easily quantitated or categorized. For
example, technician fatigue and morale should be considered.
Similarly, the tension between the need for quick response to
unexpected developments and the need for strict accountability to
funding agencies relates to planning for quality data. With these
considerations in mind, these guidelines are designed to support
project officers as they oversee the progress of their tasks from
concept to final report.
As a research project progresses, it frequently becomes apparent
that additional "trivial" data (instrument settings, exact identity of
the components of a buffer solution, etc.), which are not usually
recorded, are useful for data interpretation. As a general rule, then,
it is cost-effective to record well-organized, complete data from which
an experiment can be properly reconstructed. Lab notebook (or station
4-3
-------
logbook) records of numerical as well as anecdotal data will frequently
prove useful when experiment reconstruction becomes necessary.
The remainder of this section addresses the various elements of a
research task. It should be realized that different research projects
will involve varied applications of these QA elements. The project
officer, however, should be cautious in deleting considerations of any
element and should be certain that it will in no way affect the quality
of the data that are produced by the task. If there is any uncertainty
regarding the design of a task QA plan, the project officer may request
the aid of his QA representative or the QA coordinator. It should be
remembered that, when properly used, quality assurance planning can be
a very effective insurance policy against data of unacceptably poor
quality.
4.1.1 Control Charts as Applied to Research Projects
The theory and use of control charts as a tool for assuring arid
demonstrating data quality are described in Appendix H of reference 3.
Use of this technique applies most often to routine, repetitious labor-
atory operations. Any repetitious operation can be documented on a
control chart. Consider, for example, a task to investigate the effect
of 6 months of exposure to a 2 parts per million (ppm) sulfur dioxide
(502) atmosphere on the respiration rates of white mice. Since the
project may never have been done before and may never be repeated, it
is not repetitious in itself and, therefore, cannot be subjected to
control chart techniques. However, since the SC^ atmosphere is to
remain constant at a concentration of 2 ppm for 6 months, repeated de-
terminations of that concentration are repetitious and can be subjected
to control chart techniques.
4.2 EXPERIMENTAL DESIGN
Adequate planning prior to the startup of a task is by far the
most cost-effective program for task quality assurance. This planning
4-4
-------
should include a discussion of the experimental design including man-
power, facilities, supplies, and equipment logistics; and detailed
plans for data collection and analysis as well as statistical
experimental design per se. The protocol that results from this type
of planning serves at least three purposes:
a. It provides a planning focal point for obtaining
answers to the basic issues of:
what is to be accomplished?
how is it to be accomplished?
how can one show that the stated purpose has been
accomplished?
b. It documents for all interested parties that responsi-
ble planning has occurred.
c. It provides criteria for making logical decisions when
such decision points are reached in the later stages of
the task life.
Typical contents of a task protocol (Figure 3), minimum contents
of task protocol, as proposed by EPA as part of the Good Laboratory
Practice Standards (GLP's) for Health Effects, relative to the Toxic
Substances Control Act, [9] (Figure 4), may be used as guidelines in
the design of task-specific protocols. Additionally, the GLP's are
summarized in Figure 5.
During initial phases of research planning and during protocol
development, the project officer should solicit advice from the various
HERL support functions that will be involved. Specifically, the
statistical design of the experiment, the data collection and analysis,
and the animal care requirements should be planned in detail by the
time the research protocol is drafted. (The ongoing collaboration of
each of these functions should also be programmed in order to
successfully cope with the various unexpected difficulties that
generally occur in research.) Each of these three areas is discussed
below.
4.2.1 Statistical Experimental Design
In any HERL task that involves the gathering and analysis of data,
it is important to seek the aid of a competent statistician. The
4-5
-------
PREPARATION OF DETAILED TECHNICAL RESEARCH PLAN
1. Introduction: State the overall objective of the study and sum-
marize briefly the approach to be taken to meet this objective.
Normally a project will be divided into subtasks. List these
subtasks and proceed to describe each under the following headings:
Hypothesis, Proposed Means of Testing Hypothesis, Experimental
Design and Statistical Methodology, and Quality Control Plans and
Procedures.
2. Hypothesis: State clearly the hypothesis to be tested for the
subtask.Include a concise discussion of the facts and/or obser-
vations upon which this hypothesis is based and conflicting
hypotheses.
3. Proposed Means of Testing the Hypothesis: Describe clearly the
method or methods by which the hypothesis will be tested. Describe
each experiment to be performed in moderate detail. Make clear the
dependence of one experiment in the sequence upon another. De-
scribe the variables that are to be controlled in order to carry
out the test.
4. Experimental Design and Statistical Methodology: Describe, in
moderate detail, the statistical basis for the collection of data
and/or the testing schedule. Determine (estimate) differences in
results between test and control measurements that would be
accepted as significant; refer to previous work whenever possible
to substantiate decisions regarding these differences. Describe
measurement design, numbers of measurements, numbers of exposures
(i.e., animals to be tested), level of exposure, time of exposure,
measurement conditions, etc. which would permit identification of
significant differences between test and control measurements in a
reasonable period of time and/or in a cost-effective manner.
5. Quality Control Plans and Procedures: Describe the quality control
(QC) for measurements that may introduce significant variability or
are critical to the success of the task. The project officer must
evaluate task requirements and resources to decide specific QC
activities and their scheduling. QC activities to be considered
may include:
(a) Maintaining and testing for test subject quality
(cells, animals, etc.).
(b) Calibration and maintenance of instrumentation.
(c) Personnel (adequate training and/or experience).
(d) Facilities.
(e) Sample collection.
(f) Recordkeeping.
(g) Data handling and validation.
(h) Feedback and corrective action.
(i) Report design.
QC activities may also be described in other parts of the plan and
should be identified as such.
6. Other Activities Required to Successfully Complete This Task:
Other major resources that will be required to successfully com-
plete the study should be described. These might include exposure
measurements, animal care, consultation with regard to statistical
treatment of data, and testing the agreement of various models with
the data collected.
Figure 3. Example of major topics addressed in a research task
protocol.
4-6
-------
(A) A descriptive title and statement of the purpose of
the study.
(B) Identification of the test and control substance by
name, chemical abstract (CAS) number or code
number.
(C) The name and address of the sponsor including the
sponsor s project manager. The name and address of
the testing facility at which the study is being
conducted.
(D) The proposed starting and completion dates.
(E) Justification for selection of the test system.
(F) Where applicable, the number, body weight, range,
sex, source of supply, species, strain, substrain,
and age of the test system.
(G) The procdure for identification of the test system.
(H) A description of the study design, including the
methods for control of bias.
(I) A description and/or identification of the diet
used in the study as well as solvents, emulsifers
and/or other materials used to solubilize or
suspend the test or control substances oefore
mixinr with the carrier. The description must in-
clude specification for acceptable levels of con-
taminants that are reasonabley expected to be pre-
sent in the dietary materials and are known to be
capable of interfering with the purpose or conduct
of the study if present at levels greater than
established by the specifications.
(J) The route of administration and the reason for its
choice.
(K) Each dosage level, expressed in milligrams per
kilogram of body weight or other appropriate units.
of the test or control substance to be administered
and the method and frequency of administration.
(L) Method by which the degree of absorption of the
test and control substances by the test system will
be determined if necessary to achieve the object-
ives of the study.
(M) The type and frequency of tests, analyses, and
measurements to be made.
(N) The records to be maintained.
(0) The date of approval of the protocol by the sponsor
and the signature of the study director.
(P) A statement of the proposed statistical methods to
be used.
Figure 4. Proposed protocol contents, EPA [9].
4--7
-------
SUMMARY OF EPA's PROPOSED
GOOD LABORATORY PRACTICES FOR HEALTH EFFECTS
(FR Wed. May 9, 1979, p. 27369, ff)
a) The proposed GLP's apply to studies relating to health and safety evaluations conducted
under Section 4 of the Toxic Substances Control Act, whether conducted by the sponsor,
or under contract or grant. Fourteen terms are defined in this section.
b) Test and control substances must be characterized as to their strength, purity,
composition and stability before the initiation of a study. Their containers must
be labeled by name, chemical abstract number or code number, batch number, (expiration
date) and storage conditions requirements. Handling procedures must be used which ensure
proper identification, and minimize contamination, deterioration or damage. Mixtures
must be suitably analysed to characterize their uniformity, concentration, and stability:
expiration date is that of the earliest expiring component.
c) An adequate number of personnel having adequate and documented education, training,
and/or experience must be available to the study. Their personal habits, health and
clothing must be appropriate for their assigned duties. The designated study
director ensures that all provisions of the GLP's are fulfilled for the study. The
quality assurance unit independently ensures management that the facilities,
equipment, personnel, methods, practices, records and controls are in conformance with
the GLP's, in each phase of the study, at no more than 3 month intervals.
d) Facilities must be of suitable size, construction and location to facilitate proper
conduct of the study. For animal studies, this means proper separation, isolation and^
quarantine of animals. Separate areas are required for: biohazardous substances; for
diagnosis, treatment and control of known or suspected laboratory animal diseases; for
sanitary disposal; for feed, bedding, supplies and equipment; for handling of test and
control substances, and their mixing; for routine procedures; for administrative and
personnel use; for secure archival of raw data and specimens.
e) Equipment must be suitably designed and located for operation, inspection, cleaning,
maintanence and calibration according to written procedures; written records are kept to
document these operations.
f) Testing facility operation must be by written standard operating procedures (SOP) for
(as a minimum): animal room preparation; animal care; test and control substance
handling; test system observations; lab tests; handling of moribund/dead animals;
necropsy; specimen collection and identification; histopathology; data handling, storage
and retreival; equipment maintanence and calibration; transfer, placement and
identification of animals. All deviations must be authorized by the study director, and
documented in the raw data. Each lab must have immediately available suitable lab
manuals and SOP's, both active and historical. Reagents and solutions must be
labeled to indicate identity, concentration, storage requirements and expiration date.
SOP's for animal care include housing, feeding, handling, care, receiving quarantine,
health parameters, identification. In addition, periodic feed and water analysis must
be documented as part of the raw data; cages and racks must be cleaned at appropriate
intervals. Bedding, cleaning materials and pest controls must be documented as
noninterferring in the study.
g) Minimum protocol specifications are given (as in the HERL QA Guidelines document).
The conduct of the study is detailed in terms of the protocol, specimen identity and
records and data recording.
h) -i) Reserved.
j) Minimum contents of the final report are outlined (as in the HERL QA Guidelines j
document). Archival of all raw data, protocols, specimens and final reports is
detailed: indexed, orderly and secure storage is required for at least 10 years.
k) Inspection of the testing facility must be permitted to an employee of EPA or FDA at
reasonable times and manner: for records and specimens, not including QA records.
Figure 5. Summary of EPA's proposed
GLP's for health effects
4-8
-------
statistician should be consulted not only after the data have been
gathered but during the planning phase of the study as well. No
analysis plan, however ingenious, can compensate for a bad experimental
design. Subsequently, as the statistician is regularly involved in the
daily execution of the plans, his timely advice for cost-effective
midcourse changes will be a valuable asset to the maintenance of task
data qua!ity.
In general, the statistician's support throughout the task will be
most helpful as the project officer formulates, examines, and carries
out the following phases of the task:
a. The objectives and hypotheses to be tested.
b. The experimental design (i.e., the design of a testing
program to meet the objectives).
c. The data processing plans.
d. The data analysis plan.
These four phases and the statistician's role in them are discussed
below.
4.2.1.1 Objectives and Hypotheses to be Tested--
Determining the objectives and the hypotheses to be tested is
obviously the first step that should be taken in designing any task.
Precise written formulation of the questions to be answered enables one
to state the hypotheses to be tested in precise terms and thus to plan
a task more effectively. The aim should be to make the statement lucid
and specific, avoiding vagueness or excessive ambition. It is
advisable to classify objectives as major and minor. This
classification is particularly helpful in assigning priorities to
objectives when the task involves cooperation among people of different
interests.
4.2.1.2 The Experimental Design (The Design of a Testing Program to
Meet the Objectives)--
The testing program design should produce a clear definition of
all the variables to be considered, the size of the testing program,
4-9
-------
the experimental units (e.g., animal models, cell cultures, humans) and
exactly what data are to be collected. In designing the testing pro-
gram, the following questions should be answered:
1. Are all the relevant factors (e.g., temperature and
subject age) being considered?
2. Are the effects of the relevant variables adequately
distinguishable from the effects of other variables
(e.g., would a factorial design be more appropriate)?
One can consider an experiment as intended to determine
the effects of one or more variables (factors) on mea-
sures of experimental outcome. From substantive con-
siderations, the project officer determines the fac-
tors, and the levels of each, that should be varied in
his experimental program. In experiments involving two
or more factors, the "effect" of a specified level of a
particular factor may depend on the levels of other
factors in the experiment (the factors may "interact").
The "main effect" of a factor is determined by compari-
sons among the effects of various levels of the factor.
In designing multifactor experiments, the project offi-
cer should carefully consider what effects—main effect
and interaction effects—are of interest to him. The
experimental plan should be such that it will result in
all the data necessary to estimate the main effects and
interactions of interest at the end of the experiment.
3. Is the plan as free from bias as possible?
4. Does the plan use a historical measure of precision
(experimental error) and if so is this precision suf-
ficient to meet the objectives of the tests?
5. Is the scope of the testing plan consistent with the
objectives given in Section 4.2.1.1?
6. Is the testing plan cost-effective (would a more
limited test plan provide equivalent information at a
lower cost)?
7. Are the data collection plans appropriate to the test
objectives (are sample frequencies appropriate; should
additional, or fewer, variables be monitored)?
8. Are available resources adequate for collecting the
quality and quantity of data required?
4-10
-------
9. Is the test plan logistically sound? (Is adequate
time, space, manpower, etc., available to properly
perform the quality checks necessary to ensure the
specified data quality?)
Answering questions 1 through 9 allows the formulation of a scien-
tifically sound, statistically suitable testing program and alternative
testing designs. It is important to note here that the analysis of
data (Section 4.1.1.4 below) can be made much easier if this phase
(Section 4.2.1.2) is completed properly.
Finally, a complete description of the analysis scheme to be used
in the task should be included in an experimental design. This scheme
should include details of all pertinent parts of the task including
sampling and data reduction as well as analysis. Advance development
of such a scheme will aid in making decisions concerning other aspects
of the task su-h as equipment or personnel qualification needs.
4.2.1.3 Data Processing—
The data processing phase of a task is concerned with how the data
are handled once they have been collected, and involves examining the
following kinds of questions about the data gathered according to the
testing program formulated in the experimental design phase.
1. How are the data validated, i.e., what procedures are
used to determine what data to include in the analysis?
This question may involve developing a specific statis-
tical evaluation of the data in a task and should usu-
ally be performed by the person or persons responsible
for the analysis and interpretation of the data. Also,
it should be clearly understood that experiments must
not be repeated just because the results "don't look
good." Section 4.2.3 contains further discussion of
this point.
2. When are the data to be processed so that they can be
analyzed, i.e., during the testing program or only at
the end of data collection? This question is especial-
ly important if the test program extends over a long
period of time, since preliminary analysis of the data
4-11
-------
may indicate that the testing program should be
altered for the remaining tests.
3. If data from different instruments are to be compared,
what is the comparability of outputs (e.g., one in-
strument may give continuous readings while another
may only give output at specific intervals)'
4. What (manual) data handling is required in order to
convert "as recorded" raw data into the form in which
they will be analyzed (e.g., copying from these forms
and reading the cards into a computerized data base) ?
Also, what is a realistic estimate of the net error
rate for this process (5 percent is a realistic
value)?
4.2.1.4 Data Analysis--
Initially, this phase involves reviewing any data analysis that
has been proposed or has already been performed on the project, and
giving an outline of the analysis to be performed if no outline is
available.
An outline of the data analysis should be prepared before the test
design is completed or testing begins. If this outline is not pre-
pared, it is quite likely that some measurements that should be re-
corded for proper analysis will be overlooked or will not be recorded
in the correct manner. For example, an outline of the analysis may
reveal that it is essential to record the level of an uncontrollable
variable so that adjustments for the variable may be made when the data
are analyzed. Conversely, unnecessary data may be identified and elim-
inated during this phase, thus conserving resources. In addition, if
the project involves a large number of different types of measurements,
it is important that an overall analysis plan be devised that insures
that the objectives given in Section 4.2.1.1 are met in the most effi-
cient manner. For example, a multivariate analysis may be preferable
to several univariate analyses.
Once the data for the project have been gathered, the data analy-
sis should be carried out with the close collaboration of a statisti-
cian. This is particularly important when the testing program has
changed somewhat since the beginning of the project (which is frequent-
4-12
-------
ly the case) and/or there is a large amount of missing data. In addi-
tion, the statistician and project officer should work closely together
in presenting the results of the data analysis. In this regard the
project officer should ensure that the presentation is understandable
to nonstatisticians. The statistician should make sure that the
results are presented such that the reader is aware of the functional
relationship linking the data and the tables or graphs. The statisti-
cian should also ensure that statistical results are interpreted cor-
rectly based on the nature of the design and the statistical tests.
Since any scientific study falls short of realism, useful conclusions
usually require generalizations that tend to lie outside the realm of
strict statistical justification. Thus, the reader of the technical
report should be informed of the amount of statistical and physical
justification supporting each conclusion.
4.2.2 Quality Control Considerations
Very early in the process of experimental design, consideration
should be given to the methods that will be used to evaluate, control,
and assure the quality of the experimental data. It cannot be
overemphasized that this is one of the initial steps in the design
process. EPA's commitment to QA is reflected in the directive "Envi-
ronmental Protection Agency (EPA) Quality Assurance Policy Statement,"
a memo distributed May 30, 1979, in the publication of reference 9 and
elsewhere. Far too often, quality assurance procedures are nonexistent
simply because they were considered late in the project after all the
money and time had been allocated. A design that can be expected to
produce high quality data will have begun to incorporate effective
quality control and quality assurance procedures at about the same time
that the objectives were defined.
4.2.3 Data Collection and Analysis
Once the production of raw data has begun, the manner in which
4-13
-------
they are collected and analyzed becomes important. Data validation
(see Section 4.14) must be addressed prior to this time. Manually
collected data are frequently monitored by the person recording the
data. However, computerized data acquisition systems do not have the
potential for this treatment. They are known to pick up false voltage
transients, and failure of one component of a system may seriously bias
the data of major interest in an experiment. In a system of reasonable
complexity, a variety of warnings may be identified by careful analysis
of the relationships and patterns of values of the incoming data.
The use of control charts (see Section 4.1.1), or the concept,
should be considered for use in specific data validation procedures.
Used properly, individual out-of-range points and data trends will be
readily apparent and informed response by the project officer will be
possible.
The use of computerized data acquisition systems is increasing.
This frequently permits a statistically acceptable, cost-effective
extension of the control chart concept for real-time data validation.
There are several advantages to using such a system. It accepts truly
raw data to produce intermediate and final results in tabular or
graphical form, thus minimizing human error. Similarly, the capability
of rapidly and automatically comparing experimental data against recent
values of similar data can serve as a real-time check on data
validity.
Data analysis involves the matching of the experimental system
with a model system and evaluating the differences. Since real-world
data are never sampled exactly, one source of discrepancy between the
data and the model is due to measurement error. Only rarely will the
model exactly correspond to the test system, thus adding another com-
ponent of data-model disagreements. The experiment should be designed
so that data analysis will highlight the actual model-test system
differences rather than mask the discrepancy as "error". Appropriate
statistical design of the experiment is essential at this point. Care
must also be taken that apparently irrelevant physical aspects of the
test system do not produce data which lead to erroneous interpretations
4-14
-------
ly the case) and/or there is a large amount of missing data. In addi-
tion, the statistician and project officer should work closely together
in presenting the results of the data analysis. In this regard the
project officer should ensure that the presentation is understandable
to nonstatisticians. The statistician should make sure that the
results are presented such that the reader is aware of the functional
relationship linking the data and the tables or graphs. The statisti-
cian should also ensure that statistical results are interpreted cor-
rectly based on the nature of the design and the statistical tests.
Since any scientific study falls short of realism, useful conclusions
usually require generalizations that tend to lie outside the realm of
strict statistical justification. Thus, the reader of the technical
report should be informed of the amount of statistical and physical
justification supporting each conclusion.
4.2.2 Quality Control Considerations
Very early in the process of experimental design, consideration
should be given to the methods that will be used to evaluate, control,
and assure the quality of the experimental data. It cannot be
overemphasized that this is one of the initial steps in the design
process. EPA's commitment to QA is reflected in the directive "Envi-
ronmental Protection Agency (EPA) Quality Assurance Policy Statement,"
a memo distributed May 30, 1979, in the publication of reference 9 and
elsewhere. Far too often, quality assurance procedures are nonexistent
simply because they were considered late in the project after all the
money and time had been allocated. A design that can be expected to
produce high quality data will have begun to incorporate effective
quality control and quality assurance procedures at about the same time
that the objectives were defined.
4.2.3 Data Collection and Analysis
Once the production of raw data has begun, the manner in which
4-13
-------
they are collected and analyzed becomes important. Data validation
(see Section 4.14) must be addressed prior to this time. Manually
collected data are frequently monitored by the person recording the
data. However, computerized data acquisition systems do not have the
potential for this treatment. They are known to pick up false voltage
transients, and failure of one component of a system may seriously bias
the data of major interest in an experiment. In a system of reasonable
complexity, a variety of warnings may be identified by careful analysis
of the relationships and patterns of values of the incoming data.
The use of control charts (see Section 4.1.1), or the concept,
should be considered for use in specific data validation procedures.
Used properly, individual out-of-range points and data trends will be
readily apparent and informed response by the project officer will be
possible.
The use of computerized data acquisition systems is increasing.
This frequently permits a statistically acceptable, cost-effective
extension of the control chart concept for real-time data validation.
There are several advantages to using such a system. It accepts truly
raw data to produce intermediate and final results in tabular or
graphical form, thus minimizing human error. Similarly, the capability
of rapidly and automatically comparing experimental data against recent
values of similar data can serve as a real-time check on data
validity.
Data analysis involves the matching of the experimental system
with a model system and evaluating the differences. Since real-world
data are never sampled exactly, one source of discrepancy between the
data and the model is due to measurement error. Only rarely will the
model exactly correspond to the test system, thus adding another com-
ponent of data-model disagreements. The experiment should be designed
so that data analysis will highlight the actual model-test system
differences rather than mask the discrepancy as "error". Appropriate
statistical design of the experiment is essential at this point. Care
must also be taken that apparently irrelevant physical aspects of the
test system do not produce data which lead to erroneous interpretations
4-14
-------
(e.g., diurnal fluctuations in serum enzyme levels are frequently
larger than the response to the experimental stimuli on many biological
systems). In order to maximize the quality of data from a testing
program, the project officer should routinely consult with researchers
who have specialized in related areas.
4.2.4 Biological Systems
The majority of the research and support associated with the
HERL/RTP directly involves biological systems. While this is common
knowledge among the HERL/RTP staff, it touches upon an important, and
sometimes troublesome, difference between the experimental situation at
the HERL/RTP and the situation at laboratories which perform research
on nonbiological systems. The implication of this difference is that,
while the experimental variables analyzed and modeled in other
laboratories present a complex challenge, the experimental variables
associated with biological systems studied at the HERL/RTP are orders
of magnitude more complex. The "simple" systems under study in most
physical science research laboratories involve the effects of a few to
a few dozen experimental variables, most of which are monitored, if not
controlled. Biological systems, even the most simple, involve the
interactions among several dozen recognizable molecular species. And
if research trends continue, several hundred distinctly recognizable
molecular interactions will soon be characterized in the most simple
monocellular systems.
The challenge of such a large array of experimental variables can
presently best be met by permitting variation of only a selected few of
these variables. For this reason the project officer must exercise his
best professional abilities to recognize and fix all but the experimen-
tal variables. This is the purpose of care in selecting, maintaining,
dosing, and analyzing biological subjects, whether they be cell cul-
tures, animals, or humans.
Human subjects come from diverse and largely unknown backgrounds.
This variability among human subjects can be minimized (but not elimi-
nated) by careful pretest screening and questioning. The results thus
4-15
-------
obtained are directly applicable to human health problems. On the
other hand, cell culture lines that have been quite thoroughly charac-
terized for several generations are available for research. But the
results of cell culture studies seldom, if ever, apply without
interpretation to aspects of human health. Intermediate between these
two extremes are animal subjects, some lines of which have been quite
well characterized for several generations and which correlate closely
with certain aspects of the human system. It is thus not surprising
that a large proportion of health effects research is performed using
animal subjects. Proper maintenance requirements of animals, however,
are relatively more costly (in dollars and labor) than for cell
cultures. Since careful characterization of animal subjects is no less
important than for cell culture models, the balance of this subsection
is devoted to a very brief discussion of animal care.
Comprehensive HERL/RTP guidelines for animal care are being
developed, and general guidelines are presently available [12]. A
brief discussion of the basic aspects of animal care is included here,
due to its importance to overall task data quality. The basic concept,
common to all scientific research, is to attempt to control all but the
experimental variables. Early, intensive, and consistent consultation
with qualified professionals from the HERL Laboratory Animal Support
will maximize the quality of data that are generated using laboratory
animals.
Animal selection should be based on awareness of the species'
genetically determined immunities, as well as the specific dose-re-
sponse relationship to be investigated. The research protocol should
clearly state the basis for selection of a particular species, the
anticipated interferences with the experiment design, and any
preliminary testing required for adequate characterization of the
system unknowns (e.g., interfering antibodies).
Acceptance testing, or prescreening and surveillance, should be
sufficiently comprehensive to insure that only suitable animals are
included as experimental subjects and controls. While the added ex-
pense of such testing may limit the quantity of animals used, the
4-16
-------
increase in data quality will generally more than compensate for this
loss.
Personnel assigned to animal care and dosing should have suffici-
ent technical competency to provide reliable routine care to experimen-
tal animals. In addition, their training and responsibilities should
permit their active participation in the research (e.g., to note
unusual behavior or health of any of the test animals or to note
abnormalities in the dosing formulation).
The dosing and vehicle matrix should be chosen carefully and
should be well characterized with respect to the specific experimental
animals. If the particular choice has not been well characterized, it
should be changed, or detailed studies performed to characterize it
prior to experimental work. Choice of the control group and the speci-
fic regimen should be made on the basis of acceptable data quality,
excepting only those aspects of the control that are reliably docu-
mented (i.e., complete equivalency of the experimental and control
group regimen should be routine, excepting only the test substance).
In short, the animal subjects should be treated as any nonbiologi-
cal supply; i.e., they should be thoroughly characterized.
4.3 PERSONNEL
Task operational personnel are intimately involved in one of the
most crucial aspects of the particular research task: the generation
and recording of the experimental cause-effect relationships that
result in raw task data. The upper limit of the quality of the results
is set during this phase of research task. Statistical treatment may
be used to estimate precision and accuracy; creative thinking may
rationalize discrepancies. But the upper limit of data quality for the
task cannot be improved beyond what is produced by task personnel dur-
ing task data collection. Two aspects of the personnel relationship to
acceptable data quality are (a) technical qualifications and (b) the
intangibles.
4-17
-------
The usual approach to technical qualifications is that personnel
have the education, training, and on-the-job experience to perform the
assigned function. Similarly, training in good laboratory practice
(generally and job-oriented) is recommended [8,9]. Such stipulations
are certainly reasonable, and should be the documented practice of the
project officer. Attempts should be made to ensure that all task
personnel keep abreast of contemporary developments in their fields of
expertise. Adequate theoretical briefing should be provided to bench
technicians so that they will be capable of recognizing and recording
unusual and unanticipated events.
In complex tasks it may be helpful to discuss personnel roles rel-
ative to the total task. By doing this, operating personnel can obtain
a more complete perspective of their respective tasks, their interac-
tion with others, and an overview of the experiment design. Periodic
meetings during task implementation may help in information exchange,
procedure standardization, and improved quality control of the project.
Another aspect relating personnel and data quality is far less
tangible, but none-the-less important in obtaining high quality data.
It refers to the general mental state of task personnel. Appropriate
work loads prevent excessive mental and physical fatigue. Useless
effort is avoided with optimum laboratory and equipment configurations.
Good interpersonal relationships support full productivity. Proper
management techniques (neither too restrictive nor to permissive)
result in maximum productivity and data quality. In addition, the
complex issue of motivation [13, Section 18] is an important factor in
total personnel performance and data quality. The project officer is
in the position to recognize and address such aspects relating to task
personnel which create a healthy atmosphere for research and have a di-
rect effect on overall task quality.
Bench-level personnel should also be intimately involved in the
feedback and corrective action loop (Section 4.15). This involvement
should begin early in the life of a task, preferably with a briefing on
overall task goals, methods, and their role in assuring the necessary
data quality.
4-18
-------
4.4 FACILITIES AND EQUIPMENT
After the data required by a task have been identified, the
requirements for facilities and equipment may be defined. The defini-
tion of these areas, like all areas of experimental design, should
contain provisions for the assurance of data quality. The facilities
and equipment selected for an investigation should be documented to be
capable of producing acceptable quality data at minimum risk to task
personnel (and subjects).
With HERL/RTP, the primary purpose of research conducted is to
better model the responses of the human biological system. Frequently,
nonhuman biological systems used for experimental purposes are selected
with the intention of extrapolating results to characterize the human
system. Due to the intentional similarity of the two systems, a sig-
nificant risk of cross-contamination and infection is a constant threat
to experimental results as well as personnel health. While it may be
impractical or undesirable for the HERL/RTP investigator to strictly
follow the various published animal facility guidelines, deviations
should be made only at the advice and with the approval of the profes-
sional staff of HERL Laboratory Animal Staff (see Section 8).
Similarly, many nonbiological systems are used for health-related
research, yet with potential risk to operating personnel. Insult to
operating personnel by noxious fumes, electrical shock, etc., should be
anticipated and eliminated as conducive to the long-range, cost-effec-
tive maintenance of data quality.
The experimental facility should be examine carefully prior to the
commencement of experimentation. If it is a new facility, it will be
most cost-effective to properly design the facility for its intended
purposes. Modification of an existing facility is the usual case. In
either case, resource (i.e., dollars, manpower, time, etc.) limitations
always exist which directly and indirectly affect data quality. The
various options, and their effects on data quality, should be frankly
evaluated and discussed with the management. When the task involves a
new experimental design in a facility already used by the investiga-
4-19
-------
tors, de novo evaluation should be the norm. For a variety of reasons,
this is difficult and may not be carried out. However, if a complete
evaluation of the requirements of the experimental design, as well as
of potential error sources, is conducted at the outset of a research
project, future invalidation of much or all of the experimental work
may be prevented. (For example, reference 14 reports that, under
certain conditions, light from fluorescent fixtures has caused
mutations in the hamster cell chromosomes. If substantiated, these
findings may bring into question an entire body of research. Rigorous
attention to such seemingly trivial detail can minimize this type of
problem.)
The need for dependability of support services should be evaluated
early in considerations concerning facilities and equipment. Numerous
measurement processes exist in which loss of routine services (such as
gases, electricity, heat, steam, or water) causes significant deteri-
oration in data accumulation or quality. In such cases it is necessary
to provide redundant support services.
During the consideration of required conditions, it is necessary
to examine detailed requirements for valid sampling. In air sampling
systems the materials of which sampling lines, valves, and manifolds
are constructed often play an important role in the condition of the
sample when it reaches an analyzer. The geometry of the system also
affects the validity of certain samples; the presence of long runs of
tubing or bends and constrictions will change the character of certain
types of samples (see Section 4.7). To assure high quality data, it is
important to confirm that the monitoring system delivers to the
analyzer a sample representative of the atmosphere being
characterized.
In addition to the technical suitability of the facility for
execution of the task, it is in the project officer's interest to eval-
uate and configure the facility with due care for the physical and
mental comfort of the technical staff who will be using the facility.
The discussion in Section 4.3 (Personnel) extends here to the human
engineering of hoods (for poisonous and noxious gases), sinks, walk-
ways, counters, etc. While there will be necessary trade-offs in
4-20
-------
facility configuration, the influence on traffic patterns, the environ-
mental aspects such as temperature and lighting and other fatigue- and
confusion-producing aspects should be evaluated and related to the
effect on data quality.
Depending on the type of research involved, facility security
should be specifically considered. This will range, for a wide variety
of reasons, from areas available for common use by even nontask person-
nel to stringently restricted areas for safety purposes. Relating to
data quality, the facility configuration should be carefully controlled
(see Section 4.13, also reference 3, Section 1.4.19). As is frequently
the case, even routine instrument maintenance activities can have a
profound effect on data quality; for example, a new design of a
replacement emission source for a spectrophotometer may affect data in
a manner that only becomes apparent during later analysis. If possi-
ble, authority to approve facility configuration changes should be
limited to one professional staff member who is qualified to document
and evaluate such changes (i.e., the project officer).
As with the facility used for the task, the equipment should be
evaluated for its applicability to the task research. The relationship
of the measurement methods and the variables to be monitored should be
well characterized during the initial task activities, if not before
they have begun. Similarly, the subtleties of design and performance
of different manufacturers' equipment should be thoroughly evaluated,
preferably with the aid of a professional who has both theoretical and
practical understanding of the specific instrument operation. In this
regard, it is not uncommon to learn that unadvertised features of an
instrument will permit acquisition of significantly higher quality
and/or quantity data. As discussed below in relation to supplies,
acceptance testing for new equipment should be performed on an item-by-
item basis and documented for comparison with future testing. This
testing program should be designed in such a way that operation of the
instrument at its extreme limits (i.e., worst case), as well as routine
settings, will be thoroughly characterized before it is made available
for routine use.
4-21
-------
In relation to equipment, the desirability of full- or part-time
operator and/or maintenance support should be considered. Frequently,
sophisticated instrumentation performs poorly or not at all when many
occasional users have access to it. On the other hand, minor but fre-
quent maintenance often keeps an instrument operating at peak perfor-
mance. In such cases, the cost of a dedicated operator is justified.
4.5 RECORDKEEPING
Provision for a complete, permanent, easily accessible record of
the raw experimental data should be made prior to, during, and follow-
ing completion of task experimental work. This should include a writ-
ten record (in ink, in a bound, page-numbered, durable notebook) of ex-
plicit identification of equipment, reagents, and supplies used, animal
identification and test data, as well as a record of equipment and
modifications and other seemingly inconsequential information which
will permit more accurate analysis at later dates. A cross-referencing
system must be used if the data are to be easily accessible following
their initial use. Such a system may be of various levels of
complexity, depending on the amount of data collected and their
potential applications. Reference 8, Section J, lists rules for
nonclinical laboratory reports and records, and their generation,
storage, retrieval, and retention on a long-term basis. When data are
logged by computers, it is important that adequate provisions be made
for redundant and physically separate long-term storage of such
records.
All technical personnel should be provided with a personal note-
book in which they record all data, from weights and temperatures to
calculations and general observations. Efforts should be made to
encourage the entry of not only specific data (weights, absorbances,
volumes, etc.), but also of anecdotal data (atmospheric or meteoro-
logical conditions, status of instruments, etc.), in ink. Erroneous or
invalidated data should be indicated in such a way that the entry is
flagged but remains legible. Drawing a single line through the entry
4-22
-------
is an acceptable indication, and this flag should be initialed.
Whether the recorded data are valid or (flagged as) invalid, they may
become extremely valuable in subsequent evaluation of a completed
experiment or in initial planning of a related one. In general, the
more data accumulated, properly recorded, and organized during an
experiment, the more useful that experiment will be in satisfying
overall task objectives.
At times it may be convenient to provide station, laboratory, or
task data notebooks in addition to individual notebooks or project data
notebooks. Such records will generally take the same form and adhere
to the same recommendations as personal notebooks. The difference is
that these books act as central records for the entire station, labora-
tory, or task, while the personal books act as records of individuals'
contributions to tasks. Other means of recordkeeping include automated
means such as strip charts, computer tapes, etc. Although these
records are ncc of the same form as notebook records, the same recom-
mendations apply.
Instrument (or equipment) log books contain all data relating to a
particular piece of equipment. This log maintains a convenient record
of instrument calibrations, maintenance, failures, and idiosyncrasies
in one location. Reference to such a record provides an on-the-spot
history of an instrument or piece of equipment that is often useful in
determining trends, spare parts inventories, etc. Although equipment
calibrations and maintenance records should be kept in such a log, a
specific format or printed forms should be used for accumulation of
such data. Such a format, when completed, will minimize the possibil-
ity of omission of important steps or data.
In addition to the issues discussed above, the project officer's
investment in the design of suitable data logging forms for repetitive-
ly measured parameters will be repaid in the form of assurance of com-
plete data, high productivity of technical personnel, and later, ease
of reading the raw data. Computerized data acquisition systems have
many advantages. However, they must be closely monitored for false or
erroneous signals that may not be easily detectable.
4-23
-------
High quality recordkeeping serves at least two useful functions:
(a) it makes possible the detailed reanalysis of a set of data at a
future time when the model has changed significantly, thus increasing
the cost-effectiveness of the data; and (b) it may be used in support
of the experimental conclusions if various aspects of the study are
called into question. This latter point goes to the heart of scien-
tific research: objectively, it is often possible to interpret data in
more than one way and the raw data should be available for evaluation
by qualified professionals; subjectively, when recordkeeping habits are
sloppy, suspicion is quickly aroused that all other aspects of the
research are of similarly poor quality.
4.6 SUPPLIES
As noted in Section 4.2.4 (Biological Systems), a basic premise of
scientific research is that all but specified variables are controlled
or held constant. However, reports regularly appear in the technical
literature of impure and/or mislabeled supplies; e.g., after the end
of expirements in which they were used, supposedly "gerrn-free" animal
subjects are found to have been infected, thus invalidating the entire
experiment. There are numerous examples available describing chemi-
als, ordered to be 99.9 percent pure, which were found to have a 95
percent or, perhaps, even a 65 percent assay during acceptance
screening [15].
An acceptance testing program for all incoming expendables/
supplies—be they chemicals, biologicals, etc. — should be applied prior
to and (judiciously) during use. Resources are always limited., nence
the design of a suitable testing program is important. This is
facilitated by learning as much of the processing history of the
supplies as possible, by anticipating possible experimental
interferences using the existing model, and by conferring with other
users of the same consumable.
When a commodity is received as a supply for a task, it should be
examined at once for acceptability. This acceptance screening will
4-24
-------
assure that supplies not meeting task specifications are not integrated
into the task's supply stream. Acceptance screening for the HERL/RTP
operations will deal with one of four classes of commodities: equip-
ment, instrumentation, laboratory animals, or chemicals. The results
of a successful test should (a) confirm the substance fully corresponds
to the label specifications, and (b) confirm that known or suspected
interferents are absent. When the acceptance testing is lengthy and/or
costly, adequate amounts of a common lot should be purchased to permit
completion of the tests. Sufficient excess to permit unanticipated
testing, plus a specified amount for storage, should also be included.
Equipment and instrument screening should include all the testing
required to demonstrate that the equipment or instrument performs
according to the specifications under which it was ordered. Such
testing, while protecting data quality, will also alleviate the
problems, costs, and delays that will occur when it is shown that new
equipment, already brought on line, is not performing to specifica-
tions.
The screening of laboratory animals presents a different problem
altogether. Even though they may be regarded as a commodity, they are
living and therefore susceptible to all the random variations applica-
ble to living beings. Because of this complexity, quality assurance
relating to laboratory animal care is the subject of Section 8.
The screening of chemical or reagent commodities should contain
two elements—certification of assay and examination for impurities
[16]. Such screening is usually performed on a batch basis. Certifi-
cation of assay assures that chemicals arriving for use in a task are
of the desired concentration or strength. In many analyses, chemicals
having assays of considerably less than 100 percent may be utilized.
However, the user of these chemicals must be aware of the decreased
assay in order to make appropriate modifications in the computations.
An examination for impurities should be designed to assure that a
chemical or reagent contains no substance(s) that may interfere with
any analysis in which it is to be utilized. [A recent example of such
4-25
-------
an interference was discovered in EPA Reference Metnod 6 [17] for sui-
fur dioxide in stationary sources. This method utilized 2-propanol to
separate interfering sulfur trioxide and sulfuric acid mist from sulfur
dioxide. Certain lots of the alcohol have been found to contain oxi-
dizing substances that prematurely remove the sulfur dioxide from the
analysis screen. Thus, it is necessary to screen 2-propano'i for
oxidants prior to its use in Method 6.] Following successful
completion of the acceptance test, an expiration date should be
permanently marked on each container and it should be stored on a
first-in-first-out basis. The shelf-life of many substances is known
but in some cases it must be estimated. In most cases, simple tests
exist that can, to a first approximation, rapidly document the strength
and purity of a substance (or animal) immediately prior to use.
Reliable estimates of strength as a function of time should be used to
determine a conservative useable lifetime of solutions^ mixtures,
emulsions, etc.
In this latter instance, a well-designed central stockroom track-
ing system will facilitate rapid reference to the identity of other
users of a substance. This will be useful for informal sharing of in-
formation of interest as well as for rapidly identifying and locating
the users when a specific problem (e.g., purity or contamination) has
been detected with the particular substance.
When chemicals must be stored for a length of time, certain con-
ditions should be observed to protect the integrity of the material.
These conditions will vary according to the specific chemical or piece
of equipment and are best determined from the specifications of in-
structions for the material in question. However, parameters such as
temperature, humidity, light, and shelf-life are usually of impor-
tance.
Since many of the substances involved in HERL/RTP are antagonistic
toward humans, personnel should be protected from exposure to them.
Certain substances are known to be in this category; however, the pro-
ject officer should carefully evaluate whether additional substances
may possibly degrade personnel health, and hence, data quality.
4-26
-------
Special emphasis should be placed on the need to characterize all
incoming cylinder gases containing pollutants in specified concentra-
tions. The characterization should also include an identification of
cylinder contents with reference to both pollutant(s) and matrix. It is
well known that problems concerning the identity of cylinder contents
and accuracy of the specified concentrations are commonplace. Even the
best known and most reliable gas suppliers occasionally supply faulty
materials. In addition, after the cylinder contents have been initi-
ally verified, experience indicates that over a period of time the
contents degrade. Therefore, regular recertification must be performed
to characterize changes in concentration, formation of new species, or
loss of original species to prevent them from degrading task data
quality. Because of these considerations, all HERL/RTP gas cylinders
should be subjected to a rigorous program of initial, and regularly
recurring, certification of contents and concentrations. Presently,
the EPA Environmental Research Center is considering the establishment
of a standards laboratory to serve this function.
4.7 SAMPLE COLLECTION
In sampling, one generates a new system, because as soon as a
portion of material is removed from the whole, its history becomes
different from the whole.* Primary consideration must be given to
keeping the sample collection system as nearly representative of its
condition when sampled as possible, regarding all the parameters under
investigation. The processes involved in obtaining, holding,
preserving, transporting, and resampling can potentially introduce
significant direct and indirect changes in the material destined for
analysis. Quality control measures must be specifically designed to
quantitate and characterize any sample degradation or interaction with
its particular container and environment.
*A corollary to this is that the existing system is also altered by
sampling activities.
4-27
-------
Samples must be positively identifiable by those taking the sample
and by others who are involved in subsequent analytical or handling
steps. (This does not preclude the use of blind samples, spiked sam-
ples, or other audit methods to assure the quality of the test system
in part or in whole.)
The personnel-related requirements for the technical and support
aspects of the sample collection program vary in type and number. All
operating personnel need to know exactly what is required of them, how
it is to be done, and when it is to be done. Written instructions
answering these questions for every phase of their involvement should
be developed and provided as appropriate. Periodic "practice work" may
be necessary in order to maintain the desired level of data quality.
Each person should have a clear understanding of who will answer his
questions on test protocol.
4.8 SAMPLE ANALYSIS
Sample analysis—whether it be spectrophotometer reading or viable
colony count—involves a repeated sequence of similar, documented oper-
ations by technical personnel and/or automated instrumentation. For
this reason, sample analysis is susceptible to the use of qua'iity con-
trol techniques. Adequate, correct, and available operating procedures
used by suitably trained and motivated technical personnel are the norm
in a laboratory research context. Quality control activities on sample
analysis range from nearly reflex use of a standard polymer film to
calibrate an infrared spectrophotometer to the more visible use of
split-sample aliquots, standard samples, and other techniques generally
associated with calibration.
These latter activities require conscious and visible support ana
planning by the project officer if they are to succeed. Sample blanks
should be analyzed on a regular basis. Samples spiked with known
amounts of the analyte serve as a check on analytical bias. Split-sam-
ple aliquots can be analyzed by different analysts at different times
using a different set of reagents as another measure of data quality.
4-28
-------
Quality control measurements requiring highly developed subjective
evaluations (e.g., pathological evaluation of tissue) may require
side-by-side or round-robin analysis in order to establish the quality
of the data. The project officer should choose the specific quality
control activities appropriate to a given task in such a way as to
emphasize the need for highest quality data commensurate with existing
limitations.
4.9 INTERNAL AUDITS
During the life of a task it is desirable to regularly evaluate
the ability of the total data system to produce data of the specified
quality. In this way, timely corrective action (see Section 4.15) is
possible. Internal audits, conducted by the operating group or organi-
zation, are used to obtain data for this evaluation.
The Environmental Protection Agency defines two types of audits
which perform those functions [3,4]. A quantitative measure of the
quality of the data produced is usually evaluated by means of a
performance audit. The ability of a system to produce data of the
specified quality is evaluated by means of a system audit; this type of
audit is qualitative in nature.
The performance audit should be performed (a) by qualified task
personnel not routinely involved in the measurement process, (b) in a
manner that evaluates the data system in its totality. For example, an
automated air monitoring system should be audited by introducing an
appropriate known concentration gas into the sampling system inlet and
recording the corresponding output from the data acquisition system.
The same principles should be applied to laboratory instrument systems.
Frequently the performance audit can only be designed to evaluate some
discrete subsets of the total data system, such as sampling, analysis,
and/or data reduction. Again, the audit should be designed and inter-
preted to evaluate each subsystem only to the extent possible within
the context of the existing limitations. In either case, the audit
values are compared with those generated by the data system(s), and
4-29
-------
conclusions are drawn as to the quality of the data being generated by
the total system.
Tools available for use in performance audits generally fall into
one of four categories:
a. Reference materials are available from several sources,
most notably, the National Bureau of Standards [18,19],
i.e., NBS-SRM's. These may be included for analysis in
various types of measurement systems at relatively low
cost with little interference to the normal laboratory
routine and with the highest possible degree of confi-
dence.
b. Reference devices may be obtained [e.g., the reference
flow (ReF) device for high volume samplers] for which
the critical parameters are known to the auditor but
not the analyst. These are somewhat more disruptive of
laboratory operations, and there is no possibility of
anonymity of the sample; however, the final result is
still a measure of the performance of the total analy-
tical system, including the operator.
c. Cooperative analysis, such as round-robin analysis, is
useful for estimating the precision (not accuracy un-
less the analyte is a reference material) of measure-
ment among several different operators and/or labora-
tories.
d. Side-by-side analysis, or collaborative analysis, may
be used if important variables are not controllable in
the sample.
These basic types of audit techniques may be applied to almost any
measurement system. Both EPA and NBS are expanding their services to
allow calibration of many audit substances and devices for which no
NBS-SRM's previously were available. Frequently, however, cooperative
or side-by-side analysis will be necessary for internal audits of HERL
analyses due to the lack of suitable reference materials or devices and
the complex nature of the evaluation. In these cases, the project
officer (or project leader for extramural tasks) will need to relate
his responsibility to monitor and quantitatively document the task data
quality with the various costs involved in this type of audit.
4-30
-------
System audits are familiar to health-related researchers in the
form of site visits by qualified professionals. Professional and tech-
nical evaluation, resulting from observation and discussion, is made of
the capability of a data system (including instruments, personnel,
organization, etc.) to produce the specified quality data. Questions
such as:
-- Are there written sampling and analysis procedures and
are they being used?
— Are there written calibration procedures and are they
used as frequently as necessary?
-- Is a preventive maintenance schedule defined and
full owed?
— Are data reduction, validation, and reporting techniques
completely documented and routinely utilized?
are answered on the basis of such observations.
It is extremely important to emphasize that the purpose of an
audit is to constructively evaluate measurement process data quality
(not personnel) and to identify areas where improvements can be made.
If this intent is followed by project officers and managers and made
clear from the beginning, personnel will be more likely to cooperate in
audit and corrective action cycles.
In either situation, the program and rationale for internal audits
should be designed on the basis of individual components of the specif-
ic measurement process and clearly planned for and budgeted into the
task plans. By the use of internal audits, the project officer will be
able to objectively evaluate data quality as his task progresses.
4.10 PREVENTIVE MAINTENANCE
In order to ensure long-term data quality in a cost-effective
manner, a rational preventive maintenance (PM) program must be fol-
lowed. This assumes importance roughly in proportion to the amount of
4-31
-------
instrumental data that are recorded. Reference 3 contains a good dis-
cussion of preventive maintenance, especially as related to routine
measurements (Air Quality Monitoring). In particular, preventive
maintenance will increase the completeness of data from continuous
monitoring systems, which is an important measure of quality for such
systems.
In a laboratory research environment, PM has a less visible
benefit; the effect on minimizing and controlling equipment downtime is
none-the-less real. Preventive maintenance can be budgeted and
scheduled based on failure analysis data available to (or developed by)
the equipment manufacturer. Extended laboratory use of specific items
can be scheduled with higher reliability, and with shorter, less catas-
trophic interruptions than if maintenance only occurs following equip-
ment failure.
The laboratory equipment PM program should include: scheduling,
performance, and recordkeeping. Scheduling of PM should be developed
based on the effect of equipment failure on data quality, any relevant
site-specific effects, and equipment failure analysis (or estimates).
This schedule should be available to the person or group responsible
for performing the maintenance, as well as the person or group using
the particular item of equipment. In this way, use of the equipment
may be scheduled appropriately.
Preventive maintenance should be performed by qualified techni-
cians, using accepted, documented procedures. The specific service
should be programmed based on the considerations noted in the preceding
paragraph and should be known to both the user and maintenance groups.
A predefined set of data should be obtained both before and after the
maintenance activities to permit equipment performance evaluation.
Calibration (see Section 4.11) should also be performed following all
maintenance activities.
Documentation of maintenance—scheduled or not—is essential to
monitoring and documenting data quality. A bound notebook (see Section
4.5) should be kept with each instrument as a record of its maintenance
history. A detailed description of adjustments made and parts replaced
4-32
-------
should be recorded in it. If the notebook is the multicopy type, one
of the copies can be kept by the maintenance group for analysis. This
analysis may include such considerations as mean time between failure
(MTBF) for specific components, MTBF analysis for systems (individual
and laboratory-wide), and indication of an onsite spare parts inventory
appropriate to cost-effectively support minimum equipment downtime.
Where possible, check-off forms should be used to ensure and document
thorough maintenance activities.
4.11 CALIBRATION
4.11.1 Introduction
Calibration is the process of establishing the relationship of a
measurement system output to a known input. In essence, calibration is
the reproducible point to which all sample measurements can be corre-
lated. This process is a key element of any scientific measurement
program, since without an adequate calibration system, the validity of
the data from the measurement program will be questionable.
A sound calibration system includes provisions for documentation
of calibration procedure, frequency, conditions, and standards reflect-
ing the calibration history of a particular measurement system.
Calibration should follow well-documented, step-by-step procedures
to perform the needed referencing of a given system to a standard(s).
Whether a specific standard is utilized for referencing, or visual
analysis by trained personnel (e.g., pathologist reading a microscope
slide), a clearly written, concise procedure will minimize the bias
that may be introduced into a system due to individual differences.
Calibration procedures for many systems can be obtained from NBS or
ASTM. Other procedures may have to be developed in-house and must
undergo extensive evaluation to determine, as nearly as possible, their
accuracy, precision, replicability, repeatability, and reproducibility
[3].
4-33
-------
To assure and document that the calibration is being maintained
for a measurement system, it is essential that calibration frequency be
established on the basis of historically available data. As with
preventive maintenance, the calibration frequency should be established
on the basis of documented experience with specific equipment. Thus,
initially, calibration frequency should be sufficiently high such that
minimal drift is observed between successive calibrations. Only as
this is done is it possible to rationally deduce a cost-effective
frequency that minimizes exposure without jeopardizing data quality.
The calibration schedule should involve simple daily checks as well as
full-scale, multipoint calibrations. Provisions for action to be taken
if an unforeseen circumstance occurs should be specified. Adherence to
an exercise of this nature can minimize the generation of erroneous
and/or indefensible data.
Environmental conditions are another type of reference point that
must be dealt with during measurement systems calibration (and opera-
tion). If the system is sensitive to environmental conditions (temper-
ature, pressure, light, humidity, etc.), the calibration will not be
valid unless the documented conditions are maintained as required.
The quality of the calibration standards is the most important
aspect of any calibration program; for without high quality standards,
the accuracy of the calibration cannot be demonstrated. Standards
should be of the highest possible quality and should be traceably
referenced to a primary standard such as a National Bureau of Standards
Standard Reference Material (NBS-SRM). Various organizations [18,19,
20] list reference materials applicable to health-related research for
use by HERL/RTP project officers.
Documentation of each calibration, and the full history of all
calibrations performed on a measurement system must be recorded. This
enables personnel to perform a systematic review of the data quality
from a measurement system at a later date.
4-34
-------
4.11.2 A Calibration Model
In considering a general calibration scheme that can be applied to
many different measurement processes, it is convenient to examine it as
a model composed of three distinct phases. The input phase relates to
preparation for calibration. It includes information on what standards
and equipment are to be employed and the quality of the standards to be
used. The operations phase relates to the steps and procedures by
which calibration is to be accomplished. It includes such
considerations as the detailed calibration procedure and operation of
support equipment. Finally, the output phase describes the
relationship(s) developed by the calibration. It includes generation
of calibration curves or factors and/or derivation of confidence limits
or precision and accuracy statements.
4.11.2.1 The Input Phase--
One of the most important decisions made in determining a calibra-
tion scheme is that choice of the reference material used in the
calibration process. Standards include everything from permeation
devices and pressurized cylinders to orchard leaves and bovine liver
(Table 1). Most are already prepared; however, many are generated
in situ.
Since the standard or reference material is the authority against
which the calibration relationship (input vs. output) is developed, it
must be of the highest available quality and be characterized to the
maximum extent possible. In the United States, the National Bureau of
Standards holds the position of final authority in the preparation of
many reference materials. Their Standard Reference Material series
contains the best standards of their type. Therefore, use of NBS-SRM's
completely fulfills the requirements of high quality and full charac-
terization necessary in a standard. However, since SRM's are handmade
and individually characterized by lot, they are expensive and often in
short supply. Therefore, it is generally desirable to employ secondary
standards as the actual calibration standards. One SRM is thus main-
tained as a "calibration standard for the calibration standards."
4-35
-------
M
X
• LO
00 O
VI
+j 01
0) Ol
X >- _l
cn i/i .c xi
I— i.
<£ S
< XI
O
in
Ol (SI
> Ol
IO i—
01 xi
—1 Ol
01
o z
§OI
•1—
1- Q.
+ +
ex a.
« •»
<_>
Ny* {J
+J
C
•r~
i. IO
OJ i"*
i «
—1 in
IO
Ol -Q
C 1
•i- XI
0 Ol
CO «J
+
cn
*
CL. *"*
J->
C
S
*r-
XI
Ol
.C 1/7
VI
«t t-
i— r— f— Ol
0 O 0 •— -r-
°
O
a.
+ +
_C -Q CT* ^E
l/J Q- H
**
~l~ l/^ T~ Q_ CJ
in
s.
IO
c
+J
CO
Ol
c
Ol
•1—
cn
31
r«.
lO
l-
in
XI
c
IO IO
0 O
O %
C X
CO £
O 0
u u
S- S-
r* f—
(J U
c c
o o
o
o
Ol •—
10 U C
X -r- IO
o a -c
•^ 1 4->
O OJ Ol
i •*
a. <-<
0 0
U 0
10 >o
u o
c. c
o o
El Ol
0 C
0 S- 0)
<*- 0 •—
0 r— >,
S- -C .C
O O 4->
t— -r- Ol
_E S-
f i ^—
S
O
^
F*
(J
C
o
1
IO
i.
+j Ol
Ol XI
si
c o
S£
s. u
IO
o
Ol Ol
c c
si r
3 3
XI XI
s! s!
XI XI
Ol Ol
IM IM
ai ai
01 ai
s- s.
cn
' ' T*
0
IO US IO
XI XI XI
S- S- t-
Ol Ol Ol
•r- T- •!-
c
2:
c
r-j
X! TM
l_3 4J
s_
' IO
ai .a 3
CO Q. Cy
O
S-
_j_>
^
i3
o
^Z
fO
^r
o
S-
(O
O!
t/)
cu
S-
^__
"TO
+->
c~
E
C
O
a.
Q.
0
*.
r—
Ho.!.
o. a. o.
0
o
in
OJ
uo
o
•a-
01
o
co
*j- i— -a-
i. o
o. s:
0
o
M
o
o
•t
0
LD 1^
«
O "
M »
CO »-H
0.
o.
o
o
o
i— »
«
o
0
o
o
1— t
n
0
in
*
o
(—1
§_
Q.
O
O
o
o
o
Lf)
0
OJ
o
o
t— 1
M
o
m
E
O.
o.
o
in
«*
O
CO
*
o
«— t
"o
z:
to
CO
f.
ro
Lf)
CM
C\J
in
*—
«.
uo
4J
cn
a
CO
OJ
o
OJ
cn cn i
a. 3. i
Ln
OJ
e
cn
OJ
s_
o
(/)
oo
I
00
CO
IB
•l~"
(O
>>
S- S-
CM -i- •!-
z ^
o: u-
JO
Q.
m
OJ
Ol
O
a
s-
•M
C
01
Ol
o
10
O OJ OJ
•«- o o
+J CO Z
-------
Whenever a secondary standard is employed in a calibration, it is
necessary that a pathway (i.e., traceability) showing the relationship
of the working standard to a standard of higher quality be established
and maintained. Certain EPA regulations now specify traceability of
calibration standards to NBS-SRM's [21,22] and it is likely that this
requirement will appear in future regulations. It is, therefore,
recommended that all HERL/RTP calibration procedures specify that cali-
bration standards be traceable to NBS standards insofar as possible.
Some of the currently available standard reference materials
provided by NBS are listed in Table 1. A listing of SRM's, complete
with prices, is published in reference 19.
Unfortunately, NBS does not supply SRM's for every measurement
process; in fact, there are no SRM's available at this time for many
common measurement processes routinely used at HERL/RTP. In these
cases, the investigator must use the "best available" calibration stan-
dard or, in some cases, devise a standard. These standards must also
meet the requirements of high quality and careful characterization
applicable to SRM's. High quality standards may be obtained by using
raw materials of known high quality for construction or preparation.
Careful characterization of such standards involves rigorous character-
ization to establish the "true value" of the reference material. Such
testing includes repeated analysis of the standard, analysis by more
than one analyst or technique, round-robin interlaboratory analyses,
etc., to establish the true value within known limits of precision.
One further caution should be noted with respect to the use and
handling of calibration standards. They must be used and handled under
their specified conditions. It is a matter of record that many cali-
bration data contain errors induced by incorrect handling of standards.
The following list delineates some of the more common techniques
ignored in the use of such standards:
a. Permeation devices must be used and stored under
carefully specified environmental conditions of hu-
midity [23], temperature [24], and protected from
possible environmental contaminants [23].
4-37
-------
b. Certain gases in pressurized cylinders require special
procedures for regulator installation to prevent
cylinder and regulator contamination with atmospheric
oxygen or moisture (e.g., nitric oxide in nitrogen must
not be contaminated with atmospheric oxygen) [25].
c. Electronic standards frequently require periods of
several hours for stabilization of output (e.g., ozone
generators).
d. Most solid standards (e.g., powdered chemicals such as
potassium iodate or sodium sulfate and powdered mixtures
such as orchard leaves or coal) require conditioning at
a specified humidity prior to weighing.
These examples illustrate some potential mistreatment of otherwise
valid calibration standards. The point to be emphasized is that users
of standards should be intimately familiar with specified use condi-
tions for each standard. It is imperative that this point be recog-
nized if high quality data are to be obtained from the measurement
process.
4.11.2.2 The Operation Phase--
During the operations phase of a calibration, the measurement
process is calibrated or characterized against a standard. A written
calibration procedure describing the individual steps by which the
calibration is accomplished is required. Calibration procedures may be
prepared in-house by qualified personnel, may be derived from
instrument or process manufacturer's instructions, or may be found in
sources such as ASTM Standards [26]. These procedures should be
subjected to document control as outlined in Section 4.12 to assure
that the latest revisions are being utilized.
Personnel actually performing a calibration should be qualified to
do so. They should be intimately familiar with the measurement process
as well as the calibration procedure. Their qualifications to cali-
brate should be demonstrated to a person of higher authority who is
4-38
-------
also qualified to perform the calibration and who has sign-off respon-
sibility for the ability of the person to perform the calibration.
An aspect of the calibration operation that is often overlooked
is the calibration of support equipment. The use of a high quailty
calibration standard has already been discussed. However, most cali-
bration procedures utilize equipment and/or reagents in addition to the
standard(s). All such reagents and equipment should have been sub-
jected to calibration prior to use in the procedure. The quality of
the calibration is directly related to the quality of the data derived
using any such equipment.
4.11.2.3 The Output Phase--
After the calibration of a measurement process, the derived
relationship between the known input and the measurement process output
must be depicted in a useable manner. The relationship may take the
form of a calibration curve, a correction table, or a calibration
factor or factors. Whatever method is chosen, the input/output
relationship should be accurately expressed in a useable manner.
One of the most popular methods of expressing the calibration rela-
tionship is the calibration curve. In this method, the known values
label the abscissa (x-axis) and the process outputs label the ordinate
(y-axis). The calibration input/output pairs are plotted and an appro-
priate curve is used to connect the points. A minimum of five such
points is necessary to adequately describe the curve that should cover
the range of interest in the measurement process. A typical instrument
calibration curve is shown in Figure 6.
Calibration curves are often linear; however, some take a non-
linear form. For this reason, the data should actually be plotted. If
the curve is clearly linear, the technique of linear regression becomes
useful. This technique is described in most statistics books and is
valuable because it:
a. Allows more precise interpretation of curve data.
4-39
-------
O)
t/>
c
o
a.
^=
c: o
O)
E &«
t/i
c
100
80
60
40
20
0
0.1
0.2
0.3
0.4 0.5
Concentration of Standard
(ppm)
Figure 6 . Typical calibration curve,
4-40
-------
b. Allows all personnel to derive the same line from the
data.
c. Provides a mathematical equation for calculating input
or output data from the curve.
Regression analyses, including calculation of the standard error
of the estimate, allow one to make statements concerning the precision
of the calibration. However, nonlinear data will yield linear parame-
ters when subjected to a linear regression analysis. Regression analy-
sis should, therefore, include the calculation and evaluation of the
correlation coefficient. Deviation of this coefficient from unity
(1.0000) by any appreciable extent indicates that the data should be
catofully reexamined to establish their linearity.
Experience has shown that almost all calibration curves contain
nonlinear portions. Often, the nonlinear portions lie outside the
range of interest and linear techniques are reliable as long as the
working range is limited to those portions that are demonstrably
1inear.
With the advent of readily available microprocessors and program-
mable pocket calculators, it is becoming increasingly easier to perform
regression analyses on nonlinear systems. This type of analysis pro-
vides the same advantages as linear techniques and should be seriously
considered as a viable calibration data examination technique.
4.12 DOCUMENTATION CONTROL
Operating procedures for task measurement activities should be
clearly documented and available to task operating personnel. A formal
procedure for ensuring that procedural and system changes are incorpo-
rated into existing documentation and that those changes result in cor-
responding changes in the habits of operating personnel is essential.
Reference 3 clearly describes a comprehensive, practical document
control indexing format appropriate for use within EPA laboratories.
4-41
-------
It has the advantages that only current versions of documentation are
generally retained, and updating may occur at any time. An example of
the information placed in the upper right-hand corner of each page is
as follows:
Section No. 2.12
Revision No. 0
Date September 27, 1977
Page 1 of 5
(Note that the date given is the date of the revision.)
4.13 CONFIGURATION CONTROL
An adequate program of equipment/hardware configuration control
(e.g., equipment location, environment, component alteration and/or
replacement) will readily permit tracking all changes that are made to
a data-producing system that may affect data quality. This applies to
individual instruments as well as to entire data acquisition systems.
Authorization for configuration changes should be limited to one
person, preferably the project officer, to assure that all changes to
the facility (e.g., replacement of an electronics board in an analyzer)
are properly documented and communicated. This documentation and com-
munication is essential in understanding and explaining shifts in data
patterns following such changes. It will also ensure that all involved
personnel are notified of the changes, and that the proper procedures
required by the changes (e.g., recalibration of analyzers) are initi-
ated. Finally, it will provide a convenient way of assuring that all
preventive maintenance procedures are performed on schedule.
For extensive systems, such variables as sampling site changes and
monitoring instrument replacements should be recorded similarly to cal-
ibration and maintenance (Sections 4.10 and 4.11), i.e., in a bound,
page-numbered notebook reserved for this purpose.
Configuration control for the laboratory environment is fully as
important as for extensive monitoring systems. It includes instrument
4-42
-------
location in the laboratory as well as modifications (e.g., sample
holder of different design) that affect measurement data. Temporary
and/or permanent equipment configuration changes should be made only
when the effect is well characterized and demonstrated to improve data
quality.
4.14 DATA VALIDATION
Data validation may be defined as a systematic procedure whereby
data are filtered and accepted or rejected based on a set of criteria
for providing assurance of the validity (accuracy, precision, represen-
tativeness, completeness) of data prior to their ultimate use [3].
Criteria for each application of data validation techniques should be
documented and implemented for all task data. Automated data
acquisition systems are particularly suited for extensively comparing
reported data values with earlier stored values of the same parameter
and establishing and updating such statistics as parameter mean and
standard deviation. Similarly, checks for data completeness,
calibration performance, signal levels within reliable measurement
range (i.e., above minimum detectable and below saturation levels),
etc., should be designed into data validation systems.
Data validation must be defined with reference to the requirements
of each task. Frequently, laboratory data validation relies on the
highly trained professional judgment of the investigator or technician.
However, to rely on such capabilities in a routine monitoring system
situation invites disaster. In both situations, the data should be
flagged but not discarded unless there is definitely identifiable error
(e.g., an obvious and documented equipment malfunction).
In a laboratory environment, operating personnel who are alert and
adequately trained regularly perform this type of screening as they
manually collect data. This requires particular attention that valid
data are not rejected without adequate reason. Data should not be
rejected "because they don't look right" or other similarly subjective
reasons; it is generally the case that such data are frequently valu-
4-43
-------
able as the particular model is developed to a higher level of sophis-
tication.
In either the laboratory environment or the complex data acquisi-
tion system, provision should be made for regular review of the appro-
priateness of the specific validation criteria. This analysis should
include both technical and professional inputs in order to keep a
proper balance of theoretical and practical considerations in the set-
ting of limits on the data. In all cases, data validation procedures
should not be permitted to delete raw data, but only to flag It when a
clearly stated validation criterion is exceeded.
4.15 FEEDBACK AND CORRECTIVE ACTION
For each task, a system for deleting, reporting, and correcting
problems that may be detrimental to data quality must be established.
As noted in reference 3, this system "...can be casual when the
organization is small or the problems few. When this is not the case
...action documentation and status records are required." The exact
system design should accommodate the conflicting needs for (a) quick
response and (b) thorough communication and documentation of the
problem and its solution. Complex data acquisition systems require a
formalized closed-loop system with standard forms for various stages of
the problem and its solution. In a laboratory context, however, if a
"fix" is not immediately apparent, direct contact between the project
officer and the involved technician may be the most effective "system."
An effective system will eliminate the causes of malfunctions before
they occur. With this approach, corrective action becomes preventive,
and the data from the process assume increasingly higher quality and
greater reliability.
An important aspect in improving the potential for effective feed-
back and corrective action in task personnel is a quality assurance
briefing (see Section 4.3). The purpose of this briefing is to make
each individual involved in the task aware of how his personal contri-
bution to the task affects its overall data quality. Such briefings
4-44
-------
certainly should take place during the opening phases of the task, and
should probably be continued at specified intervals throughout the task
life.
During the initial briefings, personnel could be exposed to the
answers to such questions as:
-- What is the purpose of this task?
-- How do you fit into the overall task pattern?
-- How does your work affect task data quality?
-- What can you do to improve task data quality?
These briefings provide an excellent opportunity to establish and main-
tain an active employee-management feedback loop. Since the bench-
level personnel are the best observers of routine task operations, they
are also the most likely to detect disturbances. With an effective
feedback loop in operation, management can quickly become aware of
fluctuations that might otherwise go undetected. In general, it is
important to impress on task personnel that any contribution to data
quality is important. Their daily conduct literally controls task data
quality.
Additional feedback systems should be established, at least
informally. For example, the discovery of an impure substance by one
investigator should be communicated to all other users of the particu-
lar substance as rapidly as possible. This can be facilitated by the
use of adequate stockroom records.
A description of the problems, solution of the problems, and
estimates of the effect of the problems on data quality should be made
available to appropriate management on a regular basis.
4.16 DATA PROCESSING AND ANALYSIS
Data from health-effects research are rarely, if ever, used in the
form in which they are recorded. The initial phase of data processing
is to convert the data into a form suitable for conceptual manipulation
4-45
-------
and possibly perform preliminary statistical and other calculations.,
These intermediate results are then analyzed in terms of the particular
model of interest to the investigator. Each of these transformations
of the raw, observed data is made by a manually or electronically pro-
grammed series of manipulations. Hence, each transformation is a po-
tential source of error in the final result. The automated analysis of
large amounts of data thus carries the inherent potential for signifi-
cant error, quite apart from experimental errors, due to the processing
analysis functions.
Less obvious are the routine errors of transferring data from a
notebook to machine-readable form (e.g., punched cards). Typical error
rates for each transfer may exceed 5 percent. When several manual
transfers are necessary, the quality of even the most carefully checked
data degrades quickly and significantly.
Statistical analysis must be used judiciously. The validity of
statistical techniques depends on their proper application to the par-
ticular experiment. In this context, regular contact with an in-house
statistician who is intimately familiar with the study is essential.
The overall reliability of contemporary computer hardware systems
is extremely high, due to various routine internal (to the machine)
auditing checks. A major source of error may be traced to the soft-
ware, which provides detailed instructions for operation of the hard-
ware. Typical errors may generally be traced to insufficient testing
of the program during the development stage or improper application by
the user. Either condition is difficult to detect due to the wide
range of values that may be supplied to a program for processing and
that cause no hardware detectable error. The only insurance currently
available against the "Garbage In, Garbage Out" problem is for each
user to exercise his or her best professional capabilities to estimate
reasonable results. If reasonable results are not produced by the
software system, a concerted effort should be made to determine the
exact source of the discrepancy.
The potential for such software problems is greater with increased
use of locally (i.e., within laboratory group) written programs for in-
4-46
-------
dividual minicomputers and microcomputers. In addition to verification
of the proper handling of "good data," extensive testing of the proper
handling of "bad data" (i.e., data containing some representative,
anticipated errors) should be performed over the complete range of
possible values and thoroughly documented. Consultation with the Data
Management Staff in properly testing and debugging these programs will
be cost-effective in terms of accurate and efficient data reduction.
4.17 REPORT DESIGN
The most visible product of a research task is the document(s)
which comprises the report of the important findings. Publication
guidelines applicable to the HERL research reports are available
[27,28]; minimum technical contents for nonclinical laboratory reports
and health effects research have been promulgated [9] and are shown in
Figure 7.
As in all scientific research reports, and within the indicated
consistent style stipulations [27,28], the report should be concise and
complete, with adequate discussion of the important technical aspects
of the research to permit a qualified professional to duplicate it.
Adequate data should be included to permit at least partial calculation
of important results. The conclusions, based on the data, and the
reasoning to support those conclusions should be clearly stated. As
much graphical and illustrative data correlation (with supporting
tables, as appropriate) should be used as is feasible. Error estimates
should be included with all quantitative and qualitative values re-
ported, as well as the basis upon which the estimates were made.
Much of the research conducted under the auspices of HERL/RTP is
highly specialized and frequently at the forefront of the technology,
yet few of the individuals who make up the audience for the reports are
specialists in the particular technical area. For this reason, the
purpose(s) and conclusion(s) of the research should be stated as clear-
ly as possible (see Section 4.2). The estimated errors, as well as the
limits of applicability of results, should be stated in such a way as
4-47
-------
(A) Name and address of the facility performing the
study and the dates on which the study was initiated
and compIeted.
(B) Objectives and procedures stated in the sponsor-
approved protocol, including any changes in the
original protocol including justif1 cation(s).
(C) Statistical methods employed *or analyzing the data.
(D) The test and control substances identified by name,
chemical abstract (CAS) number or code number,
strength, purity, and composition or other appropri-
ate characteristics.
(E) Stability of the test and control substances under
the conditions of administration and storage.
(F) A description of the methods used.
(G) A description of the test system used. Where appli-
cable, the final report must include the number of
animals used, sex, body weight range, source of
supply, species, strain and substrain, age, and
procedure used for identification.
(H) A description of the dosage, dosage regimen, route
of administration, and duration.
(I) A description of all circumstances that may have
affected the quality or Integrity of the data.
(J) The name of the study director, the names of the
other scientists or professionals, and the names of
all supervisory personnel, involved in the study.
(K) A description of the transformations, calculations,
or operations performed on the data, a summary and
analysis of the data, and a statement of the con-
clusions drawn from the analysis.
(L) The signed and dated reports of each of the indivi-
dual scientists or other professionals Involved In
the study.
(M) The locations where all specimens, raw data, and the
final report are to be stored.
(N) The statement prepared and signed by the qual ity
assu ranee unit.
Figure 7. Minimum report technical content for
EPA health effects tests [9].
4-48
-------
to minimize misinterpretation. Application of the results to alterna-
tive theories (models) should be provided, with indication of the
rationale used in reaching the stated conclusions rather than the
alternative conclusions.
Quality control and quality assurance activities should be dis-
cussed in as much detail as possible. This discussion should permit
the specialist and nonspecialist alike to correctly assess the level of
the quality assurance effort invested in the research. This should, in
addition, permit subjective evaluation of the validity and accuracy of
the reported results and conclusions.
4-49
-------
SECTION 5
DATA QUALITY ASSURANCE FOR RESEARCH PROJECTS
The technical discussion to this point has focused on quality
control aspects of the quality assurance plan that influence test data
quality from the perspective of operating personnel (or organization,
in the case of extramural research). In this section, the discussion
focuses on quality assurance aspects of the QA plans from the perspec-
tive of personnel other than operating personnel. The fundamental
concept is that the project officer has at his disposal a variety of
probes, or checks, on data quality quite independent of the functioning
of the task research system. The choice of suitable probes, and their
applications to the measurement system, is the project officer's, with
the support of the HERL/RTP QA organization.
5.1 QUANTITATIVE ESTIMATES OF DATA QUALITY
Quantitative measurements and comparisons (i.e., quantitative
audits) provide the best possible objective estimates of data quality--
insofar as they are available. Current efforts by the National Bureau
of Standards are resulting in the relatively rapid production of new
environmentally related Standard Reference Materials (NBS-SRM's). A
current catalog of NBS-SRM's [18,19] may be obtained from:
Office of Standard Reference Data
National Bureau of Standards
Washington, D.C. 20234
In addition, the World Health Organization maintains information on
worldwide sources of biological standards [20].
Appropriate use of the available reference materials by the
project officer can provide an objective measure of specific parameter
data quality. A variety of techniques, all of which should be designed
5-1
-------
as blinds (i.e., with operating personnel unaware of the nature of the
reference sample) are available. Direct analysis of the reference
material and routine duplicate analysis of samples (one of which is
"spiked" with a known amount, of the reference material), are two possi-
ble uses of reference materials in analytical systems for the evalua-
tion of solution concentration, aerosol characterization, etc.
Unfortunately, NBS-SRM's do not exist for many measurements of
interest. In such cases, techniques should be devised for probing the
quality of the task research system. Round-robin analysis of aliquots
of a single sample may be performed by any number of laboratories.
While accuracy (i.e., deviation from a "true" value) cannot be mea-
sured, an estimate of analytical variability (precision) is available.
For labile samples, collaborative (side-by-side) analysis may be used
(e.g., several technicians would count normal cells on a set of
plates). This is equivalent to the round-robin test, but is performed
at one location and at approximately the same time. To give a measure
of various research system components' variability, interlaboratory and
intralaboratory analysis/measurement programs may be designed. In this
case, it is important that the statistical design of such testing
recognize such aspects as operating shift changes, diurnal biological
changes, and other nonrandom variability in the sample(s) and total
measurement system.
5.2 QUALITATIVE ESTIMATES OF DATA QUALITY
In addition to the various quantitative probes available to a
taskmaster, there are also qualitative probes of task research data
quality. The comparison, rather than between two numerical values, is
between the proposed and executed(ing) plans.
Thus the protocol (or work plan in the case of extramural support)
is a statement of the reasoned plans of the operating organization.
From qualitative measures of data quality (i.e., quantitative or system
audit), an individual, independent of the operating organization or
group, compares the planned activities with what is observed to occur.
5-2
-------
While complete agreement is no guarantee of high quality data, dis-
crepancies are an indication that all is not well, that the task is not
under the control of the project officer as it should be. Thus, the
qualitative audit includes consideration of the execution of the points
addressed in the protocol (which should be essentially the points
covered in Section 4): Are data actually being collected according to
the statistical design; are operating personnel properly qualified for
their responsibilities; are records properly recorded and maintained,
etc.
In summary, the project officer has available various quantitative
and qualitative probes to effectively demonstrate and document the
quality of data being produced in a task.
5-3
-------
ATMOSPHERE
-------
Page
6.0 GUIDELINES FOR ATMOSPHERE GENERATION AND MONITORING 6-1
6.1 Introduction 6-1
6.2 Atmosphere Generation 6-1
6.2.1 General Considerations 6-2
6.2.2 Participate or Aerosol Atmospheres 6-4
6.3 Sample Collection and Analysis 6-6
6.3.1 Introduction 6-6
6.3.2 Sample Representativity 6-7
6.3.3 Physical Characterization of the Atmosphere.. 6-8
6.3.4 Sample Quantity 6-8
6.3.5 Sample Handling and Storage 6-9
6.3.6 Recommendations for Sampling and Analysis
of Selected Pollutants 6-10
6.3.6.1 Sulfur Dioxide (S02) 6-10
6.3.6.2 Nitrogen Dioxide (N02) 6-13
6.3.6.3 Photochemical Oxidants 6-14
6.3.6.4 Carbon Monoxide (CO) 6-14
6.3.6.5 Hydrocarbons (HC) 6-15
6.3.6.6 Peroxyacetyl Nitrate (PAN) 6-16
6.3.6.7 Total Suspended Particulate Matter.. 6-16
6.3.6.8 Sulfuric Acid Mist 6-17
6.3.6.9 Sulfates 6-17
6-0
-------
SECTION 6
GUIDELINES FOR ATMOSPHERE GENERATION AND MONITORING
6.1 INTRODUCTION
In the HERL/RTP exposure facilities and in population studies, the
effects of various atmospheric pollutants on test subjects are evalu-
ated. These pollutants are in gaseous and/or aerosol form. The expo-
sure facilities are used to study the effects of synthetic atmospheres
on humans and other test subjects. Population studies evaluate the
effects of the natural atmosphere on humans.
The generation of synthetic atmospheres and the monitoring of both
artificial and natural atmospheres are extremely complex tasks. Guide-
lines for quality assurance planning for air pollution measurement
systems [3], and ambient air methods [4] have been developed by EPA.
The goal of these tasks is to produce high quality exposure effects
data, hence the details of generation, sampling, and analysis tech-
niques must be considered within the quality assurance plans. In human
exposure, quality assurance planning begins with subject safety, con-
tinues in the experimental tests, and finally provides the basis for
estimating the confidence limits on the exposure-effect relationships.
6.2 ATMOSPHERE GENERATION
The test atmosphere in an exposure chamber must be well character-
ized, in terms of both the composition and the concentration of the
components. The exposure experiments may run from a few hours duration
to several consecutive days. The total dose, as well as the instantan-
eous concentration level, is important in such experiments. Therefore,
it is essential that the exposure source output be stable over the
exposure period. Since synergistic effects can complicate interpreta-
tion of the experiments, care must be taken to assure that the desired
species are present and that interferents are controlled and/or moni-
6-1
-------
tored. The changes in the composition due to loss of specific species
or generation of another species by physical or chemical reaction must
be taken into account,
6.2.1 General Considerations
Specific test pollutants (gaseous and/or particulata) are produced
by a source mixed with diluent gas and then introduced into the space
surrounding the test subject. Gaseous pollutants are usually obtained
from high pressure gas cylinders, although some are produced in situ
(e.g.,, ozone from ultraviolet irradiation). Occasionally, pollutants
in ambient concentrations are obtained from permeation tubes (e.g.,
S02> NOg, I^S, CHgSH). Aerosols may be generated by nebulizing
a solution (or a suspension) of known composition. Aerosols in solid
form may be obtained from a "dust-feeder" type of apparatus such as the
Wright Dust-Feeder [29,30].
To assure the composition of the atmosphere, the source and the
background atmosphere into which the pollutant is released must be
well-characterized and stable. Reactivity of the pollutant with the
test chamber (including the delivery system) must be characterized and
documented.
Since atmospheres are generally prepared by introduction of 3
specific amount of pollutant into a known volume of background airs the
quality of this background air is vital. Particulate matter, organic
vapors, and other gases should be removed by appropriate filters,
adsorbents, etc., prior to pollutant introduction, A schedule for the
periodic replacement of these filters and absorbent elements should be
established as indicated in Sections 4.4 and 4.10.
One of the frequently neglected "other gases" is water vapor. The
humidity of the test atmosphere is an important variable, especially
when atmospheres containing particulate matter are being generated.
Surface reactions on particulates and aerosol composition are strongly
dependent on the amount of water vapor present. All moisture should be
removed from the background air using a mechanical dryer and absorbent.
The air can then be rehumidified to a specified level by the addition
6-2
-------
of steam or a water spray. Test subject humidity requirements (e.g.,
rodents, ca. 50 percent) must be considered in determining the relative
humidity finally obtained.
Since the test atmosphere is prepared by mixing pollutants and air
in proportions described by the ratios of their volumetric flow rates,
the accurate measurement of each of these flows becomes critical. Flow
measuring devices should be properly calibrated and operated. The
pressure and temperature of gases at the flow measuring devices must be
stable and known. It must be realized that small absolute errors in
the measurement of the characteristically low pollutant flow rates
result in large relative errors in pollutant concentration. In
addition, after pollutant and air flows have been combined, it is
important to provide for good mixing of the two components in order to
assure a homogeneous atmosphere.
Finally, it is important to characterize the test atmosphere as it
encounters the test subject, i.e., spatial as well as temporal charac-
terization. This characterization provides data concerning the actual
exposure conditions. It is also helpful to characterize the atmosphere
as it leaves the source since this information is useful in the early
detection of harmful levels of pollutants resulting from source mal-
functions. Early detection allows diversion of the defective atmos-
phere before it reaches the test subject. Interaction between indivi-
dual components of test atmospheres, always a potential source of error
in atmosphere generation, can be minimized by careful attention to
parameters such as composition, concentration, and residence time. In-
teraction between the atmosphere and conduit or chamber walls can also
be a source of error. This is especially true for aerosols and react-
tive gases such as ozone and sulfur dioxide. Even the test subject may
interact with the atmosphere in an unexpected and undesirable manner
(e.g., NH3 from animal excreta). For valid data to be accumulated
from an experiment, each of the interactions that may occur must be
carefully examined and controlled by the project officer.
6-3
-------
6.2.2 Particulate or Aerosol Atmospheres
Atmospheres that contain generated participate matter or aerosols
exhibit so many specialized problems that they warrant separate
discussion. Since the dose of inhaled particles is mass- and
size-dependent, knowledge of both the rwss of the pa^c'des arid their
size distribution are needed in order to characterize the aose level.
If the aerosol atmosphere is a mixture of several pardculate
components, the size distribution of each should be characterized.
As was mentioned earlier, aerosols may be obtained by penalization
of a solution (or a suspension) of known composition. Deviations in
the aerosol characteristics may result from inadequate flow control of
the nebulized air, excess loss of solvent, and cooling of the solution
due to solvent evaporation. Circulation of the solution from an exter-
nal large reservoir may be used to avoid the problems due tc solvent
evaporation. In the Wright Dust-Feeder, lack of homogeneity in the
powder plug may produce deviation in aerosol output [31j.
Characteristics of an aerosol may change due to partic'le-gas cr
particle-particle interactions. The particle-gas interactions in hy-
groscopic aerosols resul c in evaporation or growth of particles (for
examples see reference 32). In salt aerosols, humidities above 75 per-
cent will generally result in growth of the particles. This growth
process is extremely rapid and can lead to a several-fold particle size
increase at high humidities (>90 percent). In acid aerosols numidi-
ties below 20 percent can produce change in one other cirection oue to
evaporation. As a general rule, and dependent on iest subject health
parameters, humidity should be maintained constant Deiween 20 percent
and 70 percent to avoid particle growth or evaporation losses.
Particle-particle interactions resulting in coagulation are depen-
dent upon the particle size and concentration. Coagulaiion can lead to
significant errors for dense aerosols. In general, if the concentra-
C O
tion is less than 10° particles/cm0, coagulation may be neglected.
Other factors leading to coagulation are turbulent mixing and extreme
polydispersity. Charges on particles also significantly influence
6-4
-------
aerosol behavior. An aerosol generated by nebulization may require
charge neutralization. This will avoid the uncertainty of the effect
of charge on particle-particle interaction and deposition on surfaces.
Methods for characterization of aerosol size and concentration are
based on a variety of principles. Interconversion between two methods
is not usually possible without introduction of significant errors. If
an aerosol is used for an inhalation study, the aerodynamic size dis-
tribution based on mass is appropriate. To obtain this information,
inertial classification of particles by a method such as impaction is
necessary. However, in various size ranges, other methods based on
electrical mobility, microscopic, or light scattering analysis may be
needed to characterize the aerosol. Conversion of data from these
methods into aerodynamic size should follow recognized procedures such
as those described in reference 33. An estimate of the errors involved
should accompany the conversion.
After the aerosol has been generated and characterized at the
source, it is delivered to the test subject. Certain precautions and
pretests should be taken to prevent significant change in the atmos-
phere before it reaches the subject.
Losses of the aerosol component en route to the exposure chamber
can be significant. The most common cause of particle loss is
deposition on conduit walls. This deposition of particles on surfaces
is due to sedimentation, inertia, and diffusion processes: the extent
and nature of particle loss is size-dependent. In polydisperse aero-
sols the deposition loss of particulates will affect the particle size
distribution as well as the concentration. In general, large particles
over a few microns are preferentially lost by sedimentation and iner-
tia. These effects can be minimized by using high flow velocities and
by avoiding bends or sudden transitions. Because of this tendency
toward deposition, it is extremely important that aerosol atmospheres
be finally characterized immediately before they encounter the test
subject.
If the atmosphere contains particles larger than 1 ym in
diameter, lack of homogeneity in the chamber may be significant.
6-5
-------
Segregation may occur due to sedimentation or bypassing the inlet and
outlet. Distribution of the incoming test atmosphere over as broad an
area as possible would minimize the flow channeling problems.
Sedimentation effects may be minimized by a vertically downward
movement of the test atmosphere. Even with these precautions,
segregation may occur. Again, this tendency necessitates
characterization of the aerosol at the test subject. The sampling
position for this characterization must represent the same location and
elevation in the chamber as the test subject. This will assure the
characterization of that portion of the atmosphere that actually
reaches the subject.
6.3 SAMPLE COLLECTION AND ANALYSIS
6.3.1 Introduction
Collection of a representative sample is of utmost importance in
any measurement process as noted in Section 4.7. The analytical
results may be of excellent quality; however, if the sample is
contaminated, degraded, or is otherwise not representative of the area
or population under study, the relationship between the measured
pollutant concentration and the response of exposed subjects will not
be valid.
It is important to recognize that obtaining a representative
sample is difficult, especially when components of the ambient air are
measured. For this reason the processes of sample collection and
analysis should be included in the experimental design (see Section
4.2). Sampling methodology and the number of samples required should
be established prior to beginning the task.
Ambient air studies frequently deal with large populations and
extended airshed areas, which cannot be thoroughly monitored. Thus,
statistical sampling techniques are generally required. The number and
size of "blanks," control groups, and samples taken from the background
should be carefully evaluated. Calibration, instrument spanning, and
audits also have an impact on sample collection and analysis efforts.
6-6
-------
In most studies, more than one pollutant or parameter will be
measured. During the experimental design phase, the requirement for
measurement of co-occurring pollutants should be addressed. Important
parameters such as humidity, temperature, and atmospheric pressure are
also commonly measured.
6.3.2 Sample Representativity
A representative air sample of ambient air to which plants and
animals may be exposed is difficult to obtain. Spatial and temporal
aspects of sampling should be considered carefully prior to locating
the sampling stations. A thorough background study in support of an
ambient air monitoring program should include a study of source
inventories, historical meteorology of the area, local topography, and
examination of data from any preexisting air monitoring stations.
The point in space from which a sample is taken is an important
variable. The sample should be collected in a location which is clear-
ly representative of the air space being characterized. For example,
if the objective of the study is to assess the effects of air quality
on children, the sampling point might be located in a schoolyard or
playground 1 to 1.5 meters above the ground. The inlet to the sampling
probe must be located in such a way as to protect it from possible
damage -- by the elements or by vandals -- yet out of the microenviron-
ment of the sampling equipment. The use of mobile sampling equipment
is often very helpful in locating proper sampling points and in survey-
ing a large area at minimal expense.
The time frame in which a sample is taken also has a bearing on
sample representationty. Generally, the longer the period of sampling,
the better the sample will characterize the environment. However, the
final decisions concerning sampling duration and frequency must be made
with respect to the objectives of the task (see Section 4.2). If
continuous or semicontinuous analyzers are used, concentration trends
and any unusually high or low values will appear when the overall data
are examined. On the other hand, if "grab" samples are collected for
6-7
-------
short periods of time, it is probable that very high or very low con-
centrations will be obtained which do not represent the subject's
average exposure. If the experiment is well controlled (such as a
captured air mass in a chamber), periodic (or grab) sampling can be
utilized. However, even here with certain pollutants the use of grab
samples is discouraged due to potential chemical degradation of the
sample during transport and storage [34].
Sample integration is a helpful technique when one must collect an
air sample for later analysis. In this process the sampling vessel is
slowly filled with an air sample over a period of time. Again, sample
integration must consider the stability of the sample with time, as
well as the averaging of concentration fluctuations.
6.3.3 Physical Characterization of the Atmosphere
To obtain an accurate intercomparison of samples taken in various
cities or air regimes, it is necessary to know some of the physical
characteristics of the ambient or enclosed air mass. Such
characteristics include temperature, barometric pressure, relative
humidity, and perhaps wind speed, wind direction, and solar radiation.
Additionally, knowledge of the temperature, pressure, and humidity
within the analytical laboratory is necessary for correction of gas
flow rates to standard temperature (25° C) and pressure (1 atmosphere).
This is particularly important during the calibration and operation of
analyzers and impinger systems.
6.3.4 Sample Quantity
A sufficient volume of air must be collected or passed to an
instrument to obtain valid data. In the case of continuous analyzers,
an excess volume of sample generally flows through a glass sampling
manifold and the instrument's sampling line is attached to this
manifold. An initial flow rate at least 50 percent in excess of that
required by the analyzer(s) is recommended. If the sample flow is less
than that demanded by an analyzer, the analyzer or sampling device will
6-8
-------
pull in room air and the sample will be diluted. Sufficient sample
quantity is also needed during calibration. The rate of sample flow to
a continuous analyzer should be identical to the flow established
o
during calibration. That is, if an analyzer samples 200 cirr/min
during calibration, it should sample 200 cm^/min during analysis of
ambient air. The same is true of impinger samples in which air is
bubbled through a chemical solution. In the case of impingers
containing chemical solutions, a sufficient volume of sample must
bubble through the solution to achieve a reliably detectable
spectrophotometric or other response. For particulate collection
devices (high volume samplers, cascade impactors, etc.) the flow must
be that specified to achieve the entrainment of the desired particle
sizes. The particle sampler must sample a sufficient length of time to
build up sufficient deposit for accurate weighing and/or chemical
analysis.
6.3.5 Sample Handling and Storage
Since many of the pollutants in ambient air are highly reactive,
unstable species, they cannot be reliably collected and stored for
later analysis. Ozone, oxides of nitrogen, peroxyacetyl nitrate (PAN),
sulfur dioxide, and other sulfur species should be delivered directly
from the ambient air to the analyzer or impinger through Teflon or
glass tubing.
Other less reactive pollutants, such as carbon monoxide and hydro-
carbons, may be stored for periods of several days prior to analysis.
Teflon or Tedlar bags are adequate for carbon monoxide samples. Stain-
less steel or glass sampling containers are better for hydrocarbons.
There may be no clear consensus in the scientific community as to the
reactivity of a specific pollutant. In such cases, it is essential
that it be determined and documented as part of the study if the con-
clusions are to be valid and defensible.
Particulate samples collected on glass fiber or other types of
filters are often weighed and analyzed at a later date. For reproduci-
6-9
-------
ble weight determinations, the filters must be conditioned at a con-
stant relative humidity for a specified period prior to weighing for
tare and gross weights.
In all cases, stored samples should be protected from unusually
high temperatures and light. Some samples are best stored under
refrigeration in the dark.
6.3.6 Recommendations for Sampling and Analysis of Selected
Pollutants
Recommendations for the sampling and analysis of selected pol-
lutants commonly found in ambient atmospheres follow. Included are
suggestions and a summary (Table 2) for the six EPA criteria pollutants
as well as other species of current interest to HERL/RTP personnel.
This is not an exhaustive list—either of pollutants or of sampling and
analysis methods. Rather, it is a list of some pollutants of current
interest and their most often accepted analysis methods, major
interferences, and calibration concepts (see reference 35 for
additional pollutants).
6.3.6.1 Sulfur Dioxide (S02)~
The EPA reference method for determination of ambient levels of
sulfur dioxide is the pararosanal ine method. This manual, wet chemical
method is a complex sampling and analysis procedure. EPA has accepted
an automated version of this method that reduces the complexity of the
analysis [36]. However, it is the continuous, instrumental methods
currently available that produce the most data for the least
professional time invested. These methods include coulometric, flame
photometric, pulsed fluorescent, and second derivative spectroscopic
detection of sulfur dioxide. The coulometric, pulsed fluorescent, and
second derivative spectroscopic methods are specific for SCU- The
flame photometric method detects sulfur-containing species (e.g.,
S02, H2S, R-SH); it can be made specific for S02 by inserting a
scrubber cartridge into the sample inlet line.
6-10
-------
z
o
^
S
LU
Z
LU
ID
Z
O
1—
S
CO
i
^^
o
00
1—
z
UJ
CtL
C£.
LU
Lu
or
UJ
1—
1— (
1—
z
LU
UJ o
O£ 1C
OO LU
z
t;
^c
-J
_J
o
1
I/)
>J
CO
c
o
+J
3
r—
Q
1
IO
i-
01
•0
C
•r-
>_
O
-t- 1"
Z
M-
O
C
o
•^
0
00
-o
s_
IO
•o
c
ia
ia
r— g
O 1-
•r- c
s- £
ia E
a. 10
•O
O
JC
ai
•r—
i-
•r-
z
1
cn
oo
c
0
) *
3
^
o
"g
ia
s_
O)
-o
c
•r-
>k
°
CO
i.
•Q
f"
^
u
CM
Z
c
o
o
«
< CO
U V-
O *"**
U S-
o
JC CO
•? 10-1
JC 1
OL £
c H-I 5
"~ S- 10
CMU IO
O JC t-l
0 4-> CM
O
s_
o -o 10
CMC
a:
LU
Ct
Q >3-
Z- —
O)
X
o
c
i
c
o
i-"5
ia cj
o- —
in
c
l"» ^
I- O)
n) JD
U JO
O 3
S- S-
•o o
>> cn
C
01 O
c >,
IO C7I (J
JC C C
flc'o
Ol 'r-
5= a.«t-
o am-
z -a 01
o
Lu
i
r n
C
O
ia
ia
•M
Qj
*
CO
S-
0)
|
^«
>^
U
0)
c
10
JC
_O)
^
Ol
CO
c
o c
a. o
CO -Q
01 !-.
s- 10
u
0
Lu c"
S_ CO
01 0)
C L.
•r- (J S.
i— c ai
1 ^- JO
c E
O O 3
Z 4J C
LU
a.
o.
->Lu
Lu u
o •*
to-—'
CO
c
o
i.
10
u
s- u
T3 3:
^^
in-0
01 10
•o
c *
••- CO
r— Ol
>> CO
U rO
o>
T3 CO
s- oi ai
10 S. S-
•O 3 3
C O-4->
ro X
+J If- f-
oo o E
ai
c
o
z
oo
*F
^J
C3
c
o
4->
3
•o i-
co (O
01 C
10
s- ai
Ol r"~
"O ^
i- JC
r- -!->
IO
a)
01
a.
i.
o
Ol
c in
•r- 4->
N C
i- 01
•r- IO CM
X O
O O1OO
c
C , 3 3:
3: oo —
o
z
M-
0
c
o
4->
IO— •
-a t
•^ Ol
x -a
o c
O r—
4^ (J
^^
IO O
4-> S-
10 1-
o
ia
f.
z ^~
4J
c
••- O)
CO IO ^^
Z JC OO
O.CM
t ^_^
01 CO
T3 IO CM
c cno
•i- Z
>— -o
>^ C 0
C71 Ol Ol
**^ ^~% ^~% ^— ^ >•««
> > > > >
3333 3
OO OO CO CO OO
o o o o o
i. S- S- i. S-
o o o o o
4-) +J 4-> 4J 4->
ro IO ro fO ro
S- S- S- S- S-
c c c c c
Ol Ol Ol Ol Ol
"? *> *> *> ">"
3333 3
OO OO OO CO OO
o o o o o
CO
c
0
O 4-> 4-»
Z ro ia
„ t-
CM >, C
O 4-> 01
z -i-o
•a c
•i-O
oo E o
CM 3
x jc oo
o
00 3-1—
~
*£ ai
ca o *—
z c o
• — .01 ca
>, U 1
O &. CO Q. O)
•r- +J ai oi a. c
t, Ql C C < T-
•»-> g -r- 01 E
E 4-1 3 ">, oT -g
•r- O r— JC Lu S
i. JC -r- +J CJ JC
O Q- E 01 S-
i— Ol 1 O 1
O > JC CO «t OO
U 3 O O O
rO
u
01 CO
JC 4->
u c
oo
rO
^
o
C-
-o
O)
Jf-1
o
d)
d)
to
S-
o
^
00
-o
o
a>
^ ^
c
OJ
u
S-
3
<1J
0
c_
rO
3
oo
m
CM
0)
M— •
-Q
rO
f—
o ia
O -i- 00
JC X 0
a- o — •
6-11
-------
!
o
^^
s
UJ
UJ
CJ
2:
o
K^
i
CO
*"~*
CJ.
CO
I—
2:
Ul
fy»
UJ
U.
o:
LU
i —
, t
*~~
UJ
E 0
UJ o
CO UJ
1 ' 1
SI
1 —
2:
t=£
— 1
.J
o
TJ
CO
4J
(O
s.
cu
c
(U
Ol
o
c -o
o s-
•r- 10
4-> -a
3 C
•r- 4J
Q ul
4J
(U
0 00
CO
to • —
ul C
,
C X
(o o
o
4-> C.
O -r-
.e
CL
t— •!—
CC C
13
i-
!O
•o
c
IO
Ul 4-J
~~^ C
cu cu
r— 4J
CL C
S 0
03 tj
Ul
CU
01 S-
C 3
•r- 4->
IO O
> E
0
LU
1
C3
01
n)
z
^J
[ *
ai
0
X ^— *
o z
CU (X
0-- — •
CU
u
•1—
s_
CL
t. o
IO ••-"
"o 4->
O E
U S-
cu o
CO 4-
s-
cu
E a?
CO
a.
3
"o a;
=9 U.
_C
010
3: —
Ul
cu
OS
3
O
4-> Q-
1. CO
<0 1—
O--—
II «3-
O
4-
O
O
+J
"o
CO
n3
T^
c:
4-1
CO
E
3
•r-
S-
Ul IO
o
•r- Ol
4_j c^
o •»-
co cu
i. •—
a.
i- £
QJ O
[ \ O
^
•r- Ul
4- C
1 0
iS E
C3 . O •
c s- c c.
O -i- O •>-
CJ IO U IO
t 1
-cc.ee:
•<- 41 ••- CU
"u 13
<4- H-
o j= o j:
Ol-l- O>-r-
c 2 ex
X CM X CM
•i- O ••- O
s: to s: co
CU 4)
4-1 4^
Ul Ul
>! X
Ul Ul
c c
0 O
fO IO
a; £.
O
f
CL
IO
LU
C.
o
a
fcj
c
cu
0 •
§£
U IO
1
J= C
C71
cu
CL
CM
O
CO
A
CO
o
•>
t
Ol
(U Ul
• — • CU
O) O
c cu
•^ f>
N Ul
-o. — .
f- CM
X 0
o z
u
4->
CU
3
o
CJ
C
o
IO
s-
c
cu
o •
C (.
o ••-
o -
3 CJ
-a cu
CU CL
C£ Ul
c
o
'lo
s-
I <
c:
cu
o •
c s-
o ••-
u ia
t
_c c
0110
•r- 41
O
4-
O ^
4->
Ol-!-
X CM
•r- O
•£. CO
41
4J
Ul
Ul
C
0
4-1
4J
i_
41
CL
CM
O
CO
Ul
C
4)
O
'
j
CM
O
CO
t-
41
•r—
t^,
C
0
c
o
•r—
U
CU
r—
O
CJ
-o
•r- 4->
O Ul
-------
Some nonsulfur compounds do interfere with these methods. It is
reported that differences between carbon dioxide concentration in the
calibration/zero matrix and the sample matrix interfere with certain
flame photometric detectors [37]. Hence, the C02 concentration in
the calibration gas for this instrument should be matched to the
C02 concentration expected in the sample. The pulsed fluorescence
method will respond to certain aromatic hydrocarbons unless a special
scrubber (referred to as a hydrocarbon cutter) is placed in the sample
inlet line.
Calibration of these instruments is usually accomplished against a
SOo permeation device. Although gas cylinders of SOo are widely
used for calibrations of source level monitors, they are generally not
employed for ambient level instruments due to stability problems.
6.3.6.2 Nitrogen Dioxide (N02)--
The EPA measurement principle for the determination of ambient
nitrogen dioxide (NOo) is based on the chemiluminescence produced by
the oxidation of NO with ozone. The method is instrumental and
continuous. These analyzers detect NO and total oxides of nitrogen
(NOX) directly. A readout of NOo concentration is provided
indirectly by electronic subtraction. One automated and two manual wet
chemical methods have recently been accepted by EPA as equivalent to
the reference principle [24]. Neither the Christie (arsenite) method
nor the TGS-ANSA method is affected by the interferences listed above,
but they suffer from the difficulties inherent in all manual sampling
and analysis methods.
Recently published research has indicated that this method is
subject to interference from third-body quenching reactions including
those with carbon dioxide and water vapor [38]. Research has also
shown that the thermal converter (used in this method to reduce NOo
to NO) can reduce nitrogen-containing compounds to NO. PAN is also
converted with relatively high efficiency [39]. The efficiency of this
converter should be determined frequently, especially when high concen-
6-13
-------
trations of nitrogen dioxide are being analyzed.
Calibration of the NO and NO channels of the instrumental
A
method is generally accomplished using bottled standards of nitric
oxide in nitrogen. The N02 channel is calibrated by oxidizing some
of the NO calibration standard to N0£ before the gas is introduced
into the instrument. This oxidation is accomplished by ozone gas phase
titration (GPT). Calibration of this channel may also be accomplished
against a NOo permeation device. Much helpful information on the
calibration and use of chemil uminescence NO-N02~NO analyzers is
available in an EPA technical assistance document [25],
6.3.6.3 Photochemical Oxidants--
Ozone (Og) is the most often measured photochemical oxidant.
Wet chemical methods can be employed for measurement of ozone.
However, the recommended analytical procedures are the instrumental
methods based on ultraviolet photometry, chemiluminescence from the
reaction between ozone and ethylene, or chemil uminescence from the
reaction between ozone and rhodamine-B.
Calibration is accomplished using an ozone generator. Its output
is determined by gas phase titration of nitric oxide, ultraviolet pho-
tometry, neutral buffered potassium iodide, or boric acid buffered
potassium iodide colorimetry.
6.3.6.4 Carbon Monoxide (CO) —
The EPA measurement principle for continuous monitoring of carbon
monoxide in the atmosphere is nondispersive infrared spectrometry
(NDIR). The principle is based on the absorption of infrared radiation
by carbon monoxide in a nondispersive spectrophotometer. Another
method is based on catalytic conversion of carbon monoxide to methane
by hydrogenation. The methane is then sensed by a flame ionization
detector.
6-14
-------
The infrared adsorption spectrum of water is sufficiently similar
to that of CO to interfere for NDIR measurements. In source level
concentrations (e.g., 2000 ppm), CO^ is also an interferent.
Calibration of such analyzers is by injection of carbon monoxide
from standard cylinders. Steel cylinders have a tendency to react
slowly with carbon monoxide, forming iron carbonyl. Because of this
tendency, standards should be verified every 4 to 6 months by compari-
son to an NBS-traceable standard.
6.3.6.5 Hydrocarbons (HC) —
The EPA measurement principle for determination of hydrocarbons
corrected for methane is an instrumental method based on gas
chromatography with flame ionization detection. The method is designed
to measure both total hydrocarbons and methane so that methane can be
manually subtracted from the hydrocarbon analysis. No reference
instruments are currently designated because of problems resulting from
an inefficient methanator and a nonlinear detector. The instrument is
usually calibrated on the basis of methane supplied from low-level
standard reference tanks.
If analysis for specific hydrocarbons is sought, the chromato-
graphic column-flame ionization detector approach is preferred. The
specific compound is distinguished from others by introducing a known
concentration of this hydrocarbon and determining its column retention
time. The signal strength from the detector is correlated with concen-
tration by introducing varying known concentrations of the hydrocarbon
of interest. Permeation tubes containing certain hydrocarbons may be
used to generate standards. Mixtures of hydrocarbons in air or other
gases may also be purchased in cylinders.
The possibility of peaks from one or more compounds overlapping
during chromatographic analysis increases with the complexity of the
molecules. The extent of this problem should be investigated using
several conditions and column packings.
Hydrocarbon samples may be collected in Teflon or Tedlar bags for
later analysis. However, glass or stainless steel containers are pre-
6-15
-------
ferred. Certain hydrocarbons and other organic compounds may be ad-
sorbed on columns of polymeric material such as TENAX-GC and volati-
lized onto a chromatographic column at a later time.
For ultimate certainty in identification of hydrocarbons (and
other organic species), the method of choice is the combination of gas
chromatography and mass spectrometry.
6.3.6.6 Peroxyacetyl Nitrate (PAN) —
PAN is a photochemical oxidant often found in smoggy atmospheres.
Measurement is generally by a gas chromatographic procedure employing
an electron capture detector [40]. This method may be subject to
interference from low sample moisture content unless the relative
humidities of the samples and standards are controlled [41,42].
Standards may be synthesized by the photolysis of ethyl nitrite in
oxygen [43]. The synthesized standard, however, is not a primary one
and must be verified (e.g., by infrared spectroscopy).
6.3.6.7 Total Suspended Particulate Matter—
The EPA reference method for total suspended particulate (TSP) is
the high volume sampler method. Air is drawn into a covered housing
and through a filter by means of a high flow rate flower (1.0-1.7
m3/min). This flow rate allows suspended particulates having
diameters of less than 100 y m to pass to the filter surface. Accurate
control of the flow rate is critical to obtaining a valid sample. The
collection period for ambient air is generally 24 hours. The filter is
conditioned to a fixed relative humidity and weighed before and after
sampling. The net weight and total volume sampled are used to estimate
average suspended particulate matter in terms of micrograms per cubic
meter.
For experiments where smaller volumes of air are available for
sampling, low flow rate filters and impactors may be useful. Because
the emphasis here will probably be on chemical analysis and not weight,
care must be exercised in selecting the filter media or irnpaction sur-
face. The possibility of interferences in the analysis should be ex-
6-16
-------
amined through background and blank analyses. Optical particle coun-
ters are available for continuously monitoring the number of particles
and, in certain models, the size of particles. Manufacturers of such
counters and size discriminators should be able to show how calibration
was achieved.
6.3.6.8 Sulfuric Acid Mist—
Sulfuric acid mist may be collected on glass fiber or Teflon
o
membrane filters if it is at low concentration levels (<50 mg/m ) and
no sulfur dioxide is present. The sample can then be extracted with
deionized water and analyzed. When sulfur dioxide is present, it may
be oxidized to sulfate by moisture or an oxidant on the filter surface
thus interfering with the acid mist analysis.
6.3.6.9 Sulfates--
Airborne sulfates may be measured by analysis of the particulate
matter taken from high volume filters. The technique is generally as
described for the measurement of total suspended particulate matter
(Section 6.3.6.7). Analysis of the collected particulate for sulfate
is then performed using one of several available analytical techniques
(e.g., turbidimetry, ion chromatography). To avoid unwanted formation
of sulfates on the filter by reaction of SC^, the pH of the filter
must be controlled during manufacture to around pH 5. The analysis
method usually recommended is the automated, wet chemical method based
on the detection of the barium-methyl-thymol blue chelate [44].
6-17
-------
ANIMAL DOSING
-------
Page
7.0 ANIMAL DOSING 7-1
7.1 Preparation 7-1
7.1.1 Preparation of Animals 7-1
7.1.2 Preparation of the Test Substance 7-2
7.1.3 Preparation of the Control Substance 7-3
7.1.4 Preparation of the Vehicle 7-4
7.1.5 Mixing 7-4
7.2 Administration 7-5
7-0
-------
SECTION 7
ANIMAL DOSING
In addition to inhalation dosing described in the previous sec-
tion, research within HERL/RTP requires the administration of a wide
variety of solid and liquid test substances to animal subjects, pri-
marily mammals. The dose-response data are the basis for subsequent
analysis and evaluation; quality assurance must be included in the
planning of dosing and activities as well as in the response analysis
in order to assure the specified data quality. This section is
intended to outline major considerations in animal dosing from which
the project officer can develop task-specific plans for assuring the
quality of animal dosing data.
7.1 PREPARATION
Planning for animal dosing should be aimed at adequately control-
ling and/or documenting dose parameters of the research, as well as the
other parameters. A complicating factor is that animals respond
emotionally (and thus biochemically) to a wide variety of stimuli in
their total environment. This response is directly linked with subtle
and complex biochemical changes, which may obscure or alter the
parameters under study. Hence, the planning for animal dosing should
attempt to address and control all aspects of the known environment
which may effect these biochemical changes.
7.1.1 Preparation of Animals
Beyond the recommended receiving quarantine period, animals should
be acclimated to the total test regimen insofar as possible. They
should be exposed to routine environmental factors such as room temper-
ature, lighting levels, and feeding schedules for a time sufficient to
stabilize their responses. They should primarily be exposed to the
7-1
-------
animal handlers and appropriate handling routines, such as transporta-
tion from the animal care facility to the test laboratories. In this
way, behavioral and biochemical changes may be reliably attributed to
the testing program rather than changes in the environment.
7.1.2 Preparation of the Test Substance
As a primary mission, research at HERL/RTP involves test
substances that are suspected of being highly toxic, carcinogenic, or
otherwise injurious to humans. Hence, a primary consideration in
planning must be the safety and protection of operating personnel from
these test substances.
The screening of test substances and other supplies for purity and
quality is treated in a general way in Section 4.6. Characterization
of the test substances is even more critical, since the desired datum
results from the interaction of the test substance with the various
animals' complex biological systems. Uncharacterized changes, such as
electrolyte composition or microbiological activity, will induce a
pronounced response that will be misinterpreted as being caused by the
test substance. The effects in terms of lost time, money, arid
credibility are obvious.
The test substance should be obtained in sufficient quantity to
more than meet all requirements of the specific study and should be all
from the same lot. In this way, adequate characterization will have
extensive applicability and thus be economical of both time and money.
Characterization of the test substance in terms of purity is
essential if the calculations for the dose--as delivered—are to have
any validity. This may be performed in a number of ways; e.g.,
measuring an instrumental response of a standard preparation or
measuring its effect on standard cell cultures. Two or more methods
whose principles are unrelated should be used, as a minimum, to avoid
the possibility of compensating effects. The exact methods of
7-2
-------
verification are determined by the research methods involved and by the
professional expertise of the principal investigator.
Characterization of test substance impurities and "inactive
substances" is a more difficult task, yet it is crucial to the correct
interpretation of test results. Most suppliers make an effort to con-
trol the quality of their products, yet they frequently release an in-
ferior product; neither are they in a position to precisely accommodate
the research test requirements. For example, substitution of the
potassium salt for the sodium salt of a substance is of minor concern
to many chemists, yet there will be a significantly different response
if it is injected intravenously into an animal. As part of appropriate
experimental design, analysis for "inactive" components of the test
substance must be planned. Provision should also be made for
subsequent testing if incoming test results do not follow anticipated
trends. While it is relatively straightforward to analyze for a
specific substance, it is impossible to determine that all interferents
are absent: only the expertise of the project officer and the informed
observations of operating personnel are useable in minimizing, but not
eliminating, this problem.
Additionally, an estimate of the homogeneity of the test substance
should be made; the results of replicate analyses during acceptance
testing and purity checks provide data from which the project officer
can make this estimate. Subsequently, regular analysis during the life
of a task will provide data from which an estimate of the stability of
the test substance can be made.
By including provision for thorough characterization of the test
substance, the research conclusions will be strengthened and doubts of
their validity minimized.
7.1.3 Preparation of the Control Substances
Health research routinely involves groups, whether cell cultures
or human subjects are involved. The concept of a control group in-
volves comparison between two groups that are strictly equivalent,
7-3
-------
excepting only the active test substance. In order to compensate for
deletion of the test substance, a control substance having similar
properties must be used.
Since differenct substances cannot be strictly equivalent, the
crux of the project officer's choice of the control substance is to
decide what characteristics of the test sbus'tance are to be considered
equivalent. This depends on the exact nature of the test, and requires
case-by-case application of the project officer's expertise in conjunc-
tion with peers with whom he advises.
Specific considerations for the choice of control substance hinge
on the precise nature of the test substance as well as the particular
biological system under investigation. The cation may be significant
to some systems but not to others. Microbiological contamination, on
the other hand, will almost always be a consideration.
7.1.4 Preparation of the Vehicle
The choice and preparation of a vehicle depends on the solubility
properties of particular test substances (and control substances) as
well as intended route of administration. The possible pH range will
differ for oral and intravenous administration, among other considera-
tions. This aspect of dosing is generally a routine consideration, yet
it is an important parameter in the overall dosing scheme, which should
have the project officer's close attention especially during the plan-
ning stages of the task.
7.1.5 Mixing
As noted in Section 7.1.2, the majority of test substances en-
countered within HERL/RTP should be considered toxic, carcinogenic, or
otherwise injurious to humans. Hence, plans for mixing must include
explicit safeguards for the operating personnel and for test subjects.
7-4
-------
Depending, again, on the intended route of administration, the
mixing of the test/control substance with the vehicle to produce the
dosing matrix follows varying procedures. For inspection, demonstrably
homogeneous solutions are frequently used. Emulsions are also common,
but they are less stable in terms of long-term homogeneity, and should
be prepared as near to the time of use as possible [45], Oral dosing,
especially via feed, presents a complex set of problems in that the
test substance is frequently mixed with dry feed, which only lightly
and nonuniformly coats the pellets. The individual laboratory animal
feeding schedule and total consumption are not as controllable as, for
example, an injection. At the minimum, feed mixing should be
consistently performed by one person according to a well-designed plan,
and data should be collected regularly to characterize the uniformity
and repeatability of this operation.
An analysis plan should be devised that suitably characterizes the
total mixed system on a regular basis over the life of the task. This
includes demonstration of the (lack of) interaction of the test/control
substance and the vehicle. It also includes accumulation of data of
this type at regular intervals throughout the course of the project.
In addition, in choosing the test substance, the control sub-
stance, and the vehicle, care must be taken that no inadvertent syner-
gism is involved in inducing the response. During the experimental
design phase, use of combinations of these components should be planned
to demonstrate that synergism is not a factor.
7.2 Administration
The route of administration directly affects the dosing prepara-
tions. The available routes [45,46] are oral, intravenous, intraperi-
toneal, and subcutaneous. Each has its advantages and disadvantages,
which must be evaluated in the context of the specific research study.
Technical limitations to oral dosing include the complex interactions
of the animal's digestive system with the dosing matrix. Similarly,
intravenous injection implicitly involves the action of the liver sys-
7-5
-------
tern on the dosing matrix. Again, the choice of administration route is
the decision of the project officer, who, in consulting with his peers,
should arrive at a scientifically appropriate and defensible conclu-
sion.
7-6
-------
REFERENCES
R-0
-------
REFERENCES
1. U.S. Environmental Protection Agency, Health Effects Research
Laboratory, Management Policy for the Assurance of Research
Quality, Research Triangle Park, NC, EPA-600/1-77-036, 1977.
2. U.S. Environmental Protection Agency, Health Effect Research
Laboratory, Development of Quality Assurance Plans for Research
Tasks, Research Triangle Park, NC, EPA-600/1-78-012, 1978.
3. U.S. Environmental Protection Agency, Quality Assurance Handbook
for Air Pollution Measurement Systems, Vol I - Principles. EPA
4.
600/9-76-005, Research Triangle Park, NC, March
U.S. Environmental Protection Agency, Quality
for Air Pollution Measurement Systems, Vol
Specific Methods, EPA-600/4-77-027a, Research
May 1977.
1976.
Assurance Handbook
II - Ambient Air
Triangle Park, NC,
5. U.S. Environmental Protection Agency, Quality Assurance Handbook
for Air Pollution Measurement Systems, Vol III - Stationary Source
Specific Methods, EPA-600/4-77-027b, Research Triangle Park, NC,
August 1977.
6. The American Society of Quality Control, Glossary and Tables for
Statistical Quality Control, Jackson, J.E., and R.A. Freund, eds.,
Milwaukee, WI, 1973.
7. U.S. Environmental Protection Agency, Quality Assurance Research
Plan, FY 1978-81, EPA-600/8-77-008, Washington, DC, July 1977.
8. "Non-Clinical Laboratories Studies: Regulations for Good Labora-
tory Practice," Federal Register, December 22, 1978, pp. 59985-
60025.
3. (a) "Proposed Health Effects Test Standards for Toxic Substances
Control Act Test Rules," Federal Register, May 9, 1979,
p. 27334.
(b) "Good Laboratory Practice Standards for Health Effects,"
Federal Register, May 9, 1979, p. 26362.
(c) U.S. Environmental Protection Agency, Quality Assurance Re-
quirements for All EPA Extramural Projects Involving Environ-
mental Measurements, Administrator's Memorandum, June 14, 1979.
(d) U.S. Environmental Protection Agency, Environmental Protection
Agency (EPA) Quality Assurance Policy Statement, Administra-
tor's Memorandum, May 30, 1979.
R-l
-------
10. Inhorn, S.L., ed., Quality Assurances Practices in Health Labora-
tories, American Public Health Association, 1977.
11. U.S. Environmental Protection Agency, Quality Assurance Guidelines
for Biological Testing, EPA-600/4-78-043, Las Vegas, NV, August
1978.
12. U.S. Department of Health, Education, and Welfare, Guide for the
Care and Use of Laboratory Animals, US DHEW/PHS/NIH, DHEW Publica-
tion No. (NIH) 77-23, 1972.
13. Juran, J.M., P.M. Gryna, Jr., and R.S. Binghan, Jr., eds., Quality
Control Handbook, McGraw-Hill, 1951, p. 1780.
14. Bradley, M.O., and N.A. Sharkey, Nature. 266:724-25, 1977.
15. Scaringelli, P.P., B.F. Saltzman, and S.A. Try, Spectrophotometric
Determination of Atmospheric Sulfur Dioxide.
16. American Chemical Society, Reagent Chemicals, American Chemical
Society Publications, Washington, DC, 1968.
17. Federal Register. August 18, 1977, Section 3.1.2, p. 41783.
18. National Bureau of Standards, Catalog of MBS Standard Reference
Materials, NBS Special Publication 260, U.S. Department of Com-
merce, Washington, DC.
19. National Bureau of Standards, NBS Standard Reference Materials for
Environmental Research AnalysFs and Control, U.S. Department bT
Commerce.
20. World Health Organization, Biological Substances: International
Standards, Reference Preparations, and Reference Reagents, Geneva:
World Health Organization, 1977.
21. Measurement Principle and Procedure for the Measurement of Nitrogen
Dioxide in Atmosphere (Gas Phase Chemiluminescence). In: Title
40, Code of Federal Regulations, Part 50, Federal Register, Decem-
ber 2, 1976, p. 52688.
22. National Archives and Records Service. Traceability Requirements
for Calibration Gases, in: Title 40, Code of Federal Regulations,
Part 60.13 (d)(l).
R-2
-------
23. Scaringelli, E.P., A.E. O'Keefe, E. Rosenberg, and J.P. Bell.
Preparation of Known Concentrations of Gases and Vapors with Per-
meation Devices Calibrated Gravimetrically, Analytical Chemistry,
42 (8): July 1970.
24. Federal Register, December 14, 1977, p. 62971.
25. Ellis, E.C., Technical Assistance Document for the Chemilumines-
cence Measurement of Nitrogen Dioxide. EPA Environmental Monitor-
ing Series, EPA-600/4-75-003, Environmental Protection Agency,
Research Triangle Park, NC, December 1975.
26. American Society for Testing and Materials. Annual Books for ASTM
Standards, Philadelphia, Pennsylvania, annual publication.
27. U.S. Environmental Protection Agency, Handbook for Preparing Office
of Research and Development Reports, EPA 600/9-76-001, 1976.
28. Health Effects Research Laboratory, Health Effects Research Lab-
oratory Procedures for Publishing Office of Research and Develop-
ment Technical and Scientific Materials, Research Triangle Park,
NC, July 1977.
29. Journal of Scientific Instruments, 27:15-25, 1950.
30. Review of Scientific Instruments, 34 (9):1023-5, September 1963.
31. Messrs. L. Adams, Ltd., The Wright Dust Feed Mechanism, G.A. 180 -
Instructions of Use, Publication DF 170, Issue No. 3, 22 Minerva
Road, London, NW10GHS, England, November 1975.
32. Ahlberg, N.S. and J.W. Winchester, Atmospheric Environment, 12:
1631, 1978.
33. Mercer, T.T., Aerosol Technology in Hazard Evaluation, Academic
Press, New York, 1973.
34. Cooper, C., G. Langer, and J. Rosinski. Air Pollution Control
Assn. J. 18:57, 1979.
35. Katz, M., ed., Methods of Air Sampling and Analysis (2nd edition),
American Public Health Association, 1977.
36. Federal Register, August 13, 1975, p. 34024.
37. Von Lehmden, Darryl J., Suppression Effect on CO;? on FPD Total
Sulfur Air Analyzers and Recommended Correction Action, U.S. En-
vironmental Protection Agency, Research Triangle Park, NC, Novem-
ber 1977.
38. Matthews, R.D., R.F. Sawyer, and R.W. Schefer, Interferences In
Chemiluminescent Measurement of NO and N02 Emissions from Other
Nitrogen-Containing Compounds, Environmental Science and Technol-
ogy, 11 (12):1092-5, November 1977":
R-3
-------
39. Winer, A.M., J.W. Peters, J.P. Smith, and J.N. Pitts, Jr., Response
of Commercial Chemiluminescent NO-N02 Analyzers to Other Nitrogen
Containing Compounds, Environmental Science and Technology, 8:
1118, 1974.
40. Darley, E.F., K.A. Dettner, and E.R. Stephens, Analysis of Peroxy-
acetyl Nitrates by Gas Chromatography with Electron Capture Detec-
tion, Analytical Chemistry, 35 (4):589-91, April 1963.
41. Holdren, M.W., and R.A. Rasmussen, Moisture Anomaly in Analysis of
Peroxyacetyl Nitrate (PAN), Environmental Science and Technol-
ogy, 10 (2):185-7, February 1976.
42. Watanabe, I., and E.R. Stephens, Reexamination of Moisture Anomaly
in Analysis of Peroxyacetyl Nitrate, Environmental Science and
Technology, 12 (2): February 1978.
43. Stephens, E.R., in: Advances in Environmental Sciences and Techno-
logy, Pitts and Metcalf, eds., Volume I, Wiley, New York, 1969.
44. Bergman, F.J., and M.C. Sharp, Measurement of Atmospheric Sul-
fates: Evaluation of the Methylthymol Blue Method, Environmental
Monitoring Series, EPA-600/4-76-015, Environmental Protection
Agency, Research Triangle Park, NC, March 1976.
45. Woodward, G., in: Methods of Animal Experimentation, Vol. I,
Gay, W.I., ed., Academic Press, New York, 1965.
46. Latt, R.H., in: Handbook of Laboratory Animal Science, Vol, III,
Melby, E.C., Jr., and N.H. Altman, eds., CRC Press, Inc., Cleve-
land, Ohio, 1976.
R-4
-------
TECHNICAL REPORT DATA
(Please read Instructions on the reverse before completing)
1 ._fi_Ef O
-79-013
2.
4. TITLE AND SUBTITLE Guides for Qual 1 ty Assurance
in Environmental Health Research - Health Effects
Research Laboratory/RTP, N.C.
6. PERFORMING ORGANIZATION CODE
3. RECIPIENT'S ACCESSION NO.
7. AUTHOR(S)
C. E. Tatsch, Ph.D.
Ferris B. Benson
9. PERFORMING ORGANIZATION NAME AND ADDRESS
C. E. Tatsch, Ph.D.
Research Triangle Institute, RTP, N.C.
Ferris B. Benson, HERL/RTP, N.C.
8. PERFORMING ORGANIZATION REPORT NO.
10. PROGRAM EL
11. CONTRACT/GRANT NO.
68-022612
12. SPONSORING AGENCY NAME AND ADDRESS
U. S. Environmental Protection Agency
Office of Research and Development
Washington, D. C. 20460
13. TYPE OF REPORT AND PERIOD COVERED
Final
14. SPONSORING AGENCY CODE
EPA 600/11
15. SUPPLEMENTARY NOTES
16. ABSTRACT
This document provides conceptual guidelines for the development, im-
plementation and evaluation of research task quality assurance plans for
staff of the Health Effects Research Laboratory (HERL/RTP) of the U. S.
Environmental Protection Agency, Research Triangle Park, North Carolina.
It is designed to assist project officers in applying quality assurance
concepts to each phase of a research task, from the initial planning
through final report preparation. It is designed to assist the management
staff in evaluating these plans and their implementation,for intramural as
well as extramural tasks.
The guidelines describe the policy of HERL/RTP with respect to quality
assurance, the structure of the quality assurance organization, and out-
lines specific quality assurance responsibilities for various staff posi-
tions. They also analyse the research task with respect to the various
steps which project officers may take to ensure the highest possible data
quality commensurate with resource limitations. Following this discussion,
more specific guidelines relating to dosing activities and animal care are
provided.
17.
KEY WORDS AND DOCUMENT ANALYSIS
DESCRIPTORS
b.IDENTIFIERS/OPEN ENDED TERMS
c. COS AT I Field/Group
Quality Assurance
Management
Research Quality
Health Effects
Qua 1i ty
Quality control
Health Effects Laboratory
Quality Assurance
06 ? F
18. DISTRIBUTION STATEMENT
Release to Public
19. SECURITY CLASS (This Report)
Unclassified
21. NO. OF PAGES
121
20 SECURITY CLASS (Thispage)
Unclassified
22. PRICE
EPA Form 2220-1 (Rev. 4-77) PREVIOUS EDI TION i s OBSOLETE
------- |