United States
Environmental Protection
Agency
Health Effects Research
Laboratory
Research Triangle Park NC 27711
v>EPA
Research and Development
EPA-600/S2-81-054 May 1981
Project Summary
A Plan to Develop and
Implement a Quality
Assurance Program for the
Ames/Salmonella Test
N. Sexton, L Myers, and T. Hughes
This document discusses a plan to
develop and carry out a quality assur-
ance program for the Ames/Salmo-
nella test. The Ames test is in one of
the 13 categories of tests mentioned
in the Federal Insecticide, Fungicide,
and Rodenticide Act legislation and is
chosen as a model for presenting a QA
program for all test categories.
There are three main sections con-
tained herein. An introductory section
describes the laws requiring this test-
ing and the objectives of this quality
assurance plan. The second section
defines quality assurance terms,
describes applicable quality assurance
methods, and discusses the mailable
audit materials approach that was
chosen for these purposes. The third
section details the mailable audit
materials program and discusses
selection of chemicals, preparation of
samples, shipment of audit samples,
pilot-scale audits, scheduling, other
audit programs as models, data
gathering procedures, data analysis,
reporting of results, and costs.
This Project Summary was develop-
ed by EPA's Health Effects Research
Laboratory, Research Triangle Park.
NC. to announce key findings of the
research project that is fully docu-
mented in a separate report of the
same title (see Project Report ordering
information at back).
Introduction
The 1972 Federal Insecticide, Fungi-
cide, and Rodenticide Act (FIFRA) speci-
fied that no pesticide be registered or
reregistered that would cause unrea-
sonable risk to man or to the environ-
ment After subsequent promulgation
and public comments, tentative guide-
lines were issued containing 13 kinds of
tests grouped into three categories.
gene mutations, chromosomal aberra-
tions, and primary DNA damage.
Because of its wide usage, the Ames/
Salmonella test, a test for gene muta-
tion, was selected as a model for the
preparation of a quality assurance (QA)
plan.
Quality Assurance Strategies
The stated objective of this QA plan is
to present a means of ensuring that
laboratories submitting data to the
Office of Pesticide Programs concerning
pesticide registration can produce
information that is reliable and valid
After consideration and discussion of
the various means of implementing a
QA plan, a mailable audit materials
approach was selected because:
1. Many laboratories are submitting
data, therefore, onsite evaluations
would be costly and time
consuming.
-------
2 Neither systems audits nor split-
sample schemes would provide an
evaluation of a laboratory's accu-
racy. In the case of split-sample
schemes, where there were large
variances among laboratories
there may be no useful informa-
tion obtainedforthistask Further,
there is no practical way to insert
blind samples into a laboratory's
analysis stream.
3. There is no widely accepted
standard method for all aspects of
performing the Ames/Salmonella
test Parts of the systems audit are
geared toward comparing an
individual laboratory's perform-
ance to well-established stand-
ards and protocols This lack of a
validated protocol would also
preclude a formal certification
program
The Mailable Audits Program
Factors considered m the mailable
audits program were selection of
chemicals, preparation of samples,
shipment of samples, pilot-scale audits,
data gathering procedures, analysis of
data, schedule of audits, reporting of
results, and costs These topics are
discussed in subsequent sections
Selection of Chemicals
Some chemicals giving a positive
(mutagenic) response and some chem-
icals giving a negative response should
be included as audit materials. The
physical characteristics of the chemi-
cals should be known and should be
compatible with a mailable materials
program. A proposed set of audit
materials would include-
• A direct-acting (no metabolic acti-
vation required) compound in a
solvent
• An indirect-acting compound in a
solvent
• A mixture of direct-acting com-
pounds in solvent(s) for activating
only frameshift or only base-pair
sensitive bacterial strains
• A mixture of direct-acting com-
pounds that elicit a positive
response in all five bacterial
strains
• A mixture of pure compounds that
require metabolic activation
• A complex mixture/environ-
mental sample
• A mixture that requires chemical
fractionation to show mutagen-
icity.
• Nonmutagemc materials
The importance of preaudit verifica-
tion of audit materials was emphasized.
Preparation of Samples
American Society of Testing and
Materials (ASTM) methods could be
used to ensure the homogeneity of solid
samples Choice of solvents would be
dependent on solubility of the test
materials, stability, and toxicity of the
solvents to bacteria Laboratories
preparing samples should adhere to the
Guides for Quality Assurance in Envi-
ronmental Health Research and the
proposed "good laboratory practices"
(Federal Register, 11/19/76, 1/7/77,
and 1/28/77)
Shipment of Audit Samples
Title 49, Code of Federal Regulations,
covers all modes of transport of mate-
rials within the United States, and Parts
100-177 deal with the transport of
hazardous materials In view of Title 49
regulations, three modes of transporta-
tion were considered: Federal Express,
United Parcel Service (UPS), and U.S.
Mail Details of the additional regula-
tions covering each of the three modes
of transportation, as well as costs, are
discussed in an appendix
Pilot-Scale Audits
A pilot-scale audit of seven labora-
tories was proposed to test clarity of
instructions, stability of materials
during transport, time required for ship-
ment, applicability of data entry forms,
and schemes for data interpretation A
consensus among eight laboratories
(the verification laboratory plus seven
others) would constitute a statistically
significant probability (at the 1 percent
level) that the answers were, in fact,
correct and not purely chance occur-
rences.
Scheduling of Audits
It was proposed that laboratone
wishing to enter the QA prograr
receive samples when they mitiall
submitted data or when they asked t
participate Otherwise, all participatin
laboratories would receive a set of audi
samples each year For laboratories tha
did not show acceptable performance,
second, then a third set of audit sample
would be provided. After failing thre
attempts, a laboratory would have t
wait one year before reapplying fo
evaluation.
Data Gathering Procedures
Of the several data systems for bio
assay information that are currentl
available, the system designed and n
current use at the Environments
Protection Agency's Health Effect
Research Laboratory in Researcl
Triangle Park (HERL-RTP) was selectei
as most applicable for this QA program
Data entry forms and instructions an
presented as an appendix. A supple
mentary questionnaire, also presentee
in the appendixes, was recommende<
for initial evaluation of a laboratory.
Data Analysis
The HERL-RTP pesticide QA prograrr
served as a model In this QA prograrr
for the Ames test, five major points wil
be used for evaluation
1 The mutagenic/nonmutagenic
(+/-) determination by the testing
laboratory as compared with the
expected results. The expectec
results would be based on pub-
lished response values for the
chemicals chosen as audit mate-
rials and on the preliminary anc
pilot audit analyses.
2 The mutagenic/nonmutagenic
(+/-) determination by the testing
laboratory as compared with the
+/- determination by a reviewer ol
the raw data. Mathematical
models have been developed that
consider all raw data (plate counts
on controls and samples), factor in
toxicity, and give an assessment of
the mutagenic or nonmutagenic
nature of the sample.
3 The mutagenic/nonmutagenic
(+/-) determination by the testing
laboratory as compared to the +/-
1
-------
determination by other laborator-
ies submitting data
4 Positive and negative control
values as reported by a testing
laboratory and compared with the
appropriate normal ranges
Normal ranges for controls will be
deter mined from values published
in a consensus report by NIEHS,
from valuesdetermmed in the pre-
liminary tests and pilot audits
program, and from a statistical
analysis of the values reported by
all laboratories submitting data
5 Ranking by the testing laboratory
of the set of audit samples, from
the most mutagemc to the least
mutagemc within eachstram/S-9
combination of bacteria, with and
without metabolic activation,
compared to a similar ranking
according to expected results.
Again, expected results will be
based on determination from
preliminary and pilot studies and
values reported by all laboratories
submitting data.
It is proposed that the following
weighting scheme be applied for evalu-
ation of a laboratory's capabilities in the
Ames/Salmonella test.
Criteria
1
2
3
4
5
Points
50
25
5
10
10
100
The data interpretation model men-
tioned in point 1 is an extension of the
Poisson model developed by Stead,
Hasselblad, Creason, and Claxton of
HERL-RTP. These models are described
in an appendix
Reporting of Results
The QA program would generate five
types of reports. First, a report of a
laboratory's raw data, as encoded for
interpretation, would be mailed to that
laboratory. Then an executive summary
of a laboratory's performance, based on
the previously mentioned evaluative
criteria, would be prepared for the
Office of Pesticide Programs. Third, an
annual report on all aspects of the
program would be produced. A fourth
report would be the evaluative report
mailed back to submitting laboratories
Fifth, it is anticipated that results of
such a QA program would be of general
interest to the scientific community and
could be published in the appropriate
technical and scientific journals
Costs
Costs were considered for prelimi-
nary . planning ($27,000), the pilot
program (2,180 man-hours; sample
preparation, sample testing, small
scale audits), and the full-scale audit
program (1,100 man-hours plus $15,000
computer time, packaging and mailing,
data compilation and evaluation, report-
ing, mailing second and third sets). For
270 sample sets and given $3000 per
man-hour, the cost per sample set is
$520.00.
N Sexton, L Myers, and T Hughes are with the Research Triangle Institute,
Research Triangle Park, NC 27709.
Larry Claxton is the EPA Project Officer (see below).
The complete report, entitled "A Plan to Develop and Implement a Quality
Assurance Program for the /Imes/Salmonella Test," (Order No. PB 81-182 065;
Cost. $9.50, subject to change) will be available only from.
National Technical Information Service
5285 Port Royal Road
Springfield, VA 22161
Telephone 703-487-4650
The EPA Project Officer can be contacted at:
Health Effects Research Laboratory
U S. Environmental Protection Agency
Research Triangle Park, NC 27711
! US GOVERNMENT PRINTING OFFICE. 19(1.757-012/7094
-------
United States
Environmental Protection
Agency
Center for Environmental Research
Information
Cincinnati OH 45268
Postage and
Fees Paid
Environmental
Protection
Agency
EPA 335
Official Business
Penalty for Private Use $300
".
------- |