United States Environmental Protection Agency Health Effects Research Laboratory Research Triangle Park NC 27711 v>EPA Research and Development EPA-600/S2-81-054 May 1981 Project Summary A Plan to Develop and Implement a Quality Assurance Program for the Ames/Salmonella Test N. Sexton, L Myers, and T. Hughes This document discusses a plan to develop and carry out a quality assur- ance program for the Ames/Salmo- nella test. The Ames test is in one of the 13 categories of tests mentioned in the Federal Insecticide, Fungicide, and Rodenticide Act legislation and is chosen as a model for presenting a QA program for all test categories. There are three main sections con- tained herein. An introductory section describes the laws requiring this test- ing and the objectives of this quality assurance plan. The second section defines quality assurance terms, describes applicable quality assurance methods, and discusses the mailable audit materials approach that was chosen for these purposes. The third section details the mailable audit materials program and discusses selection of chemicals, preparation of samples, shipment of audit samples, pilot-scale audits, scheduling, other audit programs as models, data gathering procedures, data analysis, reporting of results, and costs. This Project Summary was develop- ed by EPA's Health Effects Research Laboratory, Research Triangle Park. NC. to announce key findings of the research project that is fully docu- mented in a separate report of the same title (see Project Report ordering information at back). Introduction The 1972 Federal Insecticide, Fungi- cide, and Rodenticide Act (FIFRA) speci- fied that no pesticide be registered or reregistered that would cause unrea- sonable risk to man or to the environ- ment After subsequent promulgation and public comments, tentative guide- lines were issued containing 13 kinds of tests grouped into three categories. gene mutations, chromosomal aberra- tions, and primary DNA damage. Because of its wide usage, the Ames/ Salmonella test, a test for gene muta- tion, was selected as a model for the preparation of a quality assurance (QA) plan. Quality Assurance Strategies The stated objective of this QA plan is to present a means of ensuring that laboratories submitting data to the Office of Pesticide Programs concerning pesticide registration can produce information that is reliable and valid After consideration and discussion of the various means of implementing a QA plan, a mailable audit materials approach was selected because: 1. Many laboratories are submitting data, therefore, onsite evaluations would be costly and time consuming. ------- 2 Neither systems audits nor split- sample schemes would provide an evaluation of a laboratory's accu- racy. In the case of split-sample schemes, where there were large variances among laboratories there may be no useful informa- tion obtainedforthistask Further, there is no practical way to insert blind samples into a laboratory's analysis stream. 3. There is no widely accepted standard method for all aspects of performing the Ames/Salmonella test Parts of the systems audit are geared toward comparing an individual laboratory's perform- ance to well-established stand- ards and protocols This lack of a validated protocol would also preclude a formal certification program The Mailable Audits Program Factors considered m the mailable audits program were selection of chemicals, preparation of samples, shipment of samples, pilot-scale audits, data gathering procedures, analysis of data, schedule of audits, reporting of results, and costs These topics are discussed in subsequent sections Selection of Chemicals Some chemicals giving a positive (mutagenic) response and some chem- icals giving a negative response should be included as audit materials. The physical characteristics of the chemi- cals should be known and should be compatible with a mailable materials program. A proposed set of audit materials would include- • A direct-acting (no metabolic acti- vation required) compound in a solvent • An indirect-acting compound in a solvent • A mixture of direct-acting com- pounds in solvent(s) for activating only frameshift or only base-pair sensitive bacterial strains • A mixture of direct-acting com- pounds that elicit a positive response in all five bacterial strains • A mixture of pure compounds that require metabolic activation • A complex mixture/environ- mental sample • A mixture that requires chemical fractionation to show mutagen- icity. • Nonmutagemc materials The importance of preaudit verifica- tion of audit materials was emphasized. Preparation of Samples American Society of Testing and Materials (ASTM) methods could be used to ensure the homogeneity of solid samples Choice of solvents would be dependent on solubility of the test materials, stability, and toxicity of the solvents to bacteria Laboratories preparing samples should adhere to the Guides for Quality Assurance in Envi- ronmental Health Research and the proposed "good laboratory practices" (Federal Register, 11/19/76, 1/7/77, and 1/28/77) Shipment of Audit Samples Title 49, Code of Federal Regulations, covers all modes of transport of mate- rials within the United States, and Parts 100-177 deal with the transport of hazardous materials In view of Title 49 regulations, three modes of transporta- tion were considered: Federal Express, United Parcel Service (UPS), and U.S. Mail Details of the additional regula- tions covering each of the three modes of transportation, as well as costs, are discussed in an appendix Pilot-Scale Audits A pilot-scale audit of seven labora- tories was proposed to test clarity of instructions, stability of materials during transport, time required for ship- ment, applicability of data entry forms, and schemes for data interpretation A consensus among eight laboratories (the verification laboratory plus seven others) would constitute a statistically significant probability (at the 1 percent level) that the answers were, in fact, correct and not purely chance occur- rences. Scheduling of Audits It was proposed that laboratone wishing to enter the QA prograr receive samples when they mitiall submitted data or when they asked t participate Otherwise, all participatin laboratories would receive a set of audi samples each year For laboratories tha did not show acceptable performance, second, then a third set of audit sample would be provided. After failing thre attempts, a laboratory would have t wait one year before reapplying fo evaluation. Data Gathering Procedures Of the several data systems for bio assay information that are currentl available, the system designed and n current use at the Environments Protection Agency's Health Effect Research Laboratory in Researcl Triangle Park (HERL-RTP) was selectei as most applicable for this QA program Data entry forms and instructions an presented as an appendix. A supple mentary questionnaire, also presentee in the appendixes, was recommende< for initial evaluation of a laboratory. Data Analysis The HERL-RTP pesticide QA prograrr served as a model In this QA prograrr for the Ames test, five major points wil be used for evaluation 1 The mutagenic/nonmutagenic (+/-) determination by the testing laboratory as compared with the expected results. The expectec results would be based on pub- lished response values for the chemicals chosen as audit mate- rials and on the preliminary anc pilot audit analyses. 2 The mutagenic/nonmutagenic (+/-) determination by the testing laboratory as compared with the +/- determination by a reviewer ol the raw data. Mathematical models have been developed that consider all raw data (plate counts on controls and samples), factor in toxicity, and give an assessment of the mutagenic or nonmutagenic nature of the sample. 3 The mutagenic/nonmutagenic (+/-) determination by the testing laboratory as compared to the +/- 1 ------- determination by other laborator- ies submitting data 4 Positive and negative control values as reported by a testing laboratory and compared with the appropriate normal ranges Normal ranges for controls will be deter mined from values published in a consensus report by NIEHS, from valuesdetermmed in the pre- liminary tests and pilot audits program, and from a statistical analysis of the values reported by all laboratories submitting data 5 Ranking by the testing laboratory of the set of audit samples, from the most mutagemc to the least mutagemc within eachstram/S-9 combination of bacteria, with and without metabolic activation, compared to a similar ranking according to expected results. Again, expected results will be based on determination from preliminary and pilot studies and values reported by all laboratories submitting data. It is proposed that the following weighting scheme be applied for evalu- ation of a laboratory's capabilities in the Ames/Salmonella test. Criteria 1 2 3 4 5 Points 50 25 5 10 10 100 The data interpretation model men- tioned in point 1 is an extension of the Poisson model developed by Stead, Hasselblad, Creason, and Claxton of HERL-RTP. These models are described in an appendix Reporting of Results The QA program would generate five types of reports. First, a report of a laboratory's raw data, as encoded for interpretation, would be mailed to that laboratory. Then an executive summary of a laboratory's performance, based on the previously mentioned evaluative criteria, would be prepared for the Office of Pesticide Programs. Third, an annual report on all aspects of the program would be produced. A fourth report would be the evaluative report mailed back to submitting laboratories Fifth, it is anticipated that results of such a QA program would be of general interest to the scientific community and could be published in the appropriate technical and scientific journals Costs Costs were considered for prelimi- nary . planning ($27,000), the pilot program (2,180 man-hours; sample preparation, sample testing, small scale audits), and the full-scale audit program (1,100 man-hours plus $15,000 computer time, packaging and mailing, data compilation and evaluation, report- ing, mailing second and third sets). For 270 sample sets and given $3000 per man-hour, the cost per sample set is $520.00. N Sexton, L Myers, and T Hughes are with the Research Triangle Institute, Research Triangle Park, NC 27709. Larry Claxton is the EPA Project Officer (see below). The complete report, entitled "A Plan to Develop and Implement a Quality Assurance Program for the /Imes/Salmonella Test," (Order No. PB 81-182 065; Cost. $9.50, subject to change) will be available only from. National Technical Information Service 5285 Port Royal Road Springfield, VA 22161 Telephone 703-487-4650 The EPA Project Officer can be contacted at: Health Effects Research Laboratory U S. Environmental Protection Agency Research Triangle Park, NC 27711 ! US GOVERNMENT PRINTING OFFICE. 19(1.757-012/7094 ------- United States Environmental Protection Agency Center for Environmental Research Information Cincinnati OH 45268 Postage and Fees Paid Environmental Protection Agency EPA 335 Official Business Penalty for Private Use $300 ". ------- |