December 15th, 2015 EPA-HSRB-15-03 Thomas A. Burke, Ph.D., MPH EPA Science Advisor Office of the Science Advisor 1200 Pennsylvania Avenue, NW Washington, DC 20460 Subject: October 19-20, 2015 EPA Human Studies Review Board Meeting Report Dear Dr. Burke, The United States Environmental Protection Agency (EPA or Agency) requested that the Human Studies Review Board (HSRB) provide scientific and ethics reviews of two items: • A Completed Study from the U.S. Department of Agriculture Describing Laboratory Evaluation of Bite Protection from Repellent-Impregnated Clothing for the United States Military • Testing of S.C. Johnson & Son, Inc. Personal Tick Repellent Products to Support Use of the EPA Repellency Awareness Graphic The Board's key responses to the charge questions are detailed in the enclosed final report of the meeting. Signed, Liza Dawson, PhD Chair EPA Human Studies Review Board ------- INTRODUCTION On October 19-20, 2015, the United States Environmental Protection Agency's (EPA or Agency) Human Studies Review Board (HSRB or Board) met to address the scientific and ethical charge questions related to two items: • A Completed Study from the U.S. Department of Agriculture Describing Laboratory Evaluation of Bite Protection from Repellent-Impregnated Clothing for the United States Military • Testing of S.C. Johnson & Son, Inc. Personal Tick Repellent Products to Support Use of the EPA Repellency Awareness Graphic REVIEW PROCESS The Board conducted a public meeting on October 19-20, 2015. Advance notice of the meeting was published in the Federal Register as "Human Studies Review Board; Notification of a Public Meeting" (EPA-HQ-ORD-2015-0588). Following welcoming remarks from Agency officials, the Board began its review of the first item. The Board heard presentations from EPA for each agenda items in sequence, consisting of the Agency's review of scientific and ethical aspects of the two studies. This Final Report of the meeting describes the HSRB's discussion, recommendations, rationale and consensus in response to each charge question for each of these items. For each agenda item, Agency staff first presented their review of the science and the Board asked the Agency presenters clarifying questions. The staff then described their review of the ethical aspects and the Board asked clarifying questions about those. The HSRB solicited public comments and next asked Agency staff to read the Charge Questions for the publication under consideration. The Board discussed the science questions first and then the ethics question. The Chair then called for a vote to confirm concurrence on a summary statement in response to each charge question. For their evaluation and discussion, the Board considered materials presented at the meeting, oral comments, related materials and documents provided by the study sponsors, the Agency's science and ethics reviews of the studies, oral responses from a study sponsor and protocol team, and public comments made at the meeting. A comprehensive list of background documents is available online at http://www.epa.gov/osa/human-studies-review-board. ------- CHARGE TO THE BOARD AND BOARD RESPONSE HSRB review of a Completed Study from the U.S. Department of Agriculture Describing Laboratory Evaluation of Bite Protection from Repellent-Impregnated Clothing for the United States Military Charge to the Board: Is the research reported in the completed study sufficiently sound, from a scientific perspective, to be used to evaluate the bite protection level of etofenprox-treated military clothing? Board Response: The Board concluded that the research reported in the USDA report of the completed study is sufficiently sound from a scientific perspective to be used for evaluation of etofenprox-treated military clothing HSRB Detailed Recommendations and Rationale: The protocol for the study Laboratory Evaluation of Bite Protection from Repellent- Impregnated Clothing for the United States Military was reviewed by HSRB at the April 2014 meeting. The study was completed in July 2015 and the final report of the completed study has been reviewed and approved by EPA. The original HSRB review of the protocol (April 2014) concluded that the protocol would "likely generate scientifically reliable data for estimation of bite protection efficacy of impregnated clothing, if improvements in the overall study design enumerated by EPA (Sweeney & Sherman, 2014) and the Board are adequately considered." The protocol was revised according to EPA and HSRB recommendations prior to the study and revisions were approved by IRB. The study was completed as planned following the revised protocol. Protocol deviations and amendments There were documented deviations from the protocol during the completion of the study but none of them, in the view of the Board, had a significant adverse effect on the scientific validity of the study. There were minor changes in informed consent procedures. Two amendments to the protocol related to low rates of mosquito bite through. It was postulated by the study team that low bite through in one case was related to improper laboratory procedures in maintaining the mosquitoes; the batch of mosquitoes used did not exhibit sufficient biting pressure on the control (untreated) fabric. For this subject, repeat testing was undertaken with a new batch of mosquitoes. For a second subject, there was low bite through for all tests, and the data from that subject were not included in the study analysis; the protocol was amended, with IRB approval, to recruit one additional subject for testing to replace this subject. A further amendment allowed for the inclusion of an additional set of test sleeves with 75 washes (increased from the protocol which originally specified a maximum of 50 washes). The protocol team had anticipated that 50 washes would exhaust the repellency of the treated fabric, but in fact this was not the case, leading to the need to test a higher number of washes. Another protocol deviation was the concentration of etofenprox on the treated fabric. The protocol specified that the treatment ratio of product to fabric would be 1.0% (weight/weight), ------- but the actual ratio was 0.9%. This was deemed scientifically acceptable, since the change in amount of product did not affect the reliability of the study results. One additional deviation was related to the statistical data analysis method (discussed in following section). Statistical analysis There was extensive discussion of the statistical analysis of the data from this study. The protocol had left unspecified what statistical analysis would be used; at the previous HSRB review of the protocol, the Board had recommended that a generalized linear mixed model (GLiM) procedure be used to estimate confidence intervals (CIs). In the GLiM procedure, the subject to subject variation can be treated as a random effect or fixed effect. Dr. Gbur and Dr. Fernandez recommended using the GLiM approach with the subject level treated as a random effect, which is more appropriate when the aim is to generalize the results to a broader population. The study team chose to use the t-distribution for estimating CIs, which is a simpler method and does not support generalizations beyond the study population itself. Dr. Fernandez ran a simulation comparing the GLiM and t-test procedures using parameters based on estimates from the data in the study report, and found that the CIs are wider using the GLiM approach compared to the t-distribution approach, but the overall conclusions regarding repellency are not significantly different. A copy of Dr. Fernandez' simulation results are provided in an appendix to this report. There was some discussion of the fact that differing version of statistical software can produce different results, and that the most recent versions of software will be most effective in conducting these analyses. Board members recommended that the GLiM approach be used in this study and in future studies of this type, but the analysis used, although not ideal, was sufficient to allow the data to be used for decision-making in this case. Statisticians on the board also pointed out that the GLiM approach can be used to support further objectives other than simply analyzing the mean and CI for each test product. For example, the longevity of the product (based on number of washes) could be estimated, or other secondary objectives. Board recommendations for methodologic considerations in future studies A number of issues arose in this study that the Board recommended be addressed in future protocols. First, specific criteria for adequate biting pressure for a specific assay should be outlined in the protocol, such that any judgment about inadequate biting pressure is made on objective pre-specified criteria. Along the same lines, specific conditions under which subjects may be replaced, or data not included in final analysis, should be outlined in advance to avoid subjective decisions or the possible introduction of bias into study procedures and data collection. In addition, the use of GLiM statistical tests for this type of study is recommended by all the statisticians on the Board. Use of t-tests for a study protocol of this type of complexity is not considered adequate for statistical analysis. Because in this particular case, Board members were able to ascertain that the study conclusions would not differ significantly with a different analysis ------- (see Appendix I), it is deemed acceptable to use the data as presented in the study report. However, as a general matter, a more robust and reliable analysis that more closely reflects the manner in which the study was conducted should be used and the statistical plan should be detailed in advance in the protocol document. Ethics Charge to the Board: Does available information support a determination that the Completed Study from the U.S. Department of Agriculture Describing Laboratory Evaluation of Bite Protection from Repellent-Impregnated Clothing for the United States Military was conducted in substantial compliance with subparts K and L of 40 CFR Part 26? Board Response The information provided supports a determination that the studies were conducted in substantial compliance with subparts K and L of 40 CFR Part 26. HSRB Detailed Recommendations and Rationale: The purpose of this study was to determine the bite protection level of the etofenprox-treated U.S. military Fire Resistant Army Combat Uniforms (FRACUs). The protocol for this study and subsequent amendments were approved by the overseeing institutional review board, the Western Institutional Review Board (WIRE). Because this study was initiated after April 7, 2006, prior submission of the protocol and supporting materials to EPA was required by 40 CFR §26.1125. The HSRB discussed the protocol at its April 9, 2014 meeting and determined that with revisions as suggested by the EPA and the HSRB, the study would meet the applicable requirements of 40 CFR Part 26, Subparts K and L. Adult subjects were recruited from the general population in Gainesville, Florida. Children (under the age of 18) and pregnant or lactating women were excluded. Consent was obtained before participation. Subjects received $250 for completion of testing with the full set of 10 pairs of sleeves. Several minor deviations from the protocol were reported, but did not negatively affect participants' rights or their health or safety. CHARGE TO THE BOARD AND BOARD RESPONSE HSRB review of the protocol for Testing of S.C. Johnson & Son, Inc. Personal Tick Repellent Products to Support Use of the EPA Repellency Awareness Graphic Charge to the Board Is the protocol Testing of S.C. Johnson & Son, Inc. Personal Tick Repellent Products to Support Use of the EPA Repellency Awareness Graphic likely to generate scientifically reliable data, useful for estimating the complete protection time of various EPA-registered S.C. Johnson skin-applied repellents in laboratory studies using three species of tick populations? ------- Board Response: The Board concluded that the protocol, if modified according to Agency and HSRB recommendations, is likely to generate scientifically reliable data, useful for estimating the complete protection time as defined in the protocol, of various EPA-registered S.C. Johnson skin-applied repellents. HSRB Detailed Recommendations and Rationale The objective of the i2LResearch protocol is clearly stated and follows EPA's Product Performance Test Guidelines OPPTS 810.3700: Insect Repellents to be applied to Human Skin. The field tests are designed to provide data to support the EPA's Repellency Awareness Graphic. The objective of this study is to establish the complete protection time (CPT) of up to 18 EPA- registered repellent products (test substances) on human subjects in a laboratory setting against three species of ticks (Pathogen free Amblyomma americamim, Dermacentor variabilis, and Ixodes scapularis). The First Confirmed Crossing (FCC), defined as a tick crossing into the treated area confirmed by another crossing within two 15-minute test periods (or within 30 minutes) will determine repellent failure for each tick species. The CPT for ticks to be used on the insect repellency graphic will be from the tick species with the lowest CPT value. The product dose will be standardized for product comparisons, so the study has no dosimetry phase to determine the typical consumer dose of these products when applied by human subjects participating in the study. Twelve subjects (ten test subjects and two alternates) will be selected from a pool of subjects to be recruited in the Baltimore Metro area. The number of test subjects for a tick repellency efficacy study should strike a balance among three critical and competing criteria: a) Minimization of potential hazard to test subjects, where fewer subjects is better; b) Statistical robustness, where more subjects results in greater precision of numeric estimates; c) Consistency with previous repellent efficacy studies. Current and recent practice is to utilize ten_subjects for each test site/test product. Several scientific aspects of the protocol require clarification and possible revision. The most significant scientific issue was the question of positive and negative control groups. For each subject enrolled, one arm is treated with repellent and the other is not. The untreated arm serves as a negative control; both arms are exposed to ticks. Thus the only question about this negative control is whether the test conditions (placing of the tick on the arm and measurement of subsequent behavior) are identical and unbiased under both conditions, treated and untreated. HSRB members expressed some concern about whether there was an objective measurement of tick viability and whether the removal of the tick from one arm and placement on the other arm might cause any change to the tick's behavior. In discussion at the meeting, EPA staff and members of the protocol team described the sequential placement of the tick on the untreated and treated arm as a standard procedure for these repellency studies. A second issue discussed was whether a positive control was needed, namely, a subject tested using identical test procedures, treated with a known positive control repellent. Agency representatives have described that this is not standard practice for these repellency tests and that there is no particular product typically used as a positive control; staff also expressed that the additional cost and burden of the positive control is not warranted. At the same time, it was ------- acknowledged that a positive control allows for data to be compared across sites and different test days. Members of the HSRB pointed out that the scientific rationale for lack of positive control, and the adequacy of the untreated arm as a negative control, as well as other aspects of study design, should be supported by peer-reviewed published literature in the field, and justification for the study design should be included in the study protocol. For example, a number of different study designs are described in published literature, and each appears to give somewhat different results.1 The Board specifically recommends that the strengths and limitations of the chosen study design be discussed in relation to alternative designs. These details will help future reviewers understand the scientific rationale for the protocol. The Board noted that similar concerns arise in the statistical analysis; statistical methods also evolve and older methods may not be suitable. For questions related to study design and statistical analyses, methods used in prior research may not be considered state-of-the-art for current studies. Providing more detailed rationale for the design and analysis chosen will help researchers to continually strive to improve on study designs as the research field evolves. There was extensive discussion of the pregnancy test that is required as part of the study protocol, since pregnancy is an exclusion criterion in the study. The study team has included provisions for women to take the pregnancy test up to 48 hours before the first test day, so that the procedures on the test day, which are already very lengthy, can be somewhat shorter in order to reduce burden on the participants. While the majority of the board felt this was acceptable, a board member questioned whether the pregnancy testing should be carried out the same day to eliminate the possibility of an early pregnancy (less than 48 hours) that is not detected. In an effort to find a compromise solution, it was proposed that women be given an option to take the test 48 hours in advance, or on the test day, and if they chose the early test, that they be counseled about avoiding sexual activity that could lead to pregnancy. However it should be noted that the majority of the board expressed that the 48 time window was acceptable, particularly given that the test products are registered and marketed to the public with no regulatory or safety concerns regarding use in pregnancy, and that the likelihood of an unintended pregnancy in a very short time period is low. HSRB members commented that more information should be provided on training of medical personnel for the study, who would be available in case of any adverse reactions or medical needs of the study participants related to exposure to the repellent products. There was also concern that three 15-minute breaks in an 18-hour day may not be sufficient. There was a lengthy discussion about whether the study population is representative of the population that uses insect repellency products. The study team justified their choice of study population based on Neilson survey data about use of personal repellent products, and the team's intention is to include approximately 10% Hispanic study participants to match the demographics of the US population using these products. However the team is not planning to use Spanish language consent materials or have an interpreter available for those individuals who do not speak English. The effort to include Hispanic participants will consist of using Spanish-language 1 Dautle H, Dippel C., Wekhausen A, and Diller R. Efficacy testing of several Ixodes ricinus tick repellents: Different results with different assays. Ticks and Tick borne Diseases 4(2013) 256-263; Krober T, Bourquin M and Guerin PM. A standardised in vivo and in vitro test method for evaluating tick repellents. Pesticide Biochemistry and Physiology 107 (2013) 160-168. ------- advertising, using an on-line Spanish newspaper and enrolling individuals who are bilingual, Spanish and English speaking. Members of the HSRB commented that a more direct approach would be simply to recruit Hispanic individuals who self-identify as such, rather than using language as an indirect criterion. There was discussion about whether lack of Spanish language materials was ethically problematic. Diverse views were voiced on this point. Some members of the board maintained that Spanish language translation ought to be required, while others stated that in a study offering no direct benefits to participants, concerns about unfair exclusion of non-English speakers are significantly reduced. The Board also recognized that in a small study of this type there will be almost no possibility of determining any statistically significant differences in population subgroups. In the end the Board did not recommend a specific change to study procedures on this point. While the protocol was also well developed from a statistical point of view, the Board recommended that a random number generator approach be used instead of the coin toss approach. In addition, clarification was requested regarding the methodology used for calculation of sample size determinations. It was suggested that additional detail be provided about the table and its accompanying text and data presentation. It was also recommended that the statistical analysis plan be more detailed and that specific plans for analyzing treatment versus control conditions, examining gender differences, and assessing potential confounding effects when a subject is tested using more than one product. Ethics Charge to the Board Is the research described in the protocol "Testing of S.C. Johnson Personal Tick Repellent Products to Support Their Use of the EPA Repellency Awareness Graphic" likely to meet the applicable requirements of 40 CFR part 26, subparts K and L? HSRB Recommendation The study, when modified as recommended by the HSRB in this report, is likely to meet the applicable requirements of 40 CFR part 26, subparts K and L. HSRB Detailed Recommendations and Rationale This is a protocol for third-party research involving intentional exposure of human subjects to a pesticide. As noted above, the study involves the use of consumer products for tick repellency for defined periods of time with exposure to three tick species for the purpose of determining the duration of protection of up to eighteen EPA-registered repellent products from S.C. Johnson & Son, Inc. ('Johnson'). The data is intended to be used to support these products' use of the EPA repellency awareness graphic for labels in order to better inform consumers about the relative protection times of these products. The following aspects of the research plan were deemed acceptable by the HSRB: • 40 CFR part 26, subpart K requires that research involving human subjects must be reviewed and approved by an institutional review board (IRB). This study has been ------- reviewed and conditionally approved by the Schulman Associates 1KB (SAIRB), with SAIRB's final approval contingent upon Johnson obtaining HSRB review and approval. • 40 CFR part 26, subpart L requires that EPA must not rely on data from any research involving intentional exposure of any human subject who is a pregnant woman (and therefore her fetus), a nursing woman, or a child. The protocol, informed consent document, and recruitment script indicate that participants must be 18-55 years of age and that female subjects who are pregnant or breastfeeding will be excluded from study participation. The HSRB discussed changes to the pregnancy testing process for this study, as noted above. • Risks to subjects are appropriately minimized through the following mechanisms: inclusion and exclusion criteria for potential participants that minimize physical risks related to exposure to ticks, the test products, or latex;2 training provided to potential participants prior to testing day; access to first-aid treatment in the event of a research- related injury; a 24-hour telephone number for participants to contact the Study Director in the event of a research-related reaction after participants have left the testing facility; a required break of at least two calendar days between testing days, for participants who choose to participate in more than one testing day; and the use of supporting materials on which participants can rest their arms during testing procedures. • The informed consent process includes the use of open-ended prompts to assess potential participants' understanding of the study's purpose and what study participation entails. Although the study materials provided to the HSRB did not include these open-ended prompts, the EPA Ethics Reviewer presented these prompts at the October 20, 2015 meeting. Before the research begins, study materials should be revised to address the comments from EPA and the following concerns of the HSRB: • To minimize the risk of exposure to tick-borne disease, the study protocol indicates that the study will use only laboratory-reared ticks that are not known to harbor any pathogens. Confirmation from the supplier will be obtained. The HSRB recommends that the name of the tick supplier be included in the study protocol. • Since meals will be provided to participants on testing day, HSRB recommends that during training sessions, study personnel should ask potential participants about any dietary accommodations that must be met (for example, food allergies or religious dietary law) so that all participants are able to eat during the lengthy testing day. 2 Some HSRB members question the appropriateness of excluding non-English speakers from study participation with no clear scientific rationale for this exclusion, as well as the use of Spanish-language recruitment materials to recruit bilingual subjects. However, no formal recommendations arose from this discussion. ------- HSRB recommends that study personnel should either provide entertainment (such as a small television and DVD player) or encourage participants to bring their own on testing day in order to minimize participant anxiety and potential boredom during testing procedures. In accordance with 40 CFR part 26, subpart K, the HSRB reminds Johnson that study materials that have been revised in light of EPA and HSRB recommendations, as well as related correspondence with SAIRB, must be submitted to EPA. Further, Spanish language recruitment materials should be certified by a qualified translator or translation service, and both Spanish language materials and the accompanying certification should be submitted to EPA. Finally as noted above in the section on scientific review, the issues of pregnancy testing was discussed. The board's deliberations resulted in the following options: • Female participants should undergo pregnancy testing at the beginning of each testing day or at the training session prior to testing day, provided that this training session takes place within 48 hours of the testing period. If female participants choose to undergo pregnancy testing at this training session, counseling should be provided about the need to avoid sexual activity leading to pregnancy or use appropriate contraception. CONCLUSION This study is intended to be conducted in substantial compliance with the requirements of 40 CFR 26 subparts K and L. The amended protocol, when approved by SAIRB, should met all applicable ethical standards for the protection of human subjects of research, and all requirements for documentation of ethical conduct of the research. If this study is determined to be scientifically valid and relevant, there is no regulatory barrier to EPA's reliance on it in actions under FIFRA or §408 of FFDCA. 10 ------- NOTICE This report has been written as part of the activities of the EPA Human Studies Review Board, a Federal advisory committee providing advice, information and recommendations on issues related to scientific and ethical aspects of human subjects research. This report has not been reviewed for approval by the Agency and, hence, the contents of this report do not necessarily represent the view and policies of the Environmental Protection Agency, nor of other agencies in the Executive Branch of the Federal government, nor does the mention of trade names or commercial products constitute a recommendation for use. You may obtain further information about the EPA Human Studies Review Board from its website at http://www.epa.gov/osa/hsrb. You may also contact the HSRB Designated Federal Officer, via e-mail at ord-osa- hsrb@epa.gov In preparing this document, the Board carefully considered all information provided and presented by the Agency presenters, as well as information presented by public commenters. This document addresses the information provided and presented within the structure of the charge by the Agency. 11 ------- Appendix I. Simulation Results Comparing ^-Distribution and Generalized Linear Model (GLiM) Means and Confidence Intervals (CIs)* 7Q S5 95 98 ' 69.5 84.7 54.9 98.0 7® 35 95 98 "'' 69.9 S4.9 95.0' 98.0 70 85 95 98 69.9 85,0' 95,«J 98.0 M:ifti f-SittiH •>:;>:/.i| ";^. ;•,*•*?* v*T4r^-? C»;':';;-::;;r-:S S-:i»ii- •••••- ;-/.v-:.. ;;••.>••••.•" 70 85 95 98 •••^-'S't 63.3 84.2 94.8 97,9 12.3 8.2 4.3 4.2 ^h-fr«|; 95 95 94 §3 69.6 8^.7 94.9 97.9 S': : 14,0 3.7 4.6 3.3 ^IlliScff ':. .••• .•••• ',. • •.•: 99 98 99 98 7.0 4.S 2.7 94 95 94 94 69.8 84.8 94.9 98.0 9.6 5.S 2.9 1.8 99 99 99 98 4.5 1.8 95 95 95 94 70.0 84.9 95.0 98.0 6.5 3.8 1.9 100 99 S8 98 3.4 2.5 1.5 0.9 96 95 95 94 70.0 85.0' 95.0 9S.O 4.1 2.7 1.4 0.9 99 98 98 97 * Note: Subject-Subject Background Variation (Logit SD) = 0.3, Subject-Subject Protection Variation (Logit SD) = 0, Number of simulated samples = 5000, Seed = 98135183. 12 ------- US ENVIRONMENTAL PROTECTION AGENCY HUMAN STUDIES REVIEW BOARD Chair Liza Dawson, Ph.D., Research Ethics Team Leader, Division of AIDS National Institutes of Health (NIH), National Institute of Allergy and Infectious Disease (NLALD), Bethesda, MD Vice Chair Edward Gbur, Jr., Ph.D., Professor, Director, Agricultural Statistics Laboratory, University of Arkansas, Fayetteville, AR Members Gary L. Chadwick, Pharm.D., M.P.H., C.I.P., Senior Consultant, HRP Consulting Group, Inc. Fairport, NY George Fernandez, Ph.D., Statistical Training Specialist, SAS Institute, Statistical Training and Technical Services, Sparks, NV Kyle L. Galbraith, Ph.D., Manager, Human Subjects Protection, Carle Foundation Hospital, Urbana, IL Jewell H. Halanych, M.D., Assistant Professor, Internal Medicine Residency Program, Montgomery Regional Campus, University of Alabama at Birmingham, Montgomery, AL Randy Maddalena, Ph.D., Physical Research Scientist, Indoor Environment, Lawrence Berkeley National Laboratory, Berkeley, CA Kenneth Ramos, M.D., Ph.D., PharmB, Associate Vice President, Precision Health Sciences, Professor of Medicine, Arizona Health Sciences Center, Tucson, AZ Suzanne M. Rivera, Ph.D., M.S.W., Vice President for Research and Technology Management, Case Western Reserve University, Cleveland Heights, OH Helen H. Suh, Ph.D., Associate Professor of Health Sciences, Northeastern University, Boston, MA Jun Zhu, Ph.D., Professor of Statistics and of Entomology, Department of Statistics, University of Wisconsin - Madison, Madison, WI Human Studies Review Board Staff Jim Downing, Executive Director, Human Studies Review Board Staff, Office of the Science Advisor, United States Environmental Protection Agency, Washington, DC 13 ------- |