Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 EPA-HSRB-08-03
2
3 George Gray, Ph.D.
4 Science Advisor
5 Office of the Science Advisor
6 1200 Pennsylvania Avenue, NW
7 Washington, DC 20460
8
9 Subject: June 24-25, 2008 EPA Human Studies Review Board Meeting Report
10
11 Dear Dr. Gray:
12
13 The United States Environmental Protection Agency (EPA or Agency) requested the
14 Human Studies Review Board (HSRB) to review scientific and ethical issues addressing: (1)
15 Agricultural Handlers Exposure Task Force (AHETF) pesticide handler protocols: closed-cab
16 airblast scenario and (2) completed Insect Control and Research (ICR) mosquito repellent
17 efficacy study Al 17. The enclosed HSRB report provides the Board's response to EPA charge
18 questions presented at the June 24-25, 2008 meeting. A summary of the Board's conclusions
19 concerning these two topics is provided below.
20
21 AHETF Pesticide Handler Protocols: Closed-Cab Airblast Scenario
22
23 Science
24
25 The AHETF has provided the Agency with a well-documented approach to the
26 assessment of worker exposures during closed-cab airblast applications. The Board considered
27 the AHETF study designs and protocols to successfully address many scientific and logistical
28 challenges. The Board appreciated particularly the clarity of the protocols and the extensive
29 documentation associated with these materials. The Board concurred with the Agency that
30 existing data on handler exposures during closed-cab airblast applications are inadequate and that
31 the development of more accurate information is an appropriate goal. The Board also concurred
32 with the Agency that there are only minimal risks associated with the procedures described in
33 these protocols.
34
35 The Board strongly advised the Agency to require collection of information on growers
36 who do not respond or who decline to participate, such that the representativeness of
37 participating growers can be evaluated. The Board also recommended that Local Site
38 Coordinators have demonstrable training and expertise in survey implementation so as to ensure
39 optimal recruiting for these studies. The Board judged the current sample size justification to be
40 of limited utility, since it was based on a hypothetical sampling plan that differs from the
41 sampling plan presented for these protocols; i.e., the sample size calculation was based on
42 sampling one worker from each of five growers, whereas the current protocols propose to sample
43 five workers from three growers. If the approach presented in these protocols were adopted, then
44 the sample size would need to be increased. Finally the Board urged the Agency to seriously
45 consider the alternative design presented in this report before making a final decision on study
46 design. The Board recommended that the Agency reconsider the design of the study, or develop
Page 1 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 an explicit statement of the limitations on the use of data that will be collected under the
2 proposed design.
O
4 The Board noted that many aspects of the proposed studies are likely to reduce the range
5 of exposures that would be measured with applicators under real-world conditions. While a
6 reduction in the range of exposures may be unavoidable due to practical considerations, it should
7 be considered by the Agency when evaluating the usefulness of the data produced by these
8 studies.
9
10 In conclusion, if the AHETF materials are revised in accordance with the Agency's
11 suggestions and the Board's recommendations, including implementation of the Board's
12 recommended alternative design, several limitations with the proposed research will be alleviated
13 and the research is more likely to generate scientifically reliable data, useful for assessing the
14 exposure of handlers who apply liquid pesticides using airblast equipment drawn by vehicles
15 with closed cabs.
16
17 Ethics
18
19 The Board concluded that the proposed studies meet the applicable requirements of 40
20 CRF Part 26, Subparts K and L.
21
22 Completed ICR Mosquito Repellent Efficacy Study Al 17
23
24 Science
25
26 The data of study ICR Al 17 are sufficiently sound, from a scientific perspective, to be
27 used to assess the repellent efficacy of the formulations tested against mosquitoes of the genus
28 Culex.
29
30 Ethics
31
32 A majority of the Board concurred with the initial assessment of the Agency that the
33 study submitted for review was conducted in substantial compliance with the applicable
34 requirements of 40 CFR Part 26, Subparts K and L.
35
36 Other topics
37
38 The Board also provided comments_concerning statistical analysis of arthropod
39 repellency studies and revisited its criteria for analysis of completed studies in which planned
40 protocol deviations were conducted.
41
42 Statistical analysis of arthropod repellency studies
43
44 In the review of the ICR mosquito repellent efficacy study, the Board continued to raise
45 issues concerning statistical aspects of insect repellency studies. To improve the quality of the
46 analyses of insect repellent studies, the EPA should be encouraged to provide better guidance in
Page 2 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 two areas, both of which would require some additional analyses. First, a meta-analysis, that
2 includes more studies and uses more appropriate models of the relationship between standard
3 deviation and mean time of protection, should be conducted. Second, the potential use of either
4 maximum likelihood methods for estimation of the mean and variance in the presence of heavy
5 censoring or estimation of the proportion of the population having protection times of at least a
6 pre-specified number of hours should be considered as alternatives to those currently used to
7 analyze the data.
8
9 Board criteria for analysis of completed studies in which planned protocol deviations were
10 conducted
11
12 The Board revisited its criteria for analysis of completed studies in which planned protocol
13 deviations were conducted: (1) prior to 1KB review and (2) following HSRB review of the
14 originally approved protocol. The need for such a reevaluation by the Board was informed by its
15 analysis of the completed ICR mosquito repellent efficacy study Al 17. In the Board's report
16 from its April 9-10, 2008 meeting, it advised the Agency regarding future review of a study with
17 an originally approved protocol by the HSRB:
18
19 l.Any study executed prior to IRB approval of the Informed Consent Form and the protocol, or
20 changed in ways that were not approved by the IRB will be judged by the Board as failing to
21 meet the applicable requirements of §40 CFR 26, subparts K.
22
23 2. If the EPA submits to the Board for review a completed protocol with scientific deviations
24 from the original protocol reviewed by the Board, the EPA review of the completed protocol
25 should provide the Board with EPA's opinion regarding why the deviation did not meet the
26 requirement for re-review and why the protocol still meets the applicable regulations.
27
28 The Board has revised its recommendation as follows:
29
30 1. Execution of a study prior to IRB approval of the Informed Consent Form and the protocol, or
31 changed in ways that were not approved by the IRB will be evaluated by the Board to determine
32 whether such actions were or were not in substantial compliance with applicable requirements of
33 §40 CFR 26, subparts K.
34
35 2. If the EPA submits to the Board for review a completed protocol with scientific or ethical
36 deviations from the original protocol reviewed by the Board, the EPA review of the completed
37 protocol should provide the Board with EPA's opinion regarding why the deviation did not meet
38 the requirement for re-review and why the protocol still meets the applicable regulations.
39
40 In conclusion, the EPA HSRB appreciated the opportunity to advise the Agency on the
41 scientific and ethical aspects of human studies research and looks forward to future opportunities
42 to continue advising the Agency in this endeavor.
43
44
45
Page 3 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
Sincerely,
2
4
5 Celia Fisher, Ph.D., Chair
6 EPA Human Studies Review Board
Page 4 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 NOTICE
2
3 This report has been written as part of the activities of the EPA Human Studies Review
4 Board, a Federal advisory committee providing advice, information and recommendations on
5 issues related to scientific and ethical aspects of human subjects research. This report has not
6 been reviewed for approval by the Agency and, hence, the contents of this report do not
7 necessarily represent the view and policies of the Environmental Protection Agency, nor of other
8 agencies in the Executive Branch of the Federal government, nor does the mention of trade
9 names or commercial products constitute a recommendation for use. Further information about
10 the EPA Human Studies Review Board can be obtained from its website at
11 http://www.epa.gov/osa/hsrb/. Interested persons are invited to contact Paul Lewis, Designated
12 Federal Officer, via e-mail at lewis.paul@epa.gov.
13
14 In preparing this document, the Board carefully considered all information provided and
15 presented by the Agency presenters, as well as information presented by public commenters.
16 This document addresses the information provided and presented within the structure of the
17 charge by the Agency.
18
Page 5 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 U. S. ENVIRONMENTAL PROTECTION AGENCY HUMAN STUDIES REVIEW
2 BOARD MEMBERS
O
4 Chair
5
6 Celia B. Fisher, Ph.D., Marie Ward Doty Professor of Psychology, Director, Center for Ethics
7 Education, Fordham University, Department of Psychology, Bronx, NY
8
9 Vice Chair
10
11 William S. Brimijoin, Ph.D., Chair and Professor, Molecular Pharmacology and Experimental
12 Therapeutics, Mayo Foundation, Rochester, MN *
13
14 Members
15
16 Alicia Carriquiry, Ph.D., Professor, Department of Statistics, Iowa State University
17 Snedecor Hall, Ames, IA
18
19 Gary L. Chadwick, PharmD, MPH, CIP, Associate Provost, Director, Office for Human Subjects
20 Protection, University of Rochester, Rochester, NY
21
22 Janice Chambers, Ph.D., D.A.B.T., William L. Giles Distinguished Professor, Director, Center
23 for Environmental Health Sciences, College of Veterinary Medicine, Mississippi State
24 University, Mississippi, MS
25
26 Richard Fenske, Ph.D., MPH, Professor and Associate Chair, Department of Environmental and
27 Occupational Health Sciences, University of Washington, Seattle, WA
28
29 Susan S. Fish, PharmD, MPH, Professor, Biostatistics & Epidemiology, Boston University
30 School of Public Health, Co-Director, MA in Clinical Investigation, Boston University School of
31 Medicine, Boston, MA
32
33 Suzanne C. Fitzpatrick, Ph.D., D.A.B.T, Senior Science Policy Analyst, Office of the
34 Commissioner, Office of Science and Health Coordination, U.S. Food and Drug Administration,
35 Rockville, MD
36
37 Dallas E. Johnson, Ph.D., Professor Emeritus, Department of Statistics, Kansas State University,
38 Manhattan, KS
39
40 Kannan Krishnan, Ph.D., Professor, Departement de sante environnementale et sante au travail,
41 Faculte de medicine, Universite de Montreal, Montreal, Canada
42
43 Michael D. Lebowitz, Ph.D., FCCP, Professor of Public Health & Medicine. University of
44 Arizona, Tucson, AZ
45
Page 6 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 Lois D. Lehman-Mckeeman, Ph.D., Distinguished Research Fellow, Discovery Toxicology,
2 Bristol-Myers Squibb Company, Princeton, NJ *
O
4 Jerry A. Menikoff, M.D., Director, Office of Human Subjects Research, Office of the Director,
5 National Institutes of Health, Bethesda, MD
6
7 Rebecca Parkin, Ph.D., MPH, Associate Dean for Research and Public Health Practice, School
8 of Public Health and Human Services, The George Washington University, Washington, DC
9
10 Sean Philpott, Ph.D., MS Bioethics, Science and Ethics Officer, Global Campaign for
11 Microbiocides, PATH, Washington, DC
12
13 Ernest D. Prentice, Ph.D., Associate Vice Chancellor for Academic Affairs, University of
14 Nebraska Medical Center, Omaha, NE*
15
16 Richard Sharp, Ph.D., Director of Bioethics Research, Department of Bioethics, Cleveland
17 Clinic, Cleveland, OH *
18
19 Linda J. Young, Ph.D., Professor, Department of Statistics, Institute of Food and Agricultural
20 Sciences, University of Florida, Gainesville, FL
21
22
23 Human Studies Review Board Staff
24
25 Paul I. Lewis, Ph.D., Executive Director, Human Studies Review Board Staff, Office of the
26 Science Advisor, United States Environmental Protection Agency, Washington, DC
27
28 * Not in attendance at the June 24-25, 2008 Public Meeting
29
30
Page 7 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 INTRODUCTION
2
3 On June 24-25, 2008, the United States Environmental Protection Agency's (EPA or
4 Agency) Human Studies Review Board (HSRB) met to address scientific and ethical issues
5 concerning: the Agricultural Handlers Exposure Task Force (AHETF) pesticide handler
6 protocols: closed-cab airblast scenario and (2) completed Insect Control and Research (ICR)
7 mosquito repellent efficacy study Al 17. Each of these topics is discussed more fully below.
8
9 1. AHETF Pesticide Handler Protocols: Closed-Cab Airblast Scenario
10
11 The HSRB has previously considered issues related to the design and conduct of research
12 to measure the levels of exposure received by people when handling (i.e., mixing, loading, or
13 applying) pesticides. The Agricultural Handlers Exposure Task Force (AHETF) has previously
14 submitted materials for HSRB review. In response to concerns raised by the Board at its June
15 2006 meeting, EPA asked its FIFRA Scientific Advisory Panel (SAP), an advisory committee of
16 independent expert scientific peer reviewers commenting on proposed pesticide regulatory
17 decisions, to address a number of scientific issues surrounding handler exposure research at its
18 January 2007 meeting. The Agency presented the results of the SAP review and additional
19 issues at the April and June 2007 HSRB meetings. In response to the SAP and HSRB reviews,
20 the Task Force reworked its research proposal.
21
22 At the April 2008 meeting and in earlier discussions with the HSRB, the design of the
23 sampling strategies to be used by the AHETF and by the Antimicrobials Exposure Assessment
24 Task Force II (AEATF), has drawn particular attention. As the Agency reported at the April
25 2008 meeting, after consulting with experts both within and outside EPA, and considering
26 information presented by the Task Forces, EPA informed the HSRB that the Agency had decided
27 to accept data developed through "hybrid" sampling strategies, i.e., strategies that use a
28 purposive design but which incorporate random elements whenever feasible.
29
30 The AHETF has submitted two proposed protocols, each for a different field study
31 involving pesticide application to orchard trees by airblast sprayers while the applicators are
32 within a vehicle with a fully enclosed cab. Both field studies would provide monitoring data for
33 the same scenario; the AHETF and EPA expect to present the protocols for the remaining three
34 field studies associated with this scenario at a future HSRB meeting. When all five field studies
35 have been conducted, data collection for this scenario will be complete.
36
37 EPA's regulation, 40 CFR §26.1125, requires a sponsor or investigator to submit to
38 EPA, before conducting a study involving intentional exposure of human subjects, materials
39 describing the proposed human research in order to allow EPA to conduct scientific and ethics
40 reviews. In addition, EPA's regulation, 40 CFR §26.1601, requires EPA to seek HSRB review
41 of the proposed research. Because the research proposed by the AEATF involves scripted
42 exposure, it meets the regulatory definition of "research involving intentional exposure of a
43 human subject" and thus these cited provisions of regulation apply to it.
44
45 EPA reviewed the AHETF proposals and concluded that, with a number of required
46 revisions, they appear likely to generate scientifically sound, useful information and to meet the
Page 8 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 applicable provisions of the EPA regulations in 40 CFR part 26, subparts K and L. Because the
2 AFIETF would like to conduct these field studies as soon as possible, and since EPA finds that
3 the protocols can meet applicable scientific and ethical standards, EPA presented this protocol
4 for review to the HSRB at its June 2008 meeting.
5
6 II.ICR Completed Mosquito Repellent Efficacy Study All?
7
8 In its October 2007 meeting the HSRB favorably reviewed protocol Al 17 from Insect
9 Control & Research, Inc. (ICR) to evaluate the efficacy in the laboratory of two registered
10 products containing picaridin against Culex mosquitoes.
11
12 Following that meeting, ICR revised the protocol to address EPA and HSRB comments
13 and then submitted the revised protocol for IRB approval. ICR executed the research and
14 submitted a report to EPA.
15
16 The Agency's regulation, 40 CFR §26.1602, requires EPA to seek HSRB review of an
17 EPA decision to rely on the results of these studies. The sponsor has submitted these data to
18 support applications for amended registration for the two test materials. EPA has reviewed the
19 research, applying the standard in 40 CFR §26.1705, which states:
20
21 §26.1705 Prohibition on reliance on unethical research with non-pregnant,
22 non-nursing adults conducted after April 7, 2006
23
24 Except as provided in §26.1706, in actions within the scope of §26.1701, EPA
25 shall not rely on data from any research initiated after April 7, 2006, unless EPA
26 has adequate information to determine that the research was conducted in
27 substantial compliance with subparts A through L of this part. . . This prohibition
28 is in addition to the prohibition in §26.1703.
29
30 EPA determined that the data are scientifically sound, and although there were some
31 irregularities in the conduct of recruitment, that the study appeared to meet the standard of
32 §26.1705. Unless the HSRB advises that the conduct of the study was not in substantial
33 compliance with EPA's rules for the protection of human subjects of research, EPA proposes to
34 rely on the results in considering the pending applications for amended registration.
35 REVIEW PROCESS
36
37 On June 24-25, 2008, the Board had a public face-to-face meeting in Arlington, Virginia.
38 Advance notice of the meeting was published in the Federal Register "Human Studies Review
39 Board: Notice of Public Meeting" (73 Federal Register 46, 12413). At the public meeting,
40 following welcoming remarks from Agency officials the Board heard presentations from the
41 Agency on the following topics: (1) Overview of EPA's assessment of AHETF pesticide handler
42 protocols: closed-cab airblast scenario and (2) ICR completed mosquito repellent efficacy study
43 A117.
44
45 Oral comments
Page 9 of27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 The following oral comments were presented at the meeting:
2
3 (1) Overview of EPA's Assessment of AHETF Pesticide Handler Protocols: Closed-Cab Airblast
4 Scenario
5
6 Victor Canez, Ph.D. of BASF on behalf of the AHETF
7 Richard Collier, Ph.D. of Landis International on behalf of the AHETF
8 Larry Holden, Ph.D. of Sielken and Associates on behalf of the AHETF
9 Mr. Curt Lunchick of Bayer Crop Science on behalf of the AHEATF
10
11 (2) ICR Completed Mosquito Repellent Efficacy Study All?
12 Mr. Niketas Spero on behalf of ICR,
13 Robin Todd, Ph.D. on behalf of ICR
14 Ralph Piedmont, Ph.D. of Loyola College of behalf of ICR, Inc.
15 Mr. Andrew Pechko of Avon
16
17 Written comments
18 Written comments were received by:
19
20 Richard Collier, Ph.D. on behalf of the AHETF
21 Mr. Micah Reynolds of behalf of ToXcel
22 B. Sachau, private citizen
23
24 For their deliberations, the Board considered the materials presented at the meeting,
25 written public comments and Agency background documents (e.g., the published literature,
26 Agency data evaluation record, weight of evidence review, ethics review, pesticide human study
27 protocols and Agency evaluation of the protocol or study). For a comprehensive list of
28 background documents visit the www.regulations.gov, or EPA's HSRB website at
29 http://www.epa.gov/osa/hsrb/
30
31 CHARGE TO THE BOARD AND BOARD RESPONSE
32
33 Overview of Assessment of AHETF Pesticide Handler Protocols: Closed-Cab Airblast
34 Scenario
35
36 If AHETF's proposed closed-cab airblast application scenario design, field study
37 protocols AHE55 and AHE56, and supporting documentation are revised as suggested in EPA's
38 reviews:
39
40 Science
41
42 Does the research appear likely to generate scientifically reliable data, useful for
43 assessing the exposure of handlers who apply liquid pesticides using airblast equipment drawn
44 by vehicles with closed cabs?
45
Page 10 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 Board Response to the Charge
2 General Comments
O
4 The AHETF has provided the Agency with a well-documented approach to the
5 assessment of worker exposures during closed-cab airblast applications. The Board considered
6 the AHETF study designs and protocols to successfully address many scientific and logistical
7 challenges. The Board appreciated particularly the clarity of the protocols and the extensive
8 documentation associated with these materials. The Board's concerns centered around study
9 design issues and exposure variability constraints.
10 Study Design Issues
11
12 The designs proposed for the citrus (FL) and pecan (GA) studies have many positive
13 aspects; however, two issues can significantly limit the utility of the data to be collected.
14
15 Low response rate
16
17 The AHETF anticipates a response rate by growers of about 5%. This extraordinarily low
18 anticipated response rate has implications regarding the design of the study. Without information
19 about important characteristics of the 95% of growers who are expected to refuse to participate
20 in the study, the data obtained from the 5% who agree to participate cannot be considered a
21 meaningful sample. It is not possible to determine whether the data collected from the responders
22 can be considered in any way representative of the data that would have been obtained if the
23 entire population of growers could have been observed unless: (1) it is possible to determine
24 whether the 5% responders are a random sub-sample of the population of growers in the counties
25 in the study; or (2) the missing data mechanism can be estimated from information collected
26 from the non-responders. However, as currently written the AHETF study design provides no
27 opportunity for collecting information from non-participating growers. Furthermore, no evidence
28 is provided in the protocol regarding the expertise of the Local Site Coordinator (LSC) regarding
29 his/her effectiveness as a recruiter of reluctant study participants. Given the generally
30 unsuccessful prior experiences of AHETF with respect to the recruitment of participants for
31 these types of exposure studies, reliance on a LSC without documented expertise in survey
32 implementation appears to be a major shortcoming of the proposed approach.
33
34 Deviation from the design that served as the basis for sample size calculations
35
36 Earlier documentation provided by AHETF included extensive discussion and
37 justification of a sample consisting of five monitoring units (MUs) in each "cluster", where
38 cluster was defined as a combination of crop, state and location within state. In those
39 calculations, each MU was assumed to be associated with a different operation (grower), and the
40 ICC of 0.3 (rather low) was meant to account for correlations arising from applicators working in
41 the same general geographic area, on the same crop, under similar climatic conditions, etc. The
42 ICC did not include covariances that would arise within sub-clusters; i.e., applicators working for
43 the same grower. Thus, the Board finds that it is problematic to now argue that the same sample
44 size of five MUs is still adequate when the five exposure measurements will be obtained from
Page 11 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 only three farms. If this is the new design, then the sample size needs to be re-calculated and
2 would be expected to increase if the same estimation accuracy is to be met.
O
4 In light of these observations, several aspects in the proposed design of the citrus (FL)
5 and pecan studies (GA) merit discussion. The alternative design proposed below and the
6 recommendations that follow are likely to alleviate some of the shortcomings of the current study
7 plan.
8
9 Alternative Design as Recommended by the Board
10
11 Choice of growers for participation in the study
12
13 This alternative study design focuses on two counties selected in Florida for illustration.
14 There are approximately 2,500 citrus growers in the two counties under consideration. At a
15 minimum, information on the size of the 2,500 farms is available from Federal or State records.
16 The proposed method includes the following steps (where the actual numbers are hypothetical
17 and used purely for illustration):
18
19 (1) stratify the 2,500 growers into three (or a different number) size strata. For the sake of
20 discussion, let us suppose that about 20% of growers (500) can be considered to be "small",
21 45% (1,125) can be considered to be "medium" and 35% (875) can be considered to be
22 "large";
23
24 (2) select, randomly, 2% of the growers from within each size stratum. This will result in a
25 sample of 10 small, 22 medium and 18 large growers, for a total of 50 growers. These 50
26 growers form our new "sampling frame" and from this frame we will be drawing the sample
27 for the study;
28
29 (3) contact each of these 50 growers by sending a letter that explains the purpose of the study,
30 the importance of obtaining exposure information that can be considered representative, and
31 assurance that grower participation and study information will remain confidential. The letter
32 should come from an institution trusted by growers; e.g., county extension service or local
33 land-grant university survey unit. In the letter, also explain to growers that someone from the
34 survey unit will be contacting them by phone within the next two to three weeks;
35
36 (4) within the stated period, contact all 50 growers via phone. Interviews should be conducted by
37 experienced interviewers, who can revisit issues associated with study goals and design, and
38 can reinforce the importance of participation. Relevant information to be collected during
39 this phone conversation includes: willingness to participate in the study, number of acres
40 under crop, pesticide typically used for treatments, number of applicators employed by the
41 grower and number of acres typically treated by each applicator, equipment (size, type, age)
42 available for spraying liquid pesticide, etc;
43
44 (5) from among those growers who agree to participate in the study, select three, six and six at
45 random from the small, medium and large size strata and for these fifteen growers, contact
46 the actual applicators to obtain their consent (or not) for participating in the study. The final
Page 12 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 goal is to recruit one applicator from the small grower stratum and two applicators from each
2 of the medium and large grower strata. Assuming that more than one applicator in more than
3 the required number of growers in each stratum agree to participate, select purely at random
4 the applicators (and thus the growers) who will ultimately participate in the exposure study;
5
6 (6) document the proportion (out of the 50 growers in the sample) who agree to participate and
7 those who do not, and compare the characteristics of each of the two groups to determine
8 whether non-response can be considered to be non-informative in further analyses.
9 Number of Monitoring Units (MU) per grower
10
11 The Board recommended that no more than one MU be associated with a grower. If more
12 than one MU is to be selected from a grower, then the AHETF should justify the sample size in
13 light of the additional correlation that is expected to arise between applicators that work for the
14 same grower.
15 Small Growers
16
17 The Board was concerned that the recruitment process will focus on growers with
18 multiple workers. This may eliminate small growers who employ only a single pesticide handler.
19 Yet these are often the workplaces in which the higher exposures occur due to fewer resources
20 available for safety training, PPE, etc. The Board recommended that the AFIETF better define
21 the recruitment process in regard to local resources and how these local resources will be
22 contacted. The AHETF is preparing to conduct studies in Florida citrus and in Georgia pecans.
23 The Task Force should be able to identify for the Agency the individuals or organizations that
24 will be contacted in these cases.
25 Applicator Behavior
26
27 The Board was concerned that the proposed studies will not capture the full exposure
28 variability in this population, since applicators will not necessarily practice normal behavior
29 while participating in these studies. For example, if another worker conducts all mixing and
30 loading, then it seems likely that the applicators being studied will not exit their cabs as
31 frequently as they might otherwise. The AHETF documents state that "AHETF suspects that
32 exposure potential, especially dermal exposure, may be impacted by how often the applicator
33 gets out of the closed cab." The Board agreed that this behavior is likely an important factor for
34 applicator exposure in this scenario. Transitioning between tasks is usually an opportunity for
35 exposure, but apparently this will not take place in these studies. The Board recommended that
36 the Agency require the Task Force to record the number of times a worker exits the closed cab
37 during the study period; these data can then be considered in evaluating the validity of the actual
38 exposure scenarios that occur during the study.
39
40 If applicators do leave the cab they will need to wear all PPE required by the label and
41 the EPA Worker Protection Standards (WPS). This makes sense, but will tend to underestimate
42 actual exposures, since under normal working conditions many workers will not adhere to these
43 procedures. For example, repair or adjustment of equipment is often hindered by chemical
Page 13 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 protective gloves, and applicators often remove their gloves to perform these tasks. Such
2 behavior will not be captured in these studies, leading to lower exposure values than might
3 otherwise be seen.
4
5 An excerpt from the AHETF documents states that "workers will be allowed to follow
6 their normal procedures as long as they fit the scenario definition and do not conflict with EPA's
7 Worker Protection Standard. . . . Opening windows . . . therefore, will not be allowed." Again,
8 this adherence to WPS is reasonable, but will not capture those exposures that occur when
9 windows are open, which is not an uncommon occurrence.
10 Mixer/Loaders
11
12 The Board raised the question as to whether mixer/loaders should be considered to be
13 participants in these studies, as they are being asked to perform certain tasks within the study,
14 even though they are not being monitored for exposures. If so, these workers should provide
15 informed consent.
16
17 Dormant Applications
18
19 AHETF has decided to exclude dormant applications for these studies. AHETF
20 documents state that "dormant applications are estimated to account for 15% or less of all
21 airblast applications." It was not clear to the Board why dormant sprays would be excluded,
22 since they represent nearly one-sixth of the applications on these crops. It is very likely that
23 exposures to applicators will differ between dormant and full-foliage spraying. Since the goal of
24 these studies is to capture a representative sample of exposures, the Board considered that it
25 would be reasonable to have approximately one-sixth of the applications in the study conducted
26 under dormant conditions.
27 Exclusion of Tall Hops
28
29 The AHETF documents state that "trellis crops will be considered as a group with one
30 exception - tall hops." The Task Force document did not provide a clear rationale for this
31 exclusion. If exposures during applications to tall hops are different from other exposures, then
32 the data generated by these studies may not capture the full range of applicator exposures.
33
34 Product and Packaging
35
36 The AHETF documents state that "the actual product and packaging type has no
37 influence on the potential exposures to these applicators and is, therefore, not an important
38 consideration for this scenario." The rationale for this statement was not provided. The Board
39 raised the example of wettable powders and soluble packets that might clog spray nozzles,
40 prompting the applicator to have direct contact with equipment, thereby increasing exposure
41 potential.
42
Page 14 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 HSRB Consensus and Rationale
2
3 The AHETF has provided the Agency with a well-documented approach to the
4 assessment of worker exposures during closed-cab airblast applications. The Board considered
5 the AHETF study designs and protocols to successfully address many scientific and logistical
6 challenges. The Board appreciated particularly the clarity of the protocols and the extensive
7 documentation associated with these materials. The Board concurred with the Agency that
8 existing data on handler exposures during closed-cab airblast applications are inadequate and that
9 the development of more accurate information is an appropriate goal. The Board also concurred
10 with the Agency that there are only minimal risks associated with the procedures described in
11 these protocols.
12
13 In regard to study design, the Board strongly advised the Agency to require collection of
14 information on growers who do not respond or who decline to participate, such that the
15 representativeness of participating growers can be evaluated. The Board also recommended that
16 Local Site Coordinators have demonstrable training and expertise in survey implementation so as
17 to ensure optimal recruiting for these studies. The Board judged the current sample size
18 justification to be of limited utility, since it was based on a hypothetical sampling plan that
19 differs from the sampling plan presented for these protocols; i.e., the sample size calculation was
20 based on sampling one worker from each of five growers, whereas the current protocols propose
21 to five workers from three growers. If the approach presented in these protocols were adopted,
22 then the sample size would need to be increased. Finally the Board urged the Agency to seriously
23 consider the alternative design presented earlier in this report before making a final decision on
24 study design. In summary, the Board recommended that the Agency reconsider the design of the
25 study, or develop an explicit statement of the limitations on the use of data that will be collected
26 under the proposed design.
27
28 In regard to exposure variability, the Board noted that many aspects of the proposed
29 studies are likely to reduce the range of exposures that would be measured in applicators under
30 real-world conditions; dormant sprays and tall hops applications are excluded; applicators will
31 not conduct their own mixing/loading; applicators may not exit the closed cab on a regular basis;
32 applicators will be required to follow procedures consistent with the Worker Protection Standard.
33 These constraints will likely truncate the high end of the exposure distribution. While this
34 reduction in the range of exposures may be unavoidable due to practical considerations, it should
35 be considered by the Agency when evaluating the data produced by these studies.
36
37 If the AHETF materials are revised in accordance with the Agency's suggestions and the
38 Board's recommendations, including implementation of the Board's recommended alternative
39 design, several limitations with the proposed research will be alleviated and the research is more
40 likely to generate scientifically reliable data, useful for assessing the exposure of handlers who
41 apply liquid pesticides using airblast equipment drawn by vehicles with closed cabs.
42
43
Page 15 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 Ethics
2
3 Charge to the Board
4
5 Does the research appear to meet the applicable requirements of 40 CFR part 26, subparts
6 K and L?
7
8 Board Response to the Charge
9
10 The Board concurred with the strengths and weaknesses of the proposed research
11 protocols, as detailed in the EPA's review (Carley 2008). Most of the risks of the study are
12 consistent with the research subjects' daily work lives, or have been minimized as much as
13 possible. In addition, the risks are justified by the potential benefits of eventually having a
14 database of agricultural handlers' exposure to pesticides that EPA can use to make data-driven
15 risk assessments.
16
17 The AHETF has addressed many of the major issues of concern raised at the Board's last
18 reviews. There remain a number of additional issues to be addressed. Relating to the risks of the
19 study, the agricultural work being done is not actually part of the study but rather the workers'
20 daily job, and thus the risks of the agricultural work are not risks of the research, and should not
21 be used when assessing the risk-benefit relationship. As there is a risk of fetal exposure to
22 pesticides, all women will undergo a pregnancy test prior to enrollment and if pregnant, will not
23 be enrolled in the study. The Board questions whether there will be any confidential risk
24 counseling be offered to these women, whose daily jobs include handling these pesticides.
25
26 Although the AHETF has communicated to EPA that they will include a researcher who
27 is bilingual in English and Spanish to enroll and work with Spanish-speaking subjects, there are a
28 few remaining questions related to language and the consent process. All documents (SOPs and
29 protocols) should be changed to remove reference to interpreters and witnesses, which are now
30 unnecessary because of Spanish-speaking research staff. The protocols state that the expected
31 population of potential research subjects is 90% Caucasian and 10% Hispanic in both Florida and
32 Georgia. Members of the Board felt that this seemed quite low for proportion of Hispanic
33 workers. The Board requested that the Agency confirm that these numbers are correct and are
34 identical for the two states.
35
36 Reference is made to reading ability of the subjects, but there is no discussion of how this
37 reading ability will be assessed, or whether it will be assessed in English or Spanish. In addition,
38 the contact information for the researchers (Larry Smith and David Johnson of the AHETF) and
39 Independent Institutional Review Board (IIRB) does not indicate whether or not there are
40 Spanish-speakers available to answer questions by phone. In the description of the consent
41 process, there is thorough discussion of the importance of understanding the information
42 provided and a series of quiz questions is presented. However, there is no description of how this
43 quiz will be used, whether it will be used on all subjects, and by whom. Finally there is no
44 description of how interviewers will be trained to conduct the consent process and recognize
45 whether or not potential subjects understand the information provided.
46
Page 16 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 The translation into Spanish of the consent form, the product risk statements, recruitment
2 flyers and other documents was an area of concern for the Board. Many Board members felt that
3 the IIRB should not provide the translations. Several Board members who are fluent in Spanish
4 stated that the IRB-approved translations were not adequate. Both the English and Spanish
5 consent forms were quite complex and needed simplification. The recruitment poster also was
6 overly complex, using the term "cognitively impaired" for example. Simplification was needed
7 for all documents so that understanding of the information by the potential subjects could be
8 enhanced.
9
10 The Board was concerned about confidentiality and made a number of suggestions.
11 While the consent form is clear concerning the use of photographs, the researchers should
12 consider using black-out boxes to de-identify the photos and this should then be explained to
13 subjects. The language in the consent form concerning the photographs may border on
14 exculpatory, and the Board suggested that this section be revised to include a more complete
15 discussion of why photos are being taken and that they will "only be used for purposes of this
16 research." In addition, it seems as if the grower's identity does not appear among the data
17 collected; this should be confirmed. The subject's identity should appear on the consent form, the
18 product risk statement to be signed, and the form for the subject to request his/her own data. The
19 data to be submitted to EPA with the study report should use the study ID number rather than
20 subject name. However, EPA may see or view the identities of the subjects at the time of a
21 quality assurance audit of study documents. This should be explained in the consent process and
22 form.
23
24 The section of the consent form that discusses compensation for research-related injury
25 states "AHETF will cover the cost of reasonable and appropriate medical attention that is not
26 covered by our own insurance or insurance provided through your employer." The Board
27 recommended removing the phrase "reasonable and appropriate" since that is something that the
28 researchers or subjects cannot decide but rather is decided by the health care provider.
29
30 The Board agreed with EPA's suggestion to have standard procedures described in SOPs
31 and remove them from the protocols, but rather refer to the appropriate SOP. The Adverse Event
32 Reporting SOP refers to WIRE (presumably Western IRB) which is irrelevant for these
33 protocols.
34
35 HSRB Consensus and Rationale
36
37 The Board concluded that if the AHETF follows the recommendations of EPA and the
38 Board with respect to informed consent and confidentiality the proposed studies meet the
39 applicable requirements of 40 CRF Part 26, Subparts K and L.
40
41
42
Page 17 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 EPA Review of Completed ICR Mosquito Repellent Efficacy Study All?
2
3 Science
4
5
6 Charge to the Board
7
8 Is this study sufficiently sound, from a scientific perspective, to be used, in conjunction
9 with other information, to assess the repellent efficacy of the formulations tested against
10 mosquitoes of the genus Culex!
11
12 Board Response to the Charge
13
14 Overview of the Study
15
16 ICR All? was a laboratory study of repellency to Culex quinquefasciatus mosquitoes of
17 two registered products (lotion and spray) containing 10% picaridin; these two formulations are
18 Avon Skin-So-Soft SSS Bug Guard Plus Picaridin Insect Repellent (EPA Reg. No. 806-29) and
19 Avon Skin-So-Soft SSS Bug Guard Plus Picaridin Insect Repellent Spray (EPA Reg. No. 806-
20 31). It was conducted by Insect Control & Research, Inc., of Baltimore, MD on March 4, 2008.
21 The study was managed by toXcel, LLC of Gainesville, Virginia, and was sponsored by Avon
22 Products, Inc. of Suffern, New York. The hypothesis provided by the sponsor was that
23 protection (i.e., repellency) of 8 hours (h) would be demonstrated.
24
25 The study was required by EPA to support a proposed extension of label claims to
26 include repellency of mosquitoes which can vector West Nile virus (WNV). Mosquitoes of the
27 genus Culex are a major vector of WNV in the U.S. The test products had previously been
28 shown to repel mosquitoes of other genera.
29
30 Cages were 2x2x2 feet; each cage was used simultaneously to test two subjects, each
31 of whom was treated with the lotion repellent on one forearm and the spray on the other. A 250
32 cm2 area of each arm was the tested region of skin, and the amount of repellent formulation
33 applied to each arm was the amount (0.42 ml containing 417.5 mg of picaridin for a dosage of
34 1.67mg/cm2) that had been tested previously in support of the current label. Each cage contained
35 200 laboratory-reared, disease-free, young adult female mosquitoes which had never been
36 provided a blood meal.
37
38 Thirteen adult subjects (7 male and 6 female; females neither pregnant nor nursing) were
39 recruited from the ICR database of previous subjects. The ethnicity and age of study participants
40 were not listed in the documents submitted to the Agency for review. One subject (male),
41 selected by lot, served as an untreated control. After establishing the attractiveness of their
42 untreated arms to the caged mosquitoes, the remaining 12 subjects were treated on each forearm
43 with one of the test repellents, using the dose indicated above.
44
45 At thirty-minute intervals the untreated control subject confirmed aggressiveness of the
46 mosquitoes in each of the 6 cages. If in any cage fewer than 5 mosquitoes landed on the forearm
Page 18 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 of the untreated control subject within one minute, 200 additional mosquitoes were released into
2 each cage. During the test this was necessary at 2.5 h post-treatment and 200 mosquitoes were
3 added to each of the cages.
4
5 Treated subjects exposed their arms to the caged mosquitoes for 5 minutes at intervals of
6 30 minutes, for 10 hours (20 exposure periods) post-treatment or until failure of efficacy,
7 whichever occurred first. Failure of efficacy was defined as a confirmed bite—i.e., a bite
8 followed by another confirming bite on the same arm within the same or the subsequent 5-
9 minute exposure period. Three subjects experienced efficacy failure on one arm; those arms
10 were not tested further. No subjects experienced efficacy failure on both arms. All 12 treated
11 subjects completed testing.
12
13 Over the ten-hour post-treatment duration of the test, one confirmed bite was observed
14 for one of the test repellents and two confirmed bites were observed for the other. In addition,
15 five unconfirmed bites were recorded—three for one repellent and two for the other.
16
17 Statistical analyses performed included Kaplan-Meier Product Limit analysis and
18 descriptive statistics consisting of the mean, standard error, and 95% confidence intervals were
19 provided. Medians were not calculated because less than half the sample experienced an
20 efficacy failure. Mean "Complete Protection Time" (CPT) was about 9.5 h for each repellent.
21 Based on power analyses contained in the protocol, it was concluded with 95% confidence that
22 CPT was at least 8 h ± 2 h.
23
24 Science Review
25
26 The study ICR Al 17 was carried out in accordance with the ICR protocol Al 17 which
27 was favorably reviewed by the HSRB in October 2007 and which was revised in February 2008.
28 There was no dosimetry phase to this study, consistent with the earlier studies on these products
29 and consistent with the approved protocol. Also bites were used as the endpoint (except for the
30 negative control which assessed landings) consistent with the earlier studies on these products
31 and consistent with the approved protocol. The only deviation noted was the grouping of
32 subjects in 2 groups of 6 subjects (for the sake of more orderly conduct of the study) instead of 6
33 groups of 2 subjects as was proposed in the protocol. This deviation related to grouping of
34 subjects would not have affected the quality of the data.
35
36 The Board concurs with EPA's assessment of the scientific quality of the accumulated
37 data, i.e., that the data were reliable. However, the Board expressed concern about the use of the
38 Rutledge and Gupta (1999) method of calculating statistical power and also expressed concern
39 about the validity of the mean and standard error calculated by the Kaplan-Meier analysis in light
40 of the large amount of censored data resulting from the complete efficacy of both repellent
41 formulations for the entire 10 h testing period for almost all subjects. An expanded discussion of
42 these concerns and alternate statistical methods are presented in the Appendix A to this report.
43
44 The Board remains unsure about the nature of the data and the type of statistical analysis
45 that EPA requires for making its regulatory decisions regarding labeling arthropod repellency
Page 19 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 products, and encourages EPA to provide more guidance to the Board on this matter in the
2 future.
O
4 HSRB Consensus and Rationale
5
6 The data of study ICR All? are sufficiently sound, from a scientific perspective, to be
7 used to assess the repellent efficacy of the formulations tested against mosquitoes of the genus
8 Culex. Following the recommendations in Appendix A may help alleviate some of the
9 limitations of the currently proposed statistical analysis.
10
11 Ethics
12
13 Charge to the Board
14
15 Does available information support a determination that this study was conducted in
16 substantial compliance with subparts K and L of EPA regulations at 40 CFR Part 26?
17
18 Board Response to the Charge
19
20 The Board concurred with the factual observations of the strengths and weaknesses of the
21 study, as detailed in the EPA's Ethics Review (Carley 2008). A majority of Board members also
22 concluded that the completed research, as described in the documents submitted to the Agency
23 for review (Reynolds and Kelly, 2008; Spero 2008), met the applicable requirement of 40 CFR
24 Part 26, Subpart Q, namely that the study was in substantial compliance with 40 CFR Part 26,
25 Subparts K and L.
26
27 The original protocol for this study (ICR Al 17) was reviewed at the October 2007
28 meeting of the HSRB, at which time the Board concluded that the study would meet the
29 applicable requirements established in the Agency's final human studies rule (40 CFR Part 26)
30 pending minor revisions (EPA HSRB 2007). The revisions suggested by the Agency and the
31 HSRB at the October 2007 meeting were duly incorporated into the study protocol (Carley 2008;
32 Reynolds and Kelly 2008; Spero 2008).
33
34 The submitted study documents (Reynolds and Kelly 2008; Spero 2008) assert
35 compliance with the ethical and regulatory standards of 40 CFR Part 26, Subparts K and L, as
36 well as the requirements of the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA)
37 § 12(a)(2)(P), and the EPA's Good Laboratory Practice (GLP) Standards described at 40 CFR
38 Part 160.
39
40 The study protocol was reviewed and approved by an independent human subjects review
41 committee, Essex Investigational Review Board (EIRE), Inc., of Lebanon, New Jersey prior to
42 submission to the Agency. In the documents submitted to the Agency for review (Spano 2008),
43 EIRE Inc. is described as being accredited by the Partnership for Human Research Protections,
44 Inc. (PHRP). PHRP was dissolved in late 2005, but the organization's website asserts that "the
45 Accreditation status of [affiliated] organizations remain in effect and valid"
46 (http://www.phrp.org; accessed 21 June 2008). The accreditation status of EIRE, Inc., however,
Page 20 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 is not listed on the PHRP website. EIRE, Inc. is also described in the submitted documents as
2 "being in the process of obtaining accreditation from AAHRPP (Association for the
3 Accreditation of Human Research Protection Programs)" (Spano 2008, 17). Currently, EIRE,
4 Inc. is not listed as an accredited organization on the AAHRPP website (http://www.aahrpp.org:
5 accessed 21 June 2008). A description of EIRE procedures was previously provided to the EPA
6 with a claim of confidentiality (EPA HSRB 2007), so they were not available for review by the
7 HSRB. Agency staff, however, reviewed the documentation provided by EIRE, Inc., and
8 previously determined these procedures and policies to be in compliance with the applicable
9 standards of the Common Rule (45 CFR Part 46, Subpart A). It is not known, however, if these
10 policies and procedures have changed since first reviewed by the Agency.
11
12 Overall, the risks to study participants were minimal and justified by the likely societal
13 benefits, including the production of data on the efficacy of these picaridin-based formulations as
14 a repellent for some of the key mosquito genera known to transmit WNV in the United States. A
15 total of 13 'experienced' volunteers participated in the study - 12 treated subjects and one
16 untreated control chosen at random. The study protocol justified the enrollment of twelve treated
17 participants by stating that ten volunteers are needed to obtain statistical validity; an additional
18 two participants will be enrolled as alternates to "allow for drop outs" (Spero 2008, 29).
19
20 The potential risks to study participants, adequately described in the protocol and
21 informed consent document, were four-fold: 1) harms resulting from the physical requirements
22 imposed upon volunteers; 2) reaction to test materials themselves; 3) exposure to biting
23 arthropods; and 4) exposure to arthropod-borne diseases.
24
25 One potential risk of study participation was physical strain associated with the
26 requirements imposed upon volunteers, particularly long-term exposure to the warm, humidified
27 laboratory environment necessary to rear and maintain mosquito colonies. The risk of physical
28 harm was minimized, however, by limiting each volunteer's exposure to this environment to five
29 minutes every half-hour. A plan for medical monitoring and treatment of physical strains was
30 also clearly articulated.
31
32 The risk that enrolled participants would experience adverse effects upon exposure to the
33 test materials was also minimal. The active ingredient of these two repellent formulations is
34 commercially available and is present at similar concentrations in other EPA-registered products;
35 specifically, picari din is registered and marketed as an insect repellent in the United States under
36 the registered trade name Bayrepel™ and the brand name Autan®. In addition, picaridin is
37 commercially available and has been used at higher doses as a repellent with little evidence of
38 toxic effects. The inert ingredients are widely used in cosmetic and personal care products, and
39 have previously been reviewed and approved by the Agency under FIFRA. Volunteers with
40 known allergic reactions to insect repellents and common cosmetics were excluded from
41 participating in this study, and the amount of skin treated with picaridin was limited. The study
42 protocol also included clear stopping rules and plans for the medical management of any
43 unanticipated side effects or adverse events associated with product exposure.
44
45 The endpoints of the study protocol require two or more mosquito bites to document
46 breakdown of repellent effectiveness. A total of 11 mosquito bites occurred among the thirteen
Page 21 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 study participants during the ten-hour observational period. Reactions to mosquito bites are
2 usually mild and easily treated with over-the-counter steroidal creams; each participant receiving
3 a bite was offered Caladryl® or Calamine® lotion to ease the associated discomfort. The study
4 also excluded individuals with a history of severe skin reactions to further minimize the risk of a
5 participant experiencing an extreme physical reaction to a mosquito bite. No study-related
6 adverse events were reported by any of the volunteers during the two weeks following study
7 participation.
8
9 The mosquitoes used for the study were bred and raised in a laboratory environment, and
10 had never been fed a blood meal. The mosquitoes thus are likely to be pathogen-free, minimizing
11 the risk of vector-borne disease.
12
13 Finally, the study protocol also included several mechanisms designed to minimize
14 coercive recruitment and enrollment, and compensation was not considered to be so high as to
15 unduly influence participation. As required by 40 CFR Part 26, Subpart L, minors and pregnant
16 or lactating women were explicitly excluded from volunteering (pregnancy being confirmed by
17 requiring all female volunteers to undergo a self-administered over-the-counter pregnancy test on
18 the day of the study). The potential stigmatization resulting from study exclusion was minimized
19 by the enrollment of extra 'alternate' participants, allowing for volunteers to withdraw or be
20 excluded from participating without unduly compromising their confidentiality.
21
22 Overall, the risks to study participants were minimal, but it is nevertheless important to
23 note two significant deviations that occurred during the conduct of this study. First, although the
24 HSRB-reviewed and EIRB-approved protocol states that participants will be treated with the test
25 products and then exposed to mosquitoes in pairs, in reality six participants were treated and
26 simultaneously exposed during each five-minute observation period. This protocol deviation was
27 reported to the EIRE ten days after study completion. Federal regulations, however, state that the
28 only acceptable deviations are those that are unanticipated and that are necessary to protect the
29 safety of trial participants (45 CFR 46.103(b)(4)). In public comments to the HSRB, Dr. Nick
30 Spero of ICR testified that the decision to change the protocol was made at an ICR staff meeting
31 on March 3rd, 2008, one day prior to study initiation. He also testified that the change was made
32 to "eliminate confusion among study participants about when to enter and leave the test area."
33 Although the Board agreed with the Agency's assessment (Carley 2008) that this deviation was
34 unlikely to have altered the risk-benefit ratio for the study participants, several members pointed
35 out that planned protocol deviations are not allowed solely for expedience as appeared to be the
36 case here.
37
38 Second, recruitment of study participants began prior to final approval of the informed
39 consent documents by EIRE, Inc. As communicated verbally to ICR by the EIRE Office
40 Manager, Karen Radcliffe, the revised protocol was approved on February 18, 2008. The revised
41 informed consent document was conditionally approved on the same day, pending revision. This
42 approval and conditional approval was also communicated to ICR via email on February 19,
43 2008. Recruitment of volunteers began on February 18th, with Dr. Spero identifying and calling
44 potential participants to confirm their availability on March 4th, the planned study date.
45
Page 22 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 According to documents submitted to the Agency, permission to begin participant
2 recruitment prior to submission and approval of the revised consent document was given
3 verbally by Ms. Radcliffe (Kelly 2008; Spero, personal testimony). However, the only written
4 evidence of such permission to begin recruitment prior to final approval of the informed consent
5 document was an email submitted to the Agency that was sent to ICR by EIRE, Inc. on February
6 19th but is dated June 3rd (Reynolds 2008, 3). Such permission also runs counter to formal written
7 restrictions on study execution previously sent to ICR by EIRE, Inc. For example, in a letter sent
8 to Dr. Spero dated August 2, 2007, Dr. Glenn Lambert, Chair of EIRE, Inc., wrote "Please be
9 reminded that the study may not commence any research activity (including scheduling)
10 until formal, written approval and a stamped consent form is received by the research site"
11 (Spero 2008, 92; emphasis in original). The approved and stamped consent form was not
12 received by ICR until February 26th, at which time potential study participants were again
13 contacted and the consent document verbally conveyed to them. Copies of the consent document
14 were then mailed and received by all but three of the study participants, and all volunteers were
15 offered the opportunity to visit the ICR offices (without recompense) to discuss the document in
16 greater detail. Although none of the study participants opted to do so, all volunteers were
17 reportedly given ample opportunity to review the document on the morning of the study.
18
19 At the April 2008 meeting of the HSRB, the Board concluded that: "Any study executed
20 prior to IRB approval of the Informed Consent Form and the protocol, or changed in ways that
21 were not approved by the IRB will be judged by the Board as failing to meet the applicable
22 requirements of 40 CFR Part 26, Subparts K and L" (EPA HSRB 2008, 35). However, the Board
23 also noted that the actions in question occurred prior to the April 2008 meeting. In light of this
24 and other considerations, some members of the Board felt that these two deviations, taken
25 together, raised doubt about whether the study met the necessary standard of "substantial
26 compliance" with the regulations promulgated in the Agency's Final Human Studies Rule (40
27 CFR 26.1706) . Other Board members, however, argued that it was unlikely that these deviations
28 either put participants at increased risk or significantly impaired the informed consent process.
29 (Note that the Board has revised this recommendation as described in this report).
30
31 HSRB Consensus and Rationale
32
33 A majority of the Board concurred with the initial assessment of the Agency that the
34 study submitted for review was conducted in substantial compliance with the applicable
3 5 requirements of 40 CFR Part 26, Subparts K and L.
36
37 Board criteria for analysis of completed studies in which planned protocol deviations were
38 conducted
39
40 The Board revisited its criteria for analysis of completed studies in which planned protocol
41 deviations were conducted: (1) prior to IRB review and (2) following HSRB review of the
42 originally approved protocol. Such a reevaluation by the Board was informative based on its
43 analysis of the completed ICR mosquito repellent efficacy study Al 17. In the Board's report
44 from its April 9-10, 2008 meeting, it advised the Agency regarding future review of a study with
45 an originally approved protocol by the HSRB:
46
Page 23 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 1 .Any study executed prior to 1KB approval of the Informed Consent Form and the protocol, or
2 changed in ways that were not approved by the 1KB will be judged by the Board as failing to
3 meet the applicable requirements of §40 CFR 26, subparts K.
4
5 2. If the EPA submits to the Board for review a completed protocol with scientific deviations
6 from the original protocol reviewed by the Board, the EPA review of the completed protocol
7 should provide the Board with EPA's opinion regarding why the deviation did not meet the
8 requirement for re-review and why the protocol still meets the applicable regulations.
9
10 The Board has revised its recommendation as follows:
11
12 1. Execution of a study prior to 1KB approval of the Informed Consent Form and the protocol, or
13 changed in ways that were not approved by the 1KB will be evaluated by the Board to determine
14 whether such actions were or were not in substantial compliance with applicable requirements of
15 §40 CFR 26, subparts K.
16
17 2. If the EPA submits to the Board for review a completed protocol with scientific or ethical
18 deviations from the original protocol reviewed by the Board, the EPA review of the completed
19 protocol should provide the Board with EPA's opinion regarding why the deviation did not meet
20 the requirement for re-review and why the protocol still meets the applicable regulations.
21
Page 24 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 Appendix A: Comments on Statistical Analysis of Arthropod Repellency Studies
2
3 Currently, the statistical foundation underlying most analyses in arthropod repellent
4 studies is weak. Adopting the statistics proposed in the paper by Rutledge and Gupta (1999) can
5 lead to flawed analyses. The relationship between standard deviation and mean, if any, is
6 unlikely to be linear. Although it is reasonable to weight the analysis by sample size, the two
7 large studies (from among about 20 studies included in the paper), each with 50 participants, had
8 small means and standard deviations. The relationship tends to be easy to model when the
9 means and standard deviations are small. The challenge arises when the means and standard
10 deviations increase, and this is the region that it is most important to model well. The weighting
11 by sample size had the unintended effect of putting less weight on samples in this region.
12
13 When the data are heavily censored, as is often the case in arthropod repellent studies, the
14 use of the Kaplan-Meier method is problematic. If censored values are replaced by the time of
15 censoring, the estimated mean is biased downward, and the standard deviation is under-
16 estimated. The downward bias of the mean tends toward a more conservative result, but under-
17 estimating the standard deviation may lead one to assume more confidence in the results than
18 should be given. It is not appropriate to return to the power analysis conducted prior to the study
19 as a foundation for drawing conclusions.
20
21 As an example, one can use the data reported in study ICR Al 17 in which a confirmed
22 bite was observed at 3 hours with one participant while the other 11 participants did not have a
23 confirmed bite when the study ended after 10 hours. In this case, it is not appropriate to say that
24 the mean time of protection is 8 hours, plus or minus two hours. One can estimate the
25 probability of protection lasting at least 8 hours to be 91.7%, and we are 95% confident that the
26 true proportion of the sampled population having at least 8 hours of protection is between 61.5
27 and 99.8%. Another approach, which does not seem to have been explored in this setting, is to
28 use maximum likelihood methods that account for the censoring of the values to obtain estimates
29 for the mean and standard deviation.
30
31 To improve the quality of the analyses of arthropod repellent studies, EPA should be
32 encouraged to provide better guidance in two areas, both of which would require some additional
33 analyses. First, a meta-analysis, similar to that of Rutledge and Gupta, that includes more studies
34 and uses more appropriate models of the relationship between standard deviation and mean time
35 of protection, should be conducted. Second, the potential use of either maximum likelihood
36 methods for estimation of the mean and variance in the presence of heavy censoring or
37 estimation of the proportion of the population having protection times of at least a pre-specified
38 number of hours should be considered as alternatives to those currently used to analyze the data.
39
40 All analyses have been based on the assumption that the population distribution of the
41 protection time is normally distributed. Some evaluation of the appropriateness of this
42 assumption should be conducted. A lognormal distribution is often more appropriate in studies
43 similar to these. For the maximum likelihood approach, the E-M algorithm can be used to obtain
44 the estimates. However, convergence may be an issue if the data are too heavily censored.
45 Certainly, convergence will not be obtained if none of the study participants have a confirmed
46 bite before the study's end. If one or two confirmed bites are obtained, convergence may be
Page 25 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 obtained, but the estimates may still be unreliable. The challenges associated with estimation of
2 the mean and standard deviation in the presence of heavy censoring lead one to consider other
3 options. The proportion of the population having the specified protection time is one such
4 alternative. Here, the proportion of the population that must have protection times at least as
5 great as those considered would then be the criterion utilized.
6
7
8
9
10
11
Page 26 of 27
-------
Proposed Final Draft v.l. Dated September 15, 2008; Do Not Cite or Quote
1 REFERENCES
2
3 Carley, J.M. 2008. Ethics Review of Human Study of Mosquito Repellent Performance.
4 Unpublished document prepared by Office of Pesticide Programs, United States Environmental
5 Protection Agency. Dated May 22, 2008.
6
7 EPA HSRB. 2007. October 24-26, 2007 EPA Human Studies Review Board Meeting Report.
8 Washington, DC: Environmental Protection Agency.
9
10 EPA HSRB. 2008. April 8-9, 2008 EPA Human Studies Review Board Meeting Report..
11 Washington, DC: Environmental Protection Agency.
12
13 Reynolds, M. 2008. Electronic Submission to the Docket Re: Submission of Written Comments
14 for Inclusion in the Upcoming Meeting of the Human Subjects Review Board, June 24-25, 2008.
15 EPA Docket ID: EPA-HQ-ORD-2008-0355. Unpublished letter to Dr. Paul Lewis. Dated June 6,
16 2008.
17
18 Reynolds, M., and Kelley, J. 2008. Additional Information to Fulfill 40 CFR §26.1303 for the
19 Study: Evaluation of the Efficacy of Personal Repellents Against Mosquitoes in the Laboratory.
20 Unpublished report prepared by toXcel, LLC, under Protocol No. G0590607001 Al 17 and
21 Project No. 0607-059-0157. Dated April 8, 2008.
22
23 Spero, N. 2008. Evaluation of the Efficacy of Personal Repellents Against Mosquitoes in the
24 Laboratory. Unpublished report prepared by Insect Control & Research, Inc., under Protocol No.
25 G0590607001A117 and Project No. 0607-059-0157. Dated April 4, 2
26
Page 27 of 27
------- |