April 16, 2007
EPA-HSRB-07-01
George Gray, Ph.D.
Science Advisor
Office of the Science Advisor
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Subject: January 24, 2007 EPA Human Studies Review Board Meeting Report
Dear Dr. Gray:
The United States Environmental Protection Agency (EPA or Agency) requested the
Human Studies Review Board (HSRB) to review scientific and ethical issues addressing: (1)
two completed human studies (EMD 003 and EMD 004) evaluating repellent efficacy of
formulations containing the active ingredient IR3535; and (2) a research proposal to evaluate the
efficacy in the field of multiple formulations of the repellent DEBT against mosquitoes.
The enclosed HSRB report addresses the Board's response to EPA charge questions at its
January 24, 2007 meeting. A summary of the Board's conclusions is provided below.
Completed IR3535 Insect Repellent Efficacy Studies
Studies EMD-003.1 and EMD 003.2
Scientific Considerations
• The reported studies on the efficacy of lotion and pump spray formulations of IR3535
(studies EMD-003.1 and EMD-003.2) on repelling ticks are sufficiently sound, from a
scientific perspective, to be used to assess the repellent efficacy of the two formulations
against ticks.
• The Board also recognized that advances in statistical analyses mean that there are
probably ways of measuring efficacy of individual products that would be an
improvement over traditional techniques. The Board encouraged the Agency to proceed
in its efforts to examine how a transition to more appropriate methods of calculating
efficacy for specific data sets can be introduced so mat consumers can not only compare
relative efficacy of products based on traditional methods but also have better
information on the degree of protection individual products provide.
• The Board commended Dr. Carroll for conducting the preliminary dosimetry phase of the
study. Since the Agency noted that this was the first repellency study to have a specific
dosimetry phase, the HSRB suggested EPA might wish to provide guidance concerning
Iof28
-------
whether the method employed in this study was the most valid way to determine dose.
The Board recommended inclusion of a description of the sampling frame and definition
of eligible subjects to help justify subject generalizability.
Ethical Considerations
• The Board concurred with the initial assessment of the Agency that the studies EMD—
003.1 and EMD-003.2 submitted for review by the Board meet the applicable
requirements of 40 CFR part 26, subparts K and L.
• The Board also noted that there were a series of deviations from Subpart K that, while not
adversely affecting the rights and welfare of human subjects of the study, reflected a lack
of familiarity with IRB procedures and protocol requirements described in Subpart K.
The HSRB advised the Agency to recommend that investigators undergo human research
protection training, and include evidence of completion of such training as part of their
submission of protocols or completed studies to the Agency. Examples of such training
could include the on-line training program offering by NIH/NCI or a similar program that
might be developed by the Agency.
Studies EMD-004.1 and EMD-004.2
Scientific Considerations
• The reported studies on the efficacy of lotion and pump spray formulations of 1R3535
(studies EMD-004. 1 and EMD-004.2) on repelling mosquitoes are sufficiently sound,
from a scientific perspective, to be used to assess the repellent efficacy of the two
formulations against mosquitoes.
• The Board also recognized that advances in statistical analyses mean that there are
probably ways of measuring efficacy of individual products that would be an
improvement over traditional techniques. The Board encouraged EPA to proceed in its
efforts to examine how a transition to more appropriate methods of calculating efficacy
for specific data sets can be introduced so that consumers can not only compare relative
efficacy of products based on traditional methods but also have better information on the
degree of protection individual products provide.
• The Board commended Dr. Carroll for conducting the preliminary dosimetry phase of the
study. Since the Agency noted that this was the first repellency study to have a specific
dosimetry phase, the HSRB suggested EPA might wish to provide guidance concerning
whether the method employed in this study was the most valid way to determine dose.
The Board recommended inclusion of a description of the sampling frame and definition
of eligible subjects to help justify subject generalizability.
2 of 28
-------
Ethical Considerations
• The Board concurred with the initial assessment of the Agency that studies EMD 004.1
and EMD 004.2 submitted for review by the Board met the applicable requirements of 40
CFR part 26, subparts K and L.
• The Board also noted that there were a series of deviations from Subpart K that while not
adversely affecting the rights and welfare of human subjects of the study, reflected a lack
of familiarity with IRB procedures and protocol requirements described in Subpart K.
The HSRB advised the Agency to recommend that investigators undergo human research
protection training, and include evidence of completion of such training as part of their
submission of protocols or completed studies to the Agency. Examples of such training
could include the on-line training program offering by NIHJNCI or a similar program that
might be developed by the Agency.
Insect Repellent Efficacy Protocol SCI-OO1
Scientific Considerations
• The Board raised several concerns about sample size, population generalizability and
statistical analysis in SCI-001 that should be addressed. If the recommendations
provided by EPA and those suggested by the Board are followed, protocol SCI-00 1
appears likely to generate scientifically valid data to assess the efficacy of the test
products against mosquitoes.
• The protocol would satisfy the scientific criteria recommended by the HSRB, namely,
producing important information that cannot be obtained except by research with human
subjects, and having a clear scientific objective and study design that should produce
adequate data to test the hypothesis.
Ethical Considerations
• The Board concluded that the protocol should meet the applicable requirements of 40
CFR part 26, subparts K and L if the points raised in the EPA review and in this report
are adequately addressed.
In conclusion, the EPA HSRB appreciated the opportunity to advise the Agency on the
scientific and ethical aspects of research with human subjects and looks forward to future
opportunities to continue advising the Agency in this endeavor.
Sincerely,
Celia B. Fisher, Ph.D. Chair
EPA Human Studies Review Board
3 of 28
-------
NOTICE
This report has been written as part of the activities of the EPA Human Studies Review
Board, a Federal advisory committee providing advice, information and recommendations on
issues related to scientific and ethical aspects of human subjects research. This report has not
been reviewed for approval by the Agency and, hence, the contents of this report do not
necessarily represent the view and policies of the Environmental Protection Agency, nor of other
agencies in the Executive Branch of the Federal government, nor does the mention of trade
names or commercial products constitute a recomniendation for use. Further information about
the EPA Human Studies Review Board can be obtained from its website at
http:I/www.epa.gov/osalhsrb/. Interested persons are invited to contact Paul Lewis, Designated
Federal Officer, via e-mail at lewis.paul epà.gov.
In preparing this document, the Board carefully considered all information provided and
presented by the Agency presenters, as well as information presented by public commenters.
This document addresses the information provided and presented within the structure of the
charge by the Agency.
4 of 28
-------
United States Environmental Protection Agency Human Studies Review Board
Chair
Celia B. Fisher, Ph.D., Marie Ward Doty Professor of Psychology, Director, Center for Ethics
Education, Fordham University, Department of Psychology, Bronx, NY
Vice Chair
William S. Brimijoin, Ph.D., Chair and Professor, Molecular Pharmacology and Experimental
Therapeutics, Mayo Foundation, Rochester, MN *
Members
David C. Bellinger, Ph.D., Professor of Neurology, Harvard Medical School
Professor in the Department of Environmental Health, Harvard School of Public Health
Children’s Hospital, Boston, MA
Alicia Carriquiry, Ph.D., Professor, Department of Statistics, Iowa State University, Ames, IA
Gary L. Chadwick, PharmD, MPH, CIP, Associate Provost, Director, Office for Human Subjects
Protection, University of Rochester, Rochester, NY
Janice Chambers, Ph.D., D.A.B.T., William L. Giles Distinguished Professor, Director, Center
for Environmental Health Sciences, College of Veterinary Medicine, Mississippi State
University, Mississippi State, MS
Richard Fenske, Ph.D., MPH, Professor, Department of Environmental and Occupational Health
Sciences, University of Washington, Seattle WA *
Susan S. Fish, PharmD, MPH, Professor, Biostatistics & Epidemiology, Boston University
School of Public Health, Co-Director, MA in Clinical Investigation, Boston University School of
Medicine, Boston, MA
Suzanne C. Fitzpatrick, Ph.D., DABT, Senior Science Policy Analyst, Office of the
Commissioner, Office of Science and Health Coordination, U.S. Food and Drug Administration,
Rockville, MD
Kannan Krishnan, Ph.D., Professor, Département de sante environnementale et sante au travail,
Faculté de medicine, Université de Montréal, Montréal, Canada
KyungMann Kim, Ph.D., CCRP, Professor & Associate Chair, Department of Biostatistics &
Medical Informatics, School of Medicine and Public Health, University of Wisconsin-Madison,
Madison, WI
5 of 28
-------
Michael D. Lebowitz, Ph.D., FCCP, Professor of Public Health & Medicine. University of
Arizona, Tucson, AZ
Lois D. Lehman-Mckeeman, Ph.D., Distinguished Research Fellow, Discovery Toxicology,
Bristol-Myers Squibb Company, Princeton, NJ *
Jerry A. Menikoff, M.D., Associate Professor of Law, Ethics & Medicine, Director of the
Institute for Bioethics, Law and Public Policy, University of Kansas Medical Center,
Kansas City, KS *
Sean Philpott, PhD., MS Bioethics, Policy and Ethics Officer, Global Campaign for
Microbicides, Program for Appropriate Technology in Health, Washington D.C.
Richard Sharp, PhD., Assistant Professor of Medicine, Center for Medical Ethics and Health
Policy, Baylor College of Medicine, Houston, TX
Human Studies Review Board Staff
Paul I. Lewis, Ph.D., Designated Federal Officer, United States Environmental Protection
Agency, Washington, DC
* Not in attendance at the January 24, 2007 Meeting
6 of 28
-------
TABLE OF CONTENTS
INTRODUCTION 8
REVIEW PROCESS 9
CHARGE TO THE BOARD AND BOARD RESPONSE 10
Completed 1R3535 Insect Repellent Efficacy Studies 10
Insect Repellent Efficacy Protocol SCI-001 22
REFERENCES 28
7 of 28
-------
INTRODUCTION
On January 24, 2007, the United States Environmental Protection Agency’s (EPA or
Agency) Human Studies Review Board (HSRB) met to address scientific and ethical issues
concerning two categories of research:
(1) two completed human studies (EMD 003 and EMD 004) evaluating repellent efficacy of
formulations containing the active ingredient 1R3535.
In its last two meetings (June 27-30, 2006 and October 18-19, 2006) the HSRB reviewed
and commented on materials relating to two insect repellent efficacy protocols from Carroll-
Loye Biological Research. These two protocols described proposed research to evaluate the
efficacy of new formulations of repellent products containing the active ingredient IR-3535. The
protocol identified as EMD-003 described a laboratory study of efficacy of the test formulations
against ticks. The protocol identified as EMD-004 described a field study of efficacy of the test
formulations against mosquitoes.
Following the June 2006 meeting, Dr. Carroll revised the protocols to address comments
from the HSRB. EPA reviewed Dr. Carroll’s revised protocols and concluded that they appeared
likely to generate scientifically sound, useful information and to meet the applicable provisions
of the EPA regulations in 40 CFR part 26, subparts K and L. When the HSRB reconsidered the
revised protocols at its October 2006 meeting, it concurred with EPA’s assessment and suggested
some minor additional refinements. Dr. Carroll proceeded to conduct the research and had
submitted the results to EPA for review.
The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB review of an
EPA decision to rely on the results of these studies. The sponsor has not yet submitted an
application to register these products, but with Agency concurrence submitted the completed
studies ahead of the applications so that HSRB review would not compromise EPA’s ability to
review the application within the time allowed by statute. The Agency expects to receive such
an application in the near future. In order to facilitate timely review of the application, EPA has
reviewed the studies, applying the standard in 40 CFR §26.1705. That provision states:
§ 26.1705 Prohibition on reliance on unethical research with non-pregnant, non-
nursing adults conducted after April 7, 2006
Except as provided in §26.1706, in actions within the scope of § 26.1701, EPA shall not
rely on data from any research initiated after April 7, 2006, unless EPA has adequate
information to determine that the research was conducted in substantial compliance
with subparts A through L of this part.. . This prohibition is in addition to the
prohibition in § 26.1703.
The Agency’s reviews concluded that the data were scientifically sound and that the
research was conducted in a manner that deviates at least technically from some of the
requirements of subparts K and L of EPA’s final rule establishing Protections for Subjects in
Human Research—the only subparts of the rule which apply to third-party research. The
8 of 28
-------
Agency sought the Board’s advice on whether the available information supports a determination
of “substantial compliance” with the applicable rules. Assuming a potential determination of
substantial compliance, and because EPA would like to rely on these data to support an
application for registration of these formulations, EPA presented these studies for review at the
Board’s January 2007 meeting.
(2) research proposal to evaluate the efficacy in the field of multiple formulations of the repellent
DEET against mosquitoes.
EPA requires data from efficacy studies using appropriate insect species to support
claims of greater efficacy than have previously been approved.
An applicant for new or amended registration typically conducts such research prior to
submitting an application. If such a study is to be initiated after April 7, 2006, EPA’ s regulation,
40 CFR §26.1125, requires the sponsor or investigator to submit to EPA, before conducting the
study, materials describing the proposed human research in order to allow EPA to conduct
scientific and ethics reviews. In addition, EPA’s regulation, 40 CFR §26.160 1, requires EPA to
seek HSRB review of the research proposal.
Dr. Scott Carroll submitted a description of proposed research to be performed by
Carroll-Loye Biological Research. The proposal, identified as SCI-001, described a study to
evaluate the efficacy of four formulations of repellent products containing the active ingredient
DEET (one formulation includes two other active ingredients as well). The study would measure
the efficacy of three test formulations and one “comparison article”—the US military standard
repellent—against mosquitoes under field conditions. The proposal bears many similarities to
the protocol EMD-004 that the HSRB had previously reviewed. EPA had reviewed Dr.
Carroll’s protocol and concluded that, with some required refinements, it appeared likely to
generate scientifically sound, useful information and to meet the applicable provisions of the
EPA regulations in 40 CFR part 26, subparts K and L.
In its initial review, EPA identified some relatively easily corrected deficiencies in the
protocol, which must be corrected before execution. In the interest of providing a thorough and
timely response to the proposal, and since EPA found the protocol generally meeting applicable
scientific and ethical standards, EPA presented this protocol for review at the Board’s January
2007 meeting.
This report transmits the HSRB’s comments and recommendations from its January 24,
2007 meeting.
REVIEW PROCESS
On January 24, 2007 the Board had a public face-to-face meeting in Arlington, Virginia.
Advance notice of the meeting was published in the Federal Register “Human Studies Review
Board: Notice of Public Meeting (71 Federal Register 249). At the public meeting, following
welcoming remarks from Agency officials, Celia B. Fisher, 1-IRSB Chair, summarized the
Board’s process for its review. The Board then heard presentations from the Agency on the
9 of 28
-------
following topics: (1) Insect Repellent Completed Efficacy Studies EMD-003 and EMD-004; and
(2) Insect Repellent Efficacy Protocol SCI-0O 1.
The Board received written public comments from Dr. Scott Carroll representing Carroll-
Loye Biological Research. In addition, Dr. Carroll, representing Carroll-Loye Biological
Research, provided oral comments at the meeting.
For their deliberations, the Board considered the materials presented at the meeting,
written public comments and Agency background documents (e.g. pesticide human study,
Agency data evaluation record (DER) of the pesticide human study, weight of evidence review,
ethics review, pesticide human study protocol/study and Agency evaluation of the
protocol/study).
CHARGE TO THE BOARD AND BOARD RESPONSE
Completed 1R3535 Insect Repellent Efficacy Studies
Background
Protocol EMD-003 proposed testing 3 formulations of repellent (lotion, pump spray, and
aerosol) containing the active ingredient IR-3 535 for efficacy in repelling ticks under laboratory
conditions. Protocol EMD-004 proposed testing of the same 3 formulations for efficacy in
repelling mosquitoes under field conditions in two habitats (dense forest and moist pasture or
marshland). EPA guidelines recommend testing in two habitats to assess efficacy in the presence
of different mosquito species with different behaviors. Both protocols had a dosimetry phase to
establish a “typical consumer dose” that would be used in the efficacy phases of the trials.
Twelve subjects pafticipated in dosimetry testing of the 3 formulations whose results were used
for both protocols. An error in formulation of the aerosol test material caused a delay in testing
this formulation, and the reports considered duringthis meeting addressed testing of only the
lotion and pump spray formulations. Separate reports for each formulation (lotion and pump
spray) were submitted for each protocol, and then a subsequent report including both
formulations was re-submitted.
Studies EMD-003.1 and EMD 003.2
Charge to the Board
Scientific Considerations
Are these studies sufficiently sound, from a scientific perspective, to be used to assess the
repellent efficacy of the formulations tested against ticks?
Board Response to the Charge
The active ingredient IR 3535 was tested for its ability to repel ticks on the forearms of
volunteers by the protocol presented and modified by Carroll-Loye. The protocol had been
10 of 28
-------
modified based on the suggestions and input of EPA and HSRB. The results were reported in
- EMD-003.1 and EMD-003.2
The active ingredient was formulated into two products, a pump spray and a lotion, but
data on the originally proposed aerosol was not provided because of an error in the formulation.
The products were produced using Good Manufacturing Practices. All experiments were
conducted using Good Laboratory Practices. A passive dosimetry experiment was done, as
suggested previously by the HSRB both at its June and October 2006 meeting reviewing this
protocol, to determine the amount of product that would be utilized by people using the product
as directed. This passive dosimetry experiment was used to determine a grand mean of the 12
individuals tested (3 subsamples each) per product that was then used for all 10 individuals per
product participating in the subsequent tick repellency tests for each product (It should be noted
that the dosimetry experiment was common for both this study and the mosquito repellency
study, EMD-004, since the same formulated products were used for both).
The experiment was a laboratory study and was conducted according to the approved
protocol with only very minor deviations, and none of these deviations would have affected the
quality of the data or the safety of the subjects. The number of 10 subjects was justified in the
text as leading to sufficient statistical power while exposing only a small number of people to the
potential risks. Each subject had one limb treated. Each of the subjects served as a negative
control in that each tick was tested first on the untreated limb to guarantee that the ticks
demonstrated typical questing behavior (all did) prior to being tested on the treated limb. All
ticks were laboratory reared with no history of tick-borne pathogens. Each tick was used only
once. Repellency was tested during a 3-mm interval each 15 minutes, starting 15 minutes after
product application, using the criterion of First Confirmed Crossing (FCC) for each individual
(replicate) to calculate Complete Protection Time (CPT) for the study. Stopping rules were
employed. The study identified a range of 5-12 hr with a mean CPT of 9.1 hr for the lotion and a
range of 6.5 to 15 hr with a mean CPT of 12.1 hr for the pump spray. The CPT is probably
conservative as a number of the subjects reported no crossings at all, and the experiment was
terminated before a FCC.
With respect to the science criteria established earlier by the HSRB for completed
studies:
General HSRB Scientific Criteria
• The scientific question was stated (i.e., to test the efficacy of 1R3535 in repelling ticks).
• Existing data were not adequate to answer the question of efficacy of these new
formulations.
• Because existing data were not adequate to answer the question of efficacy, new studies
involving human subjects are necessary.
• The potential benefits of the study were clear, i.e., that an effective repellent would be
available that would have either greater efficacy and/or fewer drawbacks than what was
currently approved.
• It is likely that the benefits would be realized because repellent efficacy was determined
in controlled experiments.
11 of28
-------
• The risks are minimal because the formulation products are of very low toxicity and ticks
are laboratory-reared with no evidence of pathogens.
• The most likely relevant risk would be irritation from tick bites, but participants were
instructed to remove ticks before they were bitten.
Study Design Criteria
• The purpose of the study was clearly defined (i.e., efficacy testing).
• There were specific objectives/hypotheses (i.e., that 1R3535 in the proposed formulations
is an effective repellent).
• The study as described tested this hypothesis.
• The sample size was 10 individuals per product with each individual serving as his/her
own negative control to test for tick questing behavior. A dosimetry experiment prior to
the field experiment determined the amount of repellent to be tested.
• There was a plan allocating individuals to treatments.
• It is anticipated that the findings from this study can be generalized beyond the study
sample.
Participation Criteria
• There was justification for the selection of the target population.
• The participants were representative of some of the population of concern; however,
there are others in the population unlike these participants who are likely to use these
products, but it would either be unethical to test them or would be less appropriate to test
them. The participating population is considered appropriate and reasonable.
• The inclusion/exclusion criteria were appropriate.
• The sample was not a vulnerable group.
Measurement Criteria
• The measurements were accurate and reliable.
• The measurements were appropriate to the question being asked.
• Quality assurance was addressed; however, some of the quality assurance was not as
precise as it should have been.
Statistical Analysis Criteria
• The data can be analyzed to calculate CPT with a range of variability.
• The statistical method will be commented upon in more detail in the Board’s response to
protocol SCI-00l below.
• Measures of uncertainty were addressed.
Laboratory and Field Conditions
• Laboratory experiments were appropriate.
• Field experiments were not conducted.
• The study included a stop rule plan, medical management plan, and a safety monitor.
12 of 28
-------
HSRB Consensus and Rationale
The reported studies on the efficacy of lotion and pump spray formulations of 1R3535
(studies EM1D-003.1 and EMD-003.2) on repelling ticks are sufficiently sound, from a scientific
perspective, to be used to assess the repellent efficacy of the two formulations against ticks.
The Board also recognized that advances in statistical analyses mean that there are probably
ways of measuring efficacy of individual products that would be an improvement over traditional
techniques. The Board encouraged the Agency to proceed in its efforts to examine how a
transition to more appropriate methods of calculating efficacy for specific data sets can be
introduced so that consumers can not only compare relative efficacy of products based on
traditional methods but also have better information on the degree of protection individual
products provide.
The Board commended Dr. Carroll for conducting the preliminary dosimetry phase of the
study. Since the Agency noted that this was the first repellency study to have a specific
dosimetry phase, the HSRB suggested EPA might wish to provide guidance concerning whether
the method employed in this study was the most valid way to determine dose. The Board
recommended inclusion of a description of the sampling frame and definition of eligible subjects
to help justify subject generalizability.
Charge to the Board
Ethical Considerations
Does available information support a determination that these studies were conducted in
substantial compliance with subparts K and L of EPA regulations at 40 CFR-part 26?
Brief Overview of the Study
The protocol for these two studies was initially reviewed at the June 2006 meeting of the
HSRB, at which time the Board concluded that the study failed to meet the requirements
established in the Environmental Protection Agency’s final human studies rule (40 CFR Part 26,
subpart K.) The Board also raised questions about: 1) equitable study subject selection and
recruitment; and 2) whether or not the documentation and process of study subject enrollment
was sufficient to meet prevailing standards of voluntary informed consent. A revised Institutional
Review Board (IRB)-approved protocol was submitted and reviewed at the October 2006
meeting of the HSRB, at which the Board concluded that the revised research protocol, as
submitted to the EPA, was compliant with the applicable ethical requirements of 40 CFR Part 26,
subpart K and L.
Subsequent to the aforementioned October meeting of the HSRB, two dosimetry and
efficacy studies for tick repellents containing IR-3535 were conducted from October 23 through
November 8, 2006 (Carroll 2006a; Carroll 2006b). The studies were performed in Davis,
California by researchers at Carroll-Loye Biological Research. The studies were sponsored by
EMD Chemicals, Inc., Gibbstown, New Jersey; EMD Chemicals is the North American
13 of 28
-------
subsidiary of Merck KGaA, Darmstadt, Germany. The documents provided by Carroll-Loye
specifically state that each study was conducted in compliance with the requirements of the U.S.
EPA Good Laboratory Practice Regulations for Pesticide Programs, as promulgated at 40 CFR
Part 160 (Carroll 2006a, 3; Carroll 2006b, 3). Each study was also reviewed and approved by a
commercial human subjects review committee, Independent Investigational Review Board
(IIRB), Inc., Plantation, FL. Documentation provided to the EPA by IIRB indicates that it
reviewed these studies pursuant to the standards of the Common Rule (45 C.F.R. Part 46,
Subpart A) and determined them to be in compliance with that Rule.
As submitted to the EPA, each completed study consisted of two interdependent
analyses: 1) a dosimetry study designed to determine the amount of an insect-repelling
compound, known as IR-3535, that typical users would typically apply when provided with one
of two compound formulations (lotion or pump spray); and 2) an efficacy study designed to
measure the efficacy of IR-3535 as a tick repellent for each compound formulation. Dosimetry
was determined either by passive dosimetry using self-adhesive roll-gauze (pump spray
formulation) or by direct measurement of compound application (lotion formulation). The
efficacy of IR-3535 as a tick repellent was determined by placing Western black-legged ticks
(Ixodespac /Icus) on IR-3535-treated and untreated forearms and measuring the speed and
distance that moving insects would penetrate into the treated area; thus, each subject served as
his/her own control. The scientific strengths and weaknesses of each study design were described
above.
The dosimetry study enrolled a total of 12 individuals, seven women and five men, each
of whom tested both the lotion and pump spray formulations. The efficacy study for each
formulation enrolled 10 subjects: seven women and three men tested the lotion formulation, and
four women and six men tested the pump spray formulation. Two subjects enrolled in the
dosimetry study participated in both the lotion and pump spray efficacy studies. All remaining
subjects participated in only one of the three analytic phases of EMD-003.1 and EMD-003.2,
giving a total of 28 subjects enrolled. In addition, three alternate subjects were enrolled to: 1)
replace any subject who withdrew; and 2) protect the confidentiality of any subject excluded
from the study as a result of pregnancy or other potentially stigmatizing condition, as described
below. Study documents, however, also include limb measurement information for an additional
nine subjects who were not enrolled in either the dosimetry or the efficacy studies. These
subjects appear to be enrolled in two additional studies also submitted the EPA by Carroll-Loye
Biological Research, EMD-004.1 (Completed Efficacy Studies for Mosquito Repellents
Containing IR-3535 — Lotion) and EMID-004.2 (Completed Efficacy Studies for Mosquito
Repellents Containing IR-3535 — Pump Spray) (Carroll 2006c; Carroll 2006d).
Criticiue of Study
The Board concurred with the factual observations of the ethical strengths and
weaknesses of the study, as detailed in the EPA’s Ethics Review (Carley 2006a). In general, the
research described in EMD-003.l and EMD-003.2 comports with the applicable requirements of
40 CFR Part 26, subparts K and L. The risks to study participants were minimal and were
justified by the likely societal benefits, including data on the efficacy of IR-3535 as a tick
repellent. As IR-3535 is commercially available and has been used as a repellent in Europe for
14 of 28
-------
years with no evidence of toxic effects, the subjects enrolled in this study were unlikely to be at
increased risk of experiencing adverse side effects upon exposure. The ticks used for the study
were reared in a laboratory environment and are considered to be pathogen-free, minimizing the
risk of vector-borne disease. Clear stopping rules also were developed, as were plans for the
medical management of any side effects or adverse events; no side effects or adverse events were
reported. The study protocol also included several mechanisms designed to minimize coercive
subject recruitment and enrollment, compensation was not considered to be so high as to unduly
influence participation, and minors and pregnant or lactating women were explicitly excluded
from volunteering (pregnancy being confirmed by requiring all female volunteers to undergo a
self-administered over-the-counter pregnancy test on the day of the study). The potential
stigmatization resulting from study exclusion was minimized by the use of so-called “alternate”
subjects, allowing for volunteers to withdraw or be excluded from participating without unduly
compromising their confidentiality.
The revised protocol and informed consent documents were reviewed and approved by
IIRB, Inc., on November 1, 2006, nine days after study subject enrollment began. In email
correspondence between Dr. Scott Carroll of Carroll-Loye Biological Research and Mr. John
Carley of the EPA’s Office of Pesticide Programs, dated December 18-19, 2006, Dr. Carroll
reported that subjects were enrolled using a previously approved protocol and consent form,
dated September 12, 2006; these were modified to reflect protocol and consent form changes
under review but not yet approved by the IIRB. For example, Dr. Carroll reported that, “to each
of the 12 September consent forms used for subject enrollment ... corrections were made by
hand, and acknowledged by initialing by the subject and Study Director” (Carley and Carroll
2006).
Although it is unlikely that these changes knowingly and/or seriously impaired the
informed consent process, enrollment of subjects using unapproved protocols and consent forms
represents a significant and serious departure from accepted review and approval practices. EPA
regulations regarding review and approval of human subjects research, for example, prohibit
investigators from implementing any protocol changes without prior IRE approval unless such
changes are necessary to prevent immediate, serious harm to study participants. The regulations
also require investigators to only obtain consent using IRB-approved forms. Furthermore, it is
the policy of the IIRB, available online at http://iirb.com , that all “significant protocol deviations
[ be] reported to the Independent Investigational Review Board, Inc. in a timely manner.”
Protocol violations or deviations occur when there is a variance in a research study between what
is described in the protocol approved by the IRB and the actual activities performed by the
research team. The failures of Carroll-Loye Biological Research to 1) obtain IRB approval of the
revised protocol and consent forms prior to enrollment of study subjects, and 2) report these
deviations to the Independent Investigational Review Board, are serious regulatory breaches. The
Board thus recommended that Carroll-Loye Biological Research report these deviations to the
IIRB as soon as possible and work with that organization to develop and implement a corrective
course of action.
15 of 28
-------
HSRB Consensus and Rationale
The Board concurred with the initial assessment of the Agency that the studies EMD—003.1
and EMD-003.2 submitted for review by the Board meet the applicable requirements of 40 CFR
part 26, subparts K and L.
The Board also noted that there were a series of deviations from Subpart K that, while not
adversely affecting the rights and welfare of human subjects of the study, reflected a lack of
familiarity with IRB procedures and protocol requirements described in Subpart K. The HSRB
advised the Agency to recommend that investigators undergo human research protection training,
and include evidence of completion of such training as part of their submission of protocols or
completed studies to the Agency. Examples of such training could include the on-line training
program offering by NIH/NC! or a similar program that might be developed by the Agency.
Completed 1R3535 Insect Repellent Efficacy Studies
Studies EMD-004.1 and EMD 004.2
Charge to the Board
Scientific Considerations
Are these studies sufficiently sound, from a scientific perspective, to be used to assess the
repellent efficacy of the formulations tested against mosquitoes?
Board Response to the Charge
The active ingredient IR 3535 was tested for its ability to repel mosquitoes from the
forearms or legs of volunteers by the protocol presented and modified by Carroll-Loye. The
protocol had been modified based on the suggestions and input of EPA and HSRB. The results
were reported in EMD-004.1 and EMD-004.2
The active ingredient was formulated into two products, a pump spray and a lotion, but
data on the originally proposed aerosol was not provided because of an error in the formulation.
The products were produced using Good Manufacturing Practices. All experiments were
conducted using Good Laboratory Practices. A passive dosimetry experiment was done, as
suggested by the HSRB, to determine the amount of product that would be utilized by people
using the product as directed. This passive dosimetry experiment was used to determine a grand
mean of the 12 individuals tested (3 subsamples each) per product that was then used for all 10
individuals per product participating in the subsequent mosquito repellency tests for each
product. (It should be noted that the dosimetry experiment was in common for both this study
and the tick repellency study, EMD-003, since the same formulated products were used for
both.)
16 of 28
-------
The experiment was a field study and was conducted according to the approved protocol
with only very minor deviations, and none of these deviations would have affected the quality of
the data or the safety of the subjects. Two locations in California were used, one a dense forest
and the other a moist pasture marshland; the two locations had differences in the composition
and relative abundance of mosquito species. Neither location showed evidence of the presence of
West Nile Virus (WNV). The number of 10 subjects per product was justified in the text as
leading to sufficient statistical power while exposing only a small number of people to the
potential risks.
Each subject had one limb treated, and the remainder of the body was covered with
material impervious to mosquitoes. There were two experienced persons serving as negative
controls (i.e., without any repellant product) to confirm mosquito biting pressure (and biting
pressure was maintained throughout the period of the study, defined as at least one Landing with
Intent to Bite, LIBe, per mm). Experimental subjects, in pairs, monitored L1Be’s during a one
mm interval each 15 mm, until the First Confirmed LIBe (FCLIBe) was determined. Stopping
rules were employed. The Complete Protection Time (CPT) was calculated as the mean for all
participants for each product. For the lotion the study identified a range of 6-8.5 hr with a mean
CPT of 7.3 hr for the forest, and a range of 7.75 to 10 hr with a mean CPT of 8.5 hr for the
marsh. For the pump spray the study identified a range of 5 to 8 hr with a mean CPT of 7.1 hr
for the forest and a range of 7.7 to 10 hr with a CPT of 8.4 hr for the marsh. The CPT is
probably conservative as a number of the subjects reported no LIBe’s at all, and the experiment
was terminated before a FCLIBe was observed.
With respect to the science criteria established earlier by the HSRB for completed
studies:
General HSRB Scientific Criteria
• The scientific question was stated (i.e., to test the efficacy of 1R3 535 in repelling
mosquitoes).
• Existing data were not adequate to answer the question of efficacy of these new
formulations.
• Because existing data were not adequate to answer the question of efficacy, new studies
involving human subjects are necessary.
• The potential benefits of the study were clear, i.e., that an effective repellent would be
available that would have either greater efficacy and/or fewer drawbacks than what was
currently approved.
• It is likely that the benefits would be realized because repellent efficacy was determined
in carefully designed field experiments.
• The risks are minimal because the formulation products are of very low toxicity, the
mosquitoes were aspirated before they had an opportunity to bite, and the regions
selected did not have evidence of WNV.
• The most likely relevant risk would be irritation from mosquito bites, but participants
were instructed to remove mosquitoes before they were bitten, or the possibility of
infection with WNV, but the regions selected had no evidence of the virus.
17 of 28
-------
Study Design Criteria
• The purpose of the study was clearly defined (i.e., efficacy testing).
• There were specific objectives/hypotheses (i.e., that 1R3535 in the proposed formulations
is an effective repellent). -
• The study as described tested this hypothesis.
• The sample size was 10 individuals per product along with 2 experienced individuals to
confirm mosquito biting pressure. A dosimetry experiment prior to the field experiment
determined the amount of repellent to be tested.
• There was a plan allocating individuals to treatments.
• It is anticipated that the findings from this study can be generalized beyond the study
sample.
Participation Criteria
• There was justification for the selection of the target population.
• The participants were representative of some of the population of concern; however,
there are others in the population unlike these participants who are likely to use these
products, but it would either be unethical to test them or would be less appropriate to test
them. The participating population is considered appropriate and reasonable.
• The inclusion/exclusion criteria were appropriate.
• The sample was not a vulnerable group.
Measurement Criteria -
• The measurements were accurate and reliable.
• The measurements were appropriate to the question being asked.
• Quality assurance was addressed; however, some of the quality assurance was not as
precise as it should have been.
Statistical Analysis Criteria
• The data can be analyzed to calculate CPT with a range of variability.
• The statistical method will be commented upon in more detail by the Board in its
response to protocol SCI-001 below.
• Measures of uncertainty were addressed.
Laboratory and Field Conditions
• Laboratory experiments were not conducted.
• Field experiments were appropriate.
• The study included a stop rule plan, medical management plan, and a safety monitor.
HSRB Consensus and Rationale
The reported studies on the efficacy of lotion and pump spray formulations of 1R3535
(studies EMD-004.1 and EMD-004.2) on repelling mosquitoes are sufficiently sound, from a
scientific perspective, to be used to assess the repellent efficacy of the two formulations against
mosquitoes.
18 of 28
-------
The Board also recognized that advances in statistical analyses mean that there are probably
ways of measuring efficacy of individual products that would be an improvement over traditional
techniques. The Board encouraged EPA to proceed in its efforts to examine how a transition to
more appropriate methods of calculating efficacy for specific data sets can be introduced so that
consumers can not only compare relative efficacy of products based on traditional methods but
also have better information on the degree of protection individual products provide.
The Board commended Dr. Carroll for conducting the preliminary dosimetry phase of the
study. Since the Agency noted that this was the first repellency study to have a specific
dosimetry phase, the HSRB suggested EPA might wish to provide guidance concerning whether
the method employed in this study was the most valid way to determine dose. The Board
recommended inclusion of a description of the sampling frame and definition of eligible subjects
to help justify subject generalizability.
Charge to the Board
Ethical Considerations
Does available information support a determination that these studies were conducted in
substantial compliance with subparts K and L of EPA regulations at 40 CFR part 26?
Brief Overview of the Study
The protocol for these two studies was initially reviewed at the June 2006 meeting of the
Human Studies Review Board, at which time the Board concluded that the study failed to meet
the requirements established in the Environmental Protection Agency’s final human studies rule
(40 CFR Part 26, subpart K.) The Board also raised questions about: 1) equitable study subject
selection and recruitment; 2) description and minimization of risks to study participants; and 3)
whether or not the documentation and process of study subject enrollment was sufficient to meet
prevailing standards of voluntary informed consent. A revised, Institutional Review Board
(IRB)-approved protocol was submitted and reviewed at the October 2006 meeting of the Human
Studies Review Board, at which the Board concluded that the revised research protocol, as
submitted to the EPA, was compliant with the applicable ethical requirements of 40 CFR Part 26,
subpart K and L.
Subsequent to the aforementioned October meeting of the HSRB, two dosimetry and
efficacy studies for mosquito repellents containing IR-3535 were conducted from October 23
through November 8, 2006 (Carroll 2006c; Carroll 2006d). The studies were performed at a
laboratory site in Davis, California, and at two field sites in Butte and Glenn Counties,
California, by researchers at Carroll-Loye Biological Research. The studies were sponsored by
EMD Chemicals, Inc., Gibbstown, New Jersey; EMD Chemicals is the North American
subsidiary of Merck KGaA, Darmstadt, Germany. The documents provided by Carroll-Loye
specifically state that each study was conducted in compliance with the requirements of the U.S.
EPA Good Laboratory Practice Regulations for Pesticide Programs, as promulgated at 40 CFR
Part 160 (Carroll 2006c, 3; Carroll 2006d, 3). Each study was also reviewed and approved by a
commercial human subjects review committee, Independent Investigational Review Board
19 of 28
-------
(IIRB), Inc., Plantation, FL. Documentation provided to the EPA by ILRB indicates that it
reviewed these studies pursuant to the standards of the Common Rule (45 C.F.R. Part 46,
Subpart A) and determined them to be in compliance with that Rule.
As submitted to the EPA, each completed study consists of two interdependent analyses:
1) a dosimetry study designed to determine the amount of a formulation (lotion or pump spray)
containing an insect-repelling compound, known as IR-3535, that users would typically apply
when provided with one of two compound formulations (lotion or pump spray); and 2) efficacy
studies designed to measure the efficacy of IR-3535 as a mosquito repellent for each
formulation. Dosimetry was determined either by passive dosimetry using self-adhesive roll-
gauze (pump spray formulation) or by direct measurement of compound application (lotion
formulation). The efficacy of IR-3535 as a mosquito repellent was determined by measuring the
ability of the three formulations to prevent mosquito landings (defined as “Lite with Intent to
Bite”; LIBe) under field conditions. Mosquitoes were aspirated mechanically prior to biting;
prior to initiation of the efficacy study, all volunteers were trained both to recognize a mosquito
landing with the intent to bite and to remove such mosquitoes with an aspirator using laboratory-
raised, pathogen-free mosquitoes in a controlled laboratory setting. During the field studies,
subjects worked in pairs to facilitate identification and aspiration of LIBing mosquitoes during
brief exposure periods. The scientific strengths and weaknesses of each study design are
described above.
The dosimetry study enrolled a total of 12 individuals, seven women and five men, each
of whom tested both the lotion and pump spray formulations. The field-based efficacy study for
each formulation enrolled 10 subjects: seven women and three men tested both the lotion and
pump spray formulation over two days at a “forest” site in Butte County, and four women and
six men tested the pump spray formulation both the lotion and pump spray formulation over two
days at a “marsh/pasture” site in Glenn County. One subject enrolled in the dosimetry study
participated in the “forest” efficacy study, three additional subjects participated in both the
“forest” and “marsh/pasture” studies, and a fifth subject participated in the dosimetry, “forest”,
and “marsh/pasture” studies. All remaining subjects participated in only one of the analytic
phases of EMD-004.1 and EMD-004.2. Two control subjects, described as “experienced
personnel” (Carroll 2006c, 9; Carroll 2006d, 9) and who were untreated with either repellent
formulation, also participated to determine ambient LIBe pressure, giving a total of 26 subjects
enrolled. In addition, three alternate subjects were enrolled to: 1) replace any subject who
withdrew; and 2) protect the confidentiality of any subject excluded from the study as a result of
pregnancy or other potentially stigmatizing condition, as described below. Study documents,
however, also include limb measurement information for additional subjects who were not
enrolled in either the dosimetry or the efficacy studies. These subjects appear to be enrolled in
two additional studies submitted to the EPA by Carroll-Loye Biological Research, EMD-003.1
(Completed Efficacy Studies for Tick Repellents Containing IR-3 535 — Lotion) and EMD-003.2
(Completed Efficacy Studies for Tick Repellents Containing IR-3535 — Pump Spray) (Carroll
2006a; Carroll 2006b).
Critique of Study
20 of 28
-------
The Board concurred with the factual observations of the ethical strengths and
weaknesses of the study, as detailed in the EPA’s Ethics Review (Carley 2006b). In general, the
research described in EMD-004. 1 and EMD-004.2 comports with the applicable requirements of
40 CFR Part 26, subparts K and L. The risks to study participants were minimal and were
justified by the likely societal benefits, including data on the efficacy of 1R3535 as a mosquito
repellent. 1R3 535 is commercially available and has been used as a repellent in Europe for years
with no evidence of toxic effects, so the subjects enrolled in this study were unlikely to be at
increased risk of experiencing adverse side effects upon exposure. Reactions to mosquito bites
are usually mild and easily treated with over-the-counter steroidal creams. The study also
excluded subjects who have a history of such severe skin reactions to further minimize the risk of
a subject experiencing a severe physical reaction to a mosquito bite. In addition, the study
protocol was designed specifically to minimize the likelihood that a mosquito will bite, through
the use of clear stopping rules, limited exposure periods, and paired observation; no side effects
or adverse events were reported. To minimize the risk that study subjects would be exposed to
disease causal agents like WNV, the study protocol called for field tests of repellent efficacy to
be conducted only in areas where known vector-borne diseases have not been detected by county
and state health or vector/mosquito control agencies for at least one month. Although it would
have been ideal if the mosquitoes collected during the field studies were subjected to serologic or
molecular analyses to confirm that they were free of known pathogens, it is unlikely that failure
to do so compromised participant safety in any significant way. Finally, the study protocol also
included several mechanisms designed to minimize coercive subject recruitment and enrollment,
compensation was not considered to be so high as to unduly influence participation, and minors
and pregnant or lactating women were explicitly excluded from volunteering (pregnancy being
confirmed by requiring all female volunteers to undergo a self-administered over-the-counter
pregnancy test on the day of the study). The potential stigmatization resulting from study
exclusion was minimized by the use of so-called “alternate” subjects, allowing for volunteers to
withdraw or be excluded from participating without unduly compromising their confidentiality.
As with the two tick repellent studies (EMD-003.l and EMD-003.2), the revised protocol
and informed consent documents used for these mosquito repellent studies were reviewed and
approved by IIRB several days after study subject enrollment began; some subjects participating
in these studies were re-consented using IIRB-approved documents, but not all were. Although it
is unlikely that these changes knowingly and/or seriously impaired the informed consent process,
enrollment of subjects using unapproved protocols and consent forms represents a significant and
serious departure from accepted review and approval practices. The failures of Carroll-Loye
Biological Research to 1) obtain IRB approval of the revised protocol and consent forms prior to
enrollment of study subjects, and 2) report these deviations to IIRB, are serious regulatory
breaches. The Board thus recommended that Carroll-Loye Biological Research report these
deviations to the IIRB as soon as possible and work with that organization to develop and
implement a corrective course of action.
Second, the IIRB-approved protocol and consent documents specifically stated that they
are to be conducted only in areas where known vector-borne diseases have not been detected by
county and state. health or vector/mosquito control agencies for at least one month (see, for
example, Carroll 2006c, 75). One sentinel poultry flock in the area, however, did test positive for
WNV during the month prior to conduct of the field studies (Carroll 2006c, 7). Sentinel flocks
21 of28
-------
closer to the two study sites did not test positive for arboviruses during this period, and a leading
vector control ecologist consulted by Carroll-Loye reported that “WNV activity in Northern
Calfornia [ was] effectively concluded for 2006” (Carroll 2006c, 7), so it is unlikely that
participant safety was compromised in any significant way. Nevertheless, initiation of field
studies following the detection of WNV in a sentinel chicken flock represents a deviation from
the approved protocol and should be reported to the IIRB as soon as possible.
Finally, even though two IR-3535-untreated control subjects were enrolled in the study,
the IIRB-approved consent documents provided for review do not list the unique risks that these
two volunteers faced. These control subjects were “experienced” personnel who were likely
aware of these risks, but nonetheless should have been consented using documents that listed
these dangers.
HSRB Consensus and Rationale
The Board concurred with the initial assessment of the Agency that studies EMD 004.1
and EMD 004.2 submitted for review by the Board met the applicable requirements of 40 CFR
part 26, subparts K and L.
The Board also noted that there were a series of deviations from Subpart K that while not
adversely affecting the rights and welfare of human subjects of the study, reflected a lack of
familiarity with IRB procedures and protocol requirements described in Subpart K. The HSRB
advised the Agency to recommend that investigators undergo human research protection training,
and include evidence of completion of such training as part of their submission of protocols or
completed studies to the Agency. Examples of such training could include the on-line training
program offering by NII-1/NCI or a similar program that might be developed by EPA.
Insect Repellent Efficacy Protocol SCI-OO1
Background
The objectives of this study will be to test the mosquito repellent efficacy characteristics
of three test materials, to compare them to one another, reinforce measurements of time for
which they areeffective, and to contrast them with the U.S. military issue topical insect
repellent. Test Material #lis L1poDEET, which contained 30% DEET that had lipid spheres and
inhibits evaporation, improved field, and reduced plasticizing and odor. Test Material #2,
Coulston’s Duranon, is 20% DEET in a controlled-release, low-odor formulation. Test Material
#3 is Insect Guard II, which contains as active ingredients 17.5% DEET, 5% N-octyl
bicycloheptane dicarboximide (synergist), and 2.5% Di-n-propyl isocinchomerate (fly repellent).
Test Material #4, 3M Ultrathon (military issue repellent), contained 34.34% DEET in a
polymer-based lotion to extend efficacy and reduce plasticizing.
This study will be similar to EMD-004 in terms of the dosimetry phase, efficacy
measurements (time to “first confirmed landing with intent to bite”), and training of subjects in
aspirating mosquitoes before they bite. The field conditions and timing of exposure also will be
similar (treated subjects work in pairs, untreated controls work with 2 assistants to aspirate
22 of 28
-------
landing mosquitoes, and both treated and untreated subjects are exposed to the mosquitoes for 1
minute every 15 minutes). The field testing sites will be in the California Central Valley or
Florida Keys, with expected wild mosquito populations of Aedes vexans, Ochierotatus
melanimon, 0. taeniorhynchus, and Culexpipens. The test results would be analyzed using
unspecified statistics. Measurements would be reported with 95% confidence intervals of the
mean and associated standard deviations. The efficacy of each treatment would be compared to
that of Ultrathon. The sample size reflects a compromise between financial and ethical concerns,
although it was difficult to pre-determine sample size without knowing the distribution of
outcome values. EPA guidelines recommend 6 replicates, which is considered sufficient to show
statistical significance at P
-------
This approach permits direct comparisons among products. In particular, it is possible to
obtain a point estimate and a confidence interval for the difference in CPT in a test product
and in the comparison product, one of the objectives of the study.
If the entire study is replicated in two locations using different subjects, then an even better
approach is to fit a model to the entire set of 80 measurements, but adding an effect for
location (and perhaps an interaction between location and treatment) to the model.
The investigator can easily fit the model using JMP. Just use the Fit X by Y option in the
Analyze menu and then choose ANOVA.
• Interpretation of results: Results from this study need to be interpreted judiciously. Given
the large variability in individual attractiveness to mosquitoes, the small sample size
seriously limits conclusions that the sample is representative of the population of individuals
who might eventually be users of these products. While the long list of exclusions is
justifiable, one consequence is that the population represented in the sample is different from
the population of potential users.
• Sample size: Including 10 subjects per treatment is probably sufficient, but the justification
provided by investigators is not convincing. First, in order to estimate power an estimate of
the within-treatment variance in the response variable is needed. The investigator does not
provide such an estimate in the discussion. Thus it is unclear how they can argue that “from
the standpoint of statistical power, six treated and one untreated subject are sufficient to
demonstrate a significant effect at P<0.05”. Second, the argument used to justify no more
than 10 subjects per treatment states that “adding subjects beyond six increases the precision
of the means estimate only slowly”. This argument relies on the assumption that the between-
person variance in CPT does not change as sample size increases, which in general is not
true. The information on interindividual variability drawn from studies completed by these
investigators may be used to guide and justify sample size. Submission of the completed
protocol to EPA should include evidence that steps like those described above were taken to
justify sample size.
• Sample Size Considerations for Subject Drop-Outs. In previous studies, subjects dropped out
at different points potentially confounding the quantification of the CPT. Criteria need to be
established for how long subjects must remain in the study in order for their data to be used.
Criteria for when a new subject must be run as a substitute to meet the sample size
requirements must also be determined.
• Assumption of normality of CPT measurements: In choosing statistical analyses the
investigator must select the appropriate model for the distribution of the data that will be
used. The methods for the statistical analysis of the data rely heavily on the assumption that
measurements are normal. Because of the small sample size, departures from normality can
have important consequences on the validity of the methodology proposed here. In this case,
the assumption of normality is probably justifiable and in any case can be easily tested and
corrected for. There seems to be some confusion) regarding the exponential family of
distributions and the exponential distribution. The latter is a standard probability model for
variables such as time which are strictly positive and tend to exhibit a rounded L shape when
plotted. The former has nothing to do with the study at hand. If CPTs can all be expected to
be noticeably larger than 0, then approximating the exponential model with a normal model
may be justifiable.
• Measurement variables: Although it is clear from the rest of the protocol discussion, the
investigator might consider adding CPT to the list of variables given in Section 10.1.
24 of 28
-------
• Dose: Even though it is suggested that the typical consumer exposure should be far below
the dermal toxicity benchmarks, there is no indication of such toxicological data in the
MSDS included with this submission. Typical consumer dose and known toxicity
benchmarks should be clearly identified.
HSRB Consensus and Rationale
The Board raised several concerns about sample size, population generalizability and
statistical analysis in SCI-00l that should be addressed. If the recommendations provided by
EPA and those suggested by the Board are followed, protocol SCI-O0 1 appears likely to generate
scientifically valid data to assess the efficacy of the test products against mosquitoes. In
addition, the protocol would satisfy the scientific criteria recommended by the HSRB, namely,
producing important information that cannot be obtained except by research with human
subjects, and having a clear scientific objective and study design that should produce adequate
data to test the hypothesis.
Charge to the Board
Ethical Considerations
If the proposed research described in Protocol SCI-001 from Carroll-Loye Biological
Research is revised as suggested by EPA, does the research appear to meet the applicable
requirements of 40 CFR part 26, subparts K and L?
Board Response to the Charge
The Board concurred with the factual observations of the ethical strengths and
weaknesses of the study, as detailed in the EPA’s Science and Ethics Review (Carley 2006c). In
general, the research described in Protocol SCI-OOlcomports with the applicable requirements of
40 CFR Part 26, subpart K and L. The risks to study participants are limited and appropriate
steps have been taken to minimize these risks. The risks to participants are justified by the likely
societal benefits, including data on the efficacy of new topical formulations containing DEET as
a mosquito repellent. DEET is commercially available and has been used as a repellent for years
with no evidence of substantial toxic effects, so the subjects enrolled in this study are unlikely to
be at increased risk of experiencing adverse side effects upon exposure. Reactions to mosquito
bites are usually mild and easily treated with over-the-counter steroidal creams. In addition, the
study protocol is designed to minimize the likelihood that a mosquito will bite, through the use
of clear stopping rules, limited exposure periods, and paired observation. To minimize the risk
that study subjects will be exposed to illnesses resulting from WNV, the study protocol calls for
field tests of repellent formulations to be conducted only in areas where known vector-borne
diseases have not been detected by county and state health or vector/mosquito control agencies.
The Board recommended that the investigator collect mosquitoes during the field studies
and that they be subject to serologic or molecular analyses to confirm absence of known
pathogens. Finally, the study protocol included several mechanisms designed to minimize
coercive subject recruitment and enrollment, compensation was not considered to be so high as
25 of 28
-------
to unduly influence participation, and minors and pregnant or lactating women were explicitly
excluded from volunteering (pregnancy being confirmed by requiring all female volunteers to
undergo a self-administered over-the-counter pregnancy test on the day of the study). The
potential stigmatization resulting from study exclusion was minimized by the use of so-called
“alternate” subjects, allowing for volunteers to withdraw or be excluded from participating
without unduly compromising their confidentiality.
The Board concluded that research described in Protocol SCI-OO 1 minimizes risks to
subjects and has appropriate stopping rules in place. The safety monitoring proposed seems
reasonable and appropriate in light of the level of risk to subjects. Despite this generally
favorable assessment, the Board considered several additional matters relevant to subject
recruitment and the overall conduct of the study.
First, as noted in the Agency’s review of Protocol SCI-00 1, the protocol does not
describe how untreated controls would be recruited. The protocol implies that controls will be
recruited in the same manner as subjects in the “exposure” arm—via “word-of-mouth” and a
Volunteer Data Base maintained by the Principal Investigator. The protocol should clarify how
untreated controls will be recruited. . The Board also found it a bit unusual that the IRB did not
ask to review a script of the proposed recruitment phone call as most IRBs regard recruitment as
the first step in the IC process and require that all recruitment activities be reviewed. This would
include any fliers, emails, letters, or local ads as well, which should be submitted to IIRB for
review.
Second, the Board discussed several issues related to subject recruitment and consent.
First, the risks associated with DEET exposure during the course of the study are
mischaracterized in the submitted informed-consent document, which refers to sprayed
applications containing alcohol. Since the study involved the application of lotions to the skin,
these risks should be redescribed. In addition, the informed-consent document is structured in a
manner that does not apply to unexposed control subjects. Also, the submitted informed-consent
document indicates that up to 40 subjects may participate in the study when the correct number
should be 48 (10 exposed and 2 controls per arm of the study).
Third, the Board discussed the fact that the proposed sample size is slightly larger than
what EPA has historically required (10 exposed subjects vs. the historical norm of 6 exposed
subjects). The protocol provides a rationale for this approach (pp. 13-15), which is meant to
reduce the probability that the sample over-represents individuals who are “inherently
unattractive” to mosquitoes. In light of the limited risks to subjects, this departure from the
historical norm was viewed as acceptable by the Board.
Fourth, the Board found it difficult to assess the qualifications of the IIRB based on the
materials that were supplied. Although the Board did not have significant concerns about the
overall quality of the IRB’s review of the protocol, it would be reassuring to the Board if some
type of documentation of the IRB’s qualifications were provided to the Agency for review (e.g.,
evidence of member training, accreditation by an external professional body, etc.).
26 of 28
-------
HSRB Consensus and Rationale
The Board concluded that the protocol should meet the applicable requirements of 40
CFR part 26, subparts K and L if the points raised in the EPA review and in this report are
adequately addressed.
27 of 28
-------
REFERENCES
Carley, J.M. 2006a. Ethics Review of EMD-003 Reports of Completed Efficacy Studies for Tick
Repellents Containing IR-3535. Dated December 19, 2006. Unpublished document prepared by
Office of Pesticide Programs, United States Environmental Protection Agency.
Carley, J.M. 2006b. Ethics Review of EMD-004 Reports of Completed Efficacy Studies for
Mosquito Repellents Containing IR-3535. Dated December 19, 2006. Unpublished document
prepared by Office of Pesticide Programs, United States Environmental Protection Agency.
Carley, J.M. 2006c. Ethics Review of SCI-001. Unpublished document prepared by Office of
Pesticide Programs, United States Environmental Protection Agency.
Carley, J.M., and S. Carroll. 2006. E-mail Exchange Between Scott P. Carroll of Carroll-Loye
Biological Research and John M. Carley of EPA Concerning Repellent Studies EMD-003 and
EMD-004 on December 18-19, 2006. Unpublished document prepared by Office of Pesticide
Programs, United States Environmental Protection Agency.
Carroll, S. 2006a. Test of Personal Insect Repellents: Study EMD-003.1 (Lotion) — Revised.
Dated December 15, 2006. Unpublished document prepared by
Carroll-Loye Biological Research.
Carroll, S. 2006b. Test of Personal Insect Repellents: Study EMD-003.2 (Pump Spray) —
Revised. Dated December 15, 2006. Unpublished document prepared by Carroll-Loye
Biological Research.
Carroll, S. 2006c. Test of Personal Insect Repellents: Study EMD-004.l (Lotion) — Revised.
Dated December 15, 2006. Unpublished document prepared by
Carroll-Loye Biological Research.
Carroll, S. 2006d. Test of Personal Insect Repellents: Study EMD-004.2 (Pump Spray) —
Revised. Dated December 15, 2006. Unpublished document prepared by Carroll-Loye
Biological Research.
28 of 28
------- |