March 6,2008
EPA-HSRB-08-01
George Gray, Ph.D.
Science Advisor
Office of the Science Advisor
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Subject: October 24-26, 2007 EPA Human Studies Review Board Meeting Report
Dear Dr. Gray:
The United States Environmental Protection Agency (EPA or Agency) requested the
Human Studies Review Board (HSRB) to review scientific and ethical issues addressing: (1)
the EPA Office of Research and Development's document, Scientific and Ethical Approaches
for Observational Exposure Studies; (2) a sodium azide study; (3) Carroll-Loye Biological
Research Completed Field Efficacy Studies (SCI-001 and WPC-001); (4) Carroll-Loye
Biological Research Proposed Insect Repellent Efficacy Protocols (SPC-001 and SPC-002);
and (5) ICR Proposed Repellent Efficacy Protocol (A 117). The enclosed HSRB report
provides the Board's response to EPA charge questions presented at the-October 24-26, 2007
meeting.
At previous HSRB meetings, the Board raised a number of science questions related to
mosquito repellent efficacy field research. At the Board's requested, three consultants in the
field of entomology were invited to the October 2007 HSRB meeting to discuss the frequency
and duration of exposure of subjects to potential mosquito landings. Board discussion of the
consultants' comments is briefly provided. This HSRB report includes Board discussions on
the consultants opinions presented in the consultants' responses and supplemental information
shared at the meeting. Finally, the Board also appreciated the Agency highlighting progress
on issues relating to the design of sampling strategies for handler research programs proposed
by the Agricultural Handlers Exposure Task Force and the Antimicrobials Exposure
Assessment Task Force II.
On a general note, future protocols should include a statistical analysis plan for
subsequent Board review. In addition, the Board requests that revised protocols or subsequent
studies submitted to the Agency should include a response to changes as specified by the
Agency. Such a written response would assist the Board in its review process. Finally, the
HSRB requests expert consultants at a future meeting to educate the Board and Agency on
acceptable statistical approaches to insect repellent studies.
A summary of the Board's conclusions is provided below.
Iof85
-------
Scientific and Ethical Approaches for Observational Exposure Studies
Introduction. Purpose and Scope
• The Board concurred that the document is extremely well-written, presents
information accurately and clearly, and will be extremely useful; however, the
Board provided suggestions for improvements.
• Expand the discussion on how observational research is distinguished from
intentional exposure research with specific illustrations, including discussion of the
need for ethical review and evaluation of observational research.
• The document presents more ethical than science approaches for observational
exposure studies. The Board recommended that the document be revised to present
a more balanced presentation and highlight the interplay between good science and
ethics.
• The Board also recommended providing examples of study questions, citing other
documents to guide researchers in alternative research designs, and providing
bulleted summary highlights at the end of each chapter.
• The abstract states that the document will address chemicals and other stressors, but
focuses on chemicals, which gives the impression that attention is not needed
except for research involving chemicals. The language and examples used in the
document should reflect the importance of the document for both chemicals and
other stressors.
Elements to Be Considered in Study Conceptualization and Planning
• The Board concurred that Section 2 adequately identified the major areas and issues
where ethical considerations should be addressed in the study conceptualization. The
Board suggested that rather than having distinct sections and even documents on ethics
and science there be just one document, with the study design elements being a portion
of the human subjects protocol. This would more closely parallel the information
submitted to an IRB.
• Bulleted items should also include justification for sample selection and size and
sampling method, discussion of alternative designs that were rejected to help justify the
use of human subjects in the particular way being proposed, expanded examples of
conflicts of interest, and the importance of using validated measures. Expansion of text
regarding adverse events vs. unanticipated problems, scientific misconduct, subject
attrition, reporting biases etc. were recommended.
Ensuring rotections of Vulnerable Populations
• The Board recommended additional discussion on justifications for including
vulnerable populations in research as well as expansion on discussion of who is
vulnerable outside of the federal regulations (e.g. pregnant women, prisoners, children)
such as economic, educational or social vulnerabilities; noting however that in federal
regulations vulnerability is defined in terms of susceptibility to coercion and undue
influence.
2 of 85
-------
• Expand examples of studies that might involve these populations.
• Discuss the tension between over and under sampling these populations including the
importance of including vulnerable populations in research to ensure that there is safety
data on compounds to which they are more likely than other populations to be exposed.
• Discuss the weighing of risks and benefits in these types of studies.
Privacy. Confidentiality, and other Concerns Related to Observational Exposure
Studies
• This section of the document is sensitive to many key ethical and legal considerations
relating to the safeguarding of research-subject privacy, including the need to (1)
disclose the possibility of incidental reporting requirements to potential volunteers as
part of the informed-consent process, (2) provide advance notification of research visits
to third parties who may be residing in private environments, and (3) consider potential
harms to research volunteers who display personal monitoring equipment in public
settings. It would be strengthened by including specific advice on reporting and
disclosure procedures when confidential information indicates a participant or another
person is in jeopardy, the uses of the Certificate of Confidentiality and additional
references.
Creating an Appropriate Relationship Between Participant and Researcher
• Most of the major areas and issues where ethical considerations should be addressed
were included in this section, with the exception of communication/language issues.
• However, in many cases the section takes the tone of reporting what others have said,
with and without comment; and sometimes (and more appropriately) stating “it is
recommended.” A document with clear recommendations will be more useful by
researchers than one that raises issues without giving direction.
• 0MB and other guidelines for payment/remuneration could be included.
• Provide more examples of appropriate participant-researcher relationships to
observational studies.
• The role of informed consent and the IRB or other institutional representative who
protects subjects’ rights.
Building and Maintaining Appropriate Community and Stakeholder Relationships
• Many of the major issues requiring ethical considerations were included. However,
more data to support points made and less assertion or “theory” would strengthen this
section. It is also important to differentiate the terms stakeholder and community as
well as their interrelationships and discussing the value of community advisory boards
and community sensitive piloting of procedures.
• Successful community advisory board procedures, how the scientist-community
relationship will evolve and be monitored over time, how the results of research are
disseminated and the informational benefits to the community should also be discussed.
3 of 85
-------
• The Board also cautioned about editing the section so that readers would not
erroneously conclude that the EPA is advocating that scientists become community
advocates.
Designing and Implementing Strategies for Effective Communication
• This section had a very comprehensive and informative list of reference and several
suggestions for additional references have been noted. While this section is very well-
written, it does not clearly focus on the communication methods most suitable for
observational exposure studies. The use of side bars is a very effective tool to
communicate small bits of information clearly and quickly.
• The goals of the communication should also be included. In this regard, sections on
data sharing and how to address potential scientist-community disagreements (e.g.,
interpretation of data) would be helpful.
• The context in which communication occurs (e.g. a participant’s home) should also be
discussed.
• The importance of formative (process) evaluation and the importance of considering in
advance how the data might be used should be included.
Science and Ethics of Sodium Azide Study
• Based on the inadequacies in the design, methodology and reporting, the Board
concluded that the Black study does not contain sufficient information to be used as a
point of departure to estimate a safe level of acute and chronic exposure to sodium
azide.
• Based on lack of documentation in the Black et al. study, the Board was unable to
conclude that there was clear and convincing evidence that the conduct of the study was
fundamentally unethical.
• The Board was also unable to conclude that there was clear and convincing evidence
that the study was significantly deficient relative to the ethical standards prevailing at
the time the research was conducted.
Science Issues in Mosquito Repellent Efficacy Field Research
• The Board concluded that more research is needed to determine biases and adjustments
in mosquito repellent efficacy research.
• The Board remains unclear of what the mean of the times to the first 5 bites/(landings
with intent to bite) would measure and relevance to EPA determinations of efficacy.
• The Board concluded that that it would be helpful for HSRB deliberations if protocols
contained rationales for sample size, outcome measures, number of treatment groups
and controls, why a field study is preferable, why a specific environment was selected,
how different environments differ, and how controls for environmental shifts in
temperature or time of day are determined.
4 of 85
-------
• The Board understands that the need for larger sample sizes and corresponding
increase in statistical power must be balanced with subject protection, but it is also
important to understand which variables can be controlled. The expertise of control
and treated subjects with respect to detecting mosquito landings must be balanced and
the activity of subjects also should be controlled.
Completed Insect Repellent Efficacy Study (SCI-00fl of DEET Formulations
• While the Board concluded that the participation of several subjects on the day after
they had been treated with a different test repellant was not ideal, this may not have
affected the validity of the results.
• While the Board reviewed this protocol generally favorably previously, since it was
combined with a second protocol, the conduct and analysis deviated from expectation.
For example, with only 33 subjects for 80 data points (excluding the negative controls),
the overlap of some of the same subjects for different test materials for Sites 1 and 2,
and for different dates of the experiment without proper experimental design and
control, the Board concluded that it is impossible to interpret the reported data
adequately, thus bringing the scientific validity of the results into question. In addition,
the study may not have been sufficiently sound enough to estimate population
variances. Thus, the Board concluded that the study was not sufficiently sound, from a
scientific perspective, to be used to assess the repellent efficacy of the formulations
tested against mosquitoes.
• In addition, the Board concluded that the research was conducted in a manner that
failed to meet the applicable requirements of §40 CFR 26, subparts K and L. The study
investigator failed to obtain IRB approval for fundamental changes to one of the study
protocols, namely substituting an unregistered compound for the study compound. As
a result, the Board recommended that the data collected from these three concurrently
run studies should not be considered by the Agency because the changes placed
volunteers at risk of being randomized to receive treatment with an unregistered
compound.
Completed Insect Repellent Efficacy Study with Oil of Lemon Eucalyptus (WPC-001 )
• The Board concluded that despite problems estimating variability, the Carroll-Loye
Biological Research study WPC-001 assessing the repellent efficacy of the formulation
tested was sufficiently sound for the purposes for which it was intended.
• A majority of the Board concurred with the initial assessment of the Agency that the
study submitted for review by the Board meets the applicable requirements of §40 CFR
26, subparts K and L.
Proposed Carroll-Loye Picaridin Insect Repellent Efficacy Studies (SPC-00l )
• While the Board agreed that the study rationale, formulations to be tested and data
collection procedures were scientifically sound, the protocol did not adequately
explain the relationship between the study design and analytic plan, nor did it include
an appropriate statistical analysis plan (including estimation of variability) that could
be evaluated for its validity or utility. Thus, the Board was concerned that the
5 of 85
-------
proposed research may not appear likely to generate scientifically reliable data, useful
for assessing the efficacy of the test substance for repelling mosquitoes.
• The Board concurred with the assessment of the Agency that the revised protocol
submitted for review by the Board meets the applicable requirements of §40 CFR 26,
subparts K and L.
Proposed Carroll-Love Picaridin Insect Repellent Efficacy Studies (SPC-002 )
• The Board concluded that the research appears likely to generate scientifically reliable
data, useful for assessing the efficacy of the test substances for repelling ticks, provided
that the revisions suggested by EPA are incorporated, the experimental design is made
more specific to the allocation of the test substances into three groups of subjects and
that there is no overlap of subjects from one test group to the other.
• The Board underscored that the statistical analysis plan was not well-laid out and urged
EPA to ensure there was a sufficient analytic plan before the study be conducted.
• The Board urges EPA to consider the design of newer studies and the designs already
used for existing products to make certain that labels reflect information of comparative
value to consumers.
• The Board concurred with the assessment of the Agency that the revised protocol
submitted for review by the Board meets the applicable requirements of §40 CFR 26,
subparts K and L.
Proposed ICR Picaridin Insect Repellent Efficacy Study (A 117 )
• The Board concluded that proposed research is generally clear and appropriately
designed, with the exception of the appropriate statistics.
• The Board concurred that if the proposed research is revised consistent with EPA’s
recommendations and the Board’s suggestions, the study should yield valid data
regarding the efficacy of these products in repelling Culex.
• The Board concurred with the assessment of the Agency that the protocol, if revised as
suggested in both EPA’s review and by the Board, would meet the applicable
requirements of §40 CFR 26, subparts K and L.
In conclusion, the EPA HSRB appreciated the opportunity to advise the Agency on the
scientific and ethical aspects of human studies research and looks forward to future
opportunities to continue advising the Agency in this endeavor.
Sincerely,
Celia Fisher, Ph.D., Chair
EPA Human Studies Review Board
6 of 85
-------
NOTICE
This report has been written as part of the activities of the EPA Human Studies Review
Board, a Federal advisory committee providing advice, information and recommendations on
issues related to scientific and ethical aspects of human subjects research. This report has not
been reviewed for approval by the Agency and, hence, the contents of this report do not
necessarily represent the view and policies of the Environmental Protection Agency, nor of
other agencies in the Executive Branch of the Federal government, nor does the mention of
trade names or commercial products constitute a recommendation for use. Further information
about the EPA Human Studies Review Board can be obtained from its website at
http://www.epa.gov/osalhsrb/. Interested persons are invited to contact Paul Lewis, Designated
Federal Officer, via e-mail at lewis.paul epa.gov.
In preparing this document, the Board careflully considered all information provided
and presented by the Agency presenters, as well as information presented by public
commenters. This document addresses the information provided and presented within the
structure of the charge by the Agency.
7 of 85
-------
U. S. ENVIRONMENTAL PROTECTION AGENCY HUMAN STUDIES REVIEW
BOARD MEMBERS
Chair
Celia B. Fisher, Ph.D., Marie Ward Doty Professor of Psychology, Director, Center for Ethics
Education, Fordham University, Department of Psychology, Bronx, NY
Vice Chair
William S. Brimijoin, Ph.D., Chair and Professor, Molecular Pharmacology and Experimental
Therapeutics, Mayo Foundation, Rochester, MN
Members
Alicia Carriquiry, Ph.D., Professor, Department of Statistics, Iowa State University
Snedecor Hall, Ames, IA *
Gary L. Chadwick, PharmD, MPH, CIP, Associate Provost, Director, Office for Human
Subjects Protection, University of Rochester, Rochester, NY
Janice Chambers, Ph.D., D.A.B.T., William L. Giles Distinguished Professor, Director, Center
for Environmental Health Sciences, College of Veterinary Medicine, Mississippi State
University, Mississippi State, MS
Richard Fenske, Ph.D., MPH, Professor, Department of Environmental and Occupational
Health Sciences, University of Washington, Seattle, WA*
Susan S. Fish, PharmD, MPH, Professor, Biostatistics & Epidemiology, Boston University
School of Public Health, Co-Director, MA in Clinical Investigation, Boston University School
of Medicine, Boston, MA
Suzanne C. Fitzpatrick, Ph.D., D.A.B.T, Senior Science Policy Analyst, Office of the
Commissioner, Office of Science and Health Coordination, U.S. Food and Drug
Administration, Rockville, MD
Dallas E. Johnson, Ph.D., Professor Emeritus, Department of Statistics, Kansas State
University, Manhattan, KS
KyungMann Kim, Ph.D., CCRP, Professor & Associate Chair, Department of Biostatistics &
Medical Informatics, School of Medicine and Public Health, University of Wisconsin-
Madison, Madison, WI
Kannan Krishnan, Ph.D., Professor, Département de sante environnementale et sante au travail,
Faculté de medicine, Université de Montréal, Montréal, Canada
8 of 85
-------
Michael D. Lebowitz, Ph.D., FCCP, Retired Professor of Public Health (Epidemiology) and
Medicine and Research Professor of Medicine. University of Arizona, Tucson, AZ
Lois D. Lehman-Mckeeman, Ph.D., Distinguished Research Fellow, Discovery Toxicology,
Bristol-Myers Squibb Company, Princeton, NJ
Jerry A. Menikoff, M.D., Director, Office of Human Subjects Research, Office of the Director,
National Institutes of Health, Bethesda, MD
Rebecca Parkin, Ph.D., Associate Dean for Research and Public Health Practice, School of
Public Health and Human Services, The George Washington University, Washington, DC
Sean Philpott, Ph.D., MS Bioethics, Policy and Ethics Director, Global Campaign for
Microbicides, Program for Appropriate Technology in Health, Washington, DC
Ernest D. Prentice, Ph.D., Associate Vice Chancellor for Academic Affairs, University of
Nebraska Medical Center, Omaha, NE*
Richard Sharp, Ph.D., Director of Bioethics Research, Department of Bioethics, Cleveland
Clinic, Cleveland, OH
Consultants to the Board
Scient /Ic and Ethical Approaches for Observational Exposure Studies Session
Germaine Buck-Louis, Ph.D.
Division of Epidemiology, Statistics & Prevention Research
National Institute of Children & Human Development
Rockville, MD
Barry Ryan, Ph.D.
Department of Environmental and Occupational Health
Rollins School of Public Health
Emory University
Atlanta, GA
Frequency, Duration and Timing of Exposure of Subjects to Potential Mosquito Landings
Session
Col. Raj. Gupta, Ph.D.
Director, Research Plans and Programs
Walter Reed Army Medical Center
Medical Research and Material Command
Fort Detrick, MD
9 of 85
-------
Steve Schofield, Ph.D.
Department of National Defence
Canadian Forces Health Services Group — HQ Ottawa
Force Health Protection
Communicable Disease Control Program
Ottawa, Ontario, Canada
Daniel Strickman, Ph.D.
USDA, ARS
National Program Leader
Program 104: Veterinary, Medical, and Urban Entomology
National Program Staff - APP
Beltsville, MD
Human Studies Review Board Staff
Paul I. Lewis, Ph.D., Designated Federal Officer, United States Environmental Protection
Agency, Washington, DC
* Not in attendance at October 24-26, 2007 Public Meeting
10 of 85
-------
TABLE OF CONTENTS
U. S. ENVIRONMENTAL PROTECTION AGENCY HUMAN STUDIES REVIEW
BOARD MEMBERS 8
INTRODUCTION 12
REVIEW PROCESS 21
CHARGE TO THE BOARD AND BOARD RESPONSE 22
A. Scientific and Ethical Approaches for Observational Exposure Studies 22
B. Completed Oral Therapeutic Study with Sodium Azide 43
C. Science Issues in Mosquito Repellent Efficacy Field Research 47
D. Completed Insect Repellent Efficacy Study (SCI-OO 1) of DEET Formulations 51
E. Completed Insect Repellent Efficacy Study with Oil of Lemon Eucalyptus (WPC-OO1) ... 61
F(1). Proposed Carroll-Loye Picaridin Insect Repellent Efficacy Studies (SPC-OOI) 65
F(2). Proposed Carroll-Loye Picaridin Insect Repellent Efficacy Studies (SPC-002) 69
G. Proposed ICR Picaridin Insect Repellent Efficacy Study (A 117) 74
REFERENCES 81
APPENDIX A: DISCUSSION QUESTIONS FOR MOSQUITO REPELLENT STUDIES 83
11 of 85
-------
INTRODUCTION
From October 24-26, 2007, the United States Environmental Protection Agency’s (EPA
or Agency) Human Studies Review Board (HSRB) met to address scientific and ethical issues
concerning:
A. Scientific and Ethical Approaches for Observational Exposure Studies
Scientists at the U.S. Environmental Protection Agency’s (EPA’s), Office of Research
and Development’s National Exposure Research Laboratory (NERL) have conducted
observational exposure measurement research for several decades to understand how and why
people come into contact with chemicals and other stressors in their everyday lives. These
studies are performed to determine what chemicals people are exposed to, the concentrations of
the chemicals, the most important sources contributing to people’s exposures, the routes and
pathways of exposure, and the factors that have the biggest impact on exposure.
EPA strives to follow the most up-to-date approaches in designing and performing
observational exposure studies to ensure that these studies are based on sound science and meet
the highest ethical standards. To meet that goal, researchers in NERL have prepared a draft
document that identifies key scientific and ethical issues and provides information and
resources to assist researchers as they plan and implement observational exposure studies. The
document is not meant to represent an official Agency “guidance document.” Moreover, it
recognizes that researchers will work with others — EPA’s Human Subjects Research Review
Official, Institutional Review Board (IRB) members, the participants and their community, and
other stakeholders — to identify and address all of the relevant issues for any specific study to
ensure that all participants are respected and protected.
Review material : EPA provided the following materials to the HSRB relevant to this
topic:
1. External review draft document titled Scient /Ic and Ethical Approaches for
Observational Exposure Studies
2. Charge Questions
3. Report on the Workshop to Discuss State-of-the-Science Approaches for Observational
Exposure Measurement Studies, dated January 25, 2007. The report provides background
on the framework for the document and topic areas recommended by an expert panel.
B. Completed Oral Therapeutic Study with Sodium Azide
In its registration program EPA reexamines the safety of pesticides being proposed for
new or amended registration. The Agency is currently reviewing an application for registration
of the active ingredient, sodium azide (NaN 3 ), as a limited replacement for the tI migant,
methyl bromide. The application seeks to register sodium azide for commercial production of
12 of 85
-------
ornamental cut flowers and pre-plant application via drip tape irrigation on beds under plastic
mulch; for sod farms with pre-plant application to soil with tarping after application; and for
golf course turf area renovation with pre-plant application and immediate tarping.
Sodium azide also has been used for many years as a laboratory reagent and as a raw
material for production of azide-containing compounds. It has been used as a pharmaceutical
intermediate and as a preservative of blood, laboratory reagents, and biological fluids. It has
been used as a gas generant in automotive airbags, and was commonly used in early inflator
designs. During the 1 990s, however, airbag propellants containing NaN 3 were phased out in
favor of more efficient, less expensive and less toxic alternatives. In the past, NaN 3 was also
used as a pharmaceutical to treat high blood pressure and as an anti-neoplastic agent.
EPA has identified a study published in 1954 in which human subjects received oral
doses of sodium azide to assess its potential for lowering blood pressure. The Agency intends
to use this study in its hazard assessment to derive a “point of departure” (POD) for assessing
acute and chronic toxicity resulting from both acute and chronic exposures to this chemical.
The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB review of an
EPA decision to rely on the results of any study if the research was “initiated before April 7,
2006, and the research was conducted for the purpose of identifying or measuring a toxic
effect.” EPA has reviewed the study, applying the standards in 40 CFR § 26. 1703 and
26.1704. Those provisions state:
§26.1703 Prohibition of reliance on research involving intentional exposure of
human subjects who are pregnant women (and therefore their fetuses), nursing
women, or children.
Except as provided in §26.1706, in actions within the scope of §26.1701 EPA shall
not rely on data from any research involving intentional exposure of any human
subject who is a pregnant woman (and therefore her fetus), a nursing woman, or a
child.
§26.1704 Prohibition on reliance on unethical research with non-pregnant, non-
nursing adults conducted before April 7, 2006
Except as provided in §26.1706, in actions within the scope of §26.1701, EPA shall
not rely on data from any research initiated before April 7, 2006, if there is clear and
convincing evidence that the conduct of the research was fundamentally unethical
(e.g., the research was intended to seriously harm participants or failed to obtain
informed consent), or was significantly deficient relative to the ethical standards
prevailing at the time the research was conducted. This prohibition is in addition to
the prohibition in §26.1703.
The Agency’s reviews concluded that the data were scientifically sound and that there
was no clear and convincing evidence that the conduct of the research was fundamentally
unethical or significantly deficient relative to the ethical standards prevailing at the time the
13 of 85
-------
research was conducted. Nor was there evidence to show that the subjects included nursing or
pregnant women or children.
Review materials . EPA provided the following materials to the HSRB relevant to the
completed oral therapeutic study with sodium azide:
a. MRID 47221401 Black et al 1954
Black, M.; Zweifach, B.; Speer, F. (1954) “Comparison of Hypotensive Action of
Sodium Azide in Normotensive and Hypertensive Patients.” In Proceedings of the
Society for Experimental Biology and Medicine, Jan 1954, pp. 11-16. MRID
47221401.
b. MRID 47221401 Data Evaluation Record
c. EPA WOE Sodium Azide 9-18-07
Memorandum from Nancy McCarroll to Jack Housenger, Associate Director Health
Effects Division, “Human Studies Review Board: Weight of Evidence Discussion for
Sodium azide (NaN 3 ).” September 18, 2007.
d. EPA Ethics Review MRID 47221401 9-27-07
C. Science Issues in Mosquito Repellent Efficacy Field Research
Currently, EPA requires all pesticide products that claim to repel mosquitoes to provide
data on the duration of efficacy under field conditions at two biologically distinct sites. These
data are derived from human research with subjects who have been treated with the repellent
formulations in the field. The Agency evaluates the duration of repellent efficacy for a subject
by calculating the time from application of the repellent to the occurrence of an event
indicating an efficacy failure. Historically, for field studies of mosquito repellency, EPA has
used the “first confirmed bite” as an indication of efficacy failure on a test subject. Several
recent studies have shifted to the “first confirmed landing with intent to bite;” EPA has
accepted this alternative endpoint. A “confirmed landing” on a test subject is a mosquito
landing followed by a second landing on the same subject within a specified period of time
(usually 30 minutes) after the initial landing.
Field studies typically involve 6 — 10 subjects who have been treated with a defined
amount of the test material. Each subject is then regularly and repeatedly exposed to ambient
mosquito populations for a fixed interval of time until the subject experiences an efficacy
failure followed by a confirmation with the specified period of time. Mosquito landing pressure
(representing intent to bite) at a site is monitored by concurrently exposing untreated subjects
to mosquito landings. A study is considered valid only if there are at least a specified minimum
number of mosquito Landings on untreated subjects during each exposure interval.
14 of 85
-------
On October 25, 2007, the HSRB discussed scientific aspects of the design of field
studies to assess the efficacy of mosquito repellents. Prior to the meeting, the Board requested
consultants to provide specialized information or assistance to the Board. The Board was
particularly interested in the frequency, duration and timing of exposure of subjects to potential
mosquito landings. The Board requested each consultant to respond briefly to the series of
questions below.
• What do data show about the variability of the time intervals between first and
subsequent landings in mosquito repellent field trials?
• What is the current scientific understanding of how factors other than repellent
efficacy could affect the likelihood that an initial event—a mosquito landing or
mosquito bite—would be “confirmed” by another similar event within 30 minutes?
Please address at least these factors:
o Characteristics of mosquito populations
o Characteristics of test sites
o Characteristics of test subjects
o Characteristics of test methods
• Can the impact of such factors on the likelihood or timing of an initial and confirming
event be predicted? Can it be quantified?
Review materials . EPA provided the following materials to the HSRB relevant to the
science issues in mosquito repellent efficacy field research:
Consultant Responses to Discussion Questions
I. Col. Raj Gupta’s Responses to Discussion Questions
2. Dr. Steve Schofield’s Responses to Discussion Questions
3. Dr. Daniel Strickman’s Responses to Discussion Questions
D. Completed Insect Repellent Efficacy Study (SCI-OO1) of DEET Formulations
In its January 2007 meeting, the HSRB reviewed and commented on materials related
to a comparative insect repellent efficacy protocol from Carroll-Loye Biological Research,
submitted by Dr. Scott Carroll. The proposal, identified as SC1-00l, described a study to
evaluate the efficacy of four repellent formulations containing the active ingredient DEET.
(Note: One formulation included two other active ingredients as well.) The study was
designed to measure the efficacy against mosquitoes under field conditions of three test
formulations as compared to one “comparison article”—the US military standard repellent.
The HSRB offered comments on the protocol at its January 2007 meeting Following that
meeting, Dr. Carroll revised the protocol to address comments from the HSRB. Dr. Carroll
conducted the research in July 2007, and has submitted the results to EPA for review. EPA
presented the results of this testing at the October 2007 HSRB meeting.
Although the protocol SCI-001 was executed only once, the results are presented in
three separate volumes, each one addressing a single test formulation as compared to the
15 of 85
-------
military standard repellent. Most of the material presented in each report is duplicated in the
other two reports, but there are unique elements in each volume.
The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB review of an
EPA decision to rely on the results of these studies. The sponsor has submitted applications
for amendment of two of the test materials citing these data, but the third test material
(LipoDEET 3434) is not registered, nor is it the subject of any application. EPA has reviewed
the research, applying the standard in 40 CFR §26.1705. That provision states:
§ 26.1705 Prohibition on reliance on unethical research with non-pregnant, non-
nursing adults conducted after April 7, 2006
Except as provided in §26.1706, in actions within the scope of §26.170 1, EPA shall
not rely on data from any research initiated after April 7, 2006, unless EPA has
adequate information to determine that the research was conducted in substantial
compliance with subparts A through L of this part.. . This prohibition is in addition
to the prohibition in §26.1703.
Dr. Carroll conducted the research covered by SCI-001 at the same times and at the
same locations as the research covered by protocol WPC-001, described below. Because these
two protocols were executed concurrently, in the same field locations, with the ‘same untreated
controls, and with overlapping sets of treated subjects, EPA believed that the conduct of WPC-
001 study may affect the results of SCI-001, and vice versa. Thus EPA conducted a single
ethics review addressing both studies.
The Agency’s science review raised questions about whether the data are scientifically
sound. In addition, EPA’s ethics review raised questions about whether the research under
SCI-00 1 was conducted in substantial compliance with the requirements of subparts K and L of
EPA’s final rule establishing Protections for Subjects in Human Research—the only subparts
of the rule which apply to third-party research. EPA requested the Board’s advice on whether
the research data are scientifically sound and whether the available information supports a
determination of “substantial compliance” with the applicable rules. If the Board concluded
that the data are scientifically sound and the research substantially complied with the
applicable requirements, EPA would rely on these data in support of applications for new or
amended registration of the test materials.
Review materials . EPA provided the following materials to the HSRB relevant to the
completed repellent efficacy study with four DEET formulations (SCI-00l):
a. EPA Ethics Rvw SCI-001 & WPC-001 9-26-07
This review addresses both this study and the concurrently conducted WPC-001.
b. MRID 47211901 SCI-001.1 L1poDEET 302
16 of 85
-------
Carroll, S. (2007) Test of Dermaegis LipoDEET 302 Personal Insect Repellent: EPA
Reg. #82810-1. Unpublished study prepared by Carroll-Loye Biological Research
under Project No. SCI-001 .1. 219 p.
c. MRID 47208401 SCI-001.2 L1poDEET 3434
Carroll, S. (2007) Test of Dermaegis LipoDEET 3434 Personal Insect Repellent.
Unpublished study prepared by Carroll-Loye Biological Research under Project No.
SCI-001.2. 222 p.
d. MRID 47211801 SCI-0O1.3 Coulston’s Duranon
Carroll, S. (2007) Test of Coulston’.s Duranon Personal Insect Repellent (EPA Reg.
#50404-8). Unpublished study prepared by Carroll-Loye Biological Research under
Project No. SCI-00l.3. 217 p.
e. CLBR Supplement Re LipoDEET 3434
Carroll-Loye Biological Research’s September 24, 2007 response to EPA’s request for
additional information about L1poDEET-3434 and the rationale for the amendment by
which it became one of the test repellents
f. EPA Protocol Review SCI-001 12-20-06
g. 4-16-07 HSRB Report of Jan 07 discussion of SCI-001
h. SCI-0O1 Science Review 9-27-07
E. Completed Insect Repellent Efficacy Study with Oil of Lemon Eucalyptus (WPC-001 )
In the June 2007 HSRB meeting, the Board reviewed and commented on materials
relating to an insect repellent efficacy protocol from Carroll-Loye Biological Research,
submitted by Dr. Scott Carroll. The protocol described proposed research to evaluate the
efficacy of a conditionally registered repellent product containing the active ingredient Oil of
Lemon Eucalyptus (OLE). The protocol, identified as WPC-00l, described a field study of
efficacy of the test formulation against mosquitoes.
Following the June meeting, Dr. Carroll revised the protocols to address comments
from the HSRB, conducted the study, and submitted the results.
The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB review of an
EPA decision to rely on the results of these studies. EPA has reviewed the study, applying the
standard in 40 CFR §26.1705. That provision states:
§ 26.1705 Prohibition on reliance on unethical research with non-pregnant, non-
nursing adults conducted after April 7, 2006
17 of 85
-------
Except as provided in §26.1706, in actions within the scope of §26.1701, EPA shall
not rely on data from any research initiated after April 7, 2006, unless EPA has
adequate information to determine that the research was conducted in substantial
compliance with subparts A through L of this part. . . This prohibition is in addition
to the prohibition in §26.1703..
As noted above, the principal investigator conducted the research covered by WPC-001
at the same times and at the same locations as the research covered by protocol SCI-001.
Because these two protocols were executed together, EPA questioned whether and how this
fact affects the review of the separate reports.
The Agency’s science review raised questions about whether the data were
scientifically sound. In addition, depending on whether the study covered by WPC-001 was
considered separate from the study covered by SCI-001, EPA’s ethics review (discussed above
under the heading of SCI-00 1) raised a question about whether the research under WPC-00 1
was conducted in a manner that substantially complies the requirements of subparts K and L of
EPA’s final rule establishing Protections for Subjects in Human Research—the only subparts
of the rule which apply to third-party research. The Agency requested the Board’s advice on
whether the research was scientifically sound and whether the available information supports a
determination of “substantial compliance” with the applicable rules. If the Board concluded
that the data are scientifically sound and the research substantially complied with the
applicable requirements, EPA would rely on these data to satisfy the data requirement imposed
as part of the conditional registration of this product.
Review materials . EPA provided the following materials to the HSRB relevant to the
completed repellent efficacy studies of oil of eucalyptus:
a. MRID 47217601 WPC-0O1 OLE
Carroll, S. (2007) Test of an Oil of Lemon Eucalyptus-Based Personal Insect Repellent:
EPA Reg. #305-62. Unpublished study prepared by Carroll-Loye Biological Research
under Project No. WPC-001. 225 p.
b. CLBR Supplement Re: Consent Documentation
Carroll-Loye Biological Research’s September 20, 2007 response to EPA’s request for
additional information concerning which subjects signed which version(s) of the
consent document on what date(s)
c. EPA Protocol Review WPC-0O1 3-13-07
d. 6-13-07 HSRB Report of Apr 07 discussion of WPC-O01
e. WPC-001 Science Review 9-27-07
18 of 85
-------
F. Proposed Carroll-Love Picaridin Insect Repellent Efficacy Studies (SPC-001 & SPC-
002 )
EPA requires data from efficacy studies using appropriate insect species to support
claims of greater efficacy than have previously been approved.
EPA’s regulation, 40 CFR §26.1125, requires the sponsor or investigator to submit to
EPA,before conducting a study involving intentional exposure of human subjects, materials
describing the proposed human research in order to allow EPA to conduct scientific and ethics
reviews. In addition, EPA’s regulation, 40 CFR §26.1601, requires EPA to seek HSRB review
of the research proposal.
In previous meetings the HSRB has reviewed and commented favorably on several
proposed insect repellent efficacy protocols to be conducted by Carroll-Loye Biological
Research, submitted by Dr. Scott Carroll. Dr. Carroll has submitted proposals for new research
to evaluate the efficacy of two registered repellent sprays containing the active ingredient
picaridin, as well as one lotion formulation including both picaridin and a sunscreen, for which
an application for registration is pending. The first research protocol, identified as SPC-001,
describes a field study of the efficacy of the test formulations against mosquitoes. The second
research protocol, identified as SPC-002, describes a laboratory study of the efficacy of the test
formulations against ticks. Both proposals bear many similarities to protocols that the HSRB
had previously reviewed favorably.
EPA has concluded that, with some refinements, these protocols appear likely to
generate scientifically sound, useful information and to meet the applicable provisions of the
EPA regulations in 40 CFR part 26, subparts K and L.
Review materials . EPA provided the following materials to the HSRB relevant to the
proposed Carroll-Loye picaridin insect repellent efficacy studies (SPC-001 and SPC-002):
a. HRB Minutes 7-17-2007
This single document addresses IIRB review of both protocols.
SPC-OO1: Field lest of mosquito repellency
b. Carroll-Loye Protocol SPC-001 7-13-07
c. EPA Science & Ethics Review SPC-001 9-24-07
SPC-002: Laboratory test of tick repellency
d. Carroll-Loye Protocol SPC-002 7-10-07
e. EPA Science & Ethics Review SPC-002 9-24-07
19 of 85
-------
G. Proposed ICR Picaridin Insect Repellent Efficacy Study (A 117 )
EPA requires data from efficacy studies with human subjects to support claims of
efficacy of a new pesticide product intended to repel insects that transmit human diseases.
EPA’s regulation, 40 CFR §26.1125, requires the sponsor or investigator to submit to
EPA, before conducting a study involving intentional exposure of human subjects, materials
describing the proposed human research in order to allow EPA to conduct scientific and ethics
reviews. In addition, EPA’s regulation, 40 CFR §26.1601, requires EPA to seek HSRB review
of the research proposal.
Dr. Niketas Spero has submitted a proposal for new research to evaluate the efficacy of
two registered products containing picaridin, to be conducted by Insect Control & Research,
Inc. (ICR). The research protocol, identified by Protocol ID G0590607001A1 17 describes a
laboratory study of the efficacy of the test formulations against mosquitoes of the genus Culex.
EPA has reviewed ICR’ s protocol and has concluded that, with a number of required
revisions, it appears likely to generate scientifically sound, useful information and to meet the
applicable provisions of the EPA regulations in 40 CFR part 26, subparts K and L. The
sponsor wishes to submit the data to EPA later this year in support of an application to amend
the registration of these picaridin products in order to claim specifically that the products are
effective at repelling the mosquito species that transmit West Nile Virus. In the interest of
providing a thorough and timely decision on such applications, and since EPA found the
protocol can meet applicable scientific and ethical standards, EPA presented this protocol for
review at the October 2007 HSRB meeting.
Review materials . EPA provided the following materials to the HSRB relevant to the
Insect Control & Research Inc. Repellent Efficacy Protocol A117:
a. ICR Protocol A117 Transmittal 8-8-07
b. ICR Protocol A117 8-8-07
This protocol proposes a laboratory test of repellency of Culex spp. mosquitoes by two
formulations containing picaridin
c. EPA Science & Ethics Review ICR A117 9-24-07
This report transmits the HSRB’s comments and recommendations from its October
24-26, 2007 meeting.
20 of 85
-------
REVIEW PROCESS
From October 24-26, 2007, the Board had a public face-to-face meeting in Arlington,
Virginia. Advance notice of the meeting was published in the Federal Register “Human
Studies Review Board: Notice of Public Meeting (72 Federal Register 187, 54908). At the
public meeting, following welcoming remarks from Agency officials the Board then heard
presentations from the Agency on the following topics:
• EPA’ s draft document Scient flc and Ethical Approaches for Observational Exposure
Studies. The document, prepared by researchers in EPA’s National Exposure Research
Laboratory, identifies the types of issues that should be considered in planning and
implementing observational human exposure studies and provides information and
resources to assist EPA researchers in these studies.
• A published report of a completed clinical trial measuring the effects of single and
repeated treatments with sodium azide on blood pressure in human subjects. Sodium
azide is a pesticidally active ingredient being proposed as a replacement for the
fumigant methyl bromide.
• An overview of the discussion questions related to the Science Issues in Mosquito
Repellent Efficacy Field Research.
• A research proposal from Carroll-Loye Biological Research to evaluate the field
efficacy in repelling mosquitoes of three registered products containing picaridin.
• A research proposal from Carroll-Loye Biological Research to evaluate the laboratory
efficacy in repelling ticks of three registered products containing picaridin.
• A research proposal from Insect Control & Research, Inc. to evaluate the laboratory
efficacy in repelling mosquitoes of the genus Culex of two registered products
containing picaridin.
• A report of a completed field study by Carroll-Loye Biological Research of the
mosquito repellent efficacy of a registered product containing Oil of Lemon
Eucalyptus.
• Three closely related product-specific reports from a single completed field study by
Carroll-Loye Biological Research of the mosquito repellent efficacy of four pesticides,
all containing DEET.
21 of 85
-------
• Design of sampling strategies for handler research programs proposed by the
Agricultural Handlers Exposure Task Force and the Antimicrobials Exposure
Assessment Task Force II.
The following oral comments were presented at the meeting:
(1) Judith Hauswirth, Ph.D., and Mr. Douglas Richards - representing American Pacific
Corporation and addressing the Completed Oral Therapeutic Study with Sodium Azide.
(2) Thomas Osimitz, Ph.D., and M. Keith Kennedy, Ph.D., - representing Science
Strategies and addressing the Science Issues in Mosquito Repellent Efficacy Field
Research.
(3) Scott Carroll, Ph.D. - representing Carroll-Loye Biological Research and addressing:
(1) Science Issues in Mosquito Repellent Efficacy Field Research; (2) Completed Field
Efficacy Studies by Carroll-Loye Biological Research: SCI-001 and WPC-001; and (3)
Proposed Insect Repellent Efficacy Studies SPC-001 and SPC 002.
(4) Mr. Niketas Spero and Robin Todd, Ph.D. - representing ICR, Inc. and addressing the
ICR Repellency Efficacy Protocol A 117.
For their deliberations, the Board considered the materials presented at the meeting,
written public comments and Agency background documents (e.g., the published literature,
Agency data evaluation record, weight of evidence review, ethics review, pesticide human
study protocols and Agency evaluation of the protocol). For a comprehensive list of
background documents visit the www.regulations.gov, Docket ID No. EPA .-HQ-ORD-2007-
0942, or EPA’s HSRB website at htt ://www.epa.gov/osaJhsrb/oct-24-26-2007-public-
meeting.htm .
CHARGE TO THE BOARD AND BOARD RESPONSE
A. Scientific and Ethical Approaches for Observational Exposure Studies
Charge to the Board
1. One of the goals of the document is identify the major scientific and ethical areas and
issues that researchers should address in the design and implementation of
observational human exposure measurement studies, with the emphasis on the areas
requiring ethical considerations. Does each section identify the major areas and issues
where ethical considerations should be addressed?
2. The document is intended to serve as a reference and resource of information that
researchers can use in the design and implementation of observational exposure studies.
For each section, are there additional sources of information that should be considered
for inclusion?
3. Is the information presented accurately and clearly in each section?
22 of 85
-------
Board Response
Board discussion focused on responding to the three charge questions together by section
(Section 1 to 7) of the EPA draft document.
Section 1: Introduction, Purpose and Scope
Strengths
The document received nearly universal praise from the Board for the clarity of its
writing and its thoughtful consideration of the many issues involved in observational studies of
toxicant exposure in human subjects. The introduction set the tone for this excellent document.
In particular, the Board found that it did a superb job of explaining the purpose of the
document and defining its scope. It gave attention to the proper goals of such studies and the
nature of the data to be collected. Most importantly, it outlined the range of ethical issues in the
conduct of such studies. And finally, it accomplished exactly what an introduction must do by
clearly indicating the organization and general contents of the several sections to follow in the
main body of the document.
Document Enhancement
The document can be enhanced by:
• Expanding the discussion on how observational research is distinguished from
intentional exposure research with specific illustrations, including discussion of the
need for ethical review and evaluation of observational research.
• Revising the document to present a more balanced presentation and highlight the
interplay between good science and ethics.
• Providing examples of study questions, citing other documents to guide researchers
in alternative research designs, and providing bulleted summary highlights at the
end of each chapter.
HSRB Consensus and Rationale
Sectidn 1 is well written and sets the tone for ethical consideration of observational
exposure studies. The document could be enhanced by examples distinguishing observational
from intentional exposure studies, introducing the appropriate balance between scientific and
ethical concerns, and providing study questions and bullets.
The abstract states that the document will address chemicals and other stressors, but
focuses on chemicals, which gives the impression that attention is not needed except for
research involving chemicals. The language and examples used in the document should reflect
the importance of the document for both chemicals and other stressors. Finally, while the
Agency does define and differentiate observational versus intentional exposure, such a
definition should be more prominently displayed in the body of the report versus as a footnote.
23 of 85
-------
Section 2: Elements to be Considered in Study Conceptualization and Planning
Strengths
Section 2 is very strong in its consideration of the overall conceptualization of study
planning, especially the ethical component often insufficiently conceptualized by the scientists
in their initial approach to the study. The planning and scoping of the study conceptualization
includes both the science and the ethics, as does the review process (in which each component
reviews the other component as well). The strong emphasis on the ethical issues is beneficial.
Recognizing some of the needs and scope of exposure sciences is noteworthy, as well. The text
boxes are particularly helpful, as they summarize many of the important points from the text
quite succinctly.
Document enhancements
The Board believes it had made many constructive comments with the intent of making
a good document better. As with other chapters, Section 2 has a strong emphasis on ethics, to
the apparent lack of emphasis on scientific aspects. In addition, a primary concern is that
separating the study design document from the human subjects protocol will potentially lead to
inconsistencies between the documents (Figure 2-1). The Board suggests that there be just one
document, with the study design elements being a portion of the human subjects protocol.
Science cannot easily be separated from ethics, so any single document should contain
elements of both study design and implementation, as well as other elements that may be
required for an IRB to assess whether all regulatory requirements have been met.
The major area of deficiency, given the purpose of the document, is in the paucity of
information, materials, and references regarding the purpose, design and conduct of exposure
studies. Even the initial paragraphs place more emphasis on the ethical issues (covered
extensively elsewhere) than on the scientific ones; there should be more of a balance. In
addition there are contradictions starting in Section 1.1 as to whether epidemiologic studies are
included or not.
Specific Suggestions to the Document
There is little explanation or coverage of exposure study designs and methods, and their
different attributes. Well-recognized study designs such as cross-sectional, case-control, and
cohort (both prospective and retrospective) designs should be fully described. Sufficient
examples are not provided and references to such discussions, as well as to such studies, are
inadequate. There are excellent sources for such material to guide researchers, starting with
EPA documents (e.g., actual examples and references for TEAM, PTEAM, NHEXAS, the
pesticide studies, etc.). There are also excellent NRCINAS & WHO documents that could be
utilized. There are many lessons learned from such that could be culled to provide research
guidance. Some of these sources include the NRCINAS 1991 report on Human Exposure
Assessment for Airborne Pollutants, the WHO EHC 27 (1983) report on Guidelines on Studies
24 of 85
-------
in Environmental Epidemiology, and the WHO/EURO & EC/EU documents on Exposure
Assessment from the ECEHs & EU Lab. Additional suggestions are provided below.
I. Section 2.1 .1 does not define the types of study problems and questions
scientifically (or specific EPA & other references where such can be found).
2. Section 2.1.2 does not provide any basis or criteria for justification of the science
component (or specific EPA & other references where such can be found), only the ethical
component.
3. In Text Box 2-1 Elements to be Considered in Justifying a Study, there should be a
bullet added “A discussion of alternative designs, alternative models, or alternative
populations”
4. Page 21, line 4: “considers” should be changed to “considered”
5. Section 2.2 does not outline the steps in planning the study scientifically (or specific
EPA & other references where such can be found). Section 2.2.1 has only one sentence about
the scientific aspects, and does not really discuss “innovative” scientific aspects. Text Box 2-
2, Study Elements that Could Affect People’s Behavior, is the closest the section comes to
delineating the components of the study that are relevant also to its planning.
6. Section 2.2 Planning and Scoping: The latter term is jargon and can be eliminated,
or if not, it should at least be defined.
7. Section 2.2.1 is entitled Innovative Study Designs, but is actually about adding
direct benefit (such as educational materials) for research subjects in studies for which there is
not direct benefit. Innovative designs might instead include computer modeling or Bayesian
designs. Either the section title or the content needs to be changed. In addition, the section’s
current content could be said more directly and succinctly.
8. Page 23, line 42 “. . .community’s perspective” . Sentence should read,
“...community’s perspective better, the researcher...”
9. Section 2.2.4 states that conflicts due to project funding are “the most likely to
occur.” Please verify that this is the case in EPA related to observational exposure studies, or
change the sentence. Unless this type of research is unique, it is likely that financial conflicts
are not necessarily the most common, but rather they are the easiest to identify and manage.
10. Text Box 2-3 (Elements That May Be Included in a Study Design) and its corresponding
section list elements to be included in a study design. One bullet lists items to describe
technical approach and conceptual model. The Board recommended adding to the list that
endpoint or outcome measures should include a description of their accuracy and precision.
Survey instruments and questionnaires should include a description of whether they have been
previously validated and, if not, how they will be validated prior to use within a study. A
source for additional elements that might be included in a study description can be found at
25 of 85
-------
PLoS Medicine, http://medicine.plosjournals.org, volume 4, number 10, October 2007 in two
articles concerning STROBE: The Strengthening the Reporting of Observational Studies in
Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies (von
Elm E et aL, 2007) and Strengthening the Reporting of Observational Studies in Epidemiology
( STROBE): Explanation and Elaboration (Vandenbroucke et al., 2007)
11. Section 2.3 did not provide some of the more important specifics of the scope and
technical approaches (or specific EPA and other examples and references where such can be
found). Sec. 2.3.1 does not include all the important appropriate questions regarding scientific
feasibility — its one “bullet” does not even include questions about the feasibility of
measurement methods. The sub-sections also did not include monitoring of observer errors
and biases, participant reporting biases and reliability, inappropriate (as well as inadequate)
selection criteria, representativeness (refusals and withdrawals), etc. (or specific EPA and other
examples and references where such can be found). Enrollment criteria (inclusion and
exclusion) need to be included, and a discussion of the ethical issues of subject selection
should be added.
12. Section 2.3.1.1 Sample Size Determination: The authors are to be commended for
including this section. However, the section should be written in consultation with a
biostatistician. There are, in fact, many references on this topic, and to state that there is
“surprisingly small amount of literature” is maccurate. In addition, the methodology cited may
not be optimal. One needs to quote and reference statistical specifics as provided in the
documents previously mentioned, specific survey statistical textbook questions approaches,
and also more from other statistics books), and the issue of design factors (e.g. Clickner book
and other books) as well as expected refusals and expected losses that need to be taken into
account. Issues of intra- and inter-participant and observer variability are not discussed.
13. A section on obtaining an appropriate sample or a representative sample in order to
derive generalizable data should be written. Descriptions of sampling methods with their
relevant strengths and weaknesses are critical, and would help investigators enormously.
14. Text Box 2-5, Potential Topics in a Human Subjects Research Protocol, lists potential
topics for a submission to an IRB.
• Item #6 should be changed to “Affirmation of Belmont Principles...” rather than
Belmont Report.
• Item #13 should be changed to “Sample size/power and statistical analysis plan.”
• Item #22 should either be changed to include “unanticipated problems” or a separate
item should be added.
• Item #32 should also include a comment concerning unforeseen uses, if appropriate.
• Item #42 should be changed to state “Procedures for preventing falsification of data”
with the emphasis on prevention rather than on what to do if falsification occurs.
HSRB Consensus and Rationale
The Board concurred that Section 2 adequately identified the major areas and issues
where ethical considerations should be addressed in the study conceptualization. The Board
26 of 85
-------
suggested that rather than having distinct sections and even documents on ethics and science
there be just one document, with the study design elements being a portion of the human
subjects protocol. This would more closely parallel the information submitted to an IRB.
Bulleted items should also include justification for sample selection and size and sampling
method, discussion of alternative designs that were rejected to help justify the use of human
subjects in the particular way being proposed, expanded examples of conflicts of interest, and
the importance of using validated measures. Expansion of text regarding adverse events vs.
unanticipated problems, scientific misconduct, subject attrition, reporting biases etc. were
recommended.
Section 3: Ensuring Protection of Vulnerable Groups
Strengths
Section 3 is of high quality and does a very good job in addressing the major areas and
issues related to vulnerable subjects. The section presently devotes the bulk of the discussion to
issues relating to children and women. It might be helpful to present a somewhat more
balanced discussion, with added material devoted to discussing issues relating to other
vulnerable groups.
Specific Suggestions to the Document
1. Section 3.1, Identification of Vulnerable Groups, it might be helpful to more
specifically point out the differences between the rules under which the EPA is operating with
regard to identification of vulnerable groups (primarily, the Common Rule), as compared to the
concepts of vulnerability that the lay public might have. In addition, a discussion of how
vulnerability can be context-dependent could be helpful.
2. In Section 3.2, the current discussion puts an emphasis on the “special justification”
needed for inviting vulnerable subjects to participate in research, without expanding on how
that concept plays out in observational studies. In fact, many of the observational studies with
regard to which this document will apply are studies that are intended to collect important
health information regarding the well-being of various vulnerable groups (such as children and
pregnant women). Moreover, unlike many other studies (such as interventional studies),
observational studies generally do not impose more than minimal risks on subjects. Those
circumstances—the possible large benefits to the vulnerable populations from gaining the
information to be learned, and the minimal risk regarding including them in the studies —
combine to often make a strong ethical case for including such groups in these studies. Thus, it
would be appropriate for this section.to also discuss the harms from inappropriately excluding
various groups of vulnerable subjects from this category of studies, and how those harms
would often be to those very groups. This point is already made in Section 3 in some of the
discussions of specific subject groups (e.g., in the discussion in Section 3.4 with regard to
children), but it would be helpful to highlight it as a general proposition in the analysis of the
general issue of the justification for including vulnerable subjects.
27 of 85
-------
3. It might be helpful to have a more expanded discussion of how to mitigate risks to
vulnerable subjects in these studies. This point is particularly important given the substantial
reasons (as noted in item 2 above, and already discussed in portions of Section 3) for including
such vulnerable subjects in these studies. This discussion could go beyond the risks created or
imposed by the inclusion of the subjects in the study (which are often minimal), and also
address the extent to which it is appropriate for researchers to be standing by and observing
while vulnerable groups are exposed to risks not created by the study.
4. It could be helpful to add a subsection dealing specifically with workplace-based
studies. The types of vulnerabilities of workers who participate in those studies are somewhat
unique, and can vary depending on the specific type of study.
5. Given the very substantial literature on the topic of vulnerable subjects, the list of
references in Section 3.7 might be expanded.
HSRB Consensus and Rationale
The Board determined that Section 3 is of high quality, accurate and written clearly.
The Board recommended additional discussion on justifications for including vulnerable
populations in research as well as expansion on discussion of who is vulnerable outside of the
federal regulations (e.g. pregnant women, prisoners, children) such as economic, educational or
social vulnerabilities; noting however that in federal regulations vulnerability is defined in
terms of susceptibility to coercion and undue influence.
The Board also recommended the document: (a) expand examples of studies that might
involve these populations; (b) discuss the tension between over and under sampling these
populations including the importance of including vulnerable populations in research to ensure
that there is safety data on compounds in which they are more likely than other populations to
be exposed; and (c) discuss the weighing of risks and benefits in these types of studies
Section 4: Privacy, Confidentiality, and Other Concerns Related to Observational
Exposure Measurement Studies
Strengths
Section 4 provides a good discussion of several privacy concerns associated with
observational studies, particularly those conducted in private or semi-private places such as
homes and schools. Although research volunteers allow study staff to access these
environments, others who may be residing in these environments may object or feel that their
privacy is not being appropriately respected. Additionally, during the course of collecting
research data, study staff may observe illegal or immoral activities that may challenge their
ability to maintain the confidentiality of research volunteers or others who may be living in
these environments.
Document Enhancement
28 of 85
-------
Considerations relating to the protection of vulnerable groups, where more active
intervention by study staff may be appropriate should be strengthened. For example, Section 4
suggests that researchers develop a plan for responding to the incidental observation of illegal
behaviors such as child or elder abuse. It does not recommend that research staff be trained
with regard to the recognition of such behaviors, which is essential for avoiding both missed
opportunities for intervention and inappropriate accusations of abuse. Similarly, this section
provides little guidance with respect to the observation of environmental situations associated
with imminent harm, such as observations of combustible materials near an open flame, a child
playing unattended by a pool, a firearm placed near young children, etc. Section 4 could be
strengthened by including advice for addressing such situations or offering general guidance
for determining when members of the research team should act in circumstances involving
imminent harm. It would also be helpful to reinforce the point that local and state reporting
requirements may vary considerably.
Section 4 does not consider the potential risks to members of the research staff who
may find themselves in situations involving illegal activities such as drug use or sales, nor does
it consider the potential burdens that research staff members may feel when presented with
behaviors that they find morally objectionable. In contrast to several of the other sections in
the document, the citations to relevant literature provided in this section are lacking (e.g., no
recent review articles are cited, nor are there any references to important privacy documents
such as HIPAA).
Specific Suggestions
Section 4 could be strengthened by including advice for addressing environmental
situations associated with imminent harm and offering guidance for determining when
members of the research team should act in circumstances involving imminent harm.
Purpose and use of the Certificate of Confidentiality should be included. Such
certificates do not diminish the need to protect personally identifiable information and does not
relieve the requirements for reporting illegal behaviors. While the Agency’s document states
that certificates should be used for “sensitive matters,” it does not define sensitive matters.
Finally, Federal regulations exist that clearly define populations needing certificates of
confidentiality. Such regulations should be noted in the document.
HSRB Consensus and Recommendations
This section of the document is sensitive to many key ethical and legal considerations
relating to the safeguarding of research-subject privacy, including the need to (1) disclose the
possibility of incidental reporting requirements to potential volunteers as part of the informed-
consent process, (2) provide advance notification of research visits to third parties who may be
residing in private environments, and (3) consider potential harms to research volunteers who
display personal monitoring equipment in public settings. It would be strengthened by
including specific advice on reporting and disclosure procedures when confidential information
29 of 85
-------
indicates a participant or another person is in jeopardy, the uses of the Certificate of
Confidentiality and additional references.
Section 5: Creating an Appropriate Relationship Between the Participant and
Researcher
Strengths
The Board commented that this section, like other sections of the document is
excellent; it is readable and useful. In general, Section 5 accurately and clearly discusses the
ethical considerations in the relationship between investigator and participant. A major
strength of this work is its focus on researcher responsibilities and guidance for researchers. A
strength is that it includes consideration of the context of the participant’s community, etc.
(One assumes that “community” includes all the socio-cultural aspects in which the
investigators are competent and respectful.) In relation to these strengths, it would help if the
document stressed the importance on researcher training in human subject protection, with
emphasis on observational techniques and community-based research,
The sub-section relating to remuneration (payment) is highly appropriate and well
written. Likewise, Section 5.3 on rights is well done. Section 5.4, Creating a Supportive
Environment for Research and Interaction,, as defined, is very useful and well stated also.
Section 5.5 provides good discussion of equitable selection and of IRB guidelines (Text
Box 5-3) for selecting sub-populations for study from the ethical standpoint (see below re:
scientific standpoints). Section 5.6 on retention issues and ideas, especially in longitudinal
studies, is useful and generally well done.
Document Enhancements
One of the areas that is important not only to recruitment, but to the successful conduct
of studies is language. The regulations require information be presented in a language that is
“understandable” to the subjects/potential subjects. That has always been interpreted to mean
two things; at a literacy level (oral and reading) that is appropriate for the target population
and, in a language that the listener/reader speaks/reads. Because various populations to be
studied in the U.S. tend to include persons who speak/read little, if any, English, it is very
important that researchers address how communications will be handled. At a minimum,
written documents, such as consent forms, advertising flyers, instruction sheets, etc must be
developed in more than just English. Such written materials must be reviewed and approved
by an IRB prior to use. Review bodies should obtain documentation to ensure that the
communication accurately conveys the information in the English version(s). That is only part
of the issue, however, because during the conduct of the research, investigators and the
research team must be able to communicate orally with subjects, including those who may not
speak English. Thus researchers must address in protocols how planned and ad hoc
translations will be accomplished. It should be stressed in this document that untrained
persons (e.g., co-workers etc.) generally do not meet the ethical requirement for facilitating full
understanding and protecting subject welfare.
30 of 85
-------
While many examples are correct, such examples are applicable to research in general.
Section 5 would be more useful to researchers conducting observational studies if the
discussion were more focused on the special needs/considerations in that type of research. In a
related vein, Section 5 makes some global statements, such as “additional considerations arise
.“ or “a number of issues have been identified ...,“ but there is no expansion/explanation; so
there is no “teaching point.” These generalities should be removed or used to start a
description of the concern or issue; Text Boxes 5-2, 5-3 and Appendix C are good examples of
getting the key points into this section without repeating the source verbatim.
There are additional sources of information that should be considered for inclusion in
the section, specifically the NRC/NAS 1991 report on Human Exposure Assessment for
Airborne Pollutants and the WHO EHC 27 (1983) report on Guidelines on Studies in
Environmental Epidemiology, statistical survey sampling textbooks, the Board’s discussions of
“purposive” sampling, EPA and NIEHS documents on environmental equity/justice, and other
references contained in this document on community and Community Advisory Boards (CAB)
involvement. Also, the literature on observational studies, such as the developing community
based participatory research literature may be informative. With a caution about keeping the
focus on observational research, there is also a wealth of “good clinical practice” (GCP)
references which could be used to expand some issues (such as payment and other incentives
in section 5.2.1).
Specific Suggestions to the Document
Section 5.1.1 seems to include descriptions of consent elements that are of general
application, but this section would be more useful to researchers conducting observational
studies if the discussion was focused on the special needs/considerations in that type of
research.
Section 5.5, Recruiting Strategies, does not address the scientific necessities of
sometimes including over-sampling (via stratified/cluster methods) of sub-populations,
including the underrepresented and the overexposed. Thus, there should be a similar
delineation of when such sampling and recruitment are necessary, similar to Text Box 5-3,
which discusses those needs from an exposure science standpoint. For instance, it is not
inappropriate to study minority/poor children when evaluating exposures to lead, pesticides,
etc., under existing laws/rules/statutes. Some of the “environmental equity/justice” issues need
to be discussed. CABs need to be involved in approving recruitment materials.
Section 5.5 should discuss consideration of the limitations, especially statistical issues
(representativeness & generalizability) and non-random sampling (i.e., the scientific problems
inherent in “convenience,” “purposive,” etc. sampling).
There should be a parallel section to Section 5.7 that discusses the benefits of
longitudinal follow-up for the participants, communities, and responsible agencies and the
decrease of risks that may be so obtained.
31 of 85
-------
The way some references are used (e.g., Grady (page 58, lines 13-21) seems more
suited to a journal article than a review document; that is, “what does this mean for the
researcher” and “what is the application to observational research” is lost.
Additional Specific Suggestions
The Board had a few suggestions for possible additional information that might be
included in the chapter:
1. Page 51, lines 37-39 makes the comment that the consent process must explain risks, but in
order for subjects to make decisions about participation that reflect their individual concerns, it
would be well to state that an accurate and realistic description ofpossible benefits must be
included as well. See lines 41-44.
2. Page 52, lines 2-3 refers the reader to three sources for additional information. It would be
more useful for readers if key points made in these publications were listed or summarized.
That allows the point to be made or the issue to be raised.
3. Page 52, lines 36-39 discusses one interpretation of “language understandable to subjects”
(native tongue), but it also should be pointed out that this has a readability requirement too.
Page 53, lines 1-4 and Page 55 lines 1-2 seem to relate this requirement just to a description of
purpose. This should be fixed.
4. Page 53, line 16 makes a reference to a court case; its relevance needs to be explained.
5. Page, line 24 uses the term “informed consent” incorrectly to imply the form, not the
process of information exchange needs to be presented to parents. (Note the rest of that bullet
is fairly dense and could be revised to make it more readable.)
6. Page 55, line 9 uses the term “administration procedure” when “consent procedure” would
be more appropriate.
7. Page 55, line 28 uses the term “study elements” when “study characteristics” would be
better.
8. Page 55, line 29 is the first use of the word “remuneration” in this section. This term clouds
the reason subjects are offered payment for participation in research. It is not payment for
services in the way employment is. It is an “inducement,” hopefully not unduly large, that
encourages participation. The word “remuneration” should be changed to “inducement”
throughout the section and the document (search and replace) so that the ethical issue regarding
payments is not lost.
9. Page 57, line 5 is the first use of the word “compensation” in this section. This term is a
“regulatory word” and is therefore associated with “compensation for injury.” It would be
clearer and more accurate to use the term “payment” throughout the section and the document
32 of 85
-------
— for example in the Executive Summary on page 5 - (search and replace) so that the ethical
issue regarding payments is not clouded.
Page 61, lines 4-5 indicates that community input can be obtained from research team
members. While true, to be truly representative of community attitudes, non-research team
members should be sought out as they should be free from any potential scientific biases.
HSRB Consensus and Rationale
Section 5 identifies most of the major areas and issues where ethical considerations
should be addressed, with the exception of communication/language issues. However, in many
cases the section takes the tone of reporting what others have said, with and without comment;
and sometimes (and more appropriately) stating “it is recommended.” A document with clear
recommendations will be more useful by researchers than one raising issues without giving
direction.The Board raised three overall questions in reviewing this section of the document.
The questions were as follows:
1. Can the authors define “a strong relationship”, and what is a strong scientific relationship?
2. Are there specific 0MB guidelines on remuneration that could be included?
3. Should the participant grievance procedures include also any component of EPA or the
IRB(s) who approved the study?
Section 6: Building and Maintaining Appropriate Community and Stakeholder
Relationships
Strengths
Section 6 identifies many important areas and issues that need to be considered in
addressing ethical aspects of observational exposure studies. This section addresses the
rationale for and the complexity of building and maintaining trustworthy and effective
relationships with communities and stakeholders who are either directly or indirectly affected
by observational exposure studies. The authors are to be commended for identifying the key
components of this complex and multi-faceted process.
This section identifies options for researchers with regard to community involvement
including CABs, which have been shown to be quite successful in several observational
epidemiologic studies in which exposures are quantified in relation to human health effects.
The composition of CABs is not well defined. Such boards should be representative of the
community. This section also does not clearly articulate the advantages and disadvantages of
engaging a community in research, nor does it clearly describe best practices. For example,
investigators should be advocates for the data, not for the community and a CAB is not a
substitute for ethical decisions.
Section 6 carefully defines important concepts and operationalizes them for
researchers. Definitions, by design, are relatively broad offering flexibility to meet specific
study objectives tailored for particular communities.
33 of 85
-------
Document Enhancements
This section identifies and provides some of the supporting documentation for key
components required for building relationships; e.g., within an overall conceptual framework
that builds largely upon the d [ fusions of innovations literature (Rogers, 1 995a, 1 995b).
However, much of the presentation is in the form of assertions rather than supported by sound
science. While many of the assertions are quoted from references, there may be no hard data
in the references to back up the statements. The Board recommended that the final report
include evidence-based discussions and/or tables that show better compliance, retention, and
data quality when community involvement is obtained compared to when it was not. - Scientific
rigor is essential in the exposure science field and therefore should be attended to carefully in
this document.
Both moral and “best practice” considerations are included as ethical aspects of these
studies. (For example, moral principles with aspirational results are listed on p. 67, lines 12-
15, while Text Box 6-1 lists “best practice” considerations.) Clearer distinction of the types of
ethical issues and the better use of text box listings to highlight the differences would
strengthen this section [ and document]. Further, clarification would make it easier for the
reader to understand whether “ethical issues” in this section refers to moral principles, “best
practices,” or both.
Examination of the many ethical issues raised in Section 6 suggests that the section
[ and document] may benefit from a text box or table (i.e., a box that would be parallel to Table
1-3) summarizing the ethical principles, which are essential for observational exposure studies.
The advantage of such a summary would be to pull out the principles that resonate throughout
the document; i.e., the ones that are more than single mentions. [ This summarization may be
more effective on a section-by-section basis, rather than across the entire document.]
Specific Suggestions to the Document
Important points that may merit emphasis in text boxes or other mechanisms in Section 6
include the underlying principles essential for effective relationships (now spread throughout
Section 6):
• Respect, fairness, beneficence, honesty, openness, trust, commitment, confidentiality,
and responsiveness [ Note that justice and other principles are listed on pages 2 and 5
(etc.). Perhaps a cross-reference to earlier, more complete text or supporting materials
is needed in Section 6.]
• As appropriate for the contexts, early and active community involvement, meaningful
community roles and responsibilities, and two-way communications
• Effective representation of the community’s diverse views
• Dignity, veracity, sustainability, justice and community (p. 69)
• Establishing a relationship with the community before a study (p. 71)
• Build community capacity (p. 73)
• Researchers need to be forthright with communities (e.g., p. 73)
34 of 85
-------
• Research relevant to communities (p. 74)
Introductory paragraphs
Community-based studies involve numerous ethical issues that are fundamentally
different in important ways from clinical studies (p. 67, lines 19-2 1). Some of these issues are
addressed in the earlier sections of the document, but not discussed in Section 6. The one issue
highlighted here is two-way communications (p. 67, line 30), but this emphasis is not
consistently presented throughout Sections 6 and 7.
Section 6.1
-This section is quite good and includes a compendium of ways to approach the
community and get the community involved with a study. Such information is invaluable to
the neophyte researcher attempting to gather data in a new community.
However, some of the reasoning presented in Section 6.1.1 and sub-sections borders on
philosophical, is convoluted and occasionally circular: ethical action requires trust and trust is
essential in ethical action. Still, the points are of interest and should be developed. Exposure
researchers typically have “hard science” backgrounds, and therefore need both guidance in
this area and substantial evidence supporting the points (e.g., those cited from the EPA
November 2007 workshop on observational exposure studies). Perhaps the use of examples
would be a better teaching tool and would serve EPA’s needs better.
Sections 6.1.1.1 and 6.1.1.2
The sections are not balanced in their discussions of community and stakeholders,
giving less attention to the latter. The reason for this imbalance is not apparent and may
confuse researchers about who stakeholders are and what their roles are.
Who is to be involved in community-based studies and how it involves a variety of
ethical concerns. The distinction between “community” and “stakeholder” should be made
clearer (see pp. 69-70). The “stakeholder” descriptions on p. 70 (lines 14-17) and p. 74 (lines
19-20) could be improved; e.g., stakeholders can physically speak for communities but may
not be seen as legitimate spokespersons for community interests. The key issue is whether the
community has actually or officially delegated any of its representation or speaking rights to
stakeholders (whether stakeholders are also members of the community or not); this seems to
be the intent of p. 70, lines 2 1-23, 27-29, and 38-39. Perhaps minor editing of this page will
clarify and strengthen the Agency’s guidance, or — more likely — a major reconsideration of the
definitions and differences between “community” and “stakeholder” may be needed for use
throughout the document.
Section 6.1.1.4
Reading comprehension needs to be stressed more and, possibly, operationalized
further. While there is at least one comment to strive for an 8 th grade reading level, many IRBs
35 of 85
-------
require materials prepared for a 5 th grade level. Also, if translation is required, many IRBs
require a certified translating service and increasingly are requiring back translation when
certificated services are not available. In the advisory spirit, some reference may be needed
with regard to translation.
Additionally, data collection instruments can be designed so that reading
comprehension is objectively measured. To this end, researchers could be advised to formally
evaluate grade level reading and comprehension while capturing data relevant for exposures.
Sections 6.1.1.4 and 6.1.1.5
A new section should be included since means of communication (sometimes referred
to as channels of communication) are not discussed here. Just as the level and type of language
used is important, means of communication should align with communities’ preferences.
Understanding and using the ways in which communities want to receive and share
information are essential ways of demonstrating respect for communities’ interests and
showing that their input makes a difference. The Board suggests that a brief section about this
issue be added between Sections 6.1.1.4 and 6.1.1.5.
Section 6.1.1.5
Cultural differences should include race/ethnicity but, minimally, also religious beliefs
or other unique lifestyles. Currently, the text only pertains to the former and thus is
unbalanced and potentially misleading. The Board recommended that the authors consider
adding language about providing study results to the communities (and stakeholders) before
publication.
Section 6.1.1.7
The role of the researcher as an advocate (in addition to capacity building) for the
community is alluded to in the document (e.g., p. 73, lines 73-75), but is noticeably absent
here. This is an important aspect for this section but one that is strongly resisted by many
researchers. The Agency needs to consider this section very carefully as it will embark on a
new expected path for many EPA researchers - advocacy — in addition to researcher.
Section 6.2
This section could also provide advice about a reasonable length of time for
communicating study values (including personal exposure results) to participants. Ideally,
such times should be determined upfront with community input. For example, a reasonable
time might be within 6-12 months upon quantification of biospecimens.
Perhaps, attention should be given to the planned inclusion for health alerts in real time.
Some observational epidemiologic studies have health alerts built into web-based data
management structures so that researchers and participants can be alerted to any value
36 of 85
-------
requiring further attention. This approach needs formalization and guidelines for
implementation in the overall study protocol.
This section also should address the lessons learned from various observational
epidemiologic studies that have utilized CABs or other forms of participatory research. For
example, a wealth of information has been learned from the environmentally oriented studies
involving adults and children or from the Long Is land Breast Cancer research (Gammon et al.,
2002).
Section 6.3
The use of “other stakeholders” (p. 74) is confusing to the reader; what types of
stakeholders are “other?” If the definition of “stakeholder” is broad, then the concept of
“other” becomes moot. The text of this section is not fully aligned with the stakeholder
concepts in Section 6.1 .1 .1 (p. 70, lines 13-14) or the Glossary (p. 107) and therefore needs to
be reconsidered.
Further, the stakeholder discussion in Section 6.3 is given short shrift when compared
with the Community Involvement discussion (Sections 6.! and 6.2). Buy-in from stakeholders
is often as important as buy-in from the community even if for the reason that stakeholders
often have more fiscal resources to draw upon. Without their buy-in, this would limit financial
resources, potentially hampering the ability for the study to be conducted. Stakeholders may
offer alternative points of view involving jobs for the community versus the views of some
groups in the community that may want “no pollution and no risk” without regard to fiscal and
social costs. More comprehensive information about stakeholders, particularly the needs of
local government and business stakeholders, should be included.
The importance of continually attending to the dynamics of relationships is not
recognized in this section. Beyond identifying stakeholders (p. 74, lines 25-29), an additional
benefit of Mitchell et al’s (Mitchell et al, 1997) framework is that it offers a means for ongoing
assessment of the shifts in power, legitimacy and urgency that occur during studies and that
affect views of who is which type of”stakeholder” (as defined in their article) and how they
should be involved. Further, Mitchell et al (1997) describe different ways in which
organizations may interact with different types of stakeholders throughout a dynamic process
(such as occurs in observational studies). Recognition and guidance about anticipating and
strategically managing relationships throughout observational studies would strengthen this
section.
It is important to be sure the text boxes (not just in this section) are fully documented in
the text in order to avoid careless use of the points without regard to the broader issue of
community and/or stakeholder involvement. Ordering of bulleted items should reflect logical
steps in the process or importance; nonetheless, the rationale for ordering should be stated.
Editorial Suggestions
I Page I Line/s I Section I Comment
37 of 85
-------
67
5
intro
The word exposure is missing “...observational exposure
research...”
67
36
intro
Add c after 2003. [ See reference on page 76.]
68
5
6.1
Consider revising this sentence to read “. . .nature of the
research itself and affected (or targeted) community...”
68
12
6.1
Do you really mean all qualified members? Possibly add text
to say how to select among qualified members if the number
is high (e.g., randomly select, established rotations, etc...)
68
to
69
47
to
30
6.1
The section gets a bit far a field in its discussion of new
forms for Institutional Review Boards. Perhaps a new
section or sub-section is warranted at this point.
69
7-8
6.1
Ensure that these lines reflect the author’s intent. As now
written they could be misinterpreted. “CAB members
• . . .have to be educated” sounds directive and could be
read as condescending; is that what the author said?
[ The point made on p. 69, lines 24-26 is related.]
• . . . should represent their communities honestly. .
sounds judgmental, as if the presumption is that
CABs will not do so. Is that what the author said?
• . . .need to be willing to interact.. .“ involves the same
issues as the bullet immediately preceding this one.
69
28-30
6.1
What is the evidence base for the conclusion made here? If
none, consider rephrasing this point.
70
13-14
6.1.1.1
This definition of stakeholder could overlap with the
definition of community found on p. 69, lines 39-41.
Inserting a well-accepted definition for “stakeholder” here
and in the Glossary is recommended.
70
29-32
6.1.1.2
Delete this sentence. It doesn’t add to the document.
70
46-47
6.1.1.2
Is this a “should” or “must” issue? The latter implies that it
is a critical issue, while the former does not. If the agency
believes that reaching out to multiple organizations is
essential, this sentence would benefit from rephrasing. There
are other should/must issues in the document; these also may
benefit from similar reconsideration by Agency authors.
71
4-6
6.1.1.3
Delete this sentence. It doesn’t add to the document.
71
20-21
6.1.1 .3
This sentence could be misinterpreted as being directive to
communities, rather than advisory to researchers. Rephrasing
is recommended.
72
10
6.1.1.4
“Materials distributed” implies written and/or visual tools
only. This suggests that oral/aural and interactive forms of
communication are excluded; the Board doubts that the
authors intended to exclude such forms of communication
here and suggest rephrasing.
72
19
6.1.1.4
“Explanation” is a limited concept in light of the document’s
earlier discussion of the importance of two-way
38 of 85
-------
communications. Listening to communities’ feedback about
the research should be coupled here to complement the
explanation function noted.
72
30-32
6.1.1.5
The key points from this reference should be included here.
Without them, this sentence is of little use to the reader and
does not provide a level of information about ethical issues
comparable to the rest of the paragraph.
72
3 5-39
6.1.1.6
This sentence uses the terms “relational paradigm” and
“societal context.” These terms stand in sharp contrast to the
urgings earlier to present information in a manner
understandable to subjects in a study. These words are jargon
and are not clear; this sentence should be re-worked.
73
6-7
6.1.1.5
An important ethical issue not mentioned here is the need to
attend to the dynamics of relationships; these may change
whether one or more parties judge pre-established roles and
responsibilities as acceptable. An ongoing process of
assuring mutual understanding and acceptability is often
needed because community research environments are
inherently complex and dynamic.
73
17-19
6.1.1.7
Same issue as on p. 71, lines 20-21.
HSRB Consensus and Rationale
In large part, the information is presented accurately and clearly in each section;
however, suggestions for improvements are noted. Section 6 addresses many of the major
issues requiring ethical considerations but lacks the level of scientific rigor needed by exposure
scientists. Major suggestions by the Board included:
• More data to support points made and less assertion or “theory” would strengthen this
section.
• It is important to differentiate the terms stakeholder and community as well as their
interrelationships and discussing the value of community advisory boards and
community sensitive piloting of procedures.
• Successful community advisory board procedures, how the scientist-community
relationship will evolve and be monitored over time, how the results of research are
disseminated and the informational benefits to the community should be discussed.
• The Board cautioned about editing the section so that readers would not erroneously
conclude that the EPA is advocating that scientists become community advocates.
Section 7: Designing and Implementing Strategies for Effective Communication
Strength
Section 7 discusses principles for designing and implementing effective communication
strategies between all affected stakeholders in an observational exposure study. One of the
important messages coming from this section is that an effective communication strategy
39 of 85
-------
should be a structured, formal plan and should be given extensive forethought prior to
beginning the study. Additionally the communication strategy plan should be a living
document that is constantly reexamined and updated as the study progresses. It should be
stressed that this involves a two-way communication strategy and an intentional process. It is
as (or more) important for the researchers to listen to the stakeholders as it is for them to give
information to the stakeholders.
Document Enhancements
Section 7 emphasizes one-way, media-directed, and crisis communications, which
conflicts with the rest of the document. These aspects of communications may be part of a
comprehensive, strategic communication program but often are not appropriate as the major
emphasis for community-based, observational exposure studies. Instead, community concerns
about potentially being stigmatized by research participation and/or results are often important
issues.
In Section 7.1, the importance of formative evaluation should be noted. Ongoing
evaluation is a means to identify community needs and issues, thereby permitting researchers
the opportunity not only to improve conduct of the study but also to actively demonstrate
respect to participants, showing that their input makes a difference. -
Section 7.2 states it is essential to engage all stakeholders early and often in the
process. One area that has been commented on for previous sections is the definition of
“stakeholder”. On page 74 of the document (Section 6.3) it states “stakeholders may include
business, industry, and local or state governments or agencies with jurisdiction over the
community.” In contrast, Section 7 includes the community as one of the stakeholders. It is
important upfront to define the term “stakeholder” and what groups it encompasses and to use
that definition consistently throughout the document. The community from which the subjects
are to be drawn has traditionally been considered as one of the stakeholders.
While it is critical to identify all pertinent stakeholders, there is a danger that the group
would become so large that it becomes unmanageable. In a group such as this there is the
potential for having strong divergent interests. This could generate enough conflict that it
could impede the research from moving forward in a timely manner. It is therefore advisable
to add as references some articles on managing conflict with a diverse populati6n. A sidebar
with some salient points on managing conflict might help.
Specific Suggestions to the Document
Section 7.3 discusses communication timetables. It encourages researchers to begin the
dialogue with the community as soon as possible, learn from the participants and the
community, and continue exchanges of data and information through the reporting of the
study results and beyond. It suggests using press releases as an important tool to engage the
40 of 85
-------
community. Section 7 states that observational studies should also be announced to
stakeholders and the public via media, community interactions, or other means well in advance
of study implementation. It is strongly recommended that there be some “buy-in” by the
community before any public announcement.
Section 7.4 talks about the importance of developing communication materials at
different levels of science literacy. It stresses that all materials be written in “plain language.”
However, there are no follow-up references on available “plain language” strategies and tools.
Below is the link to the Department of Health and Human Services Plain Language webpage.
Other empirically-based references could also be included in the document along with a
sidebar on “plain language” tips.
http://www.health.gov/communication/l iteracy/plainlanguagelPlainLanguage.htm
While comprehension is correctly identified as an issue, testing of tools is under-
emphasized here and at the end of Section 7.5. Empirical testing of communication methods
and content is known to be essential and demonstrates respect for communities (cf. Health
Canada’s strategic risk communication handbook available at http://www.hc-sc.gc.ca/ahc-
asc/activitlris-comm/index_e.html).
Section 7.5 discusses the development of communication materials. An example of a
pediatric assent document is given. The point that should be stressed with this example is that
it must be written at the level of understanding of the reader. The use of generic
communication materials for several different stakeholders would not be appropriate, contrary
to what the document might suggest, because each stakeholder will be unique in its
understanding, level of involvement, and connection to the research. Researchers should be
strongly encouraged to gather data about individual stakeholder needs and priorities prior to
developing communication materials geared towards that population. Again two way
communication is essential because stakeholders can best define their strengths and limitations
in understanding.
Another communication tool emphasized in this section is the use of the internet to
communicate with study subjects and with interested stakeholders. One important point that
should be stressed here is that a communication tool is only useful if it is readily available and
readily understood by the target stakeholder population. For example, if the researcher
chooses to communicate by internet, he/she should be certain that everyone in the stakeholder
population has ready access to the internet and has the requisite computer skills to navigate the
research site. Researchers will have to be even more creative in developing communication
strategies with stakeholder groups that include populations of illiterate members or those
which do not speak English.
The real emphasis should be on listening carefully to and the learning from the
stakeholders and looking carefully at the communication needs and limitations of all the
pertinent stakeholders before beginning the development of communication materials.
41 of 85
-------
The end of this section brings up the important issue of conflicting ethical values and study
elements. However, the final lines only address remuneration and communication materials
(see comments above); two elements of a study that are not tools for addressing conflicting
values and elements. The text should segue into comments about methods appropriate for
addressing conflicting values and study disputes.
Section 7.6 states that researchers need to make a commitment to communicating with and
educating the study participants and the community on the purpose of the study. Some
discussion should be included about the issue of whether behavior will be adversely changed as
a result of educating the participants. If too much education results in a behavior change, the
goals of the research will be compromised.
Section 7.7 discusses approaches for reporting results to the participants but does not note
that these approaches should be developed with community input. In communicating the
results, it is vital to make sure the study participants clearly understand the meaning of the
results. Additionally the study participants and the community should be directed to sources of
additional information, resources, and counselors where they can turn to for additional
information or follow-up information in case of additional questions or concerns.
Participants should also be given the option of not receiving any results. Their options
about receiving results should be clearly laid out and discussed prior to initiation of the study.
The community and stakeholders should be notified prior to the publication of any study
results or any press releases relating to the publications. These interested parties have a right
to know when data are going to be made public but do not have the right to change the science
or the researcher’s interpretation of the science in the publication. They can, however, dispute
the interpretations in public.
It is unclear why part of this Section focuses on crisis communications and “responding.”
There is no rationale to help researchers understand why this discussion is included, or how
this advice for a specific type of communication would fit in a comprehensive risk
communication trategy for observational exposure studies. However, judging people’s
perceptions as “accurate” is not appropriate; perceptions are what they are whether someone
else agrees with them or not. Perceptions may differ among individuals and groups; perhaps
the intent here is that lay people may have perceptions that differ from that of experts (see
Morgan et al. 2002. Risk communication: a mental models approach.). Section 7.9 deals with
communicating with the interested stakeholders when a dispute in interpretation of data results
occurs. This is an important area to stress so that researchers discuss strategies to deal with
this as they develop their implementation plan. An additional section on how to deal with
litigation, should it occur, might be helpful to researchers should they find themselves in this
situation.
HSRB Consensus and Rationale
Section 7 covered most of the important issues. This section had a very comprehensive
and informative list of references and several suggestions for additional references have been
42 of 85
-------
noted. While this section is very well-written, it does not clearly focus on the risk
communication methods most suitable for observational exposure studies. The use of side bars
is a very effective tool to communicate small bits of information clearly and quickly. Other
areas that need additional discussion include: (a) the goals of communication; (b) data sharing
and how to address potential scientist-community disagreements; (c) the context in which
communication occurs; and (d) the importance of formative evaluation.
B. Completed Oral Therapeutic Study with Sodium Azide
Charge to the Board
1. The Agency has concluded that this study contains information sufficient for assessing
human risk resulting from potential acute and chronic exposure. Please comment on
whether the study is sufficiently sound, from a scientific perspective, to be used as the
point of departure to estimate a safe level of acute and chronic exposure to sodium
azide.
Board Response to the Charge
The Black et al. (1954) study was not designed adequately to estimate a NOAEL, rather it
focused on evaluating the therapeutic nature of sodium azide. Accordingly, the study did not
investigate the critical effect and it also did not report the administered dose precisely. Further,
the study lacked clarity on baseline data and overlap of subjects in the various treatment
groups. The Board felt that the study did not meet the prevailing standard in scientific conduct
and reporting in clinical trials. For example, Sir A. Bradford Hill’s randomized, placebo-
controlled trial of streptomycin for treatment of tuberculosis reported in the British Medical
Journal in 1948 had better experimental design, conduct, analysis and reporting of results from
a clinical trial. The Black et al. (1954) study, however, suggested increased sensitivity in
several subjects with chronicity, even though the basis of that sensitivity is undefined. Whether
that was due to additional blood pressure-lowering effect or another adverse effect is really not
clear. In its discussions, the Board noted that it may also not be relevant to make direct
comparisons of these human data with animal data, without body surface correction or
appropriateness of the endpoint measured. The specific concerns of the Board are summarized
below:
• Dose: The information provided by Black et al. (1954) suggested that doses of 0.65 and
1.3 mg were given three to five times to individuals with no known body weight
gender, race, and age. In the case of chronic study, the doses were given during 5 days
to more than 2 years. In page 15 of the report, the authors refer to decreasing the dose
from 0.5 to 0.25 mg in 20 patients. However in Table I which lists 30 out of the 39
patients, only 2 patients received 0.5 mg dose. This kind of reporting does not facilitate
confident determination of the point of departure for establishing a safe level of acute
and chronic exposure to sodium azide.
43 of 85
-------
• Toxicity: The study looked for some side effects (liver, kidney) and clinical measures
(no description of methodology or control values) at the therapeutic doses given to
people. There was no focus on the critical organ or critical toxicological effect.
Further, toxicity was evaluated only in 3 individuals and it did not involve any time
course analysis. The rationale regarding why, when and what was evaluated — is
missing. One out of three subjects reported pounding of the head after taking sodium
azide, and this level of information is not adequate to determine a scientifically-
defensible NOAEL or LOAEL for acute and chronic exposures to sodium azide.
• Subjects: There was no baseline/pre-treatment information about the patients, other
than pre-treatment blood pressure, limiting the value of the results presented in Black et
a!. (1954). It is also unclear whether there was an overlap of patients for the acute and
chronic effects of sodium azide.
The blood pressure data for the 30 hypertensive patients given in Table I appeared to be
included in the data presented in Figures 2 and 3 for the acute blood pressure change
for 35 patients, but only partially. In Table I, there are 19 patients with pre-treatment
systolic blood pressure (SBP) of> 190 mmHg and 11 patients with pre-treatment SBP
of 140 but < 190 mmHg among 30 hypertensive patients. However, in Figure 2,
there are only 13 patients with pre-treatment SBP of> 190 mmHg, 13 patients with
pre-treatment SBP of 140 but < 190 mmHg, and 9 normotensive patients with pre-
treatment SBP of < 140 mmHg. The gap in the data cannot be accounted for, and as a
result, the veracity of the data is being called into question. As shown in the figure
below, the plots of the acute systolic and diastolic blood pressure changes based on the
data presented in Table I for 30 hypertensive patients in contrast to Figures 2 and 3
from Black et a!. (1954).
(4. .
.1,-
35
44 of 85
-------
FiG 2
4 AL PRESSURE - SYSTOUC ..3 ()
• 20,_
3.00.’.
; :.,t - - - -) —-;;-—— - . -
•1 -
INI0 0F o 35
FIG. 3
0 OS
o o 0
o $
000 # 0 0
0
• 0 0
40 80 80 100 1 140
n=29
HSRB Consensus and Rationale
Based on the inadequacies in the design, methodology and reporting, the Board
concluded that the Black Ct al study was not sufficiently sound, from a scientific
perspective, to be used as a point of departure to estimate a safe level of acute and chronic
exposure to sodium azide.
Charge to the Board
2. Please comment on the following:
a. Is there clear and convincing evidence that the conduct of the study was
fundamentally unethical?
45 of 85
-------
b. Is there clear and convincing evidence that the conduct of the study was
significantly deficient relative to the ethical standards prevailing at the time the
research was conducted?
Board Response
The Board noted the lack of documentation pertaining to the ethical conduct of the
research reported by Black and colleagues (Black, et al., 1954). It was not unusual for clinical
reports of this era to be vague with respect to matters related to subject recruitment and
informed consent. The publication by Black and colleagues does not provide many details
with regard to the protection of human volunteers but does hint at the following: (1) the public-
health rationale for the study was supported by prior research, (2) medical monitoring was in
place to minimize potential risks to research volunteers, and (3) initial dosing schedules were
set at levels believed to be substantially below acutely toxic levels. A full assessment of these
issues is not possible, however, due to the limited information available. Even if more
information were available, the Board’s analysis is complicated by the absence of widely
agreed upon standards for the ethical conduct of research during the time this study was
conducted.
The report makes just a single statement that is related directly to the adequacy of the
process used to obtain informed consent from research volunteers. That statement is that the
study drug “was administered without informing the patient of either the nature of the drug, or
the change to be expected.” Contemporary standards of informed consent would suggest that
this was ethically worrisome because key facts affecting the assessment of the study’s risk-to-
benefit ratio were withheld from volunteers. Without a more detailed description of the set of
procedures used to present the study, however, it is difficult to conclude that the researchers
deliberately sought to deceive research subjects about the nature of the research—which might
be seen as a reason for regarding the study as significantly deficient with regard to the
prevailing ethical standards of the mid 1950s. For example, subjects might have been
informed that, to avoid a possible placebo-like effect, they would be “blinded” to the nature of
the study drug or the clinical features being studied. If subjects were informed about other
pertinent aspects of the study, and agreed to participate, then that might be viewed as
consistent with prevailing ethical standards of the time. Alternatively, the researchers
conducting the study may have conceptualized this experimental intervention as “innovative
care” and not felt it appropriate to present the intervention as a research study.
In summary, two factors combine to limit the ability to assert that the study was
“significantly deficient relative to the ethical standards prevailing at the time the research was
conducted”: (1) lack of additional detail regarding the procedures used to recruit research
subjects and obtain their consent to participate in the Black study, and (2) lack of unambiguous
ethical standards for determining what information must be disclosed to potential research
volunteers.
HSRB Consensus and Rationale
46 of 85
-------
Based on lack of documentation in the Black et al. study, the Board was unable to
conclude that there was clear and convincing evidence that the conduct of the study was
fundamentally unethical. The Board was also unable to conclude that there was clear and
convincing evidence that the study was significantly deficient relative to the ethical standards
prevailing at the time the research was conducted.
C. Science Issues in Mosquito ReDellent Efficacy Field Research
Appendix A provides background and discussion questions for Board member and
consultant consideration concerning mosquito repellent studies. Board responses to the
questions are provided below.
Board Response
Issue 1: Factors Affecting Repellent Efficacy
Presentations by EPA and consultants indicated that factors which may affect the
variability of initial and confirmed landings include biotic factors such as characteristics of the
mosquito population (genus and species distribution, level of ambient mosquito pressure) or
abiotic factors, which are related to characteristics of the test site such as season, time of
dayfbrightness, or the microclimate (including temperature, humidity, and wind speed and
direction). Other factors that may affect landings include characteristics of the test subjects
(such as differences in mosquito attractiveness; use of alcohol, tobacco, or scented products; or
behavior). Characteristics of the test methods used that may affect variability include the
pattern and duration of exposure, area of skin treated, method used to determine the amount of
test material applied, and number of concurrent treatments per subject.
Variable factors that affect first or second landing or bite include characteristics of the
subjects’ skin, temperature, age, hair on the skin surface, and the density of the mosquito
population at the test site. Usually, mosquitoes will bite within 5 to 15 minutes after the
subject enters the area; after that, the mosquitoes have acclimated to the subject’s presence and
bite much less frequently. Temperature plays a major role in landing or biting activity; an
increase in skin temperature is associated with a decrease in repellency. Skin color or type
does not have a major effect on landing or biting, and hair on the skin surface has a small
effect. The density of the insect population has a major effect on landing or biting activity.
Factors affecting landing or biting activity also include characteristics of the mosquito
population and the test sites. There is significant disagreement among researchers concerning
the most appropriate mosquito population to use for testing. Density of the population can
affect landing or biting activity. The age of the mosquito population also plays a role, with
mosquitoes between 5 to 15 days of age being the most avid biters. Susceptibility of the test
subjects to mosquito bites also can affect landing and biting, although most people are
susceptible to bites. Selection of test sites critically affects landing and biting. Weather,
particularly temperature, can significantly affect biting behavior; repellency decreases by 8
minutes for every I degree Celsius ( C) increase in ambient temperature. Human skin
temperature (30-32 C) is considered to be constant. Wind can significantly affect the efficacy
47 of 85
-------
of vapor repellents because the wind will quickly remove the vapors; increased wind speed is
associated with decreased repellency. Humidity interferes with evaporation and repellency.
Light does not appear to have a significant impact because different mosquito populations exist
that are active either during the day or at night. The local fauna or flora can impact repellency,
especially if the local fauna are the mosquitoes’ preferred hosts.
Test subject attraction for mosquitoes is variable. Skin chemistry may have an impact,
but the delivery mechanism used to apply the repellent to the skin has a larger effect. Skin
temperature is constant and thus has no impact. Skin permeability affects repellency, but it is
dependent on the formulation of the repellent.
HSRB Consensus and Rationale
Based on the thorough and highly informative presentations by the consultants the
Board concluded that in general a protocol needs to elucidate environmental, insect, and
subject factors that would create variability and then provide a rationale for the design,
sampling method, number of subjects based on those factors. Efficacy relevant information
that might be included: a description and rationale for the activity level of the subject
population, the proximity of subjects to one another (or to a partner), the expected density of
the insect population, comparison of insect density and other environmental factors (e.g.
temperature, wind) across sites. The Board also concluded that while replication is important it
is difficult to define across different studies.
Issue 2: Designing for length based sampling
The rationales for different designs should be available by reference and the effects of
different designs should be noted. It is unknown what, if any temporal protocol for exposure is
standard (classical). That is, how were the designs derived and who accepted them as standard.
Each design has different length based sampling. One rationale for using first confirmed intent
to bite (LIBe) rather that first confirmed bite (FCB), as stated by Dr. Carroll, is that using LIBe
minimizes the probability that a subject is actually bitten by a foraging mosquito [ that might
contain a pathogen or produce an irritant or allergic reaction, etc.].
Exposures in one study were during one-minute periods of exposure in 15-minute
intervals, and in another study 5 minutes of exposure every 30-minutes. The time between
first and second (or subsequent) landings is likely to be very variable. Thirty minutes is usually
suggested as a long enough interval to allow for any mosquitoes in the area to land.
The Board raised the following general questions. Were these periods at the
beginning, the middle, or the end of the interval, or were their timing randomly determined,
and what impact did these choices have? What impact is there of using I or 5 minute periods,
and/or of 15 or 30-minute intervals, compared to other period intervals?
Investigators have calculated Complete Protection Time (CPT) as the interval between
application and the first confirmed LIBe or first confirmed bite (FCB). However, investigators
state that the first event must be confirmed by a subsequent event within a period of time (e.g.,
48 of 85
-------
“one-half hour, i.e., in either of the subsequent two exposure periods” for the LIBe). Dr.
Carroll has indicated but not confirmed that LIBe is identical to that of ‘First Confirmed Bite’,
which was classically used in measures of repellency to biting insects.
Dr. Carroll has indicated that it is most likely that a second LIBe (or bite - FCB) would
occur within that 30-minute period. However, from an analytic perspective, caution is
warranted where FCB is calculated on the basis of non-continuous field exposure because it
reduces biting pressure by a factor equivalent to: 11(1-proportional decrease in exposure). The
upshot (where RP < 100%) is a decrease in the probability of receiving a first and confirmatory
bite in any 30 mm period.
Dr. Carroll’s modeling of continuous versus two intermittent exposure scenarios
indicates that the intermittent exposures can substantially reduce the probability of seeing a
FCB at a given RP, though this has not been validated experimentally.
Continuous exposure approaches have been used widely in field studies. To make
intermittent exposure protocols standard should require comparison testing for experimental
validation. Board consultants acknowledge that a 30-minute or one-hour design is more
common in the literature, but does not state whether such designs are continuous or
intermittent exposure designs but imply (from other statements) that is continuous exposure. It
is never really accurate to do a repellent trial continuously for more than about an hour or two
because the avidity of the mosquito population will start to change significantly. The best
designs treat people the appropriate number of hours before the peak biting time and then
expose all subjects simultaneously.
The military and USDA do various types of continuous periods of exposures. The
military may do numbers of landings in 20-minute exposures (challenges) with a criterion of 1
mosquito per minute (and compare treated versus untreated leg), with exposures (challenges)
at 2,4, 6, 8, 10 and 12 hours post-application. Or they can have a protocol with staggered
treatment times during the day (e.g., 800, 1200 or 1600) with specified continuous exposures at
two times during the day (e.g., 1800, 2000). USDA may do 6-hour duration testing starting at
0730 and 1345.
There are further concerns regarding such protocols and their effects on CPT. A set of
concerns has to do with situational factors within the field setting. What differential effect is
there within such studies from light intensity, such as obtained during days in which dusk was
included? Dr. Schofield states there is an [ unspecified] effect, as there are effects due to
variations in temperature (partially related to the mosquito species in that area as well), and the
impact of subject-activity patterns. One Board member believed that there must be also effects
of field variations in relative humidity, barometric pressure, wind speed, and smoke pollution,
etc., on mosquito behavior. Such variations probably affect species-specific and general
mosquito behavior and may be relevant to CPT calculations.
The Board consultants added that it might be possible to adjust for wind and
temperature, if the studies have been done for that particular area and those species of
mosquitoes. A much more serious problem is that the biting activity of mosquitoes varies
49 of 85
-------
systematically with time of day. Variability is due to subjects, location, date and time, and one
would not obtain a normally-distributed (Gaussian) result. Lab variability has been related to
biotic factors (in subjects and mosquitoes) and abiotic factors (temperature, relative humidity,
light, dose and exposure time). These are likely to be variability factors in the field.
Given the lack of standardization between studies, the paucity of research specifically
directed towards evaluation of the above-mentioned factors and the complexity of the test
system, elaboration of anything more than a very basic general model is not possible.
HSRB Consensus and Rationale
The Board noted that the choice of intermittent vs. continuous exposure designs
depends upon the goal of the study (e.g. relative or complete protection) as well as other
factors. The Board concluded that more research is needed to determine the relative benefits of
these designs and concurred with EPA that future guidance on this matter would be helpful.
Issue 3: Complete Protection Time
Dr. Matt Kramer, a USDA statistician, has suggested that the precision of CPT
estimates in repellent testing could be significantly increased by defining failure of efficacy as
the mean time from treatment to a series of several [ e.g., five] landings or bites. The Board
consultants were asked to determine whether this approach would markedly increase the
precision of CPT estimates without requiring additional subjects and, if so, whether the
increased precision would justify the incremental risk to the subjects resulting from their
exposure to mosquitoes. The consultants also were asked to consider the practicalities
involved in testing long-lasting repellents to the point of five landings.
It is unclear what the mean of the times to the first 5 bites/LIBe’s is measuring. A
discussion on the precision seems irrelevant when one is not so sure what is being measured.
Besides, these 5 times are all correlated within each subject, and so the efficiency gain is not so
obvious due to lack of independence among measurements. With the censoring of efficacy
failure seen in a number of mosquito repellency studies, it is unrealistic to be able to observe,
say first 5 bites or LIBe’s, and probably impractical to test to the point of 5 biteslLlBe’s.
Perhaps more critical is the artifact of taking average of times to the first five
bites/LIBe’s, which is to overestimate the time to efficacy failure. This seems to make sense if
one is interested in understanding the average mosquito biting/LIBing behavior. However, it
seems unsuitable if one is interested in characterizing the “efficacy” of repellent products.
The rationale for this idea of measuring times to five bitesfL lBe’s became very clear
during the consultant’s presentation. If the population for statistical inference is the population
of mosquitoes and the statistical inference is on the mosquito biting behavior, it makes sense.
However, that is not the case in these mosquito repellency efficacy studies. The population for
50 of 85
-------
statistical inference is the population of repellent users, not mosquitoes, and the statistical
inference is on the time from application of repellent to failure of its efficacy.
Currently there is no consensus on whether continuous or intermittent methods are most
desirable. Prior to the new regulations, continuous and intermittent methods were used. A
preference for the intermittent method became dominant after the new regulations were in
place because it is more protective of individual subjects since they are exposed to less bites.
However the use of continuous methods might also allow for using fewer subjects in the
research.
HSRB Consensus and Rationale
Based upon information provided by the consultants, the Board remains unclear of
what the mean of the times to the first 5 bites/(landings with intent to bite) would measure that
would be relevant to EPA determinations of efficacy. Therefore the Board concluded that
precision of CPT estimates in repellent testing would not be significantly increased by
defining failure of efficacy as the mean time from treatment to a series of several [ e.g., five]
landings or bites.
The Board concluded it would be helpful to their deliberations if protocols submitted to
the HSRB included rationales for sample size, outcome measures, number of treatment groups
and controls, why a field study is preferable, why a specific environment was selected, how
different environments differ, and how controls for environmental shifts in temperature or
time of day are determined.
The Board understands that the need for larger sample sizes and corresponding increase
in statistical power must be balanced with subject protection, but it is also important
to understand which variables can be controlled. The expertise of control and treated subjects
with respect to detecting mosquito landings must be balanced and the activity of subjects also
should be controlled.
D. Completed Insect Repellent Efficacy Study (SCI-OO1) of DEET Formulations
Charge to the Board
1. Is this study sufficiently sound, from a scientific perspective, to be used to assess the
repellent efficacy of the formulations tested against mosquitoes? Please comment
specifically on:
a. Whether participation in field testing by several subjects on the day after they had
been treated with a different test repellent is likely to have affected the validity of
the results for those subjects on those days.
51 of85
-------
Board Response
The active ingredient DEET formulated in three forms was tested for its ability to repel
mosquitoes from the legs of volunteers by the protocol presented and modified by Carroll-
Loye. The protocol had been modified based on the suggestions and input of EPA and the
HSRB. The results were reported in SCI.001.1, LipoDEET 302; SCI.001.2, L1p0DEET 3434;
and SCI.001.3 Coulston’s Duranon, all compared to Ultrathon (approved by the military). All
experiments were conducted using Good Laboratory Practices.
The dosimetry for the three products was done in the laboratory on July 3-5, 2007. The
field tests were conducted at Site 1 in Butte County, a grassland habitat, on July 7, 8 and 15,
and at Site 2 in Glenn County, a forest habitat, on July 12, 13 and 14. Different mosquito
species composition occurred at the two sites. Ten subjects were used for the dosimetry tests.
Ten subjects were used for each of the three products. Some of the subjects participated for all
three of these test articles as well as for the Ultrathon. The subjects were required to be above
18 years of age and no more than 55 years of age, active in rural outdoor settings, and having
used no repellent on the day before the test. Only legs were tested in this study because of
greater biting pressure on legs than arms. There were two experienced persons serving as
negative controls (i.e., without any repellent product) to confirm mosquito biting pressure (and
biting pressure was maintained throughout the period of the study, defined as at least one
Landing with Intent to Bite, LIBe, per mm at Site 1). Experimental subjects, in pairs,
monitored LIBe’ s during a one minute interval each 15 minutes, until the First Confirmed
LIBe (FCLIBe) could be determined. Stopping rules were employed. No evidence of West
Nile Virus (WNV) was present in either test site. Mosquitoes landing were taken to the
laboratory for later identification and for screening for WNV, Western Equine Encephalitis,
and St. Louis Encephalitis Virus, and all mosquitoes were negative. All subjects wore Tyvek
coverall, head nets and surgical gloves, and they worked in pairs. There was a one minute
observation period during each 15 minute interval, starting 150-210 minutes post application.
Toward the end of the day, controls were reduced from 2 to 1 subject to limit exposure to
mosquitoes. Complete protection time (CPT) was measured, defined as the time to the First
Confirmed Landing with Intent to Bite (FCLIBe). Adequate LIBing pressure was maintained
throughout the test period.
LipoDEET 302 is 30% DEET on lipid spheres designed to improve the durability and
to improve the cosmetic properties. Six females and four males were tested for dosimetry, 7
females and 3 males at Site 1, 3 females and 7 males at Site 2. The MOE was 522. It yielded a
CPT of 10.3 ± 1.3 hr in Site 1 and 9.5 ± 1.8 hr in Site 2.
LipoDEET 3434 is DEET at 34.34%; this protocol SCI.001.2 was amended to test a
different test article than the original protocol described Five females and five males were
tested at Site 1, 4 females and 6 males at Site 2. The MOE was 492. It yielded a CPT of 10.6
± 1.3 hr in Site land 10.4± 1.9hrmnSite2.
Coulson’s Duranon is 20% DEET in microscopic protein spheres to reduced skin
absorption of DEET, improve cosmetic properties and inhibit evaporation. Five females and
52 of 85
-------
five males were tested at Site 1, 3 females and 7 males at Site 2. The MOE was 856. It
yielded a CPT of 8.4 ± 1.9 hr in Site 1 and 9.5 ± 1.3 hr in Site 2.
Data were presented for the comparison article, Ultrathon, CPT of 10.1 ± 2.3 hr for Site
I and 10.0 ± 2.2 hr for Site 2. While this comparison may have been of interest to the sponsor,
it is unclear why this information was provided in this report.
The report was clearly written and a greater attention to statistical analysis was
provided than has been provided in the past by Carroll-Loye Biological Research. The study
was justified in that additional insect repellents that are more efficacious and/or more
acceptable cosmetically to the public would be an advantage from both the standpoint of health
(to reduce the chances of contracting a mosquito-borne disease) and of comfort. The
information should be generalizable to the public, although the exclusions, which were highly
appropriate, excluded some subpopulations that would likely use insect repellents. The
experiment was necessary to determine the field efficacy of these test formulations, and the
experiments were set up to meet the study objective. Measurements taken were appropriate for
the objective and quality assurance considerations were in place.
The experiment was conducted according to the approved protocol with some
deviations. One of the deviations, which were not acknowledged in the description, was the
fact that some of the subjects were tested on the day following a test with another product,
despite the fact that the protocol stated “Use of insect repellent within one day of preceding the
study” was an exclusion criterion. Additionally, the tests on any given repellent (including the
repellent used in study WPC-001) were not conducted on a single day and the statistical
treatment of the data did not account for different days of testing.
The deviation of not allowing a day of non-repellent use before testing was of concern
to the Board because of potential persistence of the repellent of one day’s test into the test of
the following day. However, information provided by the Carroll-Loye Biological Research
via public comments indicated that the repellent was carefully washed off at the end of the
experiment both with soap and water and with alcohol. The public comments also indicated
that the reason for the one day wait in the protocol was not related to concerns about the
persistence of the repellent. Therefore this deviation may not have affected the validity of the
results obtained. In addition, the public comments indicated that the reason for the
compression of the tests on these several test materials into a short span of time was because of
the reports that West Nile virus-contaminated mosquitoes were moving close to the test area
and that this test region would have been unsafe in a matter ofjust a few more days. This
explanation was adequate to explain why the testing days were set up as they were. It would
have been advisable to have had this deviation and the rationale for it presented in the report.
Charge to Board
b. The effects of changes to the experimental design resulting in evaluation of repellents using
fewer than ten subjects per treatment per day, followed by pooling of results by site for
statistical analysis.
53 of 85
-------
Board Response
Strengths
A strength of the study is the inclusion of the positive control treatment with 3M’s
Ultrathon.
Weaknesses
SCI-O01 studies are very troubling from a statistical design point of view. There is
some indication that there was no desire to compare the various repellents to one another, and
yet each of reports compared a repellent to a positive control, the Ultrathon repellent. The
statistical analysis assumes that there are 20 different subjects on each repellent over two sites,
but many of the subjects wore both repellents and a few subjects were used in both sites.
Subjects that were used at both sites were subjects: 8, 13, 14, 15, 37, 40, 46, 52, 53, 60, 61, 62,
63, 67, 71, 72, and two control subjects. Almost all subjects wore more than one repellent.
This is not an inherent flaw in the design, although if subjects are going to be used more than
once, there should have been some counterbalancing of the repellents to the subjects. The
major flaw is in the statistical analysis, particularly when repellents are being compared to one
another as is done in the reports by comparing each repellent to Ultrathon. The analysis
assumes that there are 20 different subjects involved in each of the three write-ups. That is, the
analysis assumes that the experimental design is a parallel subjects design and it is clearly not.
In the combined experiments, there were only 33 unique subjects, representing 80 data
points, excluding the negative controls. Subjects do not appear to have been randomly
allocated to four test materials, between Sites 1 and 2, and over five days. This lack of
statistical independence of data between different test materials, Sites 1 and 2, and different
dates of experiment renders the analysis incorrect. For example, due to overlap of subjects
between Sites 1 and 2, one cannot pooi the results by site for statistical analysis. Also there is
substantial overlap of subjects among and between test materials, the comparison among and
between test materials cannot be made. This renders the analysis scientifically invalid.
In SCI-001, there is 0% censoring. This differs to previous studies conducted and
reported by Carroll-Loye Biological Research, Inc and reviewed by the Board. As an example,
there was 40% censoring in EMD-004.1, 10-30% censoring in EMD-004.2, and 90-100%
censoring in EMD-004.3. The Board supports the EPA’s evaluation that “Further clarification
is needed to verify the accuracy of the data generated.”
The data analysis (pairwise comparison between three test materials and the positive
control Ultrathon) is inadequate and is inconsistent with the experimental design and conduct
of the study in which subjects were allocated to four treatment groups. As was suggested by
the HSRB in its January 2007 meeting, the appropriate analysis would have been an analysis of
variance type. Wilcoxon rank sum test in the presence of censored data is known as Gehan’s
test, and is known to be inefficient for comparison of Kaplan-Meier estimates. A more
appropriate test comparison of censored time to event data is the log rank test or better yet a
regression analysis for censored time to event data based on Cox proportional hazards
54 of 85
-------
regression models. The analysis of the number of LIBe’s is inappropriate as it ignores the
duration of follow-up during which LIBe’s are recorded. An appropriate analysis would have
been Poisson analysis.
Having ten subjects spread over five days and combining the results into the current
analysis has a potential to confound the effect of repellent with the different mosquito
pressures on different days. This would have been better if it were planned in advance. The
whole idea is that the experimenter has control over the experiment by using random
mechanism for subject selection and for allocation of subjects to repellents, days and sites. One
solution would be to perform the analysis of variance as was done when the results for two
sites were pooled, i.e., by including the day as the main effect and the two-way repellent by
day interaction (repellent effect modification by day effect) in the analysis of variance, in
Table 5, on page 17 of 217, of the report for SCI-0O 1.3. Of course, this requires making the
assumption that the experimenter had adequate control of the experiment as described above
and that observations are statistically independent, which is not the case. With the proper
experimental design using site and day of experiment as blocking factors, the analysis of
variance would allow evaluation of the three main effects of repellent, day and site, along with
the two-way and three-way interactions. There is no inherent problem with pooling across
studies. Study site can be considered as a blocking factor, and as such it would generally be
appropriate to pool across the two blocks. This assumes independence of data from sites,
namely, no overlap in subjects between the sites. This was not the case for these experiments,
and as a consequence the data cannot be pooled across the two sites.
Pooling of results by site for statistical analysis is generally preferable if the other
experimental design aspects are the same, which appears to be the case with the reported
studies. The main advantage of a pooled analysis versus marginal analyses for each site is
two-fold: 1) by doubling the sample size for the analysis, you increase the statistical efficiency
by a factor of 1.41 in the point estimation of the CPT exactly or in the analysis of variance
approximately (as you lose one degree of freedom for site effect) and 2) you can assess the
effect of repellent more accurately by accounting for the potential effect of site difference in
mosquito pressure. However, this assumes that the data from two sites are independent, which
is again obviously not the case. There are three subjects (40, 18 and 62) used in both sites with
LipoDEET 302, two subjects (53 and 46) used in both sites with Ultrathon, and three subjects
(40, 8 and 63) used in both sites with Duranon.
HSRB Consensus and Rationale
While the Board concluded that the participation of several subjects on the day after
they had been treated with a different test repellant was not ideal, this may not have affected
the validity of the results.
While the Board reviewed this protocol generally favorably previously, since it was
combined with a second protocol, the conduct and analysis deviated from expectation. For
example, with only 33 subjects for 80 data points (excluding the negative controls), the overlap
of some of the same subjects for different test materials, for Sites 1 and 2, and for different
dates of the experiment without proper experimental design and control, the Board concluded
55 of 85
-------
that it is impossible to interpret the reported data adequately thus bringing the scientific
validity of the results into question. In addition, the study may not have been sufficiently
sound enough to estimate population variances. Thus, the Board concluded that the study was
not sufficiently sound, from a scientific perspective, to be used to assess the repellent efficacy
of the formulations tested against mosquitoes.
Charge to Board
2. Does available information support a determination that this study was conducted in
substantial compliance with subparts K and L of EPA regulations at 40 CFR part 26?
Please comment specifically on:
a. The decision to use a different test formulation in place of one of the test materials
described in the protocol reviewed by the IRB, EPA and the HSRB.
b. How to assess the ethical conduct of an insect repellency study involving multiple
test formulations when there is an ethical deficiency in the conduct of the study
with respect to one of the test formulations. If the ethical deficiency warrants not
relying on the results of the testing with regard to one test formulation, under what
circumstances (if any) does the ethical deficiency affect the acceptability of the
results from testing the other formulations?
Board Response
Brief Overview of the Study
The basic protocol for these studies (SCI-00l) was initially reviewed at the January
2007 meeting of the HSRB, at which time the Board concluded that the study would meet the
requirements established in the Environmental Protection Agency’s final human studies rule
(40 CFR Part 26) pending minor revision. In particular, the Board was concerned about
several matters relevant to subject recruitment and the overall conduct of the study. For
example, the protocol did not describe how untreated controls would be recruited, but instead
implied that controls will be recruited in the same manner as subjects in the “exposure” arm—
via “word-of-mouth” and a Volunteer Data Base maintained by the Principal Investigator. The
Board recommended that the protocol clarify how untreated controls will be recruited, and that
the IRB of record (Independent Investigational Review Board [ IIRB], Inc. of Plantation, FL)
review any materials used for recruiting purposes, including any telephone scripts fliers,
emails, letters, or local ads. The Board also felt that the risks associated with DEET exposure
during the course of the study are mischaracterized, and that the informed-consent document
was also structured in a manner that did not apply to unexposed control subjects. In light of
these deficiencies, the Board recommended that the informed-consent document be redesigned
and re-reviewed by IIRB (EPA HSRB 2007a).
Subsequent to the aforementioned meeting of the HSRB, the informed consent
documents were revised in accordance with Board and EPA recommendations, submitted to
56 of 85
-------
IIRB for re-review, and approved (Carley, 2007). The protocol, revised just prior to study
execution to meet Agency and HSRB recommendations, was not submitted to the IIRB for
review or approval as required by Federal regulations.
The documents provided by Carroll-Loye (Carroll, 2007a; Carroll, 2007b; Carroll,
2007c) specifically state that each study was conducted in compliance with the requirements of
the U.S. EPA Good Laboratory Practice Regulations for Pesticide Programs (40 CFR 160); 40
CFR 26 subparts K and L; FIFRA § 12(a)(2)(P); and the California State EPA Department of
Pesticide Regulations for study monitoring (California Code of Regulations Title 3, Section
6710). Each study was also reviewed and approved by a commercial human subjects review
committee, IIRB. Documentation provided to the EPA by LIRB indicated that it reviewed
these studies pursuant to the standards of the Common Rule (45 C.F.R. Part 46, Subpart A) and
determined them to be in compliance with that Rule.
As submitted to the Agency, each completed study consists of two interdependent
analyses: 1) a dosimetry study designed to determine the amount of an insect-repelling
compound (30% or 34% DEET in liposomal capsules, or 20% DEET in protein capsules) that
typical users would typically apply when provided with a lotion formulations; and 2) an
efficacy study designed to measure the effectiveness of each compound as a mosquito
repellent. For each dosimetry and efficacy study, commercially available and obtained 3M
Ultrathon (34.34% polymerized DEET) was used as a comparator. Ultrathon is the principal
insect repellent used by American military forces and is considered to be one of the most
effective insect repellents available.
Dosimetry and efficacy studies for personal insect repellents containing LipoDEET 302
and 3434 (30% and 34% DEET in liposomal capsules, respectively) and Duranon (20% DEET
in protein capsules) were conducted from July 3 through August 2, 2007 (Carroll 2007a;
Carroll 2007b; Carroll 2007c). All studies (one using each test compound) were performed
simultaneously at a laboratory site in Davis, California, and at field sites in Butte and Glenn
Counties, California, by researchers at Carroll-Loye Biological Research. The study sponsor
was Scientific Coordination, Inc., of Rockville, Maryland. The studies were conducted using
products from two manufacturers: LipoDEET 302 and 3434 was manufactured and supplied
by DermAegis, Inc. of Rockford, Illinois; Durañon was manufactured and supplied by Sawyer
Products of Safety Harbor, Florida. It is important to note, however, that the protocol
originally reviewed and approved at the January 2007 HSRB meeting listed a different set of
test compounds; at the sponsor’s request Carroll-Loye Biological Research changed one of the
test compounds from Insect Guard II (EPA Reg. No. 54287-8) to Lip0DEET 3434 (an
unregistered compound) at the sponsor’s request.
Dosimetry was determined by direct measurement of compound application. The
efficacy of each as a mosquito repellent was determined by measuring the ability of the
formulations to prevent mosquito landings (defined as “Lite with Intent to Bite”; LIBe) under
field conditions. Mosquitoes were aspirated mechanically prior to biting; prior to initiation of
the efficacy study, all volunteers were trained, using laboratory-raised, pathogen-free
mosquitoes in a controlled laboratory setting, both to recognize mosquitoes landing with the
intent to bite and to remove such mosquitoes with an aspirator. During the field studies,
57 of 85
-------
participants worked in pairs to facilitate identification and aspiration of LIBing mosquitoes
during brief exposure periods.
The dosimetry study enrolled a total of 10 individuals, each of who tested all four
formulations (the three test compounds and the comparator). Each efficacy study enrolled 10
subjects for each formulation at each of the two field sites. Many volunteers participated in
multiple phases. For example, six of the 10 participants in the dosimetry study also
participated in one or more of the field tests. In total, 39 volunteers participated in at least one
phase of SCI-001 .1, SCI-001 .2 or SCI-001 .3. Fifteen of these volunteers also participated in
the dosimetry and/or efficacy phase of Carroll-Loye study WPC-001 described below. In
addition to these 41 volunteers, two controls (described as “experienced personnel” and who
were untreated with repellent) also participated to determine ambient LIBe pressure at field
sites. On any given day at a field site, the same pair of participants served as controls for SC!-
001.1, SCI-00 1.2, SCI-00 1.3 and the concurrently run WPC-00 I study. Five individuals
served (repeatedly) as controls during each of the six field tests; two also participated in the
repellent efficacy tests, to give a cumulative total of 45 volunteers. In addition, three alternate
participants were enrolled to: 1) replace any individual who withdrew; and 2) protect the
confidentiality of any participant excluded from the study as a result of pregnancy or other
potentially stigmatizing condition, as described below.
CritiQue
The Board concurred with the factual observations of the ethical strengths and
weaknesses of the study, as detailed in the EPA’s Science and Ethics Review (Carley, 2007).
The risks to study participants, in general, were minimal and were justified by the likely
societal benefits, including data on the efficacy of these new formulations (30% or 34% DEET
in liposomal capsules, or 20% DEET in protein capsules) as personal insect repellents. Higher
concentrations of DEET are commercially available and have been used as repellents for years;
the subjects enrolled in this study were thus unlikely to be at increased risk of experiencing
adverse side effects upon exposure. Reactions to mosquito bites are usually mild and easily
treated with over-the-counter steroidal creams. The study also excluded individuals who have
a history of severe skin reactions to further minimize the risk of a participant experiencing a
severe physical reaction to a mosquito bite. In addition, the study protocol was designed
specifically to minimize the likelihood that a mosquito will bite, through the use of clear
stopping rules, limited exposure periods, and paired observation; no side effects or adverse
events were reported. To minimize the risk that study participants would be exposed to
illnesses like WNV, the study protocol called for field tests of repellent efficacy to be
conducted only in areas where known vector-borne diseases have not been detected by county
and state health or vector/mosquito control agencies for at least one month. Mosquitoes
collected during the field studies also were subjected to serologic or molecular analyses to
confirm that they were free of known pathogens. Finally, the study protocol also included
several mechanisms designed to minimize coercive recruitment and enrollment, compensation
was not considered to be so high as to unduly influence participation, and minors and pregnant
or lactating women were explicitly excluded from volunteering (pregnancy being confirmed by
requiring all female volunteers to undergo a self-administered over-the-counter pregnancy test
on the day of the study). The potential stigmatization resulting from study exclusion was
58 of 85
-------
minimized by the use of so-called “alternate” participants, allowing for volunteers to withdraw
or be excluded from participating without unduly compromising their confidentiality.
Regrettably, several serious protocol deviations occurred during the conduct of these
studies. Most notably, due to the study investigator’s failure to obtain IRB approval for
fundamental changes to one of the study protocols, the research described in SCI-001 .1, SCI-
001.2 and SCI-001.3 does not comport with the applicable requirements of 40 CFR Part 26,
subparts K and L.
First, as mentioned previously, many study volunteers participated in multiple phases
of SCI-00 1.1, SCI-00 1.2 and SCI-00 1.3, testing the field efficacy of different insect repellents
on different days. Many volunteers participated in field trials conducted on consecutive days,
and were re-randomized to receive the same or a different test compound each time. The
approved protocol, however, includes criteria excluding any participant who “ [ used an] insect
repellent within one day preceding the study” (c.f., Carroll, 2007a; Carroll, 2007b; Carroll,
2007c). There was some disagreement among Board members, as to whether the phrase “one
day preceding the study” specifically excluded participants who had used or been exposed to
an insect repellent on the day preceding a field trial, or simply prior to consent and enrollment
in the study overall. This change is unlikely to have increased the risks to study participants,
but several Board members nevertheless raised the concern that compromising these exclusion
criteria may adversely affect the prospective benefits of the study data .Dr. Carroll submitted a
response to the EPA that implied the former (Carroll, 2007d), but the Board nevertheless
recognized the uncertainty surrounding this issue. Several Board members believed that if the
exclusion criteria had been violated without submitting the protocol for approval, this may
represent a departure from accepted review and approval practices.
Second, to minimize the risk that study participants would be exposed to arthropod-
borne illnesses like WNV, for example, during its review of another Carroll-Loye protocol at
the April 2007 HSRB meeting (EPA HSRB, 2007b) the Board recommended conducting
serologic or molecular analyses to confirm that mosquitoes collected during the field studies
were free of known pathogens. Study investigators did conduct such analyses in the context of
these studies, but did so without submitting the protocol for approval or informing study
participants of these analyses in the informed consent document. Although this change is
likely to have actually reduced the risks faced by study participants, it is nevertheless a
substantial departure from accepted review and approval practices.
Finally, and most seriously, at the sponsor’s request Carroll-Loye Biological Research
changed one of the test compounds from Insect Guard II (EPA Reg. No. 54287-8) to
Lip0DEET 3434 (an unregistered compound) without IRB or HSRB review or approval. Dr.
Carroll has justified the decision to substitute a test material by stating that, in conversations
that he has previously had with the IRB and the California EPA, that IRB review and approval
is needed only for “proposed changes ... that are likely to increase risk to participants” (Carroll
2007d, 3). Because the substituted material (Lip0DEET 3434) was nearly identical to another
compound approved for testing (L1p0DEET 302), and contained a concentration of DEET
considerably less than that in already approved and marketed personal insect repellents, Dr.
Carroll felt that the substitution posed no increase in risk to study participants and fell within
59 of 85
-------
the “latitude” given to study investigators with respect to minor protocol changes. The Board
disagreed, feeling that a change of test materials was not simply a minor protocol change;
although it is unlikely that study participants were subjected to greater risk, the substitution of
LipoDEET 3434 for Insect Guard II was a protocol change of sufficient magnitude to warrant
IRB notification and review. More importantly, the failure to seek IRB review and approval is
a significant and serious departure from accepted review and approval practices, as well as a
violation of the letter and intent of the Agency’s Final Human Studies Rule.
Federal guidelines from the Office of Human Research Protections (OHRP) clearly
state that all proposed protocol changes must be reviewed by the IRB of record at convened
meetings, in accordance with HHS regulations at 45 CFR 46.108(b), although institutions may
adopt policies describing the types of minor changes in previously approved research that can
be approved under an expedited review procedure in accordance with HHS regulations at 45
CFR 46.1 10(b)(2). Except in cases when necessary to prevent imminent harm to study
participants, an investigator should never institute a protocol change without IRB review.
Federal regulations regarding review and approval of human subjects research, for example,
explicitly prohibit investigators from implementing any protocol changes without prior IRB
approval unless such changes are necessary to prevent immediate, serious harm to study
participants. It is never an investigator’s prerogative to determine which protocol changes
warrant IRB review and which do not; only the IRB (or some authority other than the
investigator) has that authority.
The failure of Carroll-Loye Biological Research to 1) obtain IRB approval of the
revised protocol; and 2) report these protocol deviations to the Independent Investigational
Review Board in a timely manner are serious regulatory breaches. The failure of Dr. Carroll to
notify the EPA and the HSRB of protocol deviations such as the substitution of LipoDEET
3434 for Insect Guard ii in submitted study documents for studies SCI-001 .1 and SCI-001 .3 is
also troubling. The Board recommended the Carroll-Loye Biological Research report these
deviations to the IIRB as soon as possible and work with that organization to develop and
implement a corrective course of action.
Because of the serious nature of these deviations, the research described in SCI-00 1.2
using the unapproved pesticide formulation failed to comport with the applicable requirements
of 40 CFR Part 26, subparts K and L, and the Board recommended that the Agency not accept
for regulatory decision-making purposes any of the data obtained during the conduct of that
particular study. Furthermore, this particular ethical deficiency also affects the acceptability of
the results from testing the other two formulations. SCI-00l.1, SCI-001.2 and SCI-001.3 were
run concurrently, and study participants enrolled in all three studies ran the risk of being
randomized to receive treatment with the unapproved investigational compound, LipoDEET
3434. It is thus impossible to separate the data collected in SCI-00 1.1 and SCI-00 1.3 from that
collected in SCI-00 1.2, and the Board recommended that the Agency not utilize any the data
obtained during the conduct of these three studies.
HSRB Consensus and Rationale
60 of 85
-------
The Board concluded that the research was conducted in a manner that failed to meet the
applicable requirements of §40 CFR 26, subparts K and L. The study investigator failed to
obtain IRB approval for fundamental changes to one of the study protocols, namely
substituting an unregistered compound for the study compound. As a result, the Board
recommended that the data collected from these three concurrently run studies should not be
considered by the Agency because the changes placed volunteers in all three studies at risk of
being randomized to receive treatment with an unregistered compound.
E. Completed Insect Repellent Efficacy Study with Oil of Lemon Eucalyptus (WPC-OOfl
Charge to the Board
1. Is the research conducted under WPC-00 1 sufficiently sound, from a scientific
perspective, to be used to assess the repellent efficacy of the formulation tested against
mosquitoes? Please comment specifically on whether participation in field testing by
several subjects on the day after they had been treated with a different test repellent is
likely to have affected the validity of the results for those subjects on those days.
Board Response
The active ingredient Oil of Lemon Eucalyptus (OLE) formulated as a pump spray was
tested for its ability to repel mosquitoes from the legs of volunteers by a study presented by
Carroll-Loye. All experiments were conducted using GLP. This study was conducted similarly
to and in conjunction with the DEET efficacy studies of SCI-001, and the descriptions of the
experimental conditions and design of SCI-001, above, are appropriate for this study and will
not be repeated here.
The test product had 30% OLE in a pump spray. Seven females and three males were
tested at Site 1, 3 females and 7 males at Site 2. It yielded a CPT of 6.1 ± 1.5 hr in Site 1 and
4.2 ± 0.8 hr in Site 2.
As was discussed for the SCI-001 study, the experiment was conducted according to
the approved protocol with some deviations, including the lack of at least a one day wait for
some of the subjects before they were tested with another product. The Board’s analysis of
this deviation and design of this experiment was discussed above for SCI-001, and is pertinent
to this experiment also and will not be repeated here. Treatment of subjects on successive days
was unlikely to have a significant effect.
HSRB Consensus and Rationale
The Board concluded that despite problems estimating variability, the Carroll-Loye
study WPC-001 assessing the repellent efficacy of the formulation tested was sufficiently
sound for the purposes for which it was intended.
Charge to Board
61 of85
-------
2. Does available information support a determination that the research covered by WPC-
001 was conducted in substantial compliance with subparts K and L of EPA regulations at
40 CFR part 26? If the conduct of any part of SCI-001 is deemed not to substantially
comply with the requirements of subparts K and L, please comment specifically on how to
assess the ethical conduct of research conducted under WPC-001 in light of the fact that it
was conducted at the same times and at the same places as the research covered under
protocol SCI-001.
Brief Overview of the Study
The protocol for this study was initially reviewed at the April 2007 meeting of the
Human Studies Review Board, at which time the Board concluded that the study would meet
the requirements established in the Environmental Protection Agency’s final human studies
rule (40 CFR Part 26) pending minor revision. In particular, the Board was concerned about
several matters relevant to subject recruitment and the overall conduct of the study. To
minimize the risk that study participants would be exposed to arthropod-borne illness, for
example, the Board recommended that investigators trap landing mosquitoes or other vectors
for pooled serologic or nucleic acid-based testing and alert research participants if they had
been inadvertently exposed to vector-borne pathogens. Secondly, the Board expressed
concerns about plans to recruit research subjects in Florida, as these recruitment procedures
were not described adequately in the protocol and supporting materials. Finally, the Board
raised questions about the informed consent procedures for control subjects, suggesting that the
informed consent procedures should be modified to more clearly explain the risks to control
(untreated) research subjects. In light of these deficiencies, the Board recommended that the
protocol and informed-consent document be revised and submitted to re-review by IIRB (EPA
HSRB, 2007b). The protocol and informed consent documents subsequently were revised in
accordance with Board and EPA recommendations, submitted to IIRB for re-review, and
approved (Carley, 2007).
The documents provided by Carroll-Loye (Carroll, 2007e) specifically state that each
study was conductcd in compliance with the requirements of the U.S. EPA Good Laboratory
Practice Regulations for Pesticide Programs (40 CFR 160); 40 CFR 26 subparts K and L;
FIFRA § 12(a)(2)(P); and the California State EPA Department of Pesticide Regulations for
study monitoring (California Code of Regulations Title 3, Section 6710). Each study was also
reviewed and approved by a commercial human subjects review committee, IIRB.
Documentation provided to the EPA by IIRB indicated that it reviewed these studies pursuant
to the standards of the Common Rule (45 CFR Part 46, Subpart A) and determined them to be
in compliance with that Rule.
As submitted to the EPA, the completed study consists of two interdependent analyses:
1) a dosimetry study designed to determine the amount of oil of lemon eucalyptus-based
personal insect repellent that typical users would typically apply when provided with a pump
spray formulation; and 2) an efficacy study designed to measure the effectiveness of the pump
spray formulation as a mosquito repellent. The dosimetry and efficacy studies were conducted
from July 10 through August 1, 2007 (Carroll, 2007e). The study was performed at a
laboratory site in Davis, California, and at field sites in Butte and Glenn Counties, California,
62 of 85
-------
by researchers at Carroll-Loye Biological Research. The study sponsor was WPC Brands, Inc.
of Bridgeton, Missouri. The study was conducted using product manufactured and supplied by
ChemRite CoPac, Inc. of Lannon, Wisconsin.
Dosimetry was determined by direct measurement of compound application. The
efficacy of each as a mosquito repellent was determined by measuring the ability of the
formulations to prevent mosquito landings (defined as “Lite with Intent to Bite”; LIBe) under
field conditions. Mosquitoes were aspirated mechanically prior to biting; prior to initiation of
the efficacy study, all volunteers were trained, using laboratory-raised, pathogen-free
mosquitoes in a controlled laboratory setting, both to recognize mosquitoes landing with the
intent to bite and to remove such mosquitoes with an aspirator. During the field studies,
participants worked in pairs to facilitate identification and aspiration of LIBing mosquitoes
during brief exposure periods.
The dosimetry study enrolled a total of 10 individuals. Each efficacy study enrolled 10
subjects at each of the two field sites. Many volunteers participated in multiple phases. For
example, six of the 10 participants in the dosimetry study also participated in one of the field
tests. In total, 23 volunteers participated in at least one phase of WPC-001. Fifteen of these
volunteers also participated in the dosimetry and/or efficacy phase of Carroll-Loye studies
SCI-00 1.1, SCI-00 1.2 or SCI-00 1.3 described previously (Carroll, 2007a; Carroll, 2007b;
Carroll, 2007c). In addition to these 23 volunteers, four controls (described as “experienced
personnel” and who were untreated with repellent) also participated to determine ambient LIBe
pressure at field sites. On any given day at a field site, the same pair of participants served as
controls for WPC-001 and the concurrently-run SCI-00l .1, SCI-001 .2, and SCI-001 .3 studies.
Three individuals served (repeatedly) as controls during each of the three field tests; all three
also participated in the repellent efficacy tests, giving a cumulative total of 27 volunteers. In
addition, three alternate participants were enrolled to: 1) replace any individual who withdrew;
and 2) protect the confidentiality of any participant excluded from the study as a result of
pregnancy or other potentially stigmatizing condition, as described below.
Critique
The Board concurred with the factual observations of the ethical strengths and
weaknesses of the study, as detailed in the EPA’s Science and Ethics Review (Carley, 2007).
In general, the research described in WPC-00l comports with the applicable requirements of
40 CFR Part 26, subparts K and L. The risks to study participants were minimal and were
justified by the likely societal benefits, including data on the efficacy of this formulation as a
personal insect repellent. Based on toxicological data currently available, the subjects enrolled
in this study were unlikely to be at increased risk of experiencing adverse side effects upon
exposure. Reactions to mosquito bites are usually mild and easily treated with over-the-
counter steroidal creams. The study also excluded individuals who have a history of such
severe skin reactions to further minimize the risk of a participant experiencing a severe
physical reaction to a mosquito bite. In addition, the study protocol was designed specifically
to minimize the likelihood that a mosquito would bite, through the use of clear stopping rules,
limited exposure periods, and paired observation; no side effects or adverse events were
reported. To minimize the risk that study participants will be exposed to illnesses like WNV,
63 of 85
-------
the study protocol called for field tests of repellent efficacy to be conducted only in areas
where known vector-borne diseases have not been detected by county and state health or
vector/mosquito control agencies for at least one month. Mosquitoes collected during the field
studies also were subjected to serologic or molecular analyses to confirm that they were free of
known pathogens. Finally, the study protocol also included several mechanisms designed to
minimize coercive recruitment and enrollment, compensation was not considered to be so high
as to unduly influence participation, and minors and pregnant or lactating women were
explicitly excluded from volunteering (pregnancy being confirmed by requiring all female
volunteers to undergo a self-administered over-the-counter pregnancy test on the day of the
study). The potential stigmatization resulting from study exclusion was minimized by the use
of so-called “alternate” participants, allowing for volunteers to withdraw or be excluded from
participating without unduly compromising their confidentiality.
Although a majority of the Board concluded that research described in WCP-OO1 was
in substantial compliance with the applicable requirements of 40 CFR Part 26, subparts A and
L, further comments are warranted on certain events that took place during the conduct of the
study. In particular, as noted in the EPA’s review of these studies, several significant protocol
deviations occurred.
As discussed in greater detail in the Board’s review of Carroll-Loye studies SCI-0Ol.1,
SCI-001 .2 and SCI-001 .3, for example, there was disagreement among Board members
regarding the change to exclusion criteria and whether or not these changes should have been
reviewed by the IIRB. Many of the study volunteers participated in multiple phases of WPC-
001 (as well as phases of SCI-00 1.1, SCI-00 1.2 and SCI-00 1.3), participating in field trials
conducted on consecutive days and being re-randomized to receive the same or a different test
compound each time. The Board recommended that Carroll-Loye Biological Research report
these deviations to the IIRB as soon as possible and work with that organization to develop and
implement a corrective course of action.
Although the research conducted under WPC-001 was conducted at the same times and
at the same places as the research covered under protocol SCI-001, there is no evidence that
study participants enrolled in this study ran the same risk of being randomized to receive
treatment with an unapproved investigational compound as did participants in SCI-00 1.1, SCI-
001.2 and SCI-00 1.3. A majority of the Board thus concluded that the research described
therein substantially complies with the applicable requirements of §40 CFR 26, subparts A and
L. Some Board members, however, concluded that the study investigator’s repeated failure in
this and other studies to obtain IRB review for protocol changes, or to report study changes and
protocol deviations to the IRB, created a pattern of departure from accepted review and
approval practices that could have put study participants at increased risk or impair the
informed consent process.
HSRB Consensus and Rationale
A majority of the Board concurred with the initial assessment of the Agency that the stpdy
submitted for review by the Board meets the applicable requirements of §40 CFR 26, subparts
KandL.
64 of 85
-------
F(1). ProDosed Carroll-Love Picaridin Insect ReDellent Efficacy Studies (SPC-OO1 )
Charge to the Board
1. If the proposed research described in Protocol SPC-001 from Carroll-Loye Biological
Research is revised as suggested in EPA’s review, does the research appear likely to
generate scientifically reliable data, useful for assessing the efficacy of the test substances
for repelling mosquitoes?
Board Response
The active ingredient Picaridin in three formulations will be tested in the field for its
ability to repel mosquitoes by the Carroll-Loye company. The active ingredient will be tested
as 1) a 7% pump spray, with results extrapolated to a 10% pump spray, an aerosol and a 5.75%
towelette; 2) a 15% pump spray, with results extrapolated to a 15% aerosol and a 12%
towelette; and 3) a 15% lotion that will also contain a sunscreen. All experiments will be
conducted using GLP. A dosimetry experiment with 10 subjects, each applying all three test
formulations, will be performed to determine the typical amount of product that would be
utilized by consumers using the product as directed; a single dosimetry experiment will
provide this information for both SPC-001 and SPC-002. The grand mean dose of all subject
mean doses for each test formulation will be applied by technicians to the subjects in the
efficacy experiment. The experiment will be a field study. Two locations in California would
be used.
Legs and/or arms will be tested. There will be two experienced persons serving as
negative controls (i.e., without any repellent product) to confirm mosquito biting/landing
pressure. Experimental subjects, in pairs, will monitor LIBe’s during a one minute interval
each 15 minute, until the First Confirmed LIBe (FCLIBe) can be determined. Stopping rules
will be employed. The Complete Protection Time (CPT) will be determined.
With respect to the pertinent science criteria established earlier by the HSRB for completed
studies:
• The scientific question was stated (i.e., to test the efficacy of Picaridin in several
formulations to repel mosquitoes).
• Because existing data were not adequate to answer the question of efficacy, new studies
involving human subjects are necessary.
• The potential benefits of the study is to allow continued registration of the product.
• The risks are minimal because the active ingredient is of very low toxicity, the other
formulation ingredients are of very low toxicity, the mosquitoes will be aspirated
before they have an opportunity to bite, and the regions selected will not have evidence
of West Nile or encephalitis viruses.
• Deficiencies related to information about the lotion formulation have been addressed.
65 of 85
-------
Study Design Criteria
• The purpose of the study is clearly defined (i.e., efficacy testing).
• There are specific objectives (i.e., to determine the CPT that Picaridin in three
formulations displays as a mosquito repellent).
• The sample size will be 10 individuals per product along with 2 experienced
individuals to confirm mosquito biting pressure. A dosimetry experiment prior to the
field experiment will quantify the amount of repellent being used.
• The Board had some concern that with r espect to data interpretation, a stronger solution
would be used to estimate CPT for the lower concentration towelettes. However, it was
explained during the meeting that the concentration quoted for the towelette also
included the weight of the fabric, and the picaridin solution was identical to the spray.
Participation Criteria :
• The participants will be representative of a subset of the population of concern;
however, there are others in the population unlike these participants who are likely to
use these products, but it would either be unethical to test them or would be less
appropriate to test them. The participating population is considered appropriate and
reasonable.
Measurement Criteria
• There was concern about using data from the tested products to extrapolate for other
products. Extrapolating information from the 7-percent pump spray to a 10-percent
pump spray formulation is acceptable, however there are problems with extrapolating
from the 7-percent pump spray to the 5.75-percent towelette because the towelette may
not administer the same dose. On the other hand, if the dosimetry phase demonstrates
the towelette administers approximately the same dose, extrapolation would be
acceptable. Quality assurance will be a part of the experimental plan. If the products
will not be compared, randomization is less of an issue if the experiments can be
performed on separate days, however randomization and allocation of subjects to
treatments needs to be fully specified.
• There were also lack of sufficient information about assignment of subjects, raising
concerns that data generated would not be appropriately analyzed.
• The measurements will be accurate and reliable.
• The measurements will be appropriate to the question being asked.
• Quality assurance will be a part of the experimental plan.
Laboratory and Field Conditions
• Laboratory experiments are not proposed, except for the dosimetry.
• Field experiments will be appropriate.
• The study will include a stop rule plan, medical management plan, and a safety
monitor.
HSRB Consensus and Rationale
While the Board agreed that the study rationale, formulations to be tested and data
collection procedures were scientifically sound, the protocol did not adequately explain the
66 of 85
-------
relationship between the study design and analytic plan nor did it include an appropriate
statistical analysis plan (including estimation of variability) that could be evaluated for its
validity or utility. Thus, the Board was concerned that the proposed research may not appear
likely to generate scientifically reliable data, useful for assessing the efficacy of the test
substance for repelling mosquitoes.
Charge to the Board
b. If the proposed research described in Protocol SPC-001 from Carroll-Loye Biological
Research is revised as suggested in EPA’s review, does the research appear to meet the
applicable requirements of 40 CFR part 26, subparts K and L?
Board Response
Brief Overview of the Study
The research is to be conducted by Carroll-Loye Biological Research, a private
laboratory in Davis, California by using healthy volunteers and a controlled environment. The
study sponsor is Spectrum Brands, Inc. of Bridgeton, Missouri, a division of United Industries
Corporation. The study protocol was reviewed and approved by a commercial human subjects
review committee, IIRB of Plantation, Florida. Minutes of IIRB meetings were provided to the
EPA as a separate document, and documentation previously provided to the EPA by IIRB
indicates that it reviewed this protocol pursuant to the standards of the Common Rule (45 CFR
Part 46, Subpart A).
The research protocol submitted consisted of two interdependent studies: 1) a
dosimetry study designed to determine the amount of an insect-repelling compound, known as
picaridin, that normal subjects would typically apply when provided with one of three
compound formulations (7% Picaridin Pump Spray [ data will be bridged to 10% Pump Spray
and Aerosol and 5.75% Towelette],15% Picaridin Pump Spray [ data will be bridged to 15%
Aerosol and 12% Towelette], and 15% Picaridin Lotion formulated with sunscreen); and 2) an
efficacy study designed to measure the effectiveness of picaridin as a mosquito repellent.
Dosimetry will be determined either by passive patch dosimetry (spray formulations) or by
direct measurement of compound application (lotion formulation). The efficacy of each
formulation as a mosquito repellent will be determined by measuring the ability of the
formulations to prevent mosquito landings (defined as “Lite with Intent to Bite”; LIBe) under
field conditions. Mosquitoes will be aspirated mechanically prior to biting; prior to initiation of
the efficacy study, all volunteers will be trained, using laboratory-raised, pathogen-free
mosquitoes in a controlled laboratory setting, both to recognize mosquitoes landing with the
intent to bite and to remove such mosquitoes with an aspirator. During the field studies,
participants will work in pairs to facilitate identification and aspiration of LIBing mosquitoes
during brief exposure periods.
The dosimetry study will enroll a total of 10 subjects, each of whom will test all three
formulations. The efficacy study will enroll 10 subjects per test formulation per field site, for a
total of 30 subjects. Efficacy will be tested independently at two different sites, representing
two different environments. Two controls (described as “experienced personnel” and who
67 of 85
-------
were untreated with repellent) will also participate to determine ambient LIBe pressure at each
field site, giving a cumulative total of 28 volunteers. Subjects may participate in either or both
studies, making the total number of volunteers enrolled no less than 32 but no greater than 74.
In addition, three alternate subjects will be enrolled to: 1) replace any subject who withdraws
from participating; and 2) protect the confidentiality of any subject excluded from the study as
a result of pregnancy or other potentially stigmatizing condition.
Critique
The Board concurred with the factual observations of the strengths and weaknesses of
the study, as detailed in the EPA’s Science and Ethics Review (Carley and Sweeney, 2007a).
The proposed research described in Protocol SPC-001 meets the applicable requirements of 40
CFR Part 26, subparts K and L.
The risks to study participants are minimal and justified by the likely societal benefits,
including data on the efficacy of picaridin as a mosquito repellent. The risks to study
participants are three-fold: 1) reaction to test materials themselves; 2) exposure to biting
arthropods; and 3) possible exposure to arthropod-borne diseases.
All three repellent formulations are currently registered and are present at similar
concentrations in other EPA-registered products; specifically, picaridin is registered and
marketed as an insect repellent in the United States under the registered trade name Bayrepel ’
and the brand name Autan. As volunteers with known allergic reactions to insect repellents
and common cosmetics are excluded from participating in this study, enrolled participants are
unlikely to be at increased risk of experiencing adverse side effects upon exposure. Clear
stopping rules also have been developed, as have plans for the medical management of any
side effects or adverse events associated with product exposure.
Reactions to mosquito bites are usually mild and easily treated with over-the-counter
steroidal creams. The study also excluded individuals who have a history of severe skin
reactions to further minimize the risk of a participant experiencing a severe physical reaction to
a mosquito bite. In addition, the study protocol was designed specifically to minimize the
likelihood that a mosquito will bite. All volunteers will be trained both to recognize
mosquitoes landing with intent to bite and to remove such mosquitoes with an aspirator before
they bite. Risk of bites is further minimized through the use of clear stopping rules, limited
exposure periods, and paired observation.
To minimize the risk that study participants will be exposed to illnesses like WNV, the
study protocol calls for field tests of repellent efficacy to be conducted only in areas where
known vector-borne diseases have not been detected by county and state health or
vector/mosquito control agencies for at least one month. Mosquitoes will be collected during
the field studies and subjected to serologic or molecular analyses to confirm that they were free
of known pathogens.
Finally, the study protocol also includes several mechanisms designed to minimize
coercive recruitment and enrollment, compensation does not appear to be so high as to unduly
68 of 85
-------
influence participation, and minors and pregnant or lactating women are explicitly excluded
from volunteering (pregnancy being confirmed by requiring all female volunteers to undergo a
self-administered over-the-counter pregnancy test on the day of the study). In reviewing
similar protocols submitted by Carroll-Loye Biological Research at previous HSRB meetings,
for example, the Board has expressed concern about the potentially coercive nature of study
subject recruitment. Although the study is to be conducted by Carroll-Loye Biological
Research, a private research laboratory in Davis, California, the Principal Investigator of the
study and Co-Owner of the research laboratory, Dr. Scott P. Carroll, also is an adjunct faculty
member of the Department of Entomology at the University of California, Davis. The
majority of research participants will be recruited from the University’s student population,
including Dr. Carroll’s own department, but the current protocol includes several mechanisms,
including the exclusion of current students or employees of the Study Director, a substantial
waiting period between recruitment and study enrollment, and an interview by Dr. Carroll,
designed to minimize coercive subject recruitment and enrollment.
In accordance with the EPA’s final human studies rule, 40 CFR §26.1703, children and
pregnant or nursing women are explicitly excluded from participation. All female volunteers
are required to undergo a self-administered over-the-counter pregnancy test on the day of the
study. The use of “alternate” subjects is an appropriate safeguard to minimize the risks of
stigmatization resulting from a positive pregnancy test; that study participants may be
designated as alternate subjects and thereby excluded from participation allows for volunteers
found to be pregnant to withdraw without compromising their confidentiality.
HSRB Consensus and Rationale
The Board concurred with the assessment of the Agency that the revised protocol
submitted for review by the Board meets the applicable requirements of §40 CFR 26, subparts
KandL.
F(2). Proposed Carroll-Love Picaridin Insect Repellent Efficacy Studies (SPC-002 )
Charge to Board
1. If the proposed research described in Protocol SPC-002 from Carroll-Loye Biological
Research is revised as suggested in EPA’s review, does the research appear likely to
generate scientifically reliable data, useful for assessing the efficacy of the test substances
for repelling ticks?
Board Response
The active ingredient picaridin in three formulations will be tested in the laboratory for
its ability to repel ticks by the Carroll-Loye company. The active ingredient will be tested in 1)
a 7% pump spray, with results extrapolated to a 10% pump spray, an aerosol and a 5.75%
towelette; 2) a 15% pump spray, with results extrapolated to a 15% aerosol and a 12%
towelette; and 3) a 15% lotion that will also contain a sunscreen. These are the same
formulations as would be tested in protocol SPC-001 (to test repellency against mosquitoes).
69 of 85
-------
All experiments will be conducted using GLP. A dosimetry experiment with 10 subjects, each
applying all three test formulations, will be performed to determine the typical amount of
product that would be utilized by consumers using the product as directed; a single dosimetry
experiment will provide this information for both SPC-001 and SPC-002. The grand mean
dose of all subject mean doses for each test formulation will be applied by technicians to the
subjects in the efficacy experiment. Subjects will be trained in the procedures. One of each
subject’s arms will serve as a negative control to validate each experimental tick for its
questing behavior.
Ticks will be laboratory reared American dog ticks (Dermacentor variabilis) and deer
ticks (Ixodes scapularis). Because these are laboratory reared, the ticks are anticipated to be
disease-free.
This protocol is very similar to protocols on tick repellency submitted by this
investigator in the past, using the metric of repellency from the treated area on the arms of the
subjects during a 3 minute observation period during each 15 minutes of the test. The
Complete Protection Time from treatment until the First Confirmed Crossing will be
determined for each subject in the efficacy phase.
With respect to the pertinent science criteria established earlier by the HSRB for
completed studies:
• The scientific question was stated (i.e., to test the efficacy of picaridin in several
formulations to repel ticks).
• Because existing data were not adequate to answer the question of efficacy, new studies
involving human subjects are necessary.
• The potential benefits of the study are clear, i.e., that an effective repellent would be
available that would have either greater efficacy and/or fewer drawbacks than what was
currently approved.
• It is likely that the benefits would be realized because repellent efficacy will be
determined in carefully designed laboratory experiments.
• The risks are minimal because the active ingredient is of very low toxicity, the other
formulation ingredients are of very low toxicity, the ticks should be removed before
they have an opportunity to bite, and the laboratory reared ticks will not possess any
diseases.
Study Design Criteria
• The purpose of the study is clearly defined (i.e., efficacy testing for tick repellency).
• There are specific objectives (i.e., to determine the Complete Protection Time that
picaridin in three formulations displays as a tick repellent).
• The sample size will be 10 individuals per product. A dosimetry experiment prior to the
field experiment will quantify the amount of repellent being used.
• It is anticipated that the findings from this study can be generalized beyond the study
sample.
Participation Criteria
70 of 85
-------
• The participants will be representative of some of the population of concern, but are not
representative of the entire population, as acknowledged by the Investigator; however,
since it would either be unethical to test them or would be less appropriate to test them,
the participating population is considered appropriate and reasonable.
Measurement Criteria
• The measurements of tick crossing will be accurate and reliable.
• The measurements will be appropriate to the question being asked.
• Quality assurance will be a part of the experimental plan.
Laboratory and Field Conditions
• Laboratory experiments are proposed.
• Field experiments are not proposed.
The EPA science review did not identify any questions or concerns that needed to be
addressed by the HSRB. As was true for protocol SPC-001, the Board had some concern that,
with respect to data interpretation, a more concentrated product appeared to be proposed to
estimate CPT for the lower concentration towelettes. However, it was explained during the
meeting that the concentration quoted for the towelette also included the weight of the fabric,
and the picaridin solution was identical to the spray.
There were a number of concerns from a statistical perspective. The protocol states in
section 8.3.1 that “ [ s]ubjects will be assigned to the treatment groups on the basis of a
randomly assigned subject number,” 1-10 to Lotion, 11-20 to 7% Pump, and 21-30 15% Pump.
However, it also states in section 8.3.2 that “individual subjects may test more than one
repellent, on separate days.” These two statements are inconsistent as the former statement
implies 30 distinct and unique subjects, whereas the latter implies an overlap, which may
render the statistical interpretation of the data very difficult, depending on how the experiment
is carried out. If there is going to an overlap of subjects among test groups, an appropriate
experimental design needs be employed to allow proper statistical inference. It is also unclear
what is meant by the sentence “The experiment will be partially randomized by subjects” in -
section 8.2 on experimental design.
As for the method of analysis, the statement that “Kaplan-Meier analyses provide median
estimates with substantially reduced error estimates” is incorrect as it can go either way. One
cannot make a direct comparison between the median and its 95% confidence interval based on
the Kaplan-Meier estimate of survival function and the mean and the 95% confidence interval
based on the normal theory. Also the statement that the median based on Kaplan-Meier
method is “less sensitive to data censoring” is incorrect.
While the current EPA guidelines define complete Protection Time defined as the mean
time across all treated subjects from application of the repellent to the First Confirmed
Crossing, this cannot be estimated due to censoring; neither can the standard deviation or the
confidence interval be accurately calculated. The more appropriate statistic in the presence of
71 of 85
-------
censoring is the median and the confidence interval based on the Kaplan-Me ier estimate of the
survival distribution for the time to efficacy failure.
While the protocol did follow EPA guidelines, the Board disagreed with EPA’s assessment
on compliance with applicable scientific standards on the following items:
• Quantification of efficacy of the test materials is inappropriate as the normality
assumption is inadequate and due to potential censoring.
• Discussion of the statistical power is irrelevant as there is no statistical test of
hypotheses is involved.
• Justification of the sample size for the repellency phase is not scientific and thus
inadequate.
The Carroll-Loye protocol SPC-002 7/10/07 document contained the Site Questionnaire
(pp 63-66 of 70) and the Study Specific Instructions (p 67 of 70) for SPC-001, which must be
in error, as they refer to mosquitoes, instead of ticks. There appears to be discrepancies in the
Material Safety Data Sheets (MSDS) in the above document: On pages 59 and 61 the EPA
registration number is identical (12 1-92), but product item numbers are different (53667 on p
59 and 53661 on p 61) for the identical formulation. There did not seem to be a MSDS
included for the formulation with sunscreen.
The LRB is universally understood as an acronym for “Institutional Review Board.”
Therefore, reference to “Independent Investigational Review Board” as “Independent IRB” is
inappropriate and misleading.
HSRB Consensus and Rationale
The Board concluded that the research appears likely to generate scientifically reliable
data, useful for assessing the efficacy of the test substances for repelling ticks, provided that
the revisions suggested by EPA are incorporated, the experimental design is made more
specific to the allocation of the test substances into three groups of subjects and there is no
overlap of subjects from one test group to the other.
The Board urged EPA to consider the design of newer studies and the designs already used
for existing products to make certain that labels reflect information of comparative value to
consumers. If there are currently inconsistencies in the information used to authorize label
information on efficacy, the Board urges EPA to develop new guidelines or revisions to the
current guidelines that will balance accuracy of data collected, safety to human subjects and
consistency in the ultimate labels produced.
Charge to Board
2. If the proposed research described in Protocol SPC-002 from Carroll-Loye Biological
Research is revised as suggested in EPA’s review, does the research appear to meet the
applicable requirements of 40 CFR part 26, subparts K and L?
Board Response
72 of 85
-------
Brief Overview of the Study
The research is to be conducted by Carroll-Loye Biological Research, a private
laboratory in Davis, California by using healthy volunteers and a controlled environment. The
study sponsor is Spectrum Brands, Inc. of Bridgeton, Missouri, a division of United Industries
Corporation. The study protocol was reviewed and approved by a commercial human subjects
review committee, Independent Investigational Review Board, Inc. (ILRB) of Plantation,
Florida. Minutes of IIRB meetings were provided to the EPA as a separate document, and
documentation previously provided to the EPA by IIRB indicates that it reviewed this protocol
pursuant to the standards of the Common Rule (45 CFR Part 46, Subpart A).
The research protocol submitted consists of two interdependent studies: 1) a dosimetry
study designed to determine the amount of an insect-repelling compound, known as picaridin,
that normal subjects would typically apply when provided with one of three compound
formulations (7% Picaridin Pump Spray [ data will be bridged to 10% Pump Spray and Aerosol
and 5.75% Towelette],15% Picaridin Pump Spray [ data will be bridged to 15% Aerosol and
12% Towelette], and 15% Picaridin Lotion formulated with sunscreen); and 2) an efficacy
study designed to measure the effectiveness of picaridin as a tick repellent. Dosimetry will be
determined either by passive patch dosimetry (spray formulations) or by direct measurement of
compound application (lotion formulation). The efficacy of picaridin as a tick repellent will be
determined by placing laboratory reared American dog ticks (Dermacentor variabilis) and deer
ticks (Ixodes scapularis) on picaridin-treated and untreated forearms and that moving insects
would migrate into the treated area.
The dosimetry study will enroll a total of 10 subjects, each of whom will test all three
formulations. The efficacy study will enroll 10 subjects per test formulation. Each subject will
serve as his/her own control. Participants in the dosimetry study may or may not participate in
the efficacy study, and participants in the efficacy study may also test different formulations on
different days; this makes the total number of volunteers enrolled in both the dosimetry and
efficacy studies no less than 10 but no greater than 40. In addition, three alternate subjects will
be enrolled to: 1) replace any subject who withdraws from participating; and 2) protect the
confidentiality of any subject excluded from the study as a result of pregnancy or other
potentially stigmatizing condition.
Critique
The Board concurred with the factual observations of the strengths and weaknesses of
the study, as detailed in the EPA’s Science and Ethics Review (Carley and Sweeney, 2007b).
Once the recommended changes outlined therein are incorporated into the protocol, the
proposed research described in Protocol SPC-002 should meet the applicable requirements of
40 CFR Part 26, subparts K and L.
The risks to study participants are minimal and justified by the likely societal benefits,
including data on the efficacy of these picaridin-based formulations as a tick repellent. The
73 of 85
-------
risks to study participants are three-fold: 1) reaction to test materials themselves; 2) exposure
to biting arthropods; and 3) possible exposure to arthropod-borne diseases.
All three repellent formulations are currently registered and are present at similar
concentrations in other EPA-registered products; specifically, picaridin is registered and
marketed as an insect repellent in the United States under the registered trade name Bayrepel ’
and the brand name Autan. As volunteers with known allergic reactions to insect repellents
and common cosmetics are excluded from participating in this study, enrolled participants are
unlikely to be at increased risk of experiencing adverse side effects upon exposure. Clear
stopping rules also have been developed, as have plans for the medical management of any
side effects or adverse events associated with product exposure.
The risks of bites are negligible and minimized by the study design; tick questing and
biting behavior is slow, and study participants are trained to remove ticks from their forearms
prior to biting.
The ticks used for the study are bred and raised in a laboratory environment and are
considered to be pathogen-free, minimizing the risk of vector-borne disease. The Agency
noted in their review (2007b) that the tick-borne rickettsial illness Rocky Mountain Spotted
Fever (RMSF) has been passed within tick colonies through a transovarian transmission
mechanism, so the laboratory colonies in question should either be tested for RMSF prior to
initiation of SPC-002, or the consent form and protocol altered accordingly to reflect the low
but not negligible risk posed to trial participants.
The study protocol also includes several mechanisms, similar to those described above
in the Board’s review of Study Protocol SPC-OO1, designed to minimize coercive recruitment
and enrollment, compensation does not appear to be so high as to unduly influence
participation, children and pregnant or lactating women are explicitly excluded from
volunteering (all female volunteers are required to undergo a self-administered over-the-
counter pregnancy test on the day of the study), and the use of so-called “alternate” subjects
allows for volunteers to withdraw or be excluded without compromising their confidentiality.
The Board’s only concern about subject recruitment was the unjustified exclusion of study
participants older than 55 years of age; most Board members believed that a clear rationale for
excluding older volunteers should be provided, or the exclusion criteria changed.
HSRB Consensus and Rationale
The Board concurred with the assessment of the Agency that the revised protocol
submitted for review by the Board meets the applicable requirements of §40CFR26, subparts K
and L.
G. Proposed ICR Picaridin Insect Repellent Efficacy Study (A 117 )
Charge to the Board
74 of 85
-------
I. If the proposed research described in ICR’ s proposed picaridin protocol is revised as
suggested in EPA’s review, does the research appear likely to generate scientifically
reliable data, useful for assessing the efficacy of the test substances for repelling
mosquitoes of the genus Culex?
Board Response
This protocol presented by ICR was designed to determine in laboratory studies
whether two picaridin-containing products will repel mosquitoes of the genus Culex. These
products are already registered and the study, if showing efficacy of the products in repelling
Culex, would allow claims for repellency of this genus to be added to the label. This is
important because Culex is one of the major carriers of WNV, and efficacy against this genus
would be of interest to consumers. The particular products to be tested are Avon Skin-So-Soft
SSS Bug Guard Plus Picaridin Insect Repellent and Avon Skin-So-Soft SSS Bug Guard Plus
Picaridin Insect Repellent spray. The hypothesis was that 8 hours of protection against Culex
quinquefasciatus, a species known to be a vector for WNV, would be demonstrated by the two
products.
The protocol was clearly written. Laboratory reared insects that were disease free
would be used. The protocol proposed the use of the guideline amount of repellent, not an
amount determined by a dosimetry study, and would monitor bites, not landings, in order to be
consistent with the methods used to obtain the label currently borne by these products. The
negative control would use only landings, avoiding bites as much as possible, and would be
used to guarantee that a landing rate of at least 5 in 60 seconds was maintained; if the landing
rate dropped below this, more mosquitoes would be added to the cages.
This protocol appeared to follow guideline procedures, and data from a study such as
this would be necessary to modify label claims to include efficacy against Culex, particularly if
Culex was not represented or was not represented well in the original field studies, which
presumably was the case. The procedures seem to be straightforward. Pre-testing the subjects
for attractiveness to Culex should yield more consistent results than if such pretesting was not
done.
There were a few points of confusion. The reference to dose range finding (pg. 27) was
not clear, particularly when the standard guideline dose level was proposed. It is unclear
whether confounding from the subject’s other arm containing a different product, or the
presence of the arms of the second subject in the cage would occur. However both subjects
would use the same two products and both products would contain the same active ingredient,
i.e., picaridin, so there is probably little, if any, confounding likely. There was a question as to
whether the laboratory temperature and humidity would be similar to’ that occurring in the
field. There was also a question of how, with an 8 hour limit to the test, an average of 8 hours
efficacy with a 2 hour deviation might be obtained. It was suggested that a Q test was not the
most appropriate statistical method, and that the Kaplan Meier test might be the better
approach.
75 of 85
-------
Two other aspects of this protocol differ with what the Board had previously concluded
regarding other insect repellency protocols, and these deserve specific discussion. One is the
use of the standard guideline dosage instead of a dosimetry test to determine the dosage to be
used. While in a totally new study, the dosimetry test would yield a dosage more
representative of the amount of product that the consumer would use, in this situation, this
study is supposed to match a previously conducted field study with the same products. It is
therefore appropriate to use the same dosage as was used in the earlier field study. Likewise,
the previous field study used bites. While the Board has been impressed with the added safety
to participants imparted by the landing endpoint over the bite endpoint, these data would need
to match the data obtained in the previous field study, which used bites as the endpoint. In
addition, the consultants indicated that landings did not necessarily predict bites, leading to
even more support for the use of bites in order to match the earlier study with bites.
HSRB Consensus and Rationale
The Board concluded that this study would be appropriate to confirm efficacy against
Culex, was designed to meet the current EPA guidelines, and was generally clear and
adequately designed, with the exception of the appropriate statistics. If revised consistent with
EPA’s recommendations and the Board’s suggestions, the study should yield valid data
regarding the efficacy of these products in repelling Culex.
Charge to Board
2. If the proposed research described in ICR’s proposed picaridin protocol is revised as
suggested in EPA’s review, does the research appear to meet the applicable requirements of
40 CFR part 26, subparts K and L?
Board Response
Brief Overview of the Study
The proposed study (Spero, 2007) would evaluate the efficacy of two different skin-
applied lotion formulations of picaridin-based insect repellents under laboratory conditions;
these two formulations are Avon Skin-So-Soft SSS Bug Guard Plus Picaridin Insect Repellent
(EPA Reg. No. 806-29) and Avon Skin-So-Soft SSS Bug Guard Plus Picaridin Insect
Repellent Spray (EPA Reg. No. 806-31.
The research is to be conducted by ICR, a commercial organization based in Baltimore,
Maryland; ICR provides testing and regulatory consulting services for companies developing
and marketing pesticides in the United States and Canada. The study is managed by toXcel,
LLC of Gainesville, Virginia. The sponsor of this study is Avon Products, Inc. of New York,
New York. The submitted documents assert that the study will be conducted in accordance
with the ethical and regulatory standards of 40 CFR 26, Subparts K and L, as well as the
requirements of FIFRA §12(a)(2)(P), and the U.S. EPA’s GLP Standards described at 40 CFR
160. The protocol was reviewed and approved by an independent human subjects review
76 of 85
-------
committee, Essex Investigational Review Board (EIRB), Inc., of Lebanon, New Jersey prior to
submission to the Agency.
Efficacy of the two picaridin-based formulations will be evaluated by using healthy
volunteers. The study will be performed at ICR’s laboratory in Baltimore, Maryland, and test
the effectiveness of the two compounds as mosquito repellents by measuring the ability of each
formulation to prevent mosquito bites under laboratory conditions.
The effectiveness study will enroll a total of 13 subjects. Of the 13 participants, twelve
will be treated and test the effectiveness of the two picaridin-based repellent formulations. The
compounds will be applied to 250 cm 2 bands of skin on the forearms of each study participant;
one compound will be applied to the right forearm and one to the left forearm, with the
effectiveness of each formulation simultaneously evaluated. Treated skin will be exposed to
mosquitoes for five minutes at half-hour intervals by having study participants insert their
forearms into 8 ft. 3 test “cages” containing 100 laboratory-reared, pathogen-free female C.
quinquefasciatus. The efficacy of each repellent formulation will be ascertained by measuring
the time from application to “breakdown” of repellency, with “breakdown” defined in the
protocol as an initial bite confirmed by either a second bite within the same five-minute
exposure period, or by a second bite in the next consecutive exposure period. Treated study
participants will work in pairs, observing mosquito landings and alerting attendant ICR staff of
potential bites; ICR staff will determine whether or not these are indeed bites. Probes (i.e.,
“bites” where the mosquito punctures the skin but does not collect blood) and bites from
mosquitoes that do not fully alight (i.e., all six legs) on the surface of the treated skin will not
be counted as bites. The cycle of intermittent exposures will continue for each subject until
breakdown is reached for both arms. Once the repellent on one arm has reached breakdown
for a particular subject, no further exposure of that arm will occur. Exposures of the subject’s
other arm will continue until breakdown of the other repellent. The study protocol justifies the
enrollment of twelve treated participants by stating that ten volunteers are needed to obtain
statistical validity; an additional two participants will be enrolled as alternates to “allow for
drop outs” (Spero and Gaynor, 2007).
One study participant, chosen by lottery (a “coin toss”), will remain untreated and will
be monitored to determine mosquito-biting pressure under laboratory conditions. A 250-cm 2
band of untreated skin will be exposed in each test cage at half-hour intervals, with the ambient
biting pressure determined by measuring the number of mosquitoes landing on the skin. A
minimum rate of 5 landings within one minute is necessary for the laboratory trial to be
conducted or continued. Landing mosquitoes will be “shaken” away by the study participant
while ICR staff counts the number of landings. If the untreated control subject experiences
fewer than 5 landings in one minute, fresh mosquitoes will be added to the cage.
Critique
The supporting documentation provided by the study investigators, sponsor and EIRB,
as submitted to the Agency, appears to meet the regulatory requirements of 40 CFR 26.1115a
and 40 CFR 26.1125. A description of EIRB procedures was provided to the EPA with a claim
of confidentiality, so it was not available for review by the HSRB. Agency staff, however,
77 of 85
-------
reviewed the documentation provided by EIRB and determined these procedures and policies
to be in compliance with the applicable standards (40 CFR 26, Subpart K). The protocol as
submitted to the Agency thus is substantially compliant with the regulatory requirements of
review and documentation, minor deficiencies notwithstanding.
The Board concurred with the factual observations of the strengths and weaknesses of
the study, as detailed in the EPA’s Science and Ethics Review (Carley and Sweeney, 2007c).
Specifically, the Board agreed with the Agency’s recommendations that ICR: 1) revise the
current protocol discussion of risks and benefits; 2) provide more information about ICR’s
subject recruitment and enrollment processes; and, 3) alter the data collection form to remove
all identifiable information. Most Board members disagreed, however, with the EPA’s
recommendation that study investigators provide a clearer justification for relying on time to
FCB as a measure of repellent efficacy. Previous field studies using time to FCB as an
endpoint for these particular repellents efficacy have already be submitted to and approved by
the Agency. These additional laboratory studies are to be conducted solely for the purpose of
justifying additional label claims as per Agency requirements. For consistency, use of time to
FCB, using pathogen-free colonies of mosquitoes under controlled conditions, is justified.
Once these changes are incorporated into the protocol, the proposed research described
in ICR Al 17 should meet the applicable requirements of 40 CFR 26, subparts K and L. In
brief, the risks to study participants are minimal and justified by the likely societal benefits,
including data on the efficacy of these picaridin-based formulations in repelling one of the key
mosquito genera known to transmit WNV in the United States. The potential risks to study
participants are three-fold: 1) reaction to test materials themselves; 2) exposure to biting
arthropods; and, 3) exposure to arthropod-borne diseases.
These two picaridin-based repellent formulations are commercially available in the
United States, and their non-repellent ingredients have been used widely in cosmetic and
personal care products with little evidence of toxic effects. Volunteers with known allergic
reactions to insect repellents and common cosmetics are excluded from participating in this
study, and the amount of skin treated with picaridin is limited, so enrolled participants are
unlikely to be at increased risk of experiencing adverse side effects upon exposure to the test
materials. Clear stopping rules also have been developed, as have plans for the medical
management of any side effects or adverse events associated with product exposure. The
Board recommends, however, that the informed consent documents be modified to more
accurately represent the known toxic risks associated with acute picaridin exposure,
particularly via ocular, oral and respiratory routes of exposure.
The endpoint of the study protocol requires a confirmed bite—i.e., an initial bite
confirmed by another bite within 30 minutes—to document breakdown of repellent
effectiveness. Since two repellents will be tested concurrently, each subject can expect to
receive at least 4 bites. Reactions to mosquito bites are usually mild and easily treated with
over-the-counter steroidal creams. Such a cream, in addition to CalomineTht and rubbing
alcohol, will be provided to study participants to alleviate minor symptoms associated
mosquito bites. The study excludes individuals who have a history of severe skin reactions to
78 of 85
-------
further minimize the risk of a participant experiencing a severe physical reaction to a mosquito
bite.
The mosquitoes used for the study are bred and raised in a laboratory environment and
are considered to be pathogen-free, minimizing the risk of vector-borne disease. In the
Agency’s Science and Ethics review of the protocol (Carley and Sweeney, 2007b), a concern
was raised that the risk of arthropod-borne illnesses is not adequately discussed in the informed
consent document; the Agency recommended that a discussion of these risks be added. The
Board agreed.
The differential risks to the untreated control subject (chosen by lottery) are listed in
both the protocol and the informed consent document. The Board noted, however, a
misleading statement in the consent document, which stated that mosquitoes will be “brushed
[ off] by ICR staff” from the untreated control subject’s arm prior to biting. The protocol stated
that the untreated control subject will “shake” landing mosquitoes off. This inconsistency must
be corrected.
Furthermore, it is clear from the protocol that all study volunteers, be they untreated
controls or treated volunteers, will be asked to expose both their arms, before they are treated,
to the mosquito colony to test “attractiveness.” The attractiveness component of the study is
not mentioned in the informed consent document, and must be added. The informed consent
document also lists one of the societal benefits of the study as bringing a “new repellent to
market.” The study as described, however, is designed to simply allow a labeling change. The
repellent formulations under evaluation are already on the market.
A more detailed explanation of study recruitment is also needed; the exact procedures
for recruiting study participants are unclear. As currently written, however, compensation for
study participation is not so high as to unduly influence enrollment, and employees and
contractors—or family members of employees or contractors—of ICR, toXcel and the sponsor
are excluded from participation. In accordance with EPA’s human studies rule (40 CFR
§26.1703), children and pregnant or nursing women are explicitly excluded from participation.
All female volunteers are required to undergo a self-administered over-the-counter pregnancy
test on the day of the field study. The use of two potential “alternate” subjects allows for
volunteers to withdraw or be excluded without compromising their confidentiality.
Finally, the Board was concerned that an overlooked risk of study participation might
be the strain associated with the physical requirements imposed upon volunteers. Each study
participant will be asked to spend a full day in a warm, humid laboratory environment.
Although the likelihood of an adverse event precipitated by the physical strains of study
participation is small, these risks nevertheless should be mentioned in the informed consent
document, and a clear plan for medical monitoring and treatment should be articulated.
HSRB Consensus and Rationale
79 of 85
-------
The Board concurred with the assessment of the Agency that the protocol ICR Al 17
submitted for review by the Board, if revised as suggested in both EPA’s review and by the
Board, would meet the applicable requirements of 40 CFR 26, subparts K and L.
80 of 85
-------
REFERENCES
Carley, J.M. 2007. Ethics Review of Reports of Completed Carroll-Loye Field Efficacy
Studies SCI-001 and WPC-001 for Mosquito Repellents. Dated September 26, 2007.
Unpublished document prepared by Office of Pesticide Programs, United States Environmental
Protection Agency.
Carley, J.M., and K. Sweeney. 2007a. Science and Ethics Review of Protocol for Human Study
of Mosquito Repellent Performance. Dated September 24, 2007. Unpublished document
prepared by Office of Pesticide Programs, United States Environmental Protection Agency.
Carley, J.M., and K. Sweeney. 2007b. Science and Ethics Review of Protocol for Human
Study of Tick Repellent Performance. Dated September 24, 2007. Unpublished document
prepared by Office of Pesticide Programs, United States Environmental Protection Agency.
Carley, J.M., and K. Sweeney. 2007c. Science and Ethics Review of Protocol for Human Study
of Mosquito Repellent Performance. Dated September 24, 2007. Unpublished document
prepared by Office of Pesticide Programs, United States Environmental Protection Agency.
Carroll, S. 2007a. Test of DermAegis Lip0DEET 302 Personal Insect Repellent: EPA Reg.
#82810-1. Date unknown. Unpublished study prepared by Carroll-Loye Biological Research
under Project No. SCI-00 1.1.
Carroll, S. 2007b. Test of DermAegis LipoDEET 3434 Personal Insect Repellent. Date
unknown. Unpublished study prepared by Carroll-Loye Biological Research under Project No.
SCI-001 .2.
Carroll, S. 2007c. Test of Coulston’s Duranon Personal Insect Repellent (EPA Reg. #50404-8).
Date unknown. Unpublished study prepared by Carroll-Loye Biological Research under
Project No. SCI-001 .3.
Carroll, S. 2007d. Response to ‘Ethics Review of Reports of Completed Carroll-Loye Field
Efficacy Studies SCI-001 and WPC-001 for Mosquito Repellents.’ Dated October 19, 2007.
Unpublished document prepared by Carroll-Loye Biological Research.
Carroll, S. 2007e. Test of an Oil Of Lemon Eucalyptus-Based Personal Insect Repellent (EPA
Reg. # 305-62). Date unknown. Unpublished document prepared by Carroll-Loye Biological
Research.
Carroll, S. 2007f. Efficacy Test Protocol SPC-001: Efficacy Test of Picaridin-Based Personal
Insect Repellents with Mosquitoes Under Field Conditions. Dates July 13, 2007. Unpublished
protocol prepared by Carroll-Loye Biological Research.
81 of 85
-------
Carroll, S. 2007g. Efficacy Test Protocol SPC-002 Efficacy Test of Picaridin-Based Personal
Tick Repellents. Dates July 10, 2007. Unpublished protocol prepared by Carroll-Loye
Biological Research.
EPA HSRB. 2007a. Draft Final HSRB Report on SCI-001 proposed studies. Washington, DC:
Environmental Protection Agency.
EPA HSRB. 2007b. Draft Final HSRB Report on WCI-001 proposed studies. Washington,
DC: Environmental Protection Agency.
Gammon, MD, Neugut, Al, Santella, RM, et al. 2002. The Long Island Breast Cancer Study
Project: Description of a Multi-Institutional Collaboration to Identify Environmental Risk
Factors for Breast Cancer. Breast Cancer Res Treat 74:235-54.
Mitchell, RK, Agle, BR, Wood, DJ. 1997. Toward a Theory of Stakeholder Identification and
Salience: Defining the Principle of Who and What Really Counts. Academy of Management
Review 22(4):853-886.
Rogers, EM. 1995a. Diffusion of Innovations, 4 th ed. New York: Free Press, 1995.
Rogers, EM. 1 995b. Lessons for Guidelines from the Diffusion of Innovations. Joint Comm J
Qua! Improv 21:324-8.
Spero, N., and W. Gaynor. 2007. Protocol for Evaluating the Efficacy of Personal Repellents
Against Mosquitoes in the Laboratory Including Supporting Materials Satisfying 40 CFR
§26.1125 for Avon Skin-So-Soft SSS Bug Guard Plus Picaridin Insect Repellent (EPA Reg.
No. 806-29) and Avon Skin-So-Soft SSS Bug Guard Plus Picaridin Insect Repellent Spray
(EPA Reg. No. 806-3 1). Dated August 8, 2007. Unpublished document prepared by Insect
Control & Research, Inc.
Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, et al. (2007)
Strengthening the Reporting of Observational Studies in Epidemiology (STROBE):
Explanation and Elaboration. PLoS Med 4(10): e297 doi:10.l371 / iournal . pmed.0040297
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, et al. (2007) The Strengthening
the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for
Reporting Observational Studies. PLoS Med 4(10): e296 doi:10.137 1/journal.pmed.0040296
82 of 85
-------
APPENDIX A: DISCUSSION QUESTIONS FOR MOSQUITO REPELLENT STUDIES
The Human Studies Review Board (HSRB or Board) has discussed and provided advice
to EPA on scientific and ethical issues related to the conduct of field studies to evaluate the
efficacy of mosquito repellent products. The HSRB has reviewed both proposals for new field
studies and the results of completed studies. The HSRB has noted that, although there are
many similarities across studies, not all studies employ the same study design. The HSRB has
identified several methodological issues for which additional background information would
assist the Board in its evaluation of such studies.
BACKGROUND
Currently, EPA requires all pesticide products that claim to repel mosquitoes to provide data
on the duration of efficacy under field conditions at two biologically distinct sites. These
data are derived from human research with subjects who have been treated with the repellent
formulations in the field. The Agency evaluates the duration of repellent efficacy for a
subject by calculating the time from application of the repellent to the occurrence of an event
indicating an efficacy failure. Historically, for field studies of mosquito repellency, EPA has
used the “first confirmed bite” as an indication of efficacy failure on a test subject. Several
recent studies have shifted to the “first confirmed landing with intent to bite;” EPA has
accepted this alternative endpoint. A “confirmed landing” on a test subject is a mosquito
landing followed by a second landing on the same subject within a specified period of time
(usually 30 minutes) after the initial landing.
Field studies typically involve 6— 10 subjects who have been treated with a defmed amount
of the test material. Each subject is then regularly and repeatedly exposed to ambient
mosquito populations for a fixed interval of time until the subject experiences an efficacy
failure followed by a confirmation with the specified period of time. Mosquito landing
pressure (representing intent to bite) at a site is monitored by concurrently exposing untreated
subjects to mosquito landings. A study is considered valid only if there are at least a
specified minimum number of mosquito landings on untreated subjects during each exposure
interval.
On October 25, 2007, the HSRB will discuss scientific aspects of the design of field studies
to assess the efficacy of mosquito repellents. For this meeting the Board has requested
consultants to provide specialized information or assistance to the Board. The Board is
particularly interested in the frequency, duration and timing of exposure of subjects to
potential mosquito landings. The Board requests each consultant to respond briefly to the
series of questions below. Please send the responses to the HSRB Chair and Designated
Federal Official (DFO) at least one week before the meeting—i.e., by no later than October
18. All responses will subsequently be provided to the other consultants, the HSRB
members, and EPA staff for their review, and will be posted on www.regulations.gov under
docket ID number, EPA-HQ-ORD-2007-0942. HSRB consultants will be available at the
83 of 85
-------
meeting to discuss their responses and address questions from the Board. The questions for
Board consultant consideration are provided below:
DISCUSSION QUESTIONS
• What do data show about the variability of the time intervals between first and
subsequent landings in mosquito repellent field trials?
• What is the current scientific understanding of how factors other than repellent efficacy
could affect the likelihood that an initial event—a mosquito landing or mosquito bite—
would be “confirmed” by another similar event within 30 minutes? Please address at least
these factors:
o Characteristics of mosquito populations
o Characteristics of test sites
o Characteristics of test subjects
o Characteristics of test methods
• Can the impact of such factors on the likelihood or timing of an initial and confirming
event be predicted? Can it be quantified?
At its June 27 - 29, 2007 meeting the Board learned that different designs with different
“length-biased” sampling for mosquito repellent field studies are in use. One design exposes
subjects to potential mosquito landings for one minute of every 15 minutes; another design
exposes subjects to potential mosquito landings for five minutes of every 30 minutes. The
DFO is separately providing a CD containing the background materials for the June 27 — 29,
2007 HSRB meeting. The protocols are loaded on the CD. These designs have different
“length-biased” sampling.
• What is the methodological rationale for the two different designs?
• Which design is used more widely in the field? Why?
• Can potential effects of variation in the pattern of intermittent exposure on the results of
efficacy testing be isolated from the effects of other variables? If so, can the direction or
magnitude of the effects be predicted? How might these influences be analyzed and
accounted for in collecting, reporting and analyzing repellent efficacy data?
Dr. Matt Kramer, a USDA statistician who has served as a consultant, has suggested that the
precision of estimates of Complete Protection Time (CPT) in repellent testing could be
significantly increased by defining a failure of efficacy as the mean time from treatment to a
series of several landings or bites. He has stated:
The precision of CPT increases when it is estimated beyond time to [ First Confirmed
Bite] FCB or FCLanding. How well CPT can be estimated depends on the distribution of
84 of 85
-------
so many bites beyond FCB. The number of mosquitoes that will bite (n) will determine
results of the test. Each person in the field should be his/her own control; that way it is
possible to know n per person, and reduce person-to-person variability.
If using the mean time to the first 5 bites, the SE will decrease proportionally as n
increases (n = 5 in this case). That is equivalent to an increase in the power of the test of
5 times. This method allows for detecting formulation differences near the CPT.
• Does this approach, indeed, increase the precision of estimates of CPT markedly
without requiring additional subjects?
• If so, would this increased precision justify the incremental risk to the subjects
resulting from their exposure to a great?
• Is it practical to test long-lasting repellents to the point of five landings?
85 of 85
------- |