March 30, 2016
EPA-HSRB-16-1
Thomas A. Burke, Ph.D., MPH
EPA Science Advisor
Office of the Science Advisor
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Subject: January 12-13, 2016 EPA Human Studies Review Board Meeting Report
Dear Dr. Burke,
The United States Environmental Protection Agency (EPA) requested that the Human Studies
Review Board (HSRB) provide scientific and ethics reviews of several items, which were
reviewed at a meeting of the Board on January 12-13, 2016.
The first item reviewed was a published study entitled Assessing Intermittent Pesticide Exposure
From Flea Control Collars Containing the Organophosphorus Insecticide Tetrachlorvinphos
(TCVP), by M. Keith Davis, J. Scott Boone, John E. Moran, John W. Tyler and Janice E.
Chambers. Journal of Exposure Science and Environmental Epidemiology (2008) 18, 564-570.
The HSRB was requested to review the scientific and ethical aspects of this published study
because of the EPA's interest in relying on some of the data in this publication for agency
decision-making.
The other items under review consisted of a series of five completed studies conducted under a
common protocol design for field testing of skin-applied mosquito repellent products to support
the use of the EPA Insect Repellency Awareness Graphic on product labels for these products.
1 | Page

-------
The HSRB reviewed the scientific and ethical aspects of these five studies, which were
conducted by the SC Johnson Company and reported to the Agency in five separate reports.
The Board's key responses to the charge questions are detailed in the enclosed final report of the
meeting.
Signed,
-—
Liza Dawson, PhD
Chair
EPA Human Studies Review Board
2 | Page

-------
INTRODUCTION
On January 12-13, 2016, the United States Environmental Protection Agency's (EPA or Agency)
Human Studies Review Board (HSRB or Board) met to address the scientific and ethical charge
questions related to several items:
•	A published study entitled Assessing Intermittent Pesticide Exposure From Flea Control
Collars Containing the Organophosphorus Insecticide Tetrachlorvinphos (TCVP), by M.
Keith Davis, J. Scott Boone, John E. Moran, John W. Tyler and Janice E. Chambers.
Journal of Exposure Science and Environmental Epidemiology (2008) 18, 564-570.
•	Field Testing of S.C. Johnson Personal Mosquito Repellent Products to Support Their
Use of the EPA Repellency Awareness Graphic, GLP Study Number 865E1, J. Palm,
September 24, 2015. Test Substance: MARK-3 OFF! Deep Woods Sportsmen Insect
Repellent I (Maximum Strength Pump Spray Deep Woods OFF! EPA Reg. No. 4822-
276)
•	Field Testing of S.C. Johnson Personal Mosquito Repellent Products to Support Their
Use of the EPA Repellency Awareness Graphic, GLP Study Number 873E1, C. Talbert,
October 21, 2015. Test Substance: MARK-8 OFF! Deep Woods Insect Repellent V
(OFF! Insect Repellent Formula, EPA Reg. No. 4822-167)
•	Field Testing of S.C. Johnson Personal Mosquito Repellent Products to Support Their
Use of the EPA Repellency Awareness Graphic, GLP Study Number 866E1, E. Laznicka,
October 21, 2015. Test Substance: MARK-4 OFF! Active Insect Repellent I (Unscented
OFF! Insect Repellent, EPA Reg. No. 4822-380)
•	Field Testing of S.C. Johnson Personal Mosquito Repellent Products to Support Their
Use of the EPA Repellency Awareness Graphic, GLP Study Number 864E1, J. Palm,
3 | P a g e

-------
September 24, 2015. Test Substance: MARK-2 OFF! Deep Woods Sportsmen Insect
Repellent II (UNSCENTED DEEP WOODS OFF! EPA Reg. No. 4822-397)
• Field Testing of S.C. Johnson Personal Mosquito Repellent Products to Support Their
Use of the EPA Repellency Awareness Graphic, GLP Study Number 867E1, E. Laznicka,
October 21, 2015. Test Substance: MARK-5 OFF! Family Care Insect Repellent IV
(Unscented) (UNSCENTED OFF! SKINTASTIC SPRAY INSECT REPELLENT, EPA
Reg. No. 4822-395)
REVIEW PROCESS
The Board conducted a public meeting on January 12-13, 2016. Advance notice of the meeting
was published in the Federal Register as "Human Studies Review Board; Notification of a
Public Meeting" (EPA-HQ-ORD-2015-0588).
This Final Report of the meeting describes the HSRB's discussion, recommendations, rationale
and consensus in response to each charge question for each of these items.
For each agenda item, Agency staff first presented their review of the science and the Board
asked the Agency presenters clarifying questions. The staff then described their review of the
ethical aspects and the Board asked clarifying questions with regard to the ethical review. The
HSRB solicited public comments and next proceeded to address the charge questions, first
discussing scientific review and then ethical review, for each study. The Chair called for a vote
to confirm concurrence on a summary statement in response to each charge question.
For their evaluation and discussion, the Board considered materials presented at the meeting,
study reports, related materials and documents provided by the study sponsors, the Agency's
science and ethics reviews of the studies, as well as oral responses from the study sponsor and
protocol teams at the HSRB meeting. A comprehensive list of background documents is
available online at http://www.epa.gov/osa/human-studies-review-board.
4 | Page

-------
The HSRB review of each of the six reported studies (one published study and five field testing
studies related to the Repellency Awareness Graphic) is presented below. This report presents
the board's finding for each study in the order in which the studies were reviewed at the meeting
Because some of the same scientific concerns were raised for several studies over the course of
the deliberations, the report makes note of the common issues raised but does not duplicate the
discussion of issues when identical concerns were raised several times.
5 | P a g e

-------
Assessing Intermittent Pesticide Exposure From Flea Control Collars Containing the
Organophosphorus Insecticide Tetrachlorvinphos (TCVP), by M. Keith Davis, J. Scott
Boone, John E. Moran, John W. Tyler and Janice E. Chambers. Journal of Exposure
Science and Environmental Epidemiology (2008) 18, 564-570.
Science Review
Charge to the Board
Is this research scientifically sound, providing reliable pet fur transferable residue data for use in
evaluating potential exposures of adults and children from contact with pets treated with
tetrachlorvinphos containing pet collars?
Board Response
The research is scientifically sound and, if used appropriately, the pet fur transferable residue
data from the rubbing protocol used in the study can provide useful information for evaluating
potential exposures of adults and children from contact with dogs treated with tetrachlorvinphos
containing pet collars.
HSRB Detailed Recommendations and Rationale
The Agency is proposing to use data from Davis et.al, (2008) on the levels of transferable
tetrachlorvinphos (TCVP) residue from the fur of dogs treated with flea collars containing
14.55% (4.8 g) of TCVP. The subject of this HSRB review is a published paper from research
funded by a USEPA Science to Achieve Results (STAR) grant.1 The research included children
but the Agency is not seeking to use the data collected on children's exposure in this case.
Rather, the Agency is interested in using data from assessment of pesticide residues on the dogs'
fur that was carried out by adults who were research staff. Therefore, the subject of this HSRB
review is limited to a part of the project that measured residue transfers from treated dogs to
gloved hand during scripted petting activity by research staff.
6 | Page

-------
The study was designed and conducted in a way that could provide scientifically sufficient data.
The paper includes two studies, with the first study using twenty three dogs and the second study
using an additional twenty two dogs. The dogs included a variety of breeds (long and short-hair)
and sizes (8 - 85 pounds). The first study collected data on residue transfers at nine time points
over 112 days (including one pre-application) and the second study measured residue transfers at
three time points over 12 days (including one pre-application). Justification by the authors for
measuring the Far at fewer time points during the second study was that the maximum transfer
was reached within 12 days of application during the first study. In addition to measuring Far,
both studies included measurement of cholinesterase inhibition in the pets and the second study
included measurements of TCVP residue transfer to t-shirts worn by children and metabolites in
urine of adults and children. The EPA is only considering use of the data on transferable residue
to gloves as a source of "transferable pet fur residue data" (no t-shirt data or biomarker data is to
be used).
The experimental methods for determining residue transfer using the scripted petting procedure
with gloved hands was well established and consistent with other studies conducted for other
active ingredients. The sampling and analysis methods for TCVP were fully described and
sufficiently verified. The measurements of the amount of TCVP transferred from treated dogs to
gloved hands is reliable but the question is whether the measurements are appropriate for the use
that is being proposed by the Agency.
The Agency is proposing to use the average mass of TCVP transferred to gloved hands over a
period of four months or 12 days following application as the transferable residue. The average
transferable residue will be divided by the initial mass applied in the collar (4,800 mg) to
calculate the available fraction of applied active ingredient (Far) as described in the EPA
Standard Operating Procedures (SOP) for Residential Pesticide Exposure Assessment (October
2012)."
To put the parameter into context, the SOP describes methods for estimating exposure (dermal,
inhalation and hand-to-mouth) to pesticides applied to pets, including flea collars (ready-to-use
liquid). The relevant equation given in the SOP for post-application exposure assessment is
E = TC*TR*ET
7 | Page

-------
Where E is exposure (mg/day), TC is the transfer coefficient (cm2/h), TR is transferable residue
(mg/cm2) and ET is exposure time (hours/day). The TR parameter gives the amount of active
ingredient on the surface of the animal that is available for transfer when the animal is touched.
TR is further defined in the SOP as
TR=AR*Far/SA
Where AR is the amount of active ingredient applied to the dog (mg), Far is the fraction of
applied active ingredient that is available as transferable residue (unitless) and SA is the surface
area of the dog (cm2). The Far is the specific parameter that the Agency is proposing to estimate
using data from Davis et.al. (2008).
Transferable residues were measured using research staff who were students from the college of
Veterinary Medicine, Mississippi State University. The students stroked/pet/rubbed the dogs
using gloved hands (pre-cleaned white cotton gloves) in a designated area (4 inches by 10
inches) following a predefined motion on the neck (with and without collar present) and on the
back near the base of the tail at each sampling time point. Results are reported graphically as the
average mass (micrograms) of TCVP transferred to glove during petting for each location and
time point.
Flea collars for the Davis et.al (2008) study were purchased new over the counter with all collars
from the same lot. The TCVP collars were designed to be used four months per manufacturer
recommendations. It is assumed that the collars were used as received (i.e., one size used for all
dogs and not cut to fit) so that the application of active ingredients was assumed to be the same
for all dogs at 4.8 g per dog. Instructions for flea collars typically requires that the collar be fit to
the dog and then the excess collar be trimmed off to get the appropriate application rate for the
sized dog. Assuming that the same mass of TCVP is applied can potentially bias the Far upwards
because the circumference and area of the neck of dogs ranging in size from 8 to 85 pounds will
vary significantly resulting in a higher mass per area for smaller dogs than larger dogs. It is
unclear how this potential difference in loading on the dog's neck will impact the amount or
residue transferred to the glove during simulated petting events.
The Agency is proposing to use the data from each study that was reported as average residue on
the gloves over all dogs and all time points over the total study period, in the first study for the
8 | P a g 6

-------
period 12 days, and for the second study, 4 months. EPA calculated the average residue by
summing the amount measured on the neck with collar and on the back then dividing by the
original mass applied. This results in values for Far of 0.3% and 0.4% for studies 1 and 2,
respectively. The amount of TCVP transferred to gloves from the back of the dogs was
insignificant (0.01 mg and 0.08 mg) compared to the neck (14.3 mg and 19.0 mg) for studies 1
and 2, respectively so can essentially be ignored in the calculation. Averaging over all the time
points in each study biases the Far value down because the transferable residue is at its maximum
around day 7 then declines dramatically over the course of the study. It is likely that the decline
in observed transferable residue is due to a decline in the amount of TCVP on the dog over time.
The measurement that is most relevant to the Far (i.e., the amount transferred relative to the
amount applied) would be closer to the initial application date of the collar when the amount
applied is still present on the dog. For example, the maximum measured transferable residue was
collected on approximately day 7 of the study. It is easier to assume that the total applied
amount of TCVP (4,800 mg) is still on the collar and the dog at the time when the maximum
transferable residue is measured. Given that the average (across all dogs) mass of TCVP on the
gloves at day 7 and day 5 is 24,000 and 22,000 for study 1 and 2, respectively (values estimated
from the figures in the paper) and assuming that the total applied mass is still on the dogs at that
time, then the resulting Far would be 0.5%.
There are a number of limitations related to the studies published in Davis et.al. (2008) that are
mostly related to a lack of information and details which are not available to the Agency. The
limitations identified by the HSRB are listed at the end of this section. However, despite these
limitations, the data as presented are sufficiently sound to support an estimate of the Far if the
maximum transferable residue is used. The data is less satisfying for averaging over the duration
of each study but can still support screening assessments.
Statistics
The following statistical review assess the adequacy of the methods employed and the scientific
validity of the reported data in Davis et al., 2008. Study authors used SAS procedure for general
linear models (PROC GLM) for glove and t-shirt residues analysis hypothesis testing, mean and
confidence interval estimates and mean comparisons based on One-way analysis of variance
9 | P a g e

-------
(ANOVA) of randomized complete block designs (RCBD). The GLM analysis is appropriate for
fixed effects hypothesis testing and mean comparisons. However, GLM is not appropriate for
estimating residue means and standard errors where random blocks were involved. To estimate
correct standard errors in the presence of random blocks, SAS mixed linear models procedure
(PROCMIXED) should have been used with random blocks. Therefore the reported confidence
intervals based on GLM underestimate TCVP residue estimates. However, the Agency is relying
solely on the mean estimates.
Urinary TCVP data were not being used by the Agency and so are not addressed here.
In the absence of performing data analysis on the transformed data, the error distribution is
assumed to have a normal distribution. This assumption was not validated in this research study.
Although the data from T-shirts is not going to be used by the Agency, the presence of extreme
large outliers (The higher mean TCVP tee shirt residues on days 9 and 11 result from one very-
high residue (17.9 and 17.3 mg/g shirt) obtained from one tee shirt on each of these two
sampling days) may suggest the residue distribution may not have a normal distribution. If this
assumption is correct then the reported point estimates (treatment means and standard error [SE])
are underestimates and any recommendation based on this study estimate may be incorrect.
Since the Agency is using data only from the gloves, and it is unclear whether the presence of
outliers in the T-shirt data has implications for the other data that were collected.
The error bars in Figure 1 and 2 are very uniform and consistent across all data points which
raises the question as to whether they are overall SE or if they are SE at the given time point
across all dogs. It would be more informative to have the error (standard deviation) at each point
so data quality decisions can be made particularly if the maximum value will be used rather than
the overall average.
Additional notes on scientific issues identified in review of the Davis et.al. (2008) paper:
1.	The application rate of active ingredient is assumed to be the same for all dogs regardless
of size/weight (i.e., 4.8 grams per dog). The Agency requested information from the
principal investigator of the study, but the information is not available.
2.	Assuming that the application rate was the same on all dogs, it is unclear how the long
tails on the collars were handled on the small dogs and whether the long tail on the collar
10 | I

-------
could be contacted by the glove during petting. If the collar was tucked in behind and
wrapped multiple times around the neck then it would potentially bias the transferable
residue upwards.
3.	The original data from the study are not available. A lack of information about dog size
and individual glove data prevents any statistical assessment of the results beyond what
is already reported (which is minimal with respect to the residue transfer data)
4.	It is assumed that the five minute petting events remove all of the transferable residue
without the gloves becoming saturated and without significant leaching through to the
skin of the research staff undertaking the data collection.
5.	Information is lacking to assess the relationship between transferable residue and dog
size, type, hair which would be important to estimate worst case Far values.
6.	It is unknown whether there is any loss of active ingredient through the cotton gloves to
the technicians' hands. This loss would reduce the value of transferable residue and in
turn reduce the resulting Far-
Ethics Review
Charge to the Board
•	Does the HSRB have any comments on EPA's determination that the samplers were not
human subjects?
•	Does the HSRB have any comments on the ethical conduct of the research?
Board Response
With regard to the first charge question, questions were raised by several committee members
about the Pi's and IRB's determinations that the samplers were not human subjects in the study;
rather, they were viewed as study staff. Some members of the board asserted that the
student/technicians, by virtue of being potentially exposed to the pesticide as part of the conduct
of the study, should have been considered human subjects. Furthermore, if they had been treated
as subjects, they might have been considered "vulnerable" due to their status as students. It was
noted that the flea control collars were commercially available at the time, and that the potential
exposure to the pesticide residues through petting the dogs for five minute periods wearing
111 I

-------
cotton gloves was likely much less than average exposure of a pet owner. There is no
information available about whether there was any "bleed through" of pesticide from cotton
gloves to the skin of the samplers and therefore the actual exposure is unknown. Considering all
of these factors, the committee felt that the risks of exposure were not greater than those
experienced in everyday life. Thus, even if the determination regarding the status of the samples
as study staff rather than subjects was mistaken, the committee did not believe this resulted in
any material harms and so this question should not prevent the EPA from using the pet fur
transferable residue data derived from the study for making a decision to impose a more stringent
regulatory restriction than could be justified without the data.
With regard to the second charge question, Board members observed that the records from
correspondence with EPA staff regarding the study suggest the consent form was amended to
include disclosure to parents about the risks of pesticide exposure, although the final approved
consent form was not provided. A question was raised about the decision made to provide
incomplete assent to the minor subjects following parental permission. Study documents suggest
this was an intentional choice ("We will not explain the connection to the pesticide residues on
the dog..."), which was made, according to study documents, in order to avoid confounding the
results by causing alterations in the children's behavior around their dogs. Board members noted
that the amount and type of information provided to children in an assent process will vary
depending on the age of the child; the children participating in the study were between the ages
of 3 and 11 years old and therefore would have had varying levels of capacity to process the
information about the study. It was noted that the Federal Insecticide, Fungicide and
Rodenticide Act (FIFRA), which existed at the time of these studies, states that it's unlawful
to use any pesticide in tests on humans unless they are fully informed of the nature and purposes
of the test. Although some board members viewed the assent as incomplete in this case, because
parents are presumed to have given fully-informed permission, and given that the flea control
collars were commercially available at the time and already in use in the households recruited to
the study, the committee felt that the risks of exposure were not greater than those experienced in
everyday life. Thus, the committee did not believe this resulted in any material harms and so this
question should not prevent the EPA from using the pet fur transferable residue data derived
12 |

-------
from the study for making a decision to impose a more stringent regulatory restriction than could
be justified without the data.
HSRB review of five completed studies of insect repellent to support the use of the EPA
Repellency Awareness Graphic.
The five completed S.C. Johnson studies each tested a single insect repellent against mosquitoes
in the field to determine the median complete protection time (CPT) of five of their skin applied
repellent products for the purpose of consumer labelling. The labelling will be done through the
EPA Repellency Awareness Graphic,111 an EPA program to raise public awareness of the health
protectiveness (efficacy) of mosquito and tick repellents applied to the skin, shows the typical
length of time the product repels mosquitoes or ticks. The purpose is to increase EPA and
consumer confidence in the efficacy claims on labels and improve consumer protection against
vector borne diseases, such as West Nile virus and Lyme disease.
Each of the five studies was evaluated individually by the Board and the scientific and ethical
aspects of each were assessed. The Board concluded that in each case, data from the studies is
sufficiently scientifically sound to be used for the repellency awareness graphic. Conduct of the
studies was in accordance with the study protocol in each case. The studies met ethical and
regulatory requirements and were conducted under appropriate IRB oversight. However, some
deficiencies in design and statistical analysis were noted in scientific review. These were not
serious enough to constitute a barrier to the use of data for the repellency graphic, but important
enough that the Board included comments in discussion and in this report for the purpose of
better design of future studies of this type.
Detailed discussion and consensus points are described for the studies below, but in brief, the
main areas of concern were the following:
• No discussion of, or justification for, the lack of a positive control substance in the tests
and the lack of an inert substance negative control, in the studies. The issue of positive
controls had not been raised during previous HSRB review of these protocols, but had
been more recently raised by the HSRB for other repellency awareness studies.
13 | I

-------
•	No discussion of whether additional sites (beyond two sites per study) might be needed to
capture the full range of likely outcomes for repellency of the products, for example, the
need to measure repellency under different conditions or in the presence of different
mosquito species;
•	Lack of sample size calculations and lack of scientific justification for the sample size
chosen;
•	No use of confidence intervals or other mechanism for expressing the degree of
variability in the data;
•	No analysis of the data collected on variables that may affect study outcomes, such as
demographic variables and environmental conditions at the time of testing;
•	No use of small sample size tests for measuring secondary questions of interest, for
example, whether there is a significant difference in repellency for women versus men
using the products;
Field Testing of SC Johnson Personal Mosquito Repellent Products to Support their Use of
the EPA Repellency Awareness Graphic, GLP Study Number 865E1, J. Palm, September
24, 2015. Test Substance: MARK-3 OFF! Deep Woods Sportsmen Insect Repellent I
(Maximum Strength Pump Spray Deep Woods OFF! EPA Reg. No. 4822-276)
Science Review
Charge to the Board
Is the study sufficiently sound, from a scientific perspective, to be used to estimate the duration
of complete protection against mosquitoes provided by the tested repellent?
Board Response
The study is sufficiently sound, from a scientific perspective, to be used to estimate the minimum
median complete protection time against mosquitoes provided by the tested repellent in support
of the EPA Repellency Awareness Graphic.
14 |

-------
HSRB Detailed Recommendations and Rationale
The HSRB was charged to conduct a science review of a study testing the efficacy of a 98.25%
DEET (diethyl toluamide) formulated as a pump spray against mosquitoes (Mark 3). The
board's review focused on the methodology and validity of the data for use to establish median
complete protection time (CPT), the unit of measure. The protocol was accepted by EPA on
April 23, 2015. Landings of wild mosquitoes on replicate subjects at two different locations were
used to evaluate repellency. The experimental design consisted of ten different treated subjects
and two untreated controls grouped into pairs and partially randomized based on random
selection from a pool of subjects and assigned based on numbers.
The HSRB concluded that the Mark 3 study was sufficiently sound, from a scientific perspective,
to be used to estimate the duration of complete protection against mosquitoes provided by the
tested repellent. However, several weaknesses in experimental design were identified that detract
from the overall quality of the study. The collection of mosquitoes to identify the populations
represented in the tested sample was remarkably different between Wisconsin and Florida, with
70 versus 302 mosquitoes collected at the two sites, respectively. Based on feedback provided
by the study sponsor, SC Johnson, the mosquitoes collected for sampling were collected by study
staff at random and not necessarily by the experimental subjects. As such, no direct correlations
could be established between the numbers of species identified in the sample pool relative to the
species actually sampled in repellency testing. This weakness leaves open the question that
repellency measures are only applicable to selected species and not generalizable to the entire
populations present in the sample and challenges statements made in the report suggesting that
collected mosquitoes represented those aspirated after the five-minute testing period.
While experimental design elements, such as the number of landings in the five-minute exposure
period considered to be the minimum necessary to ensure the mosquito population was large
enough to determine repellency or sample size calculations based on studies with different
insects, were said to have been derived from historical values and past experience, the lack of
15 | I

-------
adequate justification and experimental details provided in the report detract from the overall
quality of the reporting effort and hamper the ability of external reviewers to complete an
independent rigorous evaluation of the experimental approach. In this regard, it should be noted
that power calculations using ticks as experimental insects may or may not be applicable to
mosquitoes and this area needs to be more rigorously investigated. Another area of concern was
that clear distinctions were not made in the report to calculate target concentrations of product
from target rates of application, which clearly represent different experimental variables. It
should also be noted that while some amendments were made to the study protocol to
incorporate EPA's and HSRB's previous recommendations, not all recommendations made were
integrated into the final protocol. Also relevant to the present set of studies are concerns about
lack of positive controls. Comments on positive controls were made previously by the HSRB on
a different protocol addressing ticks (October 2015). For future studies of this type, the HSRB
recommends additional efforts to integrate positive and negative inert substance controls into the
experimental design or else provide clear scientific justification for their absence. Furthermore,
SC Johnson's handling of response to feedback provided regarding measurement of landing
pressures and generalizability of environmental conditions was not optimal.
Statistics
The purpose of the proposed study was to determine the complete protection time of up to 18
EPA-registered S.C. Johnson skin-applied repellent products. The resulting data are intended to
support the products' use of EPA's repellency awareness graphic on the product labels.
The statistical issues listed above as general issues for all five studies were raised for this
product.
SC Johnson responded to the statistical issues raised in previous reviews including experimental
design, randomization, sample size calculation, sources of variation, and data
compiling/processing. The responses are deemed to be adequate for the purpose of using the
data for the repellency graphic, although as noted above, there some areas of study design and
analysis that could be improved for increased scientific rigor.
16 |

-------
Ethics
Charge to the Board
Does available information support a determination that the study was conducted in substantial
compliance with 40 CFR Part 26, subparts K and L?
Board Response
The information provided supports a determination that the studies were conducted in substantial
compliance with subparts K and L of 40 CFR Part 26.
HSRB Detailed Recommendations and Rationale
40 CFR 26 subpart K requires that studies initiated on or after April 7, 2006 involving intentional
exposure of human subjects to a pesticide be reviewed and approved by an institutional review
board (IRB) that meets the membership and review criteria listed in that subpart. The study listed
above was approved by Schulman Associates Institutional Review Board (SAIRB) on 7/7/15
after the protocol was revised per EPA and HSRB recommendations. The study team maintained
SAIRB approval throughout the study.
40 CFR 26 subpart K mandates studies minimize risk to subjects, equitably select subjects, seek
and appropriately document informed consent, make adequate provisions to ensure safety of
subjects, and protect the privacy of subjects and confidentiality of data. The study listed above
minimized risk to subjects by adequately training subject to remove mosquitos before biting
occurred and either using lab-reared mosquitos (at the Wisconsin site) or monitoring appropriate
reporting agencies to determine that mosquito-borne diseases have not been reported by health
authorities in the previous month (at the Florida site). Subjects were equitably selected from
maintained list of recruiting firms in Wisconsin (J. Reckner Associates, Inc) and Florida (Herron
Associates, Inc) to represent the demographics of US repellent users. The Study Director or
Principal Investigator explained the study to the subjects at the training session, at which time
subject's questions were answered and consent was documented. Provisions to ensure safety and
17 | I

-------
comfort included use of bug suits to prevent mosquito biting, monitoring of the Florida site for
other environmental nuisances, food and beverages, and seating. Privacy and confidentiality
were maintained through the use of assigned code numbers, maintenance of records on a
password-protected computer server, and limiting access to records.
40 CFR 26 subpart L prohibits the EPA from relying on third-party research involving
intentional exposure to a pesticide of human subjects who are children or pregnant or nursing
women. Materials submitted to EPA and HSRB indicate that no children or pregnant or nursing
women participated in this study. The study's inclusion criteria required that subjects be 18-55
years of age and able to provide valid proof of their age (such as a driver's license or passport).
Female subjects completed pregnancy tests in a private bathroom on the training day, which was
within 48 hours of the study day. Women who chose not to participate in the study after taking
the pregnancy test were not asked to provide a reason for their decision not to participate.
However, if the subject remained interested in study participation after taking the pregnancy test,
a female study team member verified the negative pregnancy test. The pregnancy test results
were not recorded and remained confidential. As a result, the study was conducted in substantial
compliance with 40 CFR part 26, subpart L.
Field Testing of SC Johnson Personal Mosquito Repellent Products to Support their Use of
the EPA Repellency Awareness Graphic, GLP Study Number 873E1, C. Talbert, October
21, 2015. Test Substance: MARK-8 OFF! Deep Woods Insect Repellent V (OFF! Insect
Repellent Formula, EPA Reg. No. 4822-167)
Science Review
Charge to the Board
Is the study sufficiently sound, from a scientific perspective, to be used to estimate the duration
of complete protection against mosquitoes provided by the tested repellent?
18 |

-------
Board Response
The study is sufficiently sound, from a scientific perspective, to be used to estimate the minimum
median complete protection time against mosquitoes provided by the tested repellent in support
of the EPA Repellency Awareness Graphic.
HSRB Detailed Recommendations and Rationale
The HSRB was charged to conduct a science review of a study testing the efficacy of a 25%
DEET (diethyl toluamide) formulated as pressurized aerosol against mosquitoes (Mark 8). The
board's review focused on the methodology and validity of the data for use to establish median
complete protection time (CPT), the unit of measure. The protocol was accepted on April 23,
2015. Landings of wild mosquitoes on replicate subjects at two different locations were used to
evaluate repellency. The experimental design consisted of ten different treated subjects and two
untreated controls grouped into pairs and partially randomized based on random selection from a
pool of subjects and assigned based on numbers.
The strengths and weaknesses in experimental design identified for the Mark 3 study are
applicable for the present study. Of note, as stated in the review of the Mark 3 study, are
concerns related to the absence of sample size calculations and robustness of the scientific
information provided in the report. In this study, there were also questions about the effect of a
rain delay on variability of the findings reported. At one site, rainy weather led to the
cancellation of two exposure periods on a test day. The resulting calculations of CPT for that
day entailed estimating time to first confirmed landing based on the timing of the first cancelled
exposure period. The net effect of this estimation on the variability in the data is unclear.
Statistics
SCJ responded to some of the statistical design and analysis issues raised by the Board in its
previous review but their response is not clear on some issues. For example, SCJ did discuss
sample size calculations from studies in the literature (pages 94-97 of the Final Study Report).1V
19 | I

-------
However, those reported studies were reporting means and not medians, which for skewed
distributions can be quite different. The final sample size requirement in the present study was
apparently chosen based at least partially on ethical considerations—that is, with the objective of
minimizing the number of subjects exposed to mosquitoes. A complete scientific rationale for
sample size was not provided.
Other recommendations raised in the Board's previous review, such as analyzing the effect of
demographic characteristics and environmental conditions, were not taken up. These variables
(demographics, environmental conditions) were collected and reported but did not appear to be
utilized in the analysis in any way.
SCJ reported estimates of the smallest median CPT at each site as well as the overall smallest
median CPT. For this product, choosing the smaller of the two site smallest median CPTs
produced the same result as would be have been obtained as the smallest median CPT of the
combined sample after rounding downward. It appears that the smaller of the two site median
CPT values were used by SCJ (page 99 of the Final Report). SCJ confirmed this to be their
approach.
Since the objective was to produce a single estimate without any indication of a standard error or
confidence limits, the statistical analyses reported by SCJ would be deemed to be adequate.
Ethics Review
Charge to the Board
Does the available information support a determination that the research was conducted in
substantial compliance with 40 CFR part 26, subparts K and L?
Board Response
Available information indicates the research was conducted in substantial compliance with
applicable provisions of 40 CFR part 26 subparts K and L.
20 |

-------
HSRB Detailed Recommendations and Rationale
Study maintained approval from an IRB operating in accordance with 40CFR26. Because this
study was initiated after April 7, 2006, submission of the protocol and supporting materials to
EPA prior to conducting the study was required and appropriately completed.
The study excluded children. No pregnant or nursing female subjects participated in the study.
Field Testing of SC Johnson Personal Mosquito Repellent Products to Support their Use of
the EPA Repellency Awareness Graphic, GLP Study Number 864E1, J. Palm, September
24, 2015. Test Substance: MARK-2 OFF! Deep Woods Sportsmen Insect Repellent II
(UNSCENTED DEEP WOODS OFF! EPA Reg. No. 4822-397)
Science Review
Charge to the Board
Is the study sufficiently sound, from a scientific perspective, to be used to estimate the duration
of complete protection against mosquitoes provided by the tested repellent?
Board Response
The study is sufficiently sound, from a scientific perspective, to be used to estimate the minimum
median complete protection time against mosquitoes provided by the tested repellent in support
of the EPA Repellency Awareness Graphic.
HSRB Detailed Recommendations and Rationale
The HSRB was charged to conduct a science review of a study testing the efficacy of a 30%
DEET (diethyl toluamide) formulated as pressurized aerosol against mosquitoes (Mark 2). The
board's review focused on the methodology and validity of the data for use to establish median
complete protection time (CPT), the unit of measure. The protocol was accepted by EPA on
April 23, 2015. Landings of wild mosquitoes on replicate subjects at two different locations were
21 | i

-------
used to evaluate repellency. The experimental design consisted of ten different treated subjects
and two untreated controls grouped into pairs and partially randomized based on random
selection from a pool of subjects and assigned based on numbers.
The strengths and weaknesses in experimental design identified for the Mark 3 and Mark 8
studies are applicable for the present study. Of note are concerns related to the absence of sample
size calculations and robustness of the scientific information provided in the report. Given that
the Mark 8 and Mark 2 studies both examined pressurized aerosols at different concentrations,
comparisons between the two studies should be made to gain additional insight into the validity
and comparability of findings.
Statistics
The purpose of the proposed study is to determine the complete protection time of up to 18 EPA-
registered S.C. Johnson skin-applied repellent products in a laboratory setting. The resulting
data are intended to support the products' use of EPA's repellency awareness graphic on the
product labels.
Four statistical issues about the protocol were raised.
SC Johnson responded to the statistical issues raised in previous reviews including experimental
design, randomization, sample size calculation, sources of variation, and data
compiling/processing. The responses are deemed to be adequate for the purpose of using the
data for the repellency graphic, although as noted above, there are some areas of study design
and analysis that could be improved for increased scientific rigor.
Ethics review
Charge to the Board
Does available information support a determination that the study was conducted in substantial
compliance with 40 CFR Part 26, subparts K and L?
22 |

-------
Board Response
The information provided supports a determination that the studies were conducted in substantial
compliance with subparts K and L of 40 CFR Part 26.
HSRB Detailed Recommendations and Rationale:
40 CFR 26 subpart K requires that studies initiated on or after April 7, 2006 involving intentional
exposure of human subjects to a pesticide be reviewed and approved by an institutional review
board (IRB) that meets the membership and review criteria listed in that subpart. The study listed
above was approved by Schulman Associates Institutional Review Board (SAIRB) on 7/7/15
after the protocol was revised per EPA and HSRB recommendations. The study team maintained
SAIRB approval throughout the study.
40 CFR 26 subpart K mandates studies minimize risk to subjects, equitably select subjects, seek
and appropriately document informed consent, make adequate provisions to ensure safety of
subjects, and protect the privacy of subjects and confidentiality of data. The study listed above
minimized risk to subjects by adequately training subject to remove mosquitos before biting
occurred and either using lab-reared mosquitos (at the Wisconsin site) or monitoring appropriate
reporting agencies to determine whether any mosquito-borne disease had been reported in the
previous month (at the Florida site). Subjects were equitably selected from maintained list of
recruiting firms in Wisconsin (J. Reckner Associates, Inc) and Florida (Herron Associates, Inc)
to represent the demographics of US repellent users. The Study Director or Principal Investigator
explained the study to the subjects at the training session, at which time subject's questions were
answered and consent was documented. Provisions to ensure safety and comfort included use of
bug suits to prevent mosquito biting, monitoring of the Florida site for other environmental
nuisances, food and beverages, and seating. Privacy and confidentiality were maintained through
the use of assigned code numbers, maintenance of records on a password-protected computer
server, and limiting access to records.
23 |

-------
40 CFR 26 subpart L prohibits the EPA from relying on third-party research involving
intentional exposure to a pesticide of human subjects who are children or pregnant or nursing
women. Materials submitted to EPA and HSRB indicate that no children or pregnant or nursing
women participated in this study. The study's inclusion criteria required that subjects be 18-55
years of age and able to provide valid proof of their age (such as a driver's license or passport).
Female subjects completed pregnancy tests in a private bathroom on the training day, which was
within 48 hours of the study day. Women who chose not to participate in the study after taking
the pregnancy test were not asked to provide a reason for their decision not to participate.
However, if the subject remained interested in study participation after taking the pregnancy test,
a female study team member verified the negative pregnancy test. The pregnancy test results
were not recorded and remained confidential. As a result, the study was conducted in substantial
compliance with 40 CFR part 26, subpart L.
Field Testing of SC Johnson Personal Mosquito Repellent Products to Support their Use of
the EPA Repellency Awareness Graphic, GLP Study Number 866E1, E. Laznicka, October
21, 2015. Test Substance: MARK-4 OFF! Active Insect Repellent I (Unscented OFF!
Insect Repellent, EPA Reg. No. 4822-380)
Science review
Charge to the Board
Is the study sufficiently sound, from a scientific perspective, to be used to estimate the duration
of complete protection against mosquitoes provided by the tested repellent?
Board Response
The study is sufficiently sound, from a scientific perspective, to be used to estimate the minimum
median complete protection time against mosquitoes provided by the tested repellent in support
of the EPA Repellency Awareness Graphic.
24 |

-------
HSRB Detailed Recommendations and Rationale
The protocol for the field testing of Mark-4 OFF! Active Insect Repellent I (Unscented OFF!
Insect Repellent, 15% DEET) was reviewed by HSRB at the April 2015 meeting. The Board
concluded that the protocol titled "Field Testing of S.C. Johnson Personal Mosquito Repellent
Products to Support their Use of the EPA Repellency Awareness Graphic", if modified according
to Agency and HSRB recommendations, is likely to generate scientifically reliable data, useful
for estimating the complete protection time as defined in the protocol, of various EPA-registered
S.C. Johnson skin-applied mosquito repellents in the field against wild adult mosquito
populations. The protocol was amended to incorporate EPA and HSRB recommendations. The
study was completed following the approved protocol with one amendment and four reported
deviations. One deviation listed in the study report was actually an IRB approved protocol
amendment to change the study director for the study, and hence was not technically a deviation.
Another deviation was the use of light meters and stopwatches that were not GLP-qualified.
However the devices are NIST certified and so are suitable and adequate for study procedures.
There were two deviations in data collection procedures: one site had fewer subjects than
planned with an unbalanced (2 males and 6 females design; and one exposure period was
skipped due to rain. An analysis was conducted to show that the deviations had no detectable
impact on the outcome. As a result, the data produced are sufficiently sound, from a scientific
perspective, to support a median CPT of 5 hours against mosquitoes for the EPA Repellency
Awareness Graphic.
Statistics
SCJ responded to some of the statistical design and analysis issues raised by the Board in its
previous review but their response is not clear on some issues. For example, as with the studies
reviewed earlier in the meeting, SCJ did not carry out a formal sample size calculation but relied
on the same justification based on the literature despite its questionable applicability (pages 95-
96 of the Final Report). As before, demographic characteristics and environmental conditions
were reported but did not appear to be utilized in any way.
25 | I

-------
A test for a difference in median CPT between males and females was conducted using a
combined set of data from the five products in this review plus one additional product using the
Wisconsin test site data (page 117 of the Final Report). The results indicated no significant
difference. While useful, the analysis does not directly address differences for MARK-4 as a
single product. Small sample size tables for the Mann-Whitney statistic are widely available and
could have been utilized. In fact, a test of these data from MARK-4 using the Wilcoxon rank
sum version of the Mann-Whitney finds a significant difference between males and females at
the 5% level.
SCJ reported estimates of the smallest median CPT at each site as well as the overall smallest
median CPT. For this product, choosing the smaller of the two site smallest median CPTs
produced the same result as would be have been obtained as the smallest median CPT of the
combined sample after rounding. SCJ indicated during the meeting the approach was to choose
the smaller site median CPT.
Table 2 (page 9 of the Final Study Report) containing the individual subject durations indicates
large site differences in the durations. While using the smaller site median as the final CPT is the
conservative approach, the apparent large site to site variability would lead one to question
whether or not sites with much smaller medians CPTs might exist.
Since the objective was to produce a single estimate without any indication of a standard error or
confidence limits, the statistical analyses reported by SCJ would be deemed to be adequate for
this purpose.
Ethics review
Charge to the Board
Does the available information support a determination that the research was conducted in
substantial compliance with 40 CFR part 26, subparts K and L?
Board Response
The research was conducted in substantial compliance with applicable provisions of 40 CFR
part 26, subparts K and L.
26 | i

-------
HSRB Detailed Recommendations and Rationale
40 CFR 26 subpart K requires that studies initiated on or after April 7, 2006 involving
intentional exposure of human subjects to a pesticide be reviewed and approved by an
institutional review board (IRB) that meets the membership and review criteria listed in that
subpart. The study listed above was approved by the Schulman Associates convened IRB
(SAIRB) on July 7, 2015 after the sponsor revised study materials in light of EPA and HSRB
recommendations. The study team maintained SAIRB approval throughout the conduct of the
study. The letter to subjects notifying about birds testing positive for West Nile Virus being
detected in the county where the research took place also received SAIRB approval. SAIRB
meets the appropriate membership criteria and followed review criteria listed in this subpart.
As a result, the study was conducted in substantial compliance with 40 CFR part 26, subpart
K.
40 CFR 26 subpart L prohibits the EPA from relying on third-party research involving
intentional exposure to a pesticide of human subjects who are children or pregnant or nursing
women. Materials submitted to EPA and HSRB indicate that no children or pregnant or
nursing women participated in this study. The study's inclusion criteria required that subjects
be 18-55 years of age and able to provide valid proof of their age (such as a driver's license or
passport). Female subjects completed pregnancy tests in a private bathroom within 48 hours of
the study training day. Women who chose not to participate in the study after taking the
pregnancy test were not asked to provide a reason for their decision not to participate.
However, if the subject remained interested in study participation after taking the pregnancy
test, the negative pregnancy test was verified by a female study team member. The pregnancy
test results were not recorded and remained confidential. As a result, the study was conducted
in substantial compliance with 40 CFR part 26, subpart L.
Field Testing of SC Johnson Personal Mosquito Repellent Products to Support their Use of
the EPA Repellency Awareness Graphic, GLP Study Number 867E1, E. Laznicka, October
21, 2015. Test Substance: MARK-5 OFF! Family Care Insect Repellent IV (Unscented)
(UNSCENTED OFF! SKINTASTIC SPRAY INSECT REPELLENT, EPA Reg. No. 4822-
395)
27 | i

-------
Science Review
Charge to the Board
Is the study sufficiently sound, from a scientific perspective, to be used to estimate the duration
of complete protection against mosquitoes provided by the tested repellent?
Board Response
The study is sufficiently sound, from a scientific perspective, to be used to estimate the minimum
median complete protection time against mosquitoes provided by the tested repellent in support
of the EPA Repellency Awareness Graphic.
HSRB Detailed Recommendations and Rationale
The protocol for the field testing of MARK-5 (Unscented OFF! Skintastic Spray Insect Repellent
7% DEET) was reviewed by HSRB, and the protocol team responded to EPA and HSRB
recommendations. The study was completed following the EPA-approved protocol, and the data
produced are sufficiently sound, from a scientific perspective, to support a median CPT of 2
hours against mosquitoes for the EPA Repellency Awareness Graphic.
Statistics
The stated study objective is to establish the median complete protection time (CPT) against
mosquitoes for use in the EPA Repellency Awareness Graphic on the label of EPA Registered
products. The study was completed at two locations a planned and the median CPTs with 95%
confidence limits [Lower Confidence Limit (LCL) and Upper Confidence Limit (UCL)] were
calculated for each location by Kaplan Meier analysis using PROC LIFETEST in SAS, which
employs a generalization of the Brookmeyer and Crowley (1982)v method under a log-log
transformation. The statistical design used and analysis performed are adequate for the proposed
study goal. However, the analysis can be further enhanced by performing stratified analysis by
sites using the STRATA statement in SAS PROC LIFETEST. This stratified analysis will
provide the separate median CPT and CI estimates for each site but then further statistically test
28 | I

-------
whether these CPT values are statistically similar to each other. If the CPT times are determined
to be statistically similar then a pooled CPT estimates over the two sites can be derived providing
a larger sample size and increased confidence.
Ethics Review
Charge to the Board
Does the available information support a determination that the research was conducted in
substantial compliance with 40 CFR part 26, subparts K and L?
Board Response
The research was conducted in substantial compliance with applicable provisions of subparts K
and L of 40 CFR 26, met all applicable ethical standards for the protection of human subjects of
research, and satisfied the requirements for documentation of ethical conduct of research.
HSRB Detailed Recommendations and Rationale
The purpose of this study was to establish the complete protection time (CPT) of MARK-5 OFF!
Testing took place at two field sites (Wisconsin and Florida) with 24 adult (ages 18-55) human
subjects who completed the testing (12 at each site). The repellent test products are registered by
EPA and have been found to present little or no risk when used as directed. The protocol for this
study was approved by the Schulman Associates Institutional Review Board (SAIRB). The
protocol was discussed in a public meeting by the Human Studies Review Board (HSRB) on
April 22-23, 2015, which concluded that "the amended protocol ... should meet all applicable
ethical standards."
A third-party recruiting firm was used to identify potential subjects. Subjects who met the
inclusion/ exclusion criteria and were available for both the training and test dates were selected
for the studies (20 at Florida and 24 at Wisconsin). Pregnancy testing was performed by potential
29 | I

-------
female subjects alone in a private bathroom. The results were reported/verified by the subject
only.
Consent was obtained before beginning the procedures training session. Subjects were paid $60
for participating in a 3-4 hour training session conducted prior to the field testing. For each field
test day, subjects were paid $15 per hour for the test day. Subjects were advised on several
occasions that they could withdraw from the study for any reason, without penalty. No subjects
withdrew. Risks were appropriately identified and precautions were taken to mitigate identified
hazards; e.g. the study was conducted in areas where the presence of mosquito-borne disease had
not been detected by county or state health staff or mosquito abatement district staff within one
month prior to the test date. Starting thirty minutes after the test substance application to the
forearm, there were five minute exposures to mosquitoes in the field at 30 minute intervals.
Subpart K (Third-Party Human Research for Pesticides):
Study maintained approval from an IRB operating in accordance with 40CFR26. Because this
study was initiated after April 7, 2006, submission of the protocol and supporting materials to
EPA prior to conducting the study was required and appropriately completed.
Subpart L (Prohibitions):
Study excluded children. No pregnant or nursing female subjects participated in the study.
30 |

-------
NOTICE
This report has been written as part of the activities of the EPA Human Studies Review Board, a
Federal advisory committee providing advice, information and recommendations on issues
related to scientific and ethical aspects of human subjects research. This report has not been
reviewed for approval by the Agency and, hence, the contents of this report do not necessarily
represent the view and policies of the Environmental Protection Agency, nor of other agencies in
the Executive Branch of the Federal government, nor does the mention of trade names or
commercial products constitute a recommendation for use. You may obtain further information
about the EPA Human Studies Review Board from its website at http://www.epa.gov/osa/hsrb.
You may also contact the HSRB Designated Federal Officer, via e-mail at ord-osa-
hsrb@epa.gov
In preparing this document, the Board carefully considered all information provided and
presented by the Agency presenters, as well as information presented by public commenters.
This document addresses the information provided and presented within the structure of the
charge by the Agency.
31 I

-------
US ENVIRONMENTAL PROTECTION AGENCY
HUMAN STUDIES REVIEW BOARD
Chair
Liza Dawson, Ph.D., Research Ethics Team Leader, Division of AIDS National Institutes of
Health (NIH), National Institute of Allergy and Infectious Disease (NIAID), Bethesda, MD
Vice Chair
Edward Gbur, Jr., Ph.D., Professor, Director, Agricultural Statistics Laboratory, University of
Arkansas, Fayetteville, AR
Members
Gary L. Chadwick, Pharm.D., M.P.H., C.I.P., Senior Consultant, HRP Consulting Group, Inc.
Fairport, NY
George Fernandez, Ph.D., Statistical Training Specialist, SAS Institute, Statistical Training and
Technical Services, Sparks, NV
Kyle L. Galbraith, Ph.D., Manager, Human Subjects Protection, Carle Foundation Hospital,
Urbana, IL
Jewell H. Halanych, M.D., Assistant Professor, Internal Medicine Residency Program,
Montgomery Regional Campus, University of Alabama at Birmingham, Montgomery, AL
Randy Maddalena, Ph.D., Physical Research Scientist, Indoor Environment, Lawrence Berkeley
National Laboratory, Berkeley, CA
Kenneth Ramos, M.D., Ph.D., PharmB, Associate Vice President, Precision Health Sciences,
Professor of Medicine, Arizona Health Sciences Center, Tucson, AZ
Suzanne M. Rivera, Ph.D., M.S.W., Vice President for Research and Technology Management,
Case Western Reserve University, Cleveland Heights, OH
Helen H. Suh, Ph.D., Associate Professor of Health Sciences, Northeastern University, Boston,
MA
Jun Zhu, Ph.D., Professor of Statistics and of Entomology, Department of Statistics, University
of Wisconsin - Madison, Madison, WI
Human Studies Review Board Staff
Jim Downing, Executive Director, Human Studies Review Board Staff, Office of the Science
Advisor, United States Environmental Protection Agency, Washington, DC
32 |

-------
1 EPA Grant Number: R828017; Assessing Levels of Intermittent Exposures of Children to Flea Control Insecticides
from the Fur of Dogs;
https://cfpub.epa.gov/ncer abstracts/index.cfm/fuseaction/displav.abstractDetail/abstract/779/report/O
" https://www.epa.gov/pesticide-science-and-assessing-pesticide-risks/standard-operating-procedures-residential-
pesticideffsops
http://www.epa.gov/insect-repellents/repellencv-awareness-graphic
iv	Final study report, Mark 8
v	Brookmeyer, R. and Crowley, J. (1982. A confidence interval for the median survival time. Biometrics 38: 29-41.
33 |

-------