June 23, 2015
EPA-HSRB-15-02
Thomas A. Burke, Ph.D., MPH
EPA Science Advisor
Office of the Science Advisor
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Subject: April 22-23, 2015 EPA Human Studies Review Board Meeting Report
Dear Dr. Burke,
The United States Environmental Protection Agency (EPA or Agency) requested that the Human
Studies Review Board (HSRB) provide scientific and ethics reviews of two items: The Completed Study
and Monograph Report for Backpack and Handgun Application of Liquid Spray in Utility Rights
of Way (Agricultural Handlers Exposure Task Force); and Protocol for Field Testing of Skin
Applied Mosquito Repellent Products (S.C. Johnson).
The Board's responses to the charge questions are presented in the enclosed final meeting report.
Signed,
Liza Dawson, PhD
Chair
EPA Human Studies Review Board
-------
INTRODUCTION
On April 22-23, 2015, the United States Environmental Protection Agency's (EPA or Agency) Human
Studies Review Board (HSRB or Board) met to address the scientific and ethical charge questions related
to two items: The Completed Study and Monograph Report for Backpack and Handgun
Application of Liquid Spray in Utility Rights of Way (Agricultural Handlers Exposure Task Force);
and Protocol for Field Testing of Skin Applied Mosquito Repellent Products (S.C. Johnson).
REVIEW PROCESS
The Board conducted a public meeting on April 22-23, 2015. Advance notice of the meeting was
published in the Federal Register as "Human Studies Review Board; Notification of a Public Meeting"
(Federal Register / Vol. 80, No. 43 / Thursday, March 5, 2015 / Notices).
The Board heard presentations from EPA for each agenda item in sequence, consisting of the Agency's
review of scientific and ethical aspects of the two studies. This Final Report of the meeting describes the
HSRB's discussion, recommendations, rationale and consensus in response to each charge question for
each of these items.
For each agenda item, Agency staff first presented their review of the science and the Board asked the
Agency presenters clarifying questions. The staff then described their review of the ethical aspects and the
Board asked clarifying questions about those. The HSRB solicited public comments and next asked
Agency staff to read the Charge Questions for the topic under consideration. The Board discussed the
science questions first and then the ethics question. The Chair then called for a vote to confirm
concurrence on a summary statement in response to each charge question.
For their evaluation and discussion, the Board considered materials presented at the meeting, oral
comments, related materials and documents provided by the study sponsors, the Agency's science and
ethics reviews of the studies, oral responses from a study sponsor and protocol team, and public
comments made at the meeting. A comprehensive list of background documents is available online at
http://www.epa.gov/hsrb/.
-------
CHARGE TO THE BOARD AND BOARD RESPONSE
HSRB review of the Completed Study and Monograph Report for Backpack and Handgun
Application of Liquid Spray in Utility Rights of Way (Agricultural Handlers Exposure Task Force)
Overview of the completed studies and monograph report
The completed study, conducted by the Agricultural Handlers Exposure Task Force (AHETF) was a
monitoring study to assess dermal and inhalation exposure of workers using liquid pesticide sprays in
utilities' rights of way and similar settings such as parks and roadsides. The study consisted of two parts,
one assessing exposure for workers using backpack sprayers and one for handgun sprayers. The workers
did not mix the spray solutions, so the monitoring encompassed only the application of the sprays on
actual work days. Workers in the two scenarios handled a range of active ingredient from 0.03 to 45.95
Ibs, sprayed between 4.5 and 2900 gallons of solution. The period of work time ranged from 2 to 11.4
hours, and the number of acres covered ranged from less than 1 acre to approximately 20 acres.
Dermal exposure was monitored using hand washes, face and neck wipes, and whole body dosimeters
that consisted of 100% cotton union suits worn under the clothing. Inhalation exposure was monitored
using personal air sample pumps and OSHA Versatile Samplers mounted on the shirt collar.
Total dermal exposure was calculated by adding the results for the whole body dosimeters, hand washes
and face and neck wipes. Inhalation exposure was calculated from air sample data, assuming a standard
breathing rate for all workers to calculate total inhalation during the work period.
The studies were planned as a "7x3" design, meaning that a total of 7 sites were planned with 3 workers
("monitoring units" or MUs) assessed at each site, in accordance with the AHETF Governing Document
and Backpack and Handgun ROW Scenario Construction Plan. The Handgun study included the full 21
MUs, while the Backpack study included 19 MUs due to difficulties with recruitment and a protracted
study timeline.
Science
Charge to the Board
1. Was the research reported in the Agricultural Handler Exposure Task Force (AHETF) completed
monograph reports and associated field study report for AHE400 faithful to the design and objectives of
the protocol, SOPs, and governing documents?
2. Has the Agency adequately characterized, from a scientific perspective, the limitations on these data
that should be considered when using the data in estimating exposure of those who apply liquid pesticide
sprays to utilities rights-of-way using backpack or handgun spray equipment?
Board Response, Charge question #1
The Board concluded that the research reported in the Agricultural Handler Exposure Task Force
(AHETF) completed monograph reports and associated field study report for AHE400 were faithful to the
design and objectives of the protocol, SOPs, and governing documents.
HSRB Detailed Recommendations and Rationale:
The research was conducted largely as planned, with some deviations due to difficulties in recruitment
and due to laboratory changes. While there were a number of deviations from Good Laboratory Practice
Standards (GLPS), the rationale in the AHETF documentation provides a needed level of assurance that
-------
procedures and data were well tracked. The protracted study schedule added unplanned but valuable
temporal diversity by collecting samples over 3 years, and the steps taken to recruit and monitor backpack
and handgun applicators across 9 and 10 different U.S. states, respectively, slightly expanded the spatial
diversity. Randomization was used where possible, despite the constraints of recruiting within an overall
purposive sampling scheme using strata in the amounts of active ingredient handled. For example, in
both groups of applicators "When multiple workers in a given area/time were available, the monitored
workers were selected at random." (p. 8 Crowley AHE1012 and p. 8 Crowley AHE1013). There were no
repeat measurements on the same applicator. Only two pairs of applicators (one backpack and one
handgun) worked for the same employer, and these workers were monitored in different calendar years, at
different job sites, and were part of different application crews. Overall, the brands/types of backpacks
and handgun sprayers, their configurations, and spray techniques were also varied (p. 9 Crowley
AHE1012 and p. 8 Crowley AHE1013).
The Board agrees with the Agency's determination "... that the data set has captured routine behavior as
well as limiting the likelihood of "low-end" exposures via certain scripting aspects (e.g., requiring a
minimum monitoring time to avoid cases in which there is no detectable exposure); both attributes are
valuable for regulatory assessment purposes. The potential for significant carryover of residual AI from
prior days' exposures in the chaps worn by many applicators adds realism, although it complicates
explaining the high variability discussed in response to the second charge question. And the
randomization in the selection of individual applicators likely mitigated selection bias on the part of
participants or recruiters. Thus, with respect to costs, feasibility, and utility, the resulting dataset is
considered a reasonable approximation of expected exposure for this population." (p. 22 Crowley
AHE1012 and p. 21 Crowley AHE1013).
The final exposure data for the handgun applicators met both the primary objective of 3-fold accuracy for
select statistics [fRAgs] and the secondary objective of having sufficient statistical power to distinguish
whether there is proportionality between total dermal exposure both before and after being adjusted for
the recovery method efficiency [MEA] and the amount of active ingredient handled [AaiH]. The final
exposure data for the backpack applicators met the secondary objective but not the primary objective.
The explanation that not meeting the 3-fold statistical accuracy goal due was primarily due to higher than
expected variability in exposures (a measured geometric standard deviation [GSD] of 5 to 6 versus the
GSD of 4 planned within the protocol, see p. 14 Crowley AHE1012) was in fact anticipated by the Board
(p. 23, HSRB Final Report for October 2010 meeting).
The Board also agrees with the Agency that the additional recruitment of applicators (10 to 16 additional
depending on the number of clusters) that might be needed to meet the 3-fold accuracy requirement is
unlikely to change either the variability in exposures or the estimated exposure statistics themselves.
Thus, as long as the Agency can accept the existence of the high variability within this exposure scenario,
the Board also agrees "that it would not be worth the multi-year delay - nor ethically justifiable - to
monitor these additional workers to meet the accuracy benchmark." (p. 15 Crowley AHE1012)
Board Response, Charge Question #2
The Board identified three more limitations that should be considered when using this data:(l) A higher
breathing rate to estimate the inhaled dose of applicators while on foot should be used;(2) the variability
in the number of hand washes and the resulting time that deposits were allowed to reside on the skin
before attempting removal adds a small, unrecognized weakness to the data; and (3) the inability to
explain the high variability in general and the cause of the high-end exposures in particular is also an
additional limitation of the study. Neither of the first two concerns is thought to have a major impact on
the study's conclusions.
-------
Breathing Rate
The assumed breathing rate of 1.67 L/min representative of "light activity" (Table 2 p 3 of 22 of Crowley
AHE400) seems too low to apply to applicators while on foot. All of the backpack applicators were on
foot carrying a backpack with up to 3-4 gallons of water that itself would weigh in excess of 25-30
pounds. The majority of handgun applicators were on foot and manipulating heavy hoses at least part of
the time. However, even if the breathing rate while on foot were doubled, the mean inhalation UE values
for backpack applicators would be only 0.2% that of the mean dermal UE values (0.07 mg/lb ai inhaled /
31 mg/lb ai dermal); and for handgun applicators, that ratio would be 0.8% (0.014 / 1.8). Therefore,
adjusting this breathing rate upward will not have a significant effect on the overall study results.
Variability in Number of Hand Washes
The variability in the number and timing of hand washes presents a weakness in the data that was not
previously commented upon by the HSRB or others. Information about the number of wipes that were
made to collect the face/neck sample from each applicator was only descriptive: "Generally, 1-2 face/neck
wipe samples were collected for each worker then analyzed as a composite sample." (p 10 of 22 of
Crowley AHE400) However, data on the number of hand washes were provided in spread sheets, as
summarized below. As can be seen, the number of hand washes is even more variable than the number of
face/neck wipes.
# of washes
Backpack applicators
Handgun applicators
1
4
5
2
11
11
3
3
4
4
0
1
5
1
0
Unfortunately, the efficiency at which pesticides can be recovered from the skin decreases the longer that
they remain on the skin before removal is attempted. While the actual times between hand washes can
probably be obtained from the descriptive comments within the AHETF report (p. 65-174 for backpacks
and 254-369 for handguns), the average times between washes was calculated by dividing the "Dermal
Monitoring Time" by the "# Hand Washes" as listed in the spreadsheet and was found to vary from 100 to
358 minutes for backpack applicators and from 106 to 407 minutes for handgun applicators.
The variability in skin residence time adds an unknown level of uncertainty to the measured hand
exposures. The variability in residence times on the hands is more important in terms of overall
contribution to exposure, than variations in residence times on the face because measured hand exposures
for backpack applicators were on average 70 times higher than the measured face/neck exposures; those
for handgun applicators were about nine times higher. However, the importance of these variations on the
overall outcome of the study is probably not great because before a 2x method efficiency adjustment
[MEA] was applied, the hands and head together averaged only 27% of the overall dermal dose (p. 15
Crowley AHE400) and importantly, only ranged from 1.1% to 8.8% for the five backpack applicators
with the highest overall doses (see Figure la herein); the corresponding data for handguns are 30% of the
overall dermal dose and from 6.4% to 12.5% for the three handgun applicators with the highest overall
doses (see Figures 2a).
In the process of obtaining the above data, it was observed that the four highest hand exposure values (16
to 21 mg) were all among backpack applicators, although only two of the four were among those
applicators with the four highest overall UE values. A Student's t-test of the highest four backpack
applicator hand exposures (AI, A25, A23, and A16) against the other fifteen backpack applicators
(averaging 2.1 ^.g with a high of 7 (ig) yielded a p of 0.0002; a similar t-test assuming lognormal
distribution yielded a p of 0.00005, suggesting that there may have been something significantly different
-------
about the glove protection experienced by these four applicators.1 One positive implication of these high
values is that it is unlikely that there was saturation of the applicators' hands as passive dosimeters at the
thirty-six lower values, thus increasing the level of confidence in the majority of the data.
It is unknown whether the high hand exposures observed in four cases were the result of glove penetration
(meaning through openings in the glove including both the cuff and holes), permeation (basically solid-
state diffusion), or breakthrough (saturation of the glove matrix resulting in very high permeation). While
poor glove handling while doffing and re-donning contaminated gloves can result in hand exposures, it is
unlikely that much of the 16 to 21 mg of active ingredient was simply the result of mishandling.
Information provided at the meeting suggests that there was a low probability of glove permeation or
breakthrough time due to the selection of inappropriate types of gloves. If that is confirmed, then the
explanation for these high hand exposures remains unknown.
Analyses to Yield Evidence-based Conclusions
The Board concluded that the existence of high, unexplained variability, in particular the high-end
exposures, was a limitation of the study. While the Board agreed with the spirit of the investigations into
the categorical variables examined by the EPA on p. 16-20 of Crowley AHE1012 and p. 15-19 of
Crowley AHE1013 and is not recommending further formal statistical analyses, the Board believes that
different analyses might help illuminate at least some of the mechanisms underlying the high levels of
dermal exposure. The use of data or other information collected in the field to explain both the pattern of
exposures and their high variability would have several benefits. First, it could help demonstrate the
robustness of the study as a whole and provide more confidence in the representativeness of the data.
And second, it could provide the basis for evidence-based applicator training and education. The 100-to-
1 ratio of the three highest to the three lowest backpack applicator exposures should provide a strong
incentive to manufacturers, employers, and employees to believe that well-founded changes in work
practices have the potential to make large reductions in exposures.
By visual inspection, the Agency demonstrated that neither the expanded "site of application" categories,2
the three "prevalence of overhead spraying" categories, nor the use of chaps3 had a significant effect on
the overall MEA adjusted normalized total dermal exposure (MEA-nTDE in the spreadsheet) in either
group of applicators. Similarly, no effect was observable either by categorizing backpack applicators by
whether or not they conducted frill (or "hack-and-squirt") applications or by categorizing handgun
applicators by whether or not they sprayed from the vehicle, while walking, or both. However, it is
interesting (and counter intuitive if one anticipates that the use of chaps may reduce exposure) but not
statistically significant to note that the average normalized lower leg whole body dosimeters [nWBD LL
in ug/lb ai in the spreadsheet] of those seven backpack applicators who wore chaps is slightly more than
six times higher than the average of the twelve backpack applicators who didn't.
One different approach undertaken by a Board member in an attempt to gain some insight into exposure
mechanisms was to examine the distribution of the dermal doses measured on these applicators. Data
provided within the Excel spread sheets was used to construct the following four figures. Each figure
1 A similar two t-tests conducted to compare these four high hand exposures to all of the other thirty-six
applicators (including the handgun applicators who averaged 2.4 with a high of 9 ng) yielded p values of 0.0002
and 1 x 10"12, respectively, and seems to confirm the uniqueness of the four highest backpack hand exposures.
2 Student's t-test of the backpack applicator data from the three major intended categories of Distribution,
Pipeline, and Transmission versus all the other categories (drainage ditch, park, and wildlife refuge) found a p-
value of about 0.1.
3 Student's t-test of the backpack applicator data for those who used chaps versus those not using them found a p-
value of about 0.2.
-------
shows how the measured dose normalized for the AaiH for each applicator was distributed among their
lower body (comprising upper leg, lower leg, and sock dosimeters), their upper body (comprising upper
arm, lower arm, front torso, and rear torso dosimeters), their MEA hand washes, and their MEA face and
neck wipes. The sum of these four components comprises each applicator's unit exposure value [UE].
The applicators within each figure are displayed in rank-order by UE (aka MEA-nTDE within the spread
sheets). Figure la displays the data for all backpack applicators; Figure Ib displays the same data but
with the highest three doses removed and the exposure scale magnified. Figure 2a and 2b displays the
same full and truncated data for the handgun applicators.
250,000
200,000
150,000
100,000
50,000
body+ hands + head
body+ hands
I lower + upperbody
-.lower body of backpack
applicator
1 3 5 7 9 11 13 15 17 19
Figure la. Distribution of dermal doses
to all backpack applicators.
Figure 2a. Distribution of dermal doses
to all handgun applicators.
body + hands * held
body* hands
lower* upper body
lovw body o( h«ndjun
spplotor
I 3 S 7 9 11 U 15 17 19 21
25,000
5,000
body* hands* head
body* hands
lower* upperbody
lower body of backpack
= G- :=t:-
123456789 10 1112 1314 1516
.,330
53 D
body* hands* dead
body* hands
lower body of handgun
applicator
1234S67B9 10111213:415:61718
Figure Ib. Distribution of dermal doses
to the lowest 16 backpack applicators.
Figure 2b. Distribution of dermal doses
to the lowest 18 handgun applicators.
It is readily apparent from these figures that the inclusion of data from applicators with the highest doses
(visible only in the "a" versions of these figures) produces a different pattern than that seen for most
applicators (more visible in the "b" versions). While the hands are a prominent point of exposure for the
majority of both backpack and handgun applicators and the lower body exposures are often not visible in
the "b" versions, the body (and often the lower body in particular) becomes the predominant point of
exposure for those applicators with the largest doses visible in the "a" versions.
-------
The divergent exposure patterns suggest that it is unlikely that any single variable will correlate strongly
with the full array of exposures within either of these two groups of applicators. And while the challenge
of explaining the cause of the above pattern may still elude resolution from this data, one approach would
be to look for various multistep mechanisms of exposure.
For instance, it can be hypothesized that treating high or/and thick foliage might be a prerequisite to high
exposures, but only in cases in which the applicator frequently walks into or through the foliage that he
had just sprayed. Another example might be that applicators who wear chaps might correctly anticipate
encountering thick foliage, which, in conjunction with the worker practices of walking through the
sprayed foliage, will cause higher lower body exposures than applicators who don't wear them.4
A similar but as yet unenlightening variable is the spray pressure of handgun applicators. When MEA-
nTDE is plotted as a function of spray pressure (psi), the correlation is not good (p > 0.08); however, the
exposures by the two applicators with pressures in the 400 psi range (with MEA-nTDEs of 12,123 and
6,942 ug/lb) are clearly well above the bulk of handgun applicators (averaging 754 ug/lb). Again, this
single variable fails as an explanation because the exposure of the one applicator with an 800 psi system
is not at all high, and the applicator with the next (third) highest exposure was using a 45 psi system, one
of the lowest pressure systems within the study. Handgun spray pressure may be a factor but is
insufficient in itself to explain high exposures and additional factors such as occasional work practices
must play a role.
No categories of foliage or/and work practices were found in the AHETF report, and it is not clear from a
cursory review of the descriptive comments contained within it (p. 65-174 for backpacks and 254-369 for
handguns) whether sufficient information has been recorded to create such categories post-hoc. While the
Board made comments about the desirability of collecting this information in their 2010 review of the
protocol, these recommendations were not included in the Board's final report from 2010.5
The use of chaps added two complications. One (based on a response to a question from the Board) is that these
chaps could have been used on previous days and if unwashed, would have had significant carryover from those
previous uses; moreover, it seems likely (but undocumented) that those previous uses were with the same AI
(possibly even the same tank mix) as that tested. Second, if the years of experience leads an applicator to
anticipate more accurately that they will encounter high or/and thick foliage, then a third category of
"experience" could be added to the prediction; however, the small number of samples within this study are even
less expected to have the power to demonstrate the statistical significance of such higher level associations.
In October 2010, the HSRB reviewed the protocol for this research. Unfortunately, the amount of detailed
comments included within the HSRB Final Report is minimal, viz., p. 14, "However, the Board noted a few
weaknesses in the proposed study design. In particular, the variability in individual dermal and inhalation
exposure levels may be extremely high because of the diversity of terrains and locations selected for the study
and the opportunity for large (but potentially categorical) personal differences in application practices." While
weaknesses were noted, no specific changes were recommended in the HSRB's final report.
The minutes of the meeting were a bit more specific, but they are not the primary record of the Board's actions.
For example, on p. 19 of the Minutes of the October 2010 Meeting, "... ensure that the Board has an opportunity
to examine some of the pieces of information that Ms. Sherman discussed earlier: task duration records, work
pattern that the participants follow (e.g., do they advance while they are spraying or do they go out to the end of
the area and retreat; are they walking through treated foliage?); height and density of foliage; and hose length.
It would be useful to have some kind of categorical analysis because it might turn out that there are differences
between advancing and retreating application patterns." And on p. 23 of the Minutes of the October 2010
Meeting "... a tremendous amount of variability will result from this scenario because the terrain is so variable
... and therefore good notes and categorizing some of the variables will be valuable."
-------
Statistical analyses
The primary benchmark objective for each scenario in the AHETF exposure studies is to report the
geometric mean, arithmetic mean, and 95th percentile of the reference distribution of normalized
exposure. Currently these three parameter estimates are computed from a two-parameter lognormal
distribution assuming data is coming from a 1) simple random sample 2) correlated nested sample using
variance component estimates. Based on these methods, the reported back transformed geometric means
and variance adjusted arithmetic mean estimates are valid. However, the reported 95th percentile value
estimated from a two-parameter lognormal distribution geometric mean and standard deviation estimates
generally underestimates the 95th percentile upper limit. Currently advanced statistical methods are
available in UNIVARIATE procedure in SAS to estimate the 95th percentile upper limit based on three-
parameter lognormal distribution.
For simple random sample exposure data, the Board recommend using the UNIVARIATE procedure
three-parameter lognormal distribution method, estimate arithmetic mean, arithmetic standard deviation
and 95th percentile upper limit after confirming the exposure data distribution is lognormal.
For correlated nested sample data, estimate the log transformed mean and standard deviation from the
mixed model variance component method and substitute the scale and sigma in the three-parameter
lognormal distribution with these variance component values, then estimate the theta, threshold value
from the UNIVARITE procedure. Using these adjustments, estimate and report arithmetic mean,
arithmetic standard deviation and 95th percentile upper limit for correlated nested sample data using the
UNIVARIATE procedure. Furthermore, the Board recommended using these revised estimate in
computing bootstrap confidence intervals and power and sample size calculation related to AHETF
exposure studies.
Ethics
Charge to the Board
Does available information support a determination that the studies were conducted in substantial
compliance with subparts K and L of 40 CFR Part 26?
Board Response
The information provided on the Agricultural Handlers Exposure Task Force's completed study
"Backpack and Handgun Application of Liquid Spray in Utility Rights of Way (AHE400)" supports a
determination that the studies were conducted in substantial compliance with subparts K and L of 40 CFR
Part 26.
HSRB Detailed Recommendations and Rationale
As detailed above, this study monitored the exposure of trained herbicide sprayers as they applied liquid
herbicide sprays to unwanted plants in electric power line rights of way or similar outdoor areas in
scenarios that used a backpack sprayer or a handgun sprayer.
The original protocol was reviewed and approved in May 2010 by Independent Investigation Review
Board (IIRB) of Plantation, Florida; that in October 2012 IIRB became part of Schulman Associates IRB
(SAIRB), an AAHRP-accredited IRB registered with Office of Human Research Protections (OHRP).
The protocol and IRB documents were reviewed by the HSRB in October 2010. At that time, the Board
concluded that, pending modifications recommended by EPA, minor revisions to the informed consent
document, and translation of the informed consent materials into Spanish, the protocol was likely to meet
-------
the applicable requirements of 40 CFR 26, subparts K and L. OPP confirmed that these modifications
were made prior to the initiation of the study, and documents provided by AHETF for the HSRB's current
review also reflect the requested modifications.
Recruitment began in February 2011. Beginning in May 2011, the study enrolled 40 participants.
Researchers conducted 19 backpack sprayer monitoring units (MUs) and 21 handgun sprayer MUs in 12
states. Although the protocol was written to recruit both men and women, no women were enrolled and
only men participated in the study. Participants ranged from 19 to 68 years of age; no children or minors
were enrolled. Documentation from AHETF indicates that all participants received approved information
about the nature of the study, its risks and procedures to minimize risk, and that these disclosures were
provided in either of English or Spanish, depending on participant's choice, both orally and in writing.
All participants signed approved informed consent documents; one Spanish-speaking participant with
self-reported low literacy skills had the document read aloud to him. Participants received a total
compensation of $100 for their time and participation in the study. The study closed in September 2013.
Over the period of the study there were eight approved protocol amendments, which added study sites,
added certain items from participants' own personal protective equipment to the list acceptable/required
PPE; expanded the list of prescribed modified analytic methods; and expanded the list of approved
analytic laboratories and laboratory personnel. These amendments had no discernible effect on the
study's ethical characteristics.
Over the period of the study there were also five reported protocol deviations. One was administrative;
others involved incomplete collection of samples for analysis; use of alternative analytic methods not
previously listed in the protocol; monitoring periods that were shorter than times prescribed in the
protocol, including episodes in which stopping rules were invoked due to the heat index; failure of
monitoring equipment; and individual participants' actions that went against protocol and that were not
recognized by study personnel in time to be prevented. Of these protocol deviations, the only event that
appears to have any potential effect on the study's ethical protections was the potentially increased risk of
exposure incurred by a participant who smoked a cigarette during the monitoring period without first
having his hands cleaned by study personnel.
Except as noted below, the board agreed with the conclusions and observations EPA's Ethics Review
(Sherman 2015), and found that the study substantially meets the provisions of subparts K and L of 40
CFR Part 26. Specifically aspects are discussed below.
Acceptable risk-benefit ratio
Identified risks were appropriately minimized, particularly the risk of heat-related stress and
illness. There were no adverse effects reported and no withdrawals from the study. Stopping
rules for heat stress were invoked on five occasions. Following the stopping rules should not
have been considered "protocol deviations" because the rules were built into the protocol as for
the protection of participants. No other incidents were reported during monitoring.
The degree of potentially increased risk harm incurred by one participant who smoked a
cigarette without having his hands cleaned by study personnel is unknown. The potentially
increased risk of harm that accompanied periods of non-monitoring (from the intentional actions
of participants and the inadvertent failure of a personal air pump) is unknown.
10
-------
Voluntary and informed consent of all participants
All participants were provided information on the study and signed a consent document in their
choice of English or Spanish. One Spanish-only speaking participant with self-reported low
literacy skills had the Spanish-language consent form read aloud to him by a bilingual member of
the study staff.
There no were withdrawals from the study.
Participants received compensation of $ 100 ($20 for taking part in the recruitment interview and
$80 for participating in monitored spraying). This amount was not so high as to create an undue
inducement to participate, and was sufficient to justly compensate participants for their time and
inconvenience.
Equitable selection of study participants
The study was designed to recruit an appropriately diverse population representing the trained
herbicide/pesticide sprayers in each area and to minimize selection bias. A 3-phase recruitment
process was used to identify qualifying employment sites and to minimize the risk that employers
might coerce their workers either to participate or not participate.
EPA notes that "statistical inference requires assumption that the sample is representative of all
U.S. backpack and handgun applicator scenarios in applicable areas" (Crowley 2015- Slide 63,
Conclusions). Although the study protocol was written to include both male and female
participants, and had appropriate measures in place to exclude pregnant and lactating women,
ultimately the enrolled participants were exclusively male, presumably due to the composition of
the applicator workforce being almost exclusively male. The questions of whether and how this
sample is representative of exposure for female workers remains unanswered and may need to be
revisited in the future, especially if changing labor trends bring more women into this line of
work.
HSRB Review of the Protocol for Field Testing of Skin Applied Mosquito Repellent Products (S.C.
Johnson)
Overview of the protocol
This protocol outlines a study to be conducted with human volunteers using EPA-registered personal
mosquito repellent products produced by S.C. Johnson & Son, Inc. ("S.C. Johnson") in field tests against
populations of wild mosquitoes. The purpose of the study is to determine complete protection time (CPT)
of each product for inclusion on EPA's new Repellency Awareness Graphic, to be used on consumer
product labels. The study will be conducted at two sites within the U.S., in Wisconsin and Florida, and
possibly at additional locations outside the U.S. if needed. Up to 18 EPA-registered repellent products
may be included in the tests.
Human volunteers will have products applied to either arms or legs by study staff, and a standard volume
of product will be used, whether aerosol, spray, or lotion. For each product, there will be 10 human
subjects at each site using the product for a total of 20 replicates per product. CPT will be calculated by
measuring the time from product application to first confirmed landing of mosquitoes, and estimating
median CPT for each product at each site, using a Kaplan-Meier estimator. The lowest median CPT from
the test locations will be used for the repellency graphic.
11
-------
Science
Charge to the Board
Is the protocol "Field Testing of S.C. Johnson Personal Mosquito Repellent Products to Support their Use
of the EPA Repellency Awareness Graphic" likely to generate scientifically reliable data, useful for
estimating the complete protection time of various EPA-registered S.C. Johnson skin-applied mosquito
repellents in the field against wild adult mosquito populations?
Board Response
The Board concluded that the protocol titled "Field Testing of S.C. Johnson Personal Mosquito Repellent
Products to Support their Use of the EPA Repellency Awareness Graphic", if modified according to
Agency and HSRB recommendations, is likely to generate scientifically reliable data, useful for
estimating the complete protection time as defined in the protocol, of various EPA-registered S.C.
Johnson skin-applied mosquito repellents in the field against wild adult mosquito populations.
HSRB Detailed Recommendations and Rationale
The objective of the S.C. Johnson protocol is clearly stated and follows EPA's Product Performance Test
Guidelines OPPTS 810.3700: Insect Repellents to be applied to Human Skin. The field tests are designed
to provide data to support the EPA's Repellency Awareness Graphic. The Board approved of the protocol
team's responses to earlier Board and Agency recommendations. The Board recognizes that there is a
long history of insect repellant efficacy studies and applauds the efforts to bring this information and
experience into a metric or label that better protects the population by clearly informing consumers about
product efficacy. The Board concurs with the Agency recommendations except where noted below. We
also identify several opportunities to improve the quality of complete protection data in support of the
Agency graphic or to improve clarity of the protocol.
Product application rate
The protocol proposes using a standardized dose of 1 gram product per 600 cm2 treated skin for all
products (lotions, aerosols and pump sprays). The main reason given for using a standardized dose is to
reduce variability in the results that would come from applying different amounts of products. The
Agency did not agree with this approach and recommended changing the protocol to include product
specific doses based on previous dosimetry studies of consumer behavior. Public comment supported the
use of the single standardized dose recommended by S.C.J.
The Board understands that the data from this protocol will be used to calculate median CPT values
across all subjects from each of two sites rounded down to the nearest integer and the lowest value will be
used for the product graphic, (p. 11 of 39, Sweeney and Sherman). Given the stated use of the data, the
Board agrees with the use of a standardized dose of 1 gram product per 600 cm2 treated skin. However,
we strongly suggest changing the language from standardized dose to standardized application rate and
reserve "dose" to describe how much active ingredient is applied.
The protocol specifies that the actual amount of product applied will be recorded. The percent activity
ingredient in each product is also available so the Agency can in fact assess the relationship between dose
(active ingredient) and efficacy (or CPT) as a quality assurance check of the data. An assessment of the
relationship between CPT and dose (mass active ingredient per treated area) can provide an indication of
the quality of the data or point out data that might be suspect because it does not fall along the expected
dose-response trend. The Board recommends that the data analysis include a dose-response comparison
12
-------
for all products where multiple concentrations are available (e.g., DEBT content in the different products
ranges from 5% to 98.25%) to help assess data quality. In addition, the Agency might consider
normalizing the CPT results to better represent the expected application rates derived from earlier
dosimetry studies when calculating the final graphic number.
Product application method
The protocol proposes to use a variety of application methods including using pipettes for transferring
liquid contents from pump sprays and spatulas for lotions, while aerosol sprays are applied directly. In
each case, the product is applied and spread on the subject skin by a staff member. The Board was
concerned that these application methods were not representative of actual application methods but
ultimately concluded that the need for consistency outweighs the need for the protocol to be
representative of consumer behavior. However, the Board stresses the importance of accurately reporting
the application rate (mass of product per area of skin) for each subject.
A particular concern with the aerosol application method was that the iterative procedure leaves open the
possibility to repeatedly apply more than the target 1 gram. Some means should be integrated within the
protocol to limit or preclude the potential to bias the average dose upward. One (and perhaps not the best)
way to preclude such a bias would be to place some upper bound on the highest level above 1 gram that
would be allowed to proceed to the field testing phase. Such an upper bound (or similar restriction) could
be applied to all application methods, but over-exposures seem most likely to occur with the aerosol
application method.
Use of sites outside the U.S.
The protocol specifies the use of two established and ecologically distinct field sites in the United States
for testing, and the Board agrees that this will provide sufficient representation for determining CPT, but
the Board shares the concerns expressed by the Agency about using sites located outside of the U.S. The
study sponsor has described that in addition to the two established sites in the U.S., there is at least one
established site in Australia that could be used. The protocol needs to provide more information on what
constitutes an "established site" either in the U.S. or another country (i.e., climate, mosquito species
present, other hazards such as other mosquito borne diseases, presence of cell phone service,
representativeness of local demographics) and more importantly should describe how the data collected at
the alternate sites outside the U.S. will be related to the U.S. consumer demographics and the expected
mosquito populations in the U.S.
Potential for cross contamination
There are a number of places in the protocol that provide opportunities for insecticide to be inadvertently
either lost or gained from/to the treated area6 on subjects. Simple precautions can be taken to alleviate this
issue, but the protocol should specify steps that will be taken to insure that the treated area on subjects is
not impacted by activities that take place before or during the experiment (i.e., rubbing sleeve or pant leg
across the treated area).
Potential for "carryover"
No justification was provided for the adequacy of separating multiple participations by any test subject by
a minimum of one day (Science Response #6 in S.C. Johnson letter of 17 April 2015). It is important to
6 Section 10.5.1 indicates that subjects might be transported by vehicle after being treated providing opportunity for
treatment to be wiped off or transferred to another subject; Section 2.8.2.6 describes a screened enclosure that may
or may not be treated; Section 12.1.1 suggests that subjects might be used on multiple occasions where carryover of
insecticide might be an issue.
13
-------
verify that no carryover effect is present on subjects used on multiple days. The protocol suggests that a
day between treatments will be sufficient when the same subject is used a second time, but justification or
references are needed to support this. If a subject is treated with 98.25% DEBT, is there any residual
effect after 24 hours that might affect a low dose treatment (application of 5.6% DEBT wipe)?
Landing pressure
The protocol includes untreated control subjects with each test to insure that there is sufficient landing
pressure to provide valid results. However, the landing pressure is not measured in quantitative terms,
only whether it is sufficient or not (five landings in five minutes). Discussions during the meeting seemed
to imply that landing pressure will influence the measured CPT.7 If in fact the landing pressure can
influence the resulting median CPT and products tested on different days are subjected to different
landing pressures, then it would be important to collect quantitative information on landing pressure that
could be used to correct, normalize, or at least interpret the resulting CPT values. The Board recommends
that the Agency and S.C. Johnson consider how a quantitative estimate of landing pressure can be
determined without increasing the likelihood of bites if landing pressure is excessive (e.g., recording the
time of each landing, the time to reach 5 landings, or the total landings in 5 minutes) and how that
information can be used to normalize or interpret CPTs measured under different landing pressure
conditions.
Delayed start
The Board recognizes the advantages of delaying the exposure to mosquitoes for subjects treated with
products that are known from previous experience to last for a long time. However, the protocol needs to
provide more information about the criteria used to determine how long to wait before starting the test
cycles (5 minute exposure at 30 minute intervals). Regardless of how long the subject's exposure is
delayed, the protocol should require a minimum number of completed cycles to insure valid results. For
example, following a delayed exposure, the subject should complete at least three exposure cycles before
getting a confirmed landing.
Representativeness of the sample population
The Agency should resolve the perceived discrepancy between the "representativeness of the sample
population" goals of the study. The study sponsor states that participants are representative of the
population in the area where the field testing will be conducted, while the Agency seems to want the
sample to represent the population of potential repellent users in the United States, (p. 3 and 23, K.
Sweeny and K. Sherman, Science and Ethics Review, 31 Mar 2015). The Agency made a request of the
applicant to provide additional details regarding the demographics and ethnicity of the population in the
area. (p. 4 ibid) The response by S.C. Johnson was to agree to document that the demographics of the
"pool of potential volunteers ... represents the demographics of the areas that testing will occur" and to
add a criterion question that requires that each participant is an insect repellent user (Science Response #2
in S.C. Johnson letter of 17 April 2015). Although it could turn out that the composite distribution of two
7 An unrecognized risk within any protocol based on confirmed events is the impact of falsely confirmed events
created when two otherwise random events occur within 30 minutes of each other. Each falsely confirmed event
will produce an artificially short CPT for a given individual and contribute to a shorter median CPT than would have
occurred without that event, but their major adverse impact is to decrease the reproducibility of median CPT values
when measured across multiple tests. Statistical analyses indicate that CPT test results with a product that has a low
protection factor within its CPT will be less reproducible than with a product that has a high protection factor. And
a repellent's effectiveness within its CPT can be expected to depend upon the active ingredient's intrinsic
effectiveness, the AI's concentration, and the insect landing pressure measured by the untreated controls.
14
-------
test areas may approximate the population of potential repellent users in the United States, the essential
criteria are not stated within the protocol and the discrepancy appears unresolved.
Experimental design
The design as presented tests all ten subjects assigned to a product on a single day. The downside of this
design is that it does not allow results to be easily generalized to a range of environmental conditions that
may affect the attractiveness of a subject (e.g., sweating due to temperature and humidity levels). An
alternative would be to test each product on several days; e.g., five subjects on each of two days or three
subjects on each of two days and the remaining four subjects on a third day. Each day would form a block
for the analysis of that product's data. Such a design would allow testing of multiple products on a given
day. If it were of interest to compare product formulations and/or application methods, the combined data
could be analyzed as a block design with multiple replications of each product within each block (day).
These alternative designs that utilize blocking (e.g., by individual test subject) could be considered to
account for known sources of variation (e.g., individual effects).
Randomization
The randomization mechanism should be described in more detail and rationale should be given for any
given choice of randomization within the protocol. For example, it is not clear whether/how cross-
substance relations are to be evaluated in the data analysis and why randomization among test substances
is needed. An explanation of this would be helpful. In addition, when the conditions support use of arm
rather than leg for exposure, then it may be more important to consider handedness when selecting what
arm to treat, rather than randomly assigning to left or right hand, so the subject can have their dominant
hand to remove landing mosquitoes before they bite.
Sample size determination
A sample size calculation would be useful here to inform the power of testing and the width of confidence
intervals. Power and sample size calculation can be implemented using existing SAS procedures.
Information about appropriate sample size calculations is included in the EPA document "Product
performance Test Guidelines OPPTS810.3700: Insect Repellents to be Applied to Human Skin."
Sources of variation
Multiple sources of variation including, for example, site selection, treatment dosage, application rate,
mosquito type/age/condition, and landing pressure can impact the results. For the most part, however,
they are not accounted for explicitly within the study, and when the source of variation is not controlled
(e.g. as it is using a standard application rate) then the contribution to variance should be acknowledged
or discussed. The protocol does not currently specify the conditions that might cause the CPT data from
the two sites to differ; however, the researchers should consider collecting information to explain any
large and potentially significant differences in the CPT values between otherwise matched studies
conducted at two different sites.
Ethics
Charge to the Board
Is the research described in the protocol "Field Testing of S.C. Johnson Personal Mosquito Repellent
Products to Support their Use of the EPA Repellency Awareness Graphic" likely to meet the applicable
requirements of 40 CFR part 26, subparts K and L?
15
-------
HSRB Recommendation
The study, when modified as recommended by the HSRB in this report, is likely to meet the applicable
requirements of 40 CFR part 26, subparts K and L.8
HSRB Detailed Recommendations and Rationale
This is a protocol for third-party research involving intentional exposure of human subjects to a pesticide.
As described above, the study involves use of consumer products for mosquito repellency for defined
periods of time with exposure to wild populations of mosquitoes for the purpose of determining Complete
Protection Time (CPT) as defined in EPA guidance. These data will be used for product labeling to better
inform consumers about relative protection times of different products.
The following aspects of the protocol were deemed acceptable by the HSRB:
The protocol adequately describes the consent process to be used for subjects enrolling in the
study and a form has been conditionally approved by SAIRB.
The research proposal describes risks and procedures to minimize risk to human subjects. Risk
minimization procedures include, training sessions for study participants, field stopping rules to
avoid excessive heat exposure when temperatures reach 105 degrees, providing hydration/shade
for study subjects, first-aid access, and 24 hour call number in the event of any health-related
consequences of study participation. The study has been reviewed and conditionally approved by
SAIRB, which will presumably provide oversight throughout the study.
40 CFR 26 Subpart Q, at §26.1703, states that EPA must not rely on data from any research
involving intentional exposure of any human subject who is a pregnant woman (and therefore her
fetus), a nursing woman, or a child. This protocol appropriately requires that subjects be at least
18 years old and excludes female subjects who are pregnant or lactating.
Before the research is initiated, the documents should be revised to address the comments from the
EPA and the following concerns of the Human Studies Review Board:
The protocol should discuss whether the number of repeat tests per person should be limited.
There should be some plan to follow-up with subjects after their participation in the study has
ended to check for any delayed consequences of study participation. This could consist of a
phone call to check on health status.
The protocol team should provide a process to contact subjects in the unlikely event that new
information is developed or discovered as a result of the study, for example, if mosquitoes were
discovered with vector-borne disease or if a contaminated product was identified.
The protocol team should provide a justification, in the protocol or IRB documents, for excluding
non-English speakers, especially at the Florida site where Spanish-speakers are a significant
proportion of the population; or if no justification for exclusion is provided, the protocol should
be amended to include Spanish speakers. In order to assess the representativeness of the study
population, the protocol team should provide to EPA general information on population
demographics of subject pool at each site, and some information about recruitment strategies.
This is particularly important if alternate sites located outside of the U.S. are used.
8 One member of the board abstained from the recommendation due to concerns about the methodology used in
dermal toxicity studies that are used to determine the margins of exposure (MOEs) for the active ingredients in
repellent products.
16
-------
The Board discussed two additional aspects of the study that could benefit from clarification. The Board
expressed concern regarding the statement in the consent document that care and compensation for injury
would be handled similar to a workers compensation claim. It was not clear if this was an appropriate
mechanism for care and compensation in a research study. The consent form states "This study is
considered confidential." The Board questioned whether this is intended to refer to data collected on
individual subjects or whether the details of the study should not be disclosed or discussed with others.
Conclusion
This study is intended to be conducted in substantial compliance with the requirements of 40 CFR 26
subparts K and L. The amended protocol, when approved by SAIRB, should met all applicable ethical
standards for the protection of human subjects of research, and all requirements for documentation of
ethical conduct of the research. If this study is determined to be scientifically valid and relevant, there is
no regulatory barrier to EPA's reliance on it in actions under FIFRA or §408 of FFDCA.
O J o
17
-------
REFERENCES
Crowley, M. (2015) Science Review of Completed AHETF Worker Exposure Monitoring Study:
Backpack and Handgun Application of Liquid Spray in Utilities Rights-of-Way (AFIE400).
Washington, DC: U.S. Environmental Protection Agency, Office of Pesticide Programs.
Sherman, K. (2015). Ethics Review of Completed AFfETF Worker Exposure Monitoring Study:
Backpack and Handgun Application of Liquid Spray in Utilities Rights-of-Way (AHE400).
Washington, DC: U.S. Environmental Protection Agency, Office of Pesticide Programs.
Sweeney, K. and K. Sherman (2015) Science and Ethics Review of a Protocol for Field Testing of
S.C. Johnson Skin-Applied Mosquito Repellent Products. Washington, DC: U.S. Environmental
Protection Agency, Office of Pesticide Programs.
U.S. Environmental Protection Agency (EPA). (2010, July 1). 40 CFRpart 26. Code of Federal
Regulations. Washington, DC: U.S. Government Printing Office. Retrieved June 16th, 2016 from
http://www.gpo.gov/fdsvs/pkg/CFR-2010-title40-voll/xml/CFR-2010-title40-voll-part26.xml
U.S. Environmental Protection Agency (EPA). (2010, July 7). Product Performance Test Guidelines:
OPPTS 810.3700: Insect Repellents to be Applied to Human Skin [EPA 712-C-10-001] from
http://www.regulations.gov/#!documentDetail:D=EPA-HO-OPPT-2009-0150-0011. Retrieved June 16th,
2015.
U.S. Environmental Protection Agency (EPA) Human Studies Review Board (HSRB) Meeting Report,
October 27-28, 2010. Retrieved from http://epa.gov/osainter/hsrb/files/meeting-materials/oct-27-28-
2010-meeting/final-oct2010-mtg-report.pdf on June 16th, 2015.
U.S. Environmental Protection Agency (EPA) Human Studies Review Board (HSRB) Meeting Minutes,
October 27-28, 2010. Retrieved fromhttp://epa.gov/osainter/hsrb/files/meeting-materials/oct-27-28-2010-
meeting/oct2010-final-min.pdf on June 16th, 2015.
18
-------
NOTICE
This report has been written as part of the activities of the EPA Human Studies Review Board, a Federal
advisory committee providing advice, information and recommendations on issues related to scientific
and ethical aspects of human subjects research. This report has not been reviewed for approval by the
Agency and, hence, the contents of this report do not necessarily represent the view and policies of the
Environmental Protection Agency, nor of other agencies in the Executive Branch of the Federal
government, nor does the mention of trade names or commercial products constitute a recommendation
for use. You may obtain further information about the EPA Human Studies Review Board from its
website at http://www.epa.gov/osa/hsrb. You may also contact the HSRB Designated Federal Officer, via
e-mail at ord-osa-hsrb@epa.gov
In preparing this document, the Board carefully considered all information provided and presented by the
Agency presenters, as well as information presented by public commenters. This document addresses the
information provided and presented within the structure of the charge by the Agency.
19
-------
U.S. ENVIRONMENTAL PROTECTION AGENCY
HUMAN STUDIES REVIEW BOARD
Chair
Liza Dawson, Ph.D., Research Ethics Team Leader, Division of AIDS National Institutes of Health (NIH), National
Institute of Allergy and Infectious Disease (MAID), Bethesda, MD
Vice Chair
Edward Gbur, Jr., Ph.D., Professor, Agricultural Statistics Laboratory, University of Arkansas,
Fayetteville, AR
Members
Gary L. Chadwick, Pharm.D., M.P.H., C.I.P., Senior Consultant, HRP Consulting Group, Inc. Fairport, NY
George Fernandez, Ph.D., Statistical Training Specialist, SAS Institute, Statistical Training and Technical Services,
Sparks, NV
Kyle L. Galbraith, Ph.D., Manager, Human Subjects Protection, Carle Foundation Hospital,
Urbana, IL
Jewell H. Halanych, M.D., Assistant Professor, Internal Medicine Residency Program, Montgomery Regional
Campus, University of Alabama at Birmingham, Montgomery, AL
Elizabeth Heitman, Ph.D., Associate Professor of Medical Ethics, Center for Biomedical Bioethics and Society,
Vanderbilt University Medical Center, Nashville, TN
John C. Kissel, Ph.D., Professor, Department of Environmental and Occupational Health, School of Public Health,
University of Washington, Seattle, WA
Randy Maddalena, Ph.D., Physical Research Scientist, Indoor Environment, Lawrence Berkeley National
Laboratory, Berkeley, CA
William Popendorf, Ph.D., M.P.H., Professor Emeritus, Department of Biology, Utah State University, Logan, UT
Kenneth Ramos, M.D., Ph.D., PharmB, Associate Vice President, Precision Health Sciences, Professor of Medicine,
Arizona Health Sciences Center, Tucson, AZ
Suzanne M. Rivera, Ph.D., M.S.W., Associate Vice President for Research, Case Western University, Cleveland
Heights, OH
Jun Zhu, Ph.D., Professor of Statistics and of Entomology, Department of Statistics, University of Wisconsin -
Madison, Madison, WI
Human Studies Review Board Staff
Jim Downing, Executive Director, Human Studies Review Board Staff, Office of the Science Advisor, United States
Environmental Protection Agency, Washington, DC
20
------- |