Report on the Workshop to Discuss
State-of-the-Science Approaches for
Observational Exposure Measurement Studies
Durham, NC
November 28-29, 2006
Submitted to:
Human Exposure and Atmospheric Sciences Division
National Exposure Research Laboratory
U.S. Environmental Protection Agency
Research Triangle Park, NC 27709
Submitted by:
Eastern Research Group, Inc.
110 Hartwell Avenue
Lexington, MA 02421
FINAL REPORT: January 25, 2007
-------
Notice
This report was prepared by Eastern Research Group, Inc. (ERG), an EPA contractor, as a
general record of discussion during the Workshop to Discuss State-of-the-Science Approaches
for Observational Exposure Measurement Studies, held November 28-29, 2006, in Durham,
North Carolina. This report captures the main points and highlights of the meeting. It is not a
complete record of all details discussed, nor does it embellish, interpret, or enlarge upon matters
that were incomplete or unclear. Statements represent the individual views of meeting
participants.
-------
Contents
1. Introduction 1
2. Opening Session 1
3. Elements to be Considered in Study Conceptualization 5
4. Ensuring Protection of Vulnerable Groups 10
5. Addressing Privacy and Other Concerns Related to Personal Exposure Observational Studies
13
6. Creating an Appropriate Relationship Between Participant and Investigator 16
7. Building and Maintaining Appropriate Community and Stakeholder Relationships 21
8. Designing and Implementing Strategies for Effective Communication 27
9. Document Title and Other Elements 34
10. Information Resources 36
11. Observer Comments 42
Appendix A: Panelists A-l
Appendix B: Charge to Panelists B-l
Appendix C: Panelists' Pre-Meeting Input C-l
Appendix D: Meeting Agenda D-l
Appendix E: Meeting Attendees E-l
Appendix F: Presentation by Linda Sheldon, EPA F-l
Appendix G: Presentation by Roy Fortmann, EPA G-l
Appendix H: Presentation by Tim Buckley, Panel Chair H-l
Appendix I: Topics and Subtopics Discussed by the Panel 1-1
i
-------
ii
-------
1. Introduction
Researchers conduct observational human exposure studies to measure human exposure to
chemicals in the environment. The U.S. Environmental Protection Agency's (EPA's) National
Exposure Research Laboratory (NERL) has used these studies for several decades in order to
gain understanding about exposures and ultimately to support efforts to improve public health.
Because they deal with human subjects, however, these studies are complex, with numerous
scientific and ethical issues that must be addressed during design and implementation. To ensure
that EPA's research continues to be based on the most up-to-date science and the highest ethical
standards, the Agency plans to develop a document that will compile "state-of-the-science"
approaches for conducting observational exposure studies.
On November 28 and 29, 2006, Eastern Research Group, Inc. (ERG), an EPA contractor,
convened an independent panel of eleven experts (Appendix A) in a peer consultation workshop
to provide input on the development and content of the Agency's proposed state-of-the-science
document. This panel workshop brought together nationally recognized researchers, ethicists,
medical practitioners, and community leaders, providing a broad range of expertise on the
scientific and ethical dimensions of observational exposure studies.
In its charge to the panelists (Appendix B), EPA asked them to provide recommendations on
how to structure the document, what topics to address, and what information sources to consult.
EPA also asked the panel to identify elements of observational studies for which the state of the
science is most uncertain, and to provide recommendations on approaches for these elements.
Prior to the meeting, several panelists submitted potential discussion topics to the panel chair.
These preliminary suggestions are provided in Appendix C.
Appendix D contains the agenda for the workshop, and Appendix E provides a list of those who
attended the meeting as observers. A phone line was available for any observers who wished to
attend remotely. The workshop agenda included two opportunities for observer comment.
This report summarizes the presentations and panel discussions that took place in plenary
session. For organizational efficiency, this report groups many of the discussions by topic, rather
than strictly chronologically.
2. Opening Session
Jan Connery of ERG opened the workshop by welcoming the panelists and observers. She asked
panel members to introduce themselves and then reviewed the meeting agenda.
Connery introduced Kevin Teichman, Acting Deputy Assistant Administrator for Science at
EPA's Office of Research and Development, who thanked the panelists and observers for
attending the meeting. Teichman explained that observational human exposure studies are
critical to EPA's efforts to understand exposure.
1
-------
Additional remarks were provided by Warren Lux, the Human Subjects Research Review
Official in EPA's Office of the Science Advisor. Lux pointed out that EPA not only conducts its
own observational exposure studies, but also relies on studies conducted by other parties—some
of whom are funded by EPA—as a basis for certain Agency decisions. Noting the importance of
applying consistent standards to this research, he outlined his vision for the future of EPA's
observational exposure studies program: a human research program driven first and foremost by
an ethical framework that can serve as a basis for regulations, with venues for investigators to
discuss issues and adapt to changing science. He encouraged the panel to consider these issues,
noting that he looked forward to receiving their recommendations and seeing to it that these
recommendations are implemented.
Following the opening remarks, Linda Sheldon, the Acting Director of NERL's Human
Exposure and Atmospheric Sciences Division (HEASD), presented an overview of human
observational studies, which EPA has used for more than 30 years (see Appendix F for her
presentation slides). She noted that EPA's mission is to protect public health, which requires
understanding of risks to health. Exposure—that is, coming in contact with a chemical—is one
half of the risk assessment process; thus it is critical that EPA understand and quantify how
exposure actually occurs.
In observational exposure measurement studies, EPA observes people's contact with chemicals
under normal day-to-day conditions, collecting samples of environmental media (e.g., air, dust,
water) and keeping a record of subjects' activities (e.g., through videotaping or self-reporting
diaries). Sheldon presented examples of important observational exposure studies conducted in
the past, including EPA's Total Exposure Assessment Methodology (TEAM) studies—which
demonstrated the significance of indoor exposures. Other examples include radon studies, which
led to increased awareness and development of methods to reduce exposure, and PM2.5 studies
that demonstrated the association between ambient concentrations and exposure, ultimately
leading to a new National Ambient Air Quality Standard (NAAQS). Sheldon also outlined
important considerations for future studies, including the need to adhere to the highest and most
up-to-date ethical and scientific standards, to avoid encouraging subjects to change their
behavior in ways that would increase their exposure, and to work with communities to address
their concerns.
Roy Fortmann (NERL/HEASD) explained the purpose of the workshop, as well as EPA's overall
goals in developing the state-of-the-science document (see Appendix G for his presentation
slides). To ensure that EPA's observational exposure studies continue to meet the highest
scientific and ethical standards, it is important to follow the most up-to-date methods, standards,
and approaches in study design and implementation. In the document that EPA will develop, the
Agency will aim to identify issues that researchers need to consider, compile and evaluate
information on state-of-the-science approaches (particularly for ethical considerations), and
present this information in a peer-reviewed form for researchers inside and outside EPA.
Observational exposure studies involve complex scientific and ethical considerations, Fortmann
explained—particularly those related to the protection of study participants. He listed several key
study elements, including researcher responsibilities during design and implementation,
communication, balancing risks and benefits, identifying and working with stakeholders, and
involving the community throughout the process (design, review, during the study, etc.). He also
2
-------
noted that several information sources are available, including existing reports, policies, and
guidelines; peer-reviewed literature; and case studies.
Fortmann reviewed the charge to panelists and provided a tentative schedule for producing the
document after the workshop. Using the workshop report, EPA intends to prepare the document
for external peer review and public comment in May 2007 and for release in July 2007. EPA will
give the panelists an opportunity to review a draft of the document during the spring of 2007.
In response to questions from panelists, Fortmann provided the following information about the
purpose and scope of the project:
• Although a portion of the document will be geared to a public audience, the main
purpose—and the focus of this panel workshop—should be to help researchers.
• Because every study has its own unique set of considerations, EPA envisions this
document not as a specific "cookbook" for designing a study, but more as a general
resource outlining issues that researchers should address. The document can refer the
reader to other sources for more specific information.
• The document should consider science and ethics, including issues about community
engagement and considerations for special populations. It should remain broad,
addressing observational exposure studies in general, not just studies of children or other
sensitive groups. It does not need to discuss state-of-the-science for specific measurement
techniques.
• This project is motivated in part by some of the perceived concerns surrounding NERL's
Children's Environmental Exposure Research Study (CHEERS), which was suspended in
2004 and cancelled in 2005. The "state-of-the-science" document will be a first step in
moving EPA's research program forward again. However, this document is not intended
as a critical review of CHEERS.
• The panel should provide recommendations about content and how to frame the
document so it is most useful to the intended audience. The panel can also recommend a
title.
Following EPA's opening remarks, Connery facilitated the first public comment session (see
Section 11, "Observer Comments").
Following the public comment, panel chair Tim Buckley presented an overview of the charge
and the meeting format (Appendix H provides his presentation slides). He then led the panel in
determining topic areas to focus their subsequent discussions. Based on EPA's charge and on
3
-------
pre-meeting suggestions from panelists, the panel agreed that EPA's state-of-the-science
document should address the following six major topic areas:1
• Elements to be considered in study conceptualization
• Ensuring protection of vulnerable groups
• Addressing privacy and other concerns related to personal exposure observational studies
• Creating an appropriate relationship between participant and investigator
• Building and maintaining appropriate community and stakeholder relationships
• Designing and implementing strategies for effective communication
After agreeing on the list of topics, panelists developed a preliminary list of subtopics to guide
discussion in each area. Members of the panel also suggested some common themes to consider
throughout their discussions. For example, one panelist observed that all of the topic areas
require some degree of balance between scientific benefits and risks to the subjects and the
community. Panelists also suggested that EPA's document present issues for researchers to
consider before designing a study. One panelist suggested that when establishing a sound
scientific baseline for environmental decisions, deeper philosophical questions arise. What
degree of environmental safety is necessary? How much scientific evidence is needed to support
the decisions? Before researchers develop a study, they should consider what evidence is needed
and what degree of study is justified.
Panelists also noted that EPA has a complicated role in observational exposure studies because
the Agency essentially "wears two hats"—it is responsible for both research and regulation. As
one panelist observed, there is a tension between pure research (i.e., the best scientific evidence)
and constraints upon actually using that evidence to develop regulations (e.g., feasibility).
NIOSH and OSHA do not have this problem because their process is transparent; NIOSH
publishes a standard based on the scientific evidence and OSHA promulgates a standard that
considers feasibility. Ideas for reducing this conflict included having ORD publish a standard
based entirely on the scientific evidence and then the EPA administrator promulgating a standard
that considers feasibility, or simply recommending that researchers state up front that scientific
results do not always translate directly into policy.2
1 The panel recommended that EPA's document cover each topic area as a separate chapter. Titles for these topic
areas evolved over the course of the meeting. For consistency, this report refers to the six topic areas using the final
chapter names agreed to by the panel at the end of the meeting.
2 In post-meeting comments, one panelist (Michael Lebowitz) stated that EPA does have defined pathways for
translating research into regulation. For example, observational studies inform the NAAQS through scientific
criteria documents, the Science Advisory Board, and then a staff paper that goes to OMB for a review of economics
and feasibility. Ultimately, all the information is presented to the Administrator, who legally makes any changes in
the NAAQS.
4
-------
Panelists then began discussions in the six topic areas.
For the first four topics, the panel split into two
breakout groups (Exhibit 1), with each group separately
covering two of the four topics. Breakout group
rapporteurs presented summaries of these discussions
to the full panel in plenary session. The last two topic
areas were discussed by the full panel on the second
day of the workshop.
After lunch on the second day, Connery facilitated a
second public comment session. The panel then began
their final discussion session, during which they
combined the topics and subtopics from their prior
discussions into a single master document, provided in
Appendix I. The panel also discussed the title, chapter
names, and other elements EPA should consider
including in the state-of-the-science document.
Exhibit 1. Breakout Groups
Breakout Group A
- Bruce Lanphear, Chair
- David Carpenter, Rapporteur
- Giselle Corbie-Smith
- Natalie Freeman
- Jerry Menikoff
Breakout Group B
- Michael Lebowitz, Chair
- Rebecca Parkin, Rapporteur
- Sophie Balk
- Alan Fleischman
- Loretta Jones
The panel chair (Tim Buckley)
rotated between the two breakout
groups.
The remainder of this report provides an overview of
all discussions that took place in plenary session,
including results from each of the breakout groups as
reported back to the full panel. For organizational
efficiency, this report groups discussions by topic,
rather than strictly chronologically. Sections 3 through 8 correspond to the six key discussion
topics listed above, while Section 9 documents the panel's discussions about the title and other
elements of the document. Sections 10 and 11 cover suggested information resources and
observer comments, respectively.
3. Elements to Be Considered in Study Conceptualization
This topic area was discussed initially by the full panel in the opening plenary session, and then
discussed in detail by the members of Breakout Group A (see Exhibit 1).
Preliminary Comments
In their opening discussion, panelists developed the following preliminary list of subtopics to
discuss within this topic area:
• Ethical principles.
• Need for a clear research plan.
• How is the study problem defined, and by whom? What groups are engaged in defining
the problem?
5
-------
• Need for a clearly articulated study justification. In communication about the study, it is
important for the researcher to explain what questions he/she seeks to answer and justify
the investment of time and money. This step should precede study design.
• Funding, particularly conflicts of interest.
• Innovative and alternative study designs. Examples from around the country have shown
how an innovative design can lead to greater benefits to the community studied.
However, as another panelist pointed out, the study still must be able to test a hypothesis
and provide a meaningful scientific outcome.
• Feasibility—Can this study be done within a particular community? What Institutional
Review Board (IRB) restrictions are in place?
• Statistical power—i.e., How much scientific evidence does the researcher need?
• What constitutes "acceptable"? When observing subjects' behavior, the researcher must
be able to distinguish between conditions that society has determined are acceptable
(even if the subjects may be doing things that are not good for them) and conditions that
are unacceptable. The researcher must "draw a line" and establish decision points in the
study design.
• Transparency of the IRB—Who is on it, and what questions did they ask?
• The "therapeutic misconception." As one panelist explained, subjects believe that
scientists have the subjects' best interests at heart, and often expect to benefit from the
study, even if explicitly told that they will not. However, as another panelist pointed out,
the study usually does not have any large downside either.
• Harm reduction and the potential for backlash. As one panelist noted, people generally
assume that consumer products have been tested extensively, and may be upset to learn
that their assumptions are incorrect. Another panelist added that the community may
blame the researcher for being part of a "flawed" regulatory system, or may expect the
researcher to be able to solve a problem that society has thus far failed to solve—e.g.,
removing a known toxin from household products.
Results from the Breakout Discussion
Following the breakout discussion, Breakout Group A reported back to the full panel. An outline
of their presentation appears in Appendix I; detailed points are captured below:
Ethical principles. Fundamentally, many members of the group believe that one should not
impose a higher duty on the investigator than society in general imposes to protect children or
other vulnerable groups. For example, if society says it is acceptable for a child to grow up in a
home with lead paint or where pesticides are used, then it is socially acceptable for the researcher
to observe a child in such an environment. However, at least one panelist disagreed with this
6
-------
assertion, elaborating on her disagreement in post-meeting comments.3 The researcher must still
grapple with the question of whether it is ethically wrong to let this behavior continue. It is
important to consider how the community perceives risks and benefits; for example, whether
pesticide spraying is as harmful as the alternative, a home full of cockroaches that could trigger
an asthma attack. If the researcher observes an imminent hazard such as child abuse, however,
he/she does have an ethical responsibility to take action, and may have a legal responsibility to
act as well. Thus, it is important to establish a clear line between what is acceptable and what is
not.
Overall, panelists agreed that the study should have minimal risk to the participant. One concern
is that study design and disclosure could encourage participants to increase their exposure,
although several panelists noted that this outcome may be more of a perception than a reality.
One way to minimize the possibility of increasing exposure would be to avoid telling the
subjects about the study objectives, but this would be contrary to the goal of full disclosure of the
study's purpose. After debating the issue, the panel agreed that while "not enunciating clearly"
might be in the subjects' best interest, full disclosure is fundamentally more important. As one
panelist noted, there are other ways to reduce the risk of encouraging increased exposures. For
3 In post-meeting comments, one panelist (Sophie Balk) expressed reservations about not imposing a higher duty on
the researcher:
"The 'higher duty' argument depends on your perspective. I disagree with the contention in this slide that 'We
should not impose on the investigator a higher duty than our society imposes to protect children.' Society tolerates
all sorts of unacceptable circumstances for some people, mostly poor people, including children and most of us
would agree that this is not a 'right' thing. I believe that I voiced this opinion at the meeting and I would like the
summary to include this viewpoint.
"When pediatricians know what is 'right' and the matter is not minor, we will do our best to find a solution. When I
see a child with asthma and I hear that there are housing problems affecting the asthma - holes in the wall letting in
cold air, mold growing on walls from water damage -1 will refer the parent to a social worker or a lawyer so the
family can pressure the landlord to remediate the problems. If pressure doesn't work, the family is advised to sue the
landlord. From my perspective, an investigator going into a home where there are known health risks or possible
health risks has an obligation to at least educate the family about these risks and to refer the family to their health
care provider or community advocacy group for further assistance. In my opinion, if this means that an observational
study cannot be done because we can't merely observe an intolerable condition, so be it."
Another panelist (Bruce Lanphear) responded to Sophie Balk's point as follows:
"If we know what is right (i.e., there is evidence about the hazard and how to reduce or control the hazard), then I
agree with Sophie. But the point of the research is to figure out whether an agent is a hazard or how to control a
child's exposure safely. In other words, there has to be uncertainty about the hazard or the safety and efficacy of an
intervention for the study to be ethical. This deserves additional discussion, particularly because it is such an
important point and because I am not convinced we are really in disagreement."
A third panelist (Michael Lebowitz) added the following thoughts:
"Almost all observational exposure studies are to determine exposures to known hazards and other chemicals that
are monitored for other reasons, and not to determine health hazards per se; they are used for risk assessments (to
determine the extent of the risk to hazardous exposures and to lead to risk management). Observational studies of
potential hazards are epidemiological and not observational studies of exposure per se."
7
-------
example, if a study covers many classes of chemicals, participants may be less likely to feel that
a single risky behavior (e.g., spraying more pesticides) will increase their reward.
Panelists also discussed incentives and compensation. They agreed that unequal compensation is
unacceptable—e.g., a scenario in which the researcher gives different incentives to different
subjects, such as paying an affluent working mother more for her time than a poor unemployed
mother. Rather, incentives should be based on the time it takes to participate—for example,
minimum hourly wage—and the degree to which the individual may derive other benefits from
participating in the study. Although few studies have been conducted on compensation and
retention, the panel generally agreed that adequate compensation is an important factor in
retaining participants in long-term studies.
Defining the study problem. The study problem can be defined by a variety of stakeholders,
including the researcher, an agency, or the community. Ideally, all three will be involved in
defining the study problem, particularly because different groups may have different perceptions
of risks and benefits. In general, broad-based studies tend to be more useful than narrow studies.
One panelist added that the study problem must be defined in an operational fashion. Another
pointed out that investigators may not be able to fully define the problem because they often
respond to a problem that has already been defined (e.g., in a request for proposals), and
suggested that it might be more useful to define the problem broadly in the solicitation, then let
the investigator work with the community to develop a more specific definition that meets their
needs.
Justifying the study. The study should have a specific goal, address a specific public health
need, and be clearly justified. It should address community needs. However, as several panelists
observed, public health importance can be considered at different scales (broad, local, etc.).
Funding. In an ideal world, the government might provide all funding for observational
exposure studies (e.g., through a fee imposed on chemical use), but in reality, an increasing
number of studies are being funded by industry or through public-private partnerships.
Researchers should ensure that anyone with a conflict of interest, including industry sponsors,
will not have the opportunity to influence the outcome of the study.4 Many federal agencies and
universities have already developed mechanisms to minimize the potential for undue influence—
for example, by not allowing industry representatives to participate in certain decisions. Even if
no real conflict of interest is present, however, there can still be perceived conflicts of interest
whenever industry is involved, because people tend to assume that industry-funded research will
produce results that are favorable to industry. It is important to discuss these concerns with the
community and make sure they have a say in whether to accept industry funding.
Several members of the group noted that they have worked with communities who wanted more
than could be funded—for example, repeated visits or a long-term commitment that might
outlive the funding source. Researchers should be aware of these expectations.
4 In post-meeting comments, one panelist (Alan Fleischman) added that it is also important that the sponsor not
unduly influence the design of the study.
8
-------
Innovative and alternative study designs. The panel expressed interest in studies that test
variables with natural histories—for example, comparing homes that have regular application of
pesticides versus those that do not—and intervention or lag studies with a control population.5 In
any case, the community should be involved in designing the study.
Later in the discussion, one panelist noted that randomized trials of preventive interventions not
only can increase participation, but also can enhance causal inference and test the efficacy of
exposure reduction, thus making it easier to translate results into benefits for the participants.
Randomized elements could include removing the source or taking steps to prevent exposure. As
another panelist pointed out, studies that improve understanding of causal linkages could also be
more useful when EPA makes policy determinations. Several panelists questioned whether
randomized trials are within the scope of the document EPA plans to write. In response, Warren
Lux (EPA) clarified that EPA defines "observational exposure study" to include any exposure
study not considered an "intentional exposure study" (a study in which participants are exposed
to a substance they would not be exposed to outside of the study). This definition is based on
EPA's version of the Common Rule, and specifically refers to the substance under study—not
substances that might be introduced as a result of sampling or intervention. The randomized
designs described above would fall under EPA's definition of an "observational exposure study."
Several panelists agreed to recommend that additional work be done in this area. One panelist
suggested that the preamble to EPA's document should define "observational exposure study"
and should recommend that researchers at least be aware of alternative designs.
Other issues. Panelists mentioned several additional issues related to study conceptualization:
• Feasibility: The study must be accepted by the community, and must lead to results.6
• Statistical power: A study should be conducted only if it has sufficient statistical power to
yield appropriate hypothesis testing and, thus, the potential to yield significant results.7
• Communication planning: All aspects of the study require communication. The
researcher should engage in ongoing dialogue with the community, even after funding
ends.
5 In post-meeting comments, one panelist (Sophie Balk) expressed reservations about allowing any routine pesticide
application in homes with children:
"'Routine' spraying is not something a pediatrician would recommend. We know enough about the bad effects of
pesticides on children to worry about this, and tell people to minimize exposure whenever possible. We recognize,
however, that this may not always be possible but in many instances, it is possible."
6 In post-meeting comments, one panelist (Michael Lebowitz) suggested a more detailed consideration of what it
means to "lead to results." Not all studies will produce significant or meaningful results, as even an appropriately
powered study may not reject the null hypothesis or provide results that can be used to change anything. EPA may
want to clarify as "... must provide results to those studied," if this is indeed the meaning of the statement.
7 In post-meeting comments, one panelist (Bruce Lanphear) added, ".. .unless it is a pilot study."
9
-------
• When is it necessary to stop the study? The researcher needs to establish criteria in
advance.
• Researchers need to consider whether a study requires a Data and Safety Monitoring
Board.
• The researcher should explain how the IRB works and who represents the community's
interests on the IRB. The community should advocate for representation that reflects their
concerns.8
• Harm reduction: The researcher needs to explain the concept of harm reduction to the
community, noting that the study will not necessarily eliminate all dangerous exposure.
• As a general rule, communities should be consulted up front, be it for study design,
obtaining funding, determining compensation, setting decision criteria, or other
considerations.
• It is important to have diversity within the study population, reflecting the range of
socioeconomic, ethnic, and other groups who may be impacted by the exposure in
question.
Chapter Title
The panel recommended that EPA cover this topic area in a chapter entitled "Elements to Be
Considered in Study Conceptualization." Panelists originally considered the term "study design,"
but agreed with one member's suggestion that "study design" may be too specific, with different
implications in different fields. They agreed that "study conceptualization" would be a more
inclusive term, encompassing steps that must take place before the study design.
4. Ensuring Protection of Vulnerable Groups
This topic area was discussed initially by the full panel in the opening plenary session, and then
discussed in detail by the members of Breakout Group B (see Exhibit 1).
Preliminary Comments
In their opening discussion, panelists developed the following preliminary list of subtopics to
discuss within this topic area:
8 In post-meeting comments, one panelist (Sophie Balk) elaborated on transparency, suggesting that the following
items must be posted on the Internet for public scrutiny: investigators' names, credentials, and affiliations; details of
study design; funding sources; information about who is on the IRB and their credentials and affiliations; what
questions were asked of the IRB and how they were answered; and consent forms.
10
-------
• How to define vulnerable groups. One panelist noted that "vulnerable groups" can be
defined in many ways, including life stages (young and old), predisposing diseases, or
societal factors such as access to care. Another suggested a slightly different way to look
at the term, pointing out that all people are technically vulnerable to exposure, and the
only difference is the degree of vulnerability. Thus, the basic ethical principles should
apply to all subjects. A third panelist agreed that the ethical principles apply to all, but
noted that laws and regulations specifically talk about the vulnerabilities of children,
people with pre-existing conditions, and people who do not have the means to access
health care for diagnosis, treatment, or general knowledge about exposure and its effects.
As several panelists pointed out, people with low socioeconomic status (SES) are more
likely to have elevated exposures—for example, because they may be more likely to live
in polluted areas (which they may not have the means to leave), more likely to use
appliances that pollute indoor air, and less likely to know that smoking in the home is
harmful.
• The culture of the group. Cultural nuances can increase certain people's vulnerability to
exposure.
Results from the Breakout Discussion
Following the breakout discussion, Breakout Group B reported back to the full panel. An outline
of their presentation appears in Appendix I; detailed points are captured below:
What constitutes a "vulnerable group"? Definitions vary, but in general, vulnerable groups
can be defined in terms of disparities, which may be demographic, socioeconomic, or
educational in nature. Some groups may be more vulnerable to exposure, while others may be
more susceptible to harmful effects resulting from exposure—for example, those with
preexisting biological conditions. These groups can be defined on either an individual or a
societal scale. One important variable is the ability to voluntarily participate. For example, some
people may be vulnerable to coercion, or more likely to enroll in a study because of economic
need.9'10
Protection. Additional protection is justified for vulnerable groups. Protection occurs on two
levels—protecting the individual and protecting the community—and is based on the two
9 In post-meeting comments, one panelist (Sophie Balk) suggested paying specific attention to children and the
"politically powerless," including people with psychiatric conditions and addictions.
10 In post-meeting comments, one panelist (Alan Fleischman) added, "It is important when discussing 'vulnerable'
populations to make the distinction at the outset of the two ways this term is used. Environmental exposure scientists
think of vulnerable populations as those who are vulnerable to the exposure (e.g. children, elderly, immune
compromised, etc.). In the research ethics world, vulnerability means a group with diminished ability to protect their
own interests, manifested generally through a compromised capacity to provide informed or voluntary consent (e.g.
children, cognitively impaired, mentally ill, prisoners, educationally and economically disadvantaged)."
11
-------
underlying concepts of beneficence and autonomy,n To address beneficence—perhaps the more
important of the two concepts with respect to vulnerable populations—the researcher must
consider the balance of risks and benefits to the participant. Once the researcher can adequately
address the issue of beneficence, then he/she should consider autonomy. The IRB plays a role in
considering special protections and concerns.
Minimizing undue burdens. Justice is a key principle. During the breakout session, members of
the group gave examples of studies in which participants were asked to travel long distances in
order to provide samples. The panelists agreed that participation should not require an excessive
amount of time and effort. Also, some researchers may feel the need to "collect everything" in
one visit, which can put an undue burden on the participant. In addition to minimizing physical
burdens, the researcher should also avoid stigmatizing the community or appearing to "blame the
victims." The participants should also be the first to realize any benefits from the study. It is not
fair to study the poor but then steer the benefits mainly to the rich.
How to study vulnerable groups. Like any participants in a human observational exposure
study, vulnerable groups need to be provided with details of the study.12 It is also important for
the researcher to develop an authentic, substantive relationship with the group, based on trust,
respect, and empathy. The researcher should be visibly involved, meeting community members
(not just sending assistants into the community), learning about the community's needs and
concerns, and learning about their culture and how they live their everyday lives. Study design
teams and IRBs should include appropriate experts and participant advocates; for example, the
IRB for a children's study should include a pediatrician with a working knowledge of the
community and the hazard being studied. Members of vulnerable communities can be a valuable
addition to research teams because of their understanding of cultural norms, and as one panelist
explained, this should not be limited to simply hiring minorities as assistants to interact with the
community. Ideally, people from diverse backgrounds would be included at all levels of the
research structure, and researchers would endeavor to build that capacity within vulnerable
groups.
One panelist pointed out that researchers face many burdens, and may not have the time and
funding to develop deep relationships while meeting publishing deadlines and other constraints.
However, another suggested that taking the time to develop these relationships will ultimately
pay off for the researcher because he/she will be able to ask better questions and get more
complete, honest answers in return. To deal with time constraints, one panelist suggested looking
at models of paired investigations and apprenticeships, which also could help to build capacity
among junior researchers.
11 In post-meeting comments, one panelist (Sophie Balk) recommended formally defining "beneficence" for those
without an extensive background in ethics.
12 See footnote 8 for post-meeting suggestions from one panelist (Sophie Balk) on ways to maintain transparency.
12
-------
Chapter Title
The panel recommended that EPA cover this topic area in a chapter entitled "Ensuring Protection
of Vulnerable Groups." Panelists noted that the chapter titles should be grammatically consistent,
and suggested that using active language in all headings may help to convey a sense of actually
applying these strategies. Panelists also discussed the word "vulnerable," which may be defined
differently depending on one's perspective. From a more scientific perspective, one might view
the issue in terms of vulnerability to exposure (e.g., biological exposure), which led one panelist
to suggest clarifying the title as "Study and Protection of Vulnerable Groups." However, others
argued that the ethical issues really pertain to a different definition—vulnerability due to special
characteristics of the group—and suggested leaving the title as recommended.
5. Addressing Privacy and Other Concerns Related to Personal
Exposure Observational Studies
This topic area was discussed initially by the full panel in the opening plenary session, and then
discussed in detail by the members of Breakout Group B (see Exhibit 1).
Preliminary Comments
In their opening discussion, panelists developed the following preliminary list of subtopics to
discuss within this topic area:
• Emphasizing that despite all the limitations and drawbacks, the home environment is
critical to understanding exposures.
• Privacy.
• "Collateral" observations.
• Effects on third parties.
• Breaches of privacy, which are inherent whenever one observes people in their daily
routine outside of a controlled laboratory setting—e.g., at home, school, day care, work,
or a long-term care center. Privacy is not the same as confidentiality.
• Watching people in their "natural" environments, where one may observe behaviors that
are not in the subjects' best interests.
• Observing hazards that are unrelated to the research (e.g., physical hazards such as a
child about to fall out of a window). It is important to train researchers and set up a
system to deal with these types of observations.
• Working with community organizations and agencies ahead of time to determine how the
researcher should deal with issues of reportability. It is important to understand the
13
-------
community's standards and consider the ramifications of reporting a "collateral"
observation. For example, if the researcher reports a cockroach-infested home to the
Department of Youth/Family Services and the family does not have the means to remedy
the situation (e.g., because they are renters), the agency could end up removing children
from the home.
• Training researchers to be sensitive to cultural differences, including those that may
relate to potential hazards.
• Considering sociobehavioral factors.
Results from the Breakout Discussion
Following the breakout discussion, Breakout Group B reported back to the full panel. An outline
of their presentation appears in Appendix I; detailed points are captured below:
What is personal exposure? At the outset of the study, the researcher needs to define "personal
exposure," working with the community to explain this important concept in language they can
understand. The community should understand how samples will be obtained and why the study
must take place in personal spaces rather than in the laboratory. For example, as one panelist
added, personal exposure monitoring has fewer uncertainties than studies based on ambient
monitoring. In addition to the scientific fundamentals, researchers and communities should also
understand the range of ethical issues that may arise in this line of research (including privacy),
and should agree on how these concerns will be addressed.
Invasion of privacy. The issue of privacy invokes strong feelings, and invasion of personal
space can seem at odds with the long-standing American ideal of "my home is my castle."
Ultimately, to reduce the perception of invading personal space, the researcher's goal should be
to be invited in. One way to work toward this goal is by addressing privacy issues up front.
Cultural sensitivity is also important. All members of the research team—including the principal
investigator—should be sensitive to cultural differences and should recognize that people's
individual and cultural experiences can affect the way they perceive "invasion of privacy." On a
related note, members of the group suggested a need for genuine diversity throughout the
research structure, not just the research team.
Confidentiality. Confidentiality differs from personal-space privacy protection, as it relates
more to how the researcher treats information that he/she receives from participants. In some
cases, a certificate of confidentiality may be required by the IRB or by the circumstances of the
study. Certain groups such as adolescents may have unique confidentiality issues.
Risk-benefit balance. It is important to consider risks and benefits in both the scientific and
ethical domains. The ideal outcome is for all parties engaged in the process to realize some
benefit—i.e., a "win-win" situation. However, the individual may not derive any extra benefit
from participating, which should be made clear to the participant at the outset. Some IRBs
prescribe language that must be used when explaining risks and benefits to the participant.
14
-------
Members of the group noted that some researchers have a tendency to exaggerate benefits or
minimize risks to bolster recruitment, but these practices should be avoided.
Researchers should be aware that risk—and the perception of risk—may vary by individual (e.g.,
by age of the participant), by type of sampling, by the degree of invasiveness, or by cultural
norms. For example, certain cultures may consider a hair or fingernail sample to be extremely
invasive. The researcher should make participants aware that they have a right to refuse a
particular component of sampling if they wish.
Disclosure. Key considerations include when to inform individuals, what information to provide,
and how to provide it. Participants often want their individual results reported back in a timely
fashion, although as members of the group noted, some individuals may not want to receive
results, and they do have a right to refuse. The researcher should work with the IRB, sponsors,
and the community to determine what information to provide back and how to do so. In general,
participants should be given information about known and unknown hazards.
"Collateral" observations. Collateral observations are observations that are unrelated to the
purpose of the study. These observations may or may not involve potential harm. The researcher
must anticipate that these observations will occur, and must have an action plan to deal
accordingly with these observations—including unexpected observations. The action plan should
include training research staff in what to do if they observe something of concern. For example,
the observer may be required to report to the principal investigator, who is ultimately responsible
for any issues of intervention or mandatory reporting. When planning the study, the research
team should identify resources to help with decision-making, like individuals who are familiar
with the laws about mandatory reporting and appropriate follow-up actions.
Duty to care. Panelists discussed the researcher's responsibility to do things related to the care
of the individual. The researcher does not have the same level of obligation as a physician in a
clinical setting. Nonetheless, it is important for researchers to be aware of their duties ahead of
time, and to understand that they may have biases and perceptions that can influence their sense
of "duty to care."13
When to stop the study? Circumstances may require that a study be stopped on an individual
level or across the board. A Data and Safety Monitoring Board (or similar oversight body) can
play an important role in this process.
Third parties. An observational exposure study can potentially affect a number of "third
parties," including:
• Household members not in the study
• Neighbors and other members of the community
• Insurers
13 See footnote 3 for post-meeting comments from several panelists regarding the researcher's duty to care.
15
-------
• Landlords and rental agents
• School officials and teachers
• Caregivers of children, the elderly, the disabled, or others
The researcher should anticipate that third parties will be present, recognize that they may affect
participation in the study, and plan accordingly to minimize impacts on these people and to
address their needs and concerns. It is important to respect the presence of third parties—for
example, by announcing a home entry in advance in order to avoid catching people off guard.
During the planning phase, the community can provide useful input on how to deal with third
parties.
In some cases, third parties may be so involved in the study that they essentially become
participants themselves, in which case they may need to sign an informed consent form.
However, as one panelist noted, in many cases it may be sufficient to receive a waiver for third
party consent from the IRB, especially for studies with minimal risk.
Justification for the study. In explaining the study to the participants, the researcher should be
clear about why the study is important. Justification can include benefits to the individual,
benefits to the community, and broader benefits to scientific knowledge.
Chapter Title
The panel recommended that EPA cover this topic area in a chapter entitled "Addressing Privacy
and Other Concerns Related to Personal Exposure Observational Studies." Panelists originally
suggested the term "home hazards" but as several panelists pointed out, the fundamental issue is
privacy, which extends beyond the home. Breaches of privacy are inevitable when observing a
subject in a place where the researcher might make collateral observations, etc. However, the
panel decided to add "other concerns" to the title after some panelists pointed out that there are
still issues beyond privacy. Other panelists suggested adding "personal exposure" to the title in
order to encompass all exposures and all locations where the researcher may be intruding. The
panel also considered the term "natural environments," but as one member pointed out, these
words might imply an outdoor setting.
6. Creating an Appropriate Relationship Between Participant and
Investigator
This topic area was discussed initially by the full panel in the opening plenary session, and then
discussed in detail by the members of Breakout Group A (see Exhibit 1).
16
-------
Preliminary Comments
In their opening discussion, panelists developed the following preliminary list of subtopics to
discuss within this topic area:
• Informed consent, including issues around the storage and use of samples (e.g., DNA).
• Incentives and compensation.
• Advertisements.
• Grievance procedures. If participants have a problem with the study, do they know where
to raise their concerns and obtain support? The IRB plays a role in designing a thorough
informed consent form, but as one panelist noted, subjects may not remember what was
in the informed consent form. A broader solution would be to go beyond simply writing
down conditions and try to create a truly "supportive" environment for research and
interaction.
• Ensuring that recruitment/retention methods will not change subjects' behavior. In
particular, the study should not encourage risky behavior. The opposite issue—
discouraging exposure—poses a greater ethical challenge because reducing exposure
may be a desirable outcome, yet intervention to reduce risky behavior could skew the
results of a study. As one panelist pointed out, if the researcher does not tell the whole
truth to the community, it can breed distrust. However, another cautioned that providing
too much information can be overwhelming, in addition to potentially jeopardizing the
study. Panelists generally agreed that all subjects should be informed of the basic risks up
front (e.g., in the informed consent), and should be reassured that if the researcher
identifies an imminent hazard, he/she will bring it to their attention. To ensure that
intervention will not jeopardize the integrity of the study, the researcher should anticipate
potential behavior changes and the need for intervention in advance, and should design
the study accordingly. Another solution might be to avoid an observational study
altogether if another type of randomized study design might be more appropriate.
• Maintaining communication with subjects to keep them involved. For example, the
researcher must inform participants and the community of the results. If the samples will
take a long time to process, the researcher must inform the participant that there will be a
time lag before he/she can learn the results.
• Strategies for recruiting from the target population.
• Deciding when to stop the study, perhaps by adapting strategies from clinical trials. For
example, one panelist suggested that if exposures exceed established guidelines, the study
should be stopped or switched to an "intervention" phase, similar to what happens when
bad results are observed in a clinical drug trial.
• Involving community leaders to make sure recruitment, retention, and study methods are
culturally acceptable.
17
-------
• Defining and balancing risks and benefits to the researcher, the subject, and the
community.
Results from the Breakout Discussion
Following the breakout discussion, Breakout Group A reported back to the full panel. An outline
of their presentation appears in Appendix I; detailed points are captured below:
Informed consent. All members of the breakout group agreed that informed consent is
important, although they expressed differing views on exactly how extensive the informed
consent must be. In addition to explaining risks related to the substance(s) being studied, the
group generally agreed that if the researcher observes a preexisting risk such as a mother
smoking in the home, he/she should err on the side of providing adequate warning.14 However,
they also agreed that it is impossible to convey the entire universe of conceivable risks to the
participant. Panelists also raised concerns about risks that may not be fully understood, such as a
chemical that is believed to be harmful but has not been studied thoroughly.
One panelist suggested that in theory, the researcher's level of duty may fall into four categories:
• No duty15
• Duty regarding risk imposed by research on subject
• Duty regarding risks known to be caused by preexisting behavior of subject
• Duty regarding other risks unrelated to the study
Several aspects of the informed consent require careful attention. The researcher should not
promise anything he/she cannot deliver—e.g., a long-term commitment in cases with short-term
funding. It is also helpful to talk with both parents in two-parent homes and to involve child
subjects in the consent process if they are of sufficient age, although as one panelist clarified,
laws and regulations generally require only the mother's consent—and the child's assent when
appropriate—in order to study a child. The consent form should explain procedures for storage
and use of samples (particularly DNA). Some panelists also felt that the informed consent form
should clearly state that the family can change their behavior without losing compensation,
although others expressed concern about how this might affect retention and sample size.
In general, the consent form should use language appropriate to the population, and the
researcher should test participants to make sure they truly understand. There is an extensive body
of literature on the effective use of informed consent. As one panelist noted, however, in many
14 In post-meeting comments, one panelist (Bruce Lanphear) cautioned that "a researcher has to be sensitive about
lecturing a participant about a 'socially accepted' behavior."
15 Panelists agreed that in practice, "no duty" is not a viable option for a researcher in this field.
18
-------
academic settings the IRB dictates the language of the informed consent document. This
language may be difficult for the community to comprehend, making it hard for the researcher to
communicate effectively.
Incentives and compensation. Compensation is critical for recruitment and retention, but can
also be an area of controversy because of the concern that it might unduly influence participants'
behavior. If compensation is too high, it might affect recruitment in such a way as to influence
the representativeness of the sample and the generalizability of the results. Compensation can
also affect the validity of results if, for example, the incentives are so high as to influence
participants' answers.
Panelists disagreed on the extent to which "undue influence" can actually cause participants to
increase risky behavior, and they disagreed on the extent to which one needs to worry about this
outcome in studies with minimal risk. However, as one panelist pointed out, compensation could
increase risk above minimal risk. Another noted that even the perception that compensation
could increase people's risk might weaken a study's credibility.
As far as an appropriate level of compensation, the panel agreed that all participants in a study
should be compensated at an equal rate, but disagreed on whether minimum wage is appropriate
or whether it may be so high as to unduly influence behavior among some groups. A large body
of literature is available regarding the influence of compensation and the relational ethics of the
process. For example, some studies have compared the way different incentives affect retention;
one method used was a raffle, although that approach may raise questions about legality.
To reduce the possibility that compensation could unduly influence participants (or the
possibility that it could be perceived as such), the researcher should design the study to minimize
the potential to alter behavior in a way that might lead to greater risk-taking by study
participants. The researcher should also emphasize up front that participants are being
reimbursed for the burden of participation, not for assuming a risk. Panelists agreed that
researchers should avoid overemphasizing compensation in advertisements; one noted that
community engagement tends to be a more effective recruitment strategy anyway. The
researcher should also talk with the community about details such as whether participants would
prefer to be paid in a lump sum or by the hour.
Grievance procedures. Participants should have an avenue to express their concerns to a third
party (i.e., not the investigator or the IRB co-chair). It may be useful to appoint an ombudsman
or use a community advisory board—although some panelists noted that community advisory
boards may represent a small group of activists rather than a true cross-section of the community.
Creating a supportive environment for research and interaction. To create a supportive
environment, the researcher must develop community relationships prior to initiating the study.
Planning grants can help the researcher work with the community to develop a communication
plan, thereby building trust.
Ensuring that recruitment/retention methods will not change subjects' behavior. Although
the informed consent process may change behavior, it is ethically imperative that this process be
followed. Although decreasing exposure may affect the results of the study, it may still be a net
19
-------
benefit. Communication is helpful in this area; for example, providing participants with a
"risk/risk" statement that acknowledges the hazards of exposure and the hazards of the
alternative.16 One example would be a statement that explains the risks of pesticides as well as
the dangers of having pests in the home.
Ongoing communication and reporting results. Ongoing communication is critical for
retention, especially in long-term cohort studies. Reporting results can also be an incentive. If the
study involves collecting biological data that may reflect immediate hazards to participants,
results should generally be provided with a fast turnaround time. Less critical information may
take longer to process; for example, many studies have a three-year time lag. In any case, it is
critical to let the community know ahead of time what type of turnaround they can expect.
Some panelists suggested that the researcher should only report data from which meaningful
conclusions can be drawn, while others would prefer to provide all results, regardless of clinical
significance. In either case, it is important to establish clear procedures up front, working with
the community to determine what data they would like to be given.
Recruitment strategies. Churches and other community associations, agencies, etc., can
promote the recruitment strategy to which the community agreed, but are not themselves a
source of representative participants.
Deciding when to stop the study. The researcher should work with the community ahead of
time to establish standards for when the study will be stopped because of risk. This issue is
discussed further under "study conceptualization."
Procedures for collateral observations. The group agreed that the investigator must identify
and take action on imminent hazards he/she observes even if independent of the subject of the
study, but thresholds should be set relatively high because the community may evaluate dangers
differently. It is important to discuss this issue with the community in advance.17
Chapter Title
The panel recommended that EPA cover this topic area in a chapter entitled "Creating an
Appropriate Relationship Between Participant and Investigator." This topic was originally
identified as "Recruitment," but panelists also noted the importance of retention—or, in a
broader sense, participation. The panel agreed to start the title with an active word ("creating"),
and they agreed that this section also should cover the basic factors to consider with respect to
the relationship between the researcher and the participant.
16 In post-meeting comments, one panelist (Sophie Balk) added that participants should also be given information
about how to abate these hazards.
17 In post-meeting comments, one panelist (Sophie Balk) cautioned that this does not necessarily mean that
researchers have a higher standard than the community does for defining danger—just that they may have different
standards.
20
-------
7. Building and Maintaining Appropriate Community and Stakeholder
Relationships
Panelists discussed this topic area during the opening plenary session and then again in greater
depth during a plenary session, chaired by Loretta Jones, on the second day of the workshop.
Preliminary Comments
In their opening discussion, the panel developed the following preliminary list of subtopics to
discuss within this topic area:
• How to define "community."
• Determining who represents the community.
• Engaging the community throughout the process. The researcher should work with the
community to gather information and share results, then help address their concerns. As
another panelist pointed out, there have been many cases in which a researcher lost the
community's trust because he/she failed to engage the community at a particular step of
the process.
• Building relationships and building trust before beginning the study. One panelist
explained that three major factors contribute to building trust: (1) respect for the
individual and the culture; (2) equity of grant resources; and (3) a commitment to
empower the community.
• Building a lasting infrastructure. The community may expect a long-term, rewarding
relationship. As several panelists noted, a project should invest in the community so that
even after grant funding ends or the last paper is published, the community will have the
resources to address their concerns without relying on the researcher for assistance.
Building a lasting infrastructure may be an obligation of the sponsor.
• Developing a community contract that outlines the roles and expectations of the
researcher and the community, including expectations of control.
• Honesty, partnerships, and power relationships.
• Issues around revealing findings.
• Importance of language. One panelist provided an example of a study in the U.S./Mexico
border region, in which materials had to be translated into the local dialect ("border
Spanish"). The researcher can involve community members as interviewers and
translators.
21
-------
• The role of the community on the IRB. As one panelist noted, the Office of Human
Subjects Protection requires that every IRB include at least one community
representative, but institutions differ in how they define "community representative" and
the extent to which community representatives truly have a voice on the IRB. Some IRBs
include more than one community member.
• Researchers should make an effort to look at communities' resiliencies and report
positive observations along with the negatives. Communities tire of hearing about only
the things they are doing wrong. They also want to be told what they are doing right.
• How to define "stakeholders" and determine what role they should play. Stakeholders
include business, industry, and various levels of government. Relations with stakeholders
may be confrontational, and people with conflicts of interest should not be making key
decisions.
Subsequent Discussion
Why be concerned about community engagement? Panelists began the discussion by
confirming the importance of engaging the community in the research process. One panelist
noted that by its very nature, observational exposure research must take place in community
settings. As the recent NAS report pointed out, ethical principles from the "ivory tower" of
academia may take on new meanings and reveal unique nuances when applied in the field
("Ethical Considerations for Research on Housing-Related Health Hazards Involving
Children"—see Section 10, "Information Resources," for a full citation). Another panelist
explained that many of the communities targeted by observational exposure research may not
feel fully "accepted" by society, thus making it even more critical to partner with those
communities to identify acceptable research models.
Panelists agreed that fundamentally, the purpose of engaging the community is to ensure that the
following four aspects are in place:
• Respect for the individual
• Respect for the culture
• Equity of resources
• Empowering the community to endure
There are many ways to involve the community during the research process. On an advisory
level, community members can help the researcher make sure he/she is "doing the right thing."
They may also be able to help conduct research. Some aspects of community engagement may
be unique to observational exposure studies; for example, as one panelist noted, emergency
research studies may have a very different notion of how the community should be involved.
Defining "community." Communities can be defined in many ways—e.g., by geography,
shared cultural and social characteristics, or common values and interests. For example, a
22
-------
"community" of Native Americans may consist of a tribe whose members are scattered
throughout the country. How the researcher defines "community" will depend on the nature of
the study.
Panelists noted that "community" can be defined broadly or narrowly. In the broadest sense,
"community" might include anyone who is not part of the research team, or anyone who does not
have responsibility or authority for the study. This broad definition could include local agencies,
stakeholders, and the press, for example. For EPA's document, however, the panel suggested
defining "community" more narrowly as "the population from which study participants are
selected." This definition captures participants and their social/cultural community, including
community "experts," but excludes government agencies, industry, and others who do not
necessarily represent the interests of the participants.
It is important to distinguish between community (in the narrower sense of the word) and
stakeholders, and the panel recommended that EPA define both terms clearly. As one panelist
explained, a critical difference is that the community has a right to speak for its own interests,
while stakeholders cannot speak directly for the community. Nonetheless, stakeholders have
useful information and expertise, and should be engaged accordingly. For example, stakeholders
can play an important role on advisory boards because they understand impacts and can help
implement change. One panelist cautioned the group not to be overly restrictive when defining
terms, recognizing that some people's definitions of "community" and "stakeholder" may
overlap.
Who represents the community? In defining "community," several panelists raised the issue of
"voice"—namely, the community's right to speak for its own interests, and the importance of
identifying voices within the community who can articulate those interests on the community's
behalf. Identifying the appropriate voices may be harder than it appears because as one panelist
noted, the loudest voices (i.e., the activists) may not always be the most representative, and in
many cases, it may take several voices to adequately represent the diversity of viewpoints within
the community (e.g., through a community advisory committee).
One of the researcher's first steps in engaging the community should be to ask them who they
see as a legitimate voice who can speak for them. For example, one panelist recounted an
example in which she interviewed several community members and asked each to suggest five
people who could speak for their interests, then compared notes to identify names that were
suggested repeatedly. Community "leaders" can also help identify appropriate voices. For
example, the researcher may find it helpful to consult with elected officials, religious leaders,
community associations, or a community elder or matriarch (e.g., the "great mother").
For assistance, researchers also can consult a growing body of literature on community
representation. One panelist suggested reviewing sociological and ethnographic research on
identifying community leaders, while another pointed to a large body of literature on
community-based intervention research, including discussions on community advisory boards
and guiding principles in CDC's online journal, Preventing Chronic Disease (see Section 10,
"Information Resources"). Community-campus partnerships may provide useful case studies.
23
-------
Panelists also discussed the value of tuning in to the recent debate on emergency research
studies. Unlike observational exposure studies, emergency research studies involve participants
who cannot consent to participate because they are in a state of extreme physical trauma at the
time (e.g., suffering a massive loss of blood). However, two panelists noted that there are also
many parallels between observational exposure studies and emergency research, particularly the
need for community input ahead of time. In the case of emergency research, it is particularly
important to identify appropriate voices within the community because these voices will
essentially be providing advance consent on behalf of those who will become participants. There
are also some lessons to be learned from missteps that have occurred in this line of research.
Importance of language and culture. Language and culture are sometimes thought of as a
barrier to community engagement, but if the researcher understands and appreciates the unique
linguistic and cultural characteristics of the community, this can facilitate engagement. Panelists
noted the importance of avoiding misconceptions and overgeneralization; instead, the researcher
should recognize and appreciate heterogeneity within communities or groups. Differences extend
beyond nationality or ethnic background; for example, different tribal groups from the same
region may have very different cultural traditions.
Spoken language is one variable but not the only one. Groups may have different cultural norms
too; for example, one panelist noted that some Native American groups do not like pictures taken
of them, while some voodoo practitioners may be reluctant to provide hair or fingernail samples.
Groups can also differ in the way they make decisions. One panelist described an Amish
community that traditionally deliberates for a long time (compared to typical U.S. norms) before
answering a question; another recalled working with Native American groups who place a strong
emphasis on reaching decisions by consensus. In both cases, the research partnership was more
effective once the researcher learned to work within the community's decision-making culture.
Building trust. Throughout the discussion, panelists echoed a common theme: that trust must be
built, not assumed. It is incumbent upon the researcher to demonstrate trustworthiness
throughout his/her relationship with the community.
A key first step in developing trust is to establish a relationship with the community before the
study. This relationship—and the ensuing research partnership—should be built on honesty and
equity of resources (i.e., sharing equally). Panelists also emphasized the importance of honoring
voices from both sides. Beyond just listening to the other side or respecting what they have to
say, honoring voices means actually taking those voices into consideration, which the
community has a right to expect.
Another way to build trust is by establishing a contract with the community. As one panelist
pointed out, a community contract can clarify the expectations, roles, and responsibilities of both
sides of the partnership. A contract can also establish limits. Many communities see value in
formalizing this contract as a memorandum of understanding (MOU) because, as another
panelist noted, having these expectations and limits in writing will reduce the likelihood of
misunderstanding. A formal MOU can serve as a platform for resolving disputes, and can also
build expectations for a lasting infrastructure, which panelists highlighted as an important
ingredient in a successful relationship.
24
-------
For more information on building trust, one panelist noted that a great deal of literature has been
written on developing relationships, setting expectations, and other aspects of trust building.
Another pointed to a formal protocol signed by members of an interagency/tribal group in
Arizona as part of the "Turning Point" program (see Section 10, "Information Resources").
Several panelists recommended that EPA provide researchers with sample MOUs, noting that
many examples are available.
Honesty, power relationships, and partnerships. Because every study is different and every
community is unique, is hard to define exactly what an effective research partnership will look
like. However, a few general principles should be considered. As described in the previous
section, panelists noted that honesty is an important part of any partnership. Several panelists
suggested that the researcher also consider power dynamics—specifically the balance of power.
As one panelist observed, researchers often spend a great deal of time searching for funding or
working to meet the demands of an IRB, which can leave them feeling like they have very little
power. However, in the context of community relations, the researcher must recognize that
he/she actually has a great deal of power. This power can take many forms.
One of the more tangible forms of power is monetary power. A related consideration is access to
resources. As one panelist noted, if the community cannot access the resources they need in
order to understand or participate in the study, or they feel they cannot access these resources, it
can result in an imbalance of power.
Other forms of power may be more subtle, but are still important for the researcher to consider.
As one panelist noted, peer pressure can influence decisions about participation in a study, as can
the power of expectation—the feeling that one is expected to participate, and the fear of
repercussions if one does not participate. The power of expectation and fear of repercussions can
be compounded by confusion over the role of government agencies in the study; for example, if
the participant believes that the people running the study may be connected with those who
provide benefits such as food stamps. Housing relationships are particularly ripe for
misunderstanding and imbalance of power. For example, as one panelist noted, people in public
housing could feel compelled to participate out of fear that not doing so could jeopardize their
relationship with the Housing Authority.
In addition to the researcher's real or perceived power to affect the physical well-being of
participants, the researcher also possesses the power to stigmatize the community through his/her
actions. One panelist noted that the researcher may stigmatize the community by reporting
findings in a particular way; another added that simply conducting research in a particular
community might be perceived as labeling that community in a negative light.
Not all forms of power are necessarily tipped in favor of the researcher. One particularly
important characteristic of the researcher-community relationship is the imbalance of knowledge,
which works in both directions. The researcher has some knowledge that the community does
not have, but as several panelists pointed out, the community also possesses some knowledge
that the researcher lacks. One benefit of partnership, therefore, is that it allows both sides to pool
their knowledge and resources—i.e., to engage in knowledge transfer.
25
-------
Studies involving children can raise additional issues related to power dynamics. As one panelist
noted, because children have less power than adults, it is up to the parent to understand as much
as they can about the study. In some cases, it may be helpful to consult outside experts (e.g.,
pediatricians) to improve understanding.
Leaving a lasting infrastructure. Panelists agreed that building a lasting infrastructure is an
important part of the relationship between the researcher and the community. Rather than simply
"take" from the community, the researcher should give back by helping to build a sustainable
infrastructure that empowers the community.
Empowerment and capacity-building begin with the planning stage of the project. Several
panelists echoed the feeling that infrastructure-building must occur throughout the project—not
just at the end—and therefore the researcher must include this consideration in the overall plan.
Panelists also emphasized the importance of involving the community in the planning process,
including negotiations about capacity-building. One panelist specifically emphasized the need to
be forthright with the community regarding funding limitations. The community may not like the
ephemeral nature of funding and they may be apprehensive about taking on more responsibilities
in the future, but as the panelist pointed out, a well-designed program will cultivate their
confidence and expertise over time.
Panelists discussed several ways to build capacity during the study, such as involving members
of the community in certain roles—e.g., performing interventions. Ultimately, the goal is to train
community members so that in the future, they can perform some of the functions that are
initially performed by the research team. Panelists noted that certain research grants specifically
support training. Panelists also noted that training "works both ways." As two panelists pointed
out, allowing the community to train the researchers not only is a sign of respect, but also can
lead to important new understanding. For example, one panelist recalled working with a Native
American tribe who insisted upon training the entire research team on cultural sensitivity; the
result was greater awareness and a stronger partnership.
Once funding for the study ends, the researcher can still continue his/her relationship with the
community. One panelist described an example in which the research team remained accessible
to the community if they needed technical support after funding ended. Researchers can "build
bridges" by helping community members identify new funding opportunities and write grant
applications. In general, panelists agreed that the end of funding should not mean the end of the
relationship.
While the researcher plays an important role in building a lasting relationship, the panel
emphasized that the sponsor bears some responsibility as well. As one panelist pointed out, many
sponsoring institutions (for example, the Kellogg and Robert Wood Johnson Foundations)
already recognize the importance of enduring commitment, and have used a variety of
approaches to ensure that these relationships are able to continue. For example, some grants may
tail off over a five-year period, allowing time for a more gradual transition to self-sufficiency.
This panelist recommended that universities and federal agencies consider similar "umbrella"
funding mechanisms.
26
-------
Another important step is to formalize the relationship between the community and the
institution conducting or sponsoring the research, not just between the community and the
individual researcher. As several panelists noted, institutional relationships can survive even if
individuals leave—for example, if the researcher moves to a different university. Thus, panelists
also emphasized the need for researchers to build support within their own institutions. One
panelist cautioned that institutions—particularly universities—may be reluctant to build enduring
relationships with communities if they do not see long-term financial value in this investment.
Another agreed that getting long-term commitments from universities can be particularly
difficult, but suggested a growing awareness among many high-level university officials that
relationships with the community really do produce long-term gains. Researchers may be able to
get more support from university officials if they can document their successes.
Panelists recommended that EPA's document highlight some examples of sustainable
partnerships. For example, one panelist mentioned the "Turning Point" program (see Section 10,
"Information Resources").
Chapter Title
The panel recommended that EPA cover this topic area in a chapter entitled "Building and
Maintaining Appropriate Community and Stakeholder Relationships." This topic area was
originally labeled "Community Participation," but panelists agreed that "engagement" is a more
appropriate term than "participation." As one panelist noted, "participation" happens on many
levels, sometimes simple or superficial, while "engagement" reflects a more long-term, lasting
relationship. Panelists agreed that it is important to distinguish between stakeholders and the
community (in the narrower sense of the word), noting that all of these parties must be engaged
at some point in the course of the study. Two panelists noted that because researchers are
themselves a part of the engagement process, the title should also reflect the importance of a
two-way relationship.
8. Designing and Implementing Strategies for Effective
Communication
Panelists discussed this topic area during the opening plenary session and then again in greater
depth during a plenary session, co-chaired by Michael Lebowitz and Giselle Corbie-Smith, on
the second day of the workshop.
Preliminary Comments
In their opening discussion, panelists developed the following preliminary list of subtopics to
discuss within this topic area:
• What are the roles and responsibilities of the researcher? The researcher should know
whom he/she is obligated to communicate with (participants, community, sponsors, the
27
-------
press, etc.), and should know what, when, and how to communicate with these parties.
Can the researcher acknowledge participants at the end of the study without breaching
privacy?
• Developing lay language or—more broadly—communicating at different levels. The
researcher must be able to communicate effectively with scientists, journalists,
politicians, and the community. Each group has its own degree of scientific literacy.
• Communicating with others—i.e., recognizing that communication is a two-way
interaction.
• Posting plans so they are accessible to the community, not just on a web site.
• What is exposure? What pathways need to be considered? Several panelists suggested
defining key terms in any communications with the community.
• Educating the community about the principles of research (e.g., "do no harm") and
discussing these principles at an accessible level.
• Explaining what constitutes "acceptable" conditions. The researcher should let the
community know in advance where he/she intends to "draw the line"—i.e., at what point
the researcher will inform participants of dangerous conditions.
• How to reveal findings to individuals and reveal aggregate findings to the community.
• Issues around public use datasets. Participants may be concerned about sample storage
and the use of data in subsequent studies, including studies of environmental-genetic
interactions. DNA is emerging as a hot-button issue, as people grow concerned about
who may have access to their genetic data, and for how long. Protocols must be
communicated clearly.
• Legal ownership of data. As one panelist noted, communities are increasingly aware that
if studies are publicly funded, the people own the data; thus, they expect to be able to
mine the de-identified dataset.
• Communication extends beyond the principal investigator; it should also include the IRB,
for example.
Subsequent Discussion
Roles and responsibilities. Panelists discussed the researcher's responsibilities related to
communication—namely what, when, and how to communicate. Members of the group agreed
that communication is an important aspect of engagement with participants and the community,
and it is critical to maintaining transparency throughout the study. They also noted the
importance of effective communication with stakeholders (including relevant agencies whose
jurisdiction includes the community), sponsors, the media, and others.
28
-------
Members of the panel outlined some general steps for researchers to consider. Several panelists
suggested that even before securing funding, the researcher should open a dialogue with the
community, letting them know what he/she would like to study and soliciting their input on how
the study could be designed. As one panelist noted, engaging the community at this stage can be
valuable even if some aspects of the study may be "set in stone" in the request for proposals
(RFP). Next, the researcher can approach the sponsor with a preliminary plan in place, including
ideas about how the community will be involved. After securing funding, the researcher can go
back to the community for help refining the study design. At this stage, a community advisory
group also can help the researcher recruit or select participants.
Panelists also discussed interaction with the media. Before the study begins, the researcher may
want to alert the media through press releases, although as two panelists noted, the study should
not be publicized widely until funding is securely in place (i.e., the study will definitely
happen).18 The amount of detail to publicize depends on the study design. For example, a truly
random sample design requires a lot of advance publicity—perhaps including public service
announcements—in order to let members of the community know that they may be contacted by
the research team. In other cases, too much publicity may be detrimental to the study. One
panelist suggested media training for all study partners, noting that the entire team needs to
convey a clear, consistent message to the press and to the public throughout the course of the
study.
Once participants have been selected, panelists agreed, it is important to provide them with
information about the study methodology, including how the researcher plans to provide results.
This information is most often conveyed through the informed consent. As one panelist noted, it
is important not only to define the study methodology, but also to justify it. Another panelist
discussed the value of describing the study to a broader audience, noting that criticism from
activists and politicians may be a symptom of insufficient communication (see discussion on
"anticipating and responding to criticism," below).
To maintain transparency, the researcher should continue communicating with participants and
the community on a regular basis throughout the course of the study. One way to maintain
communication is through written reports; for example, one panelist noted that sponsors may
require regular progress reports. The research team can also hold community meetings. These
reports and meetings can keep the community informed about project status (e.g., recruitment
success, trajectories); meetings can also be a valuable way to solicit feedback from participants,
the community, and stakeholders. In addition, panelists noted the value of educating participants
and the community both before and during the study. Educational information can be general in
nature—for example, one panelist suggested enhancing participants' scientific literacy by
educating them about the scientific method and the basics of epidemiology. The researcher also
can provide general information about risks and indoor hazards, as well as more specific
information about how an observational exposure study works, what types of data will be
18 In post-meeting comments, one panelist (Michael Lebowitz) noted that advance publicity can also occur through
community associations and agencies, and probably through other willing stakeholders as well.
29
-------
obtained, and what the results will look like.19 One panelist discussed the value of educating
participants about issues of confidentiality and data ownership, noting that communities
occasionally ask to know everyone's results without realizing that this would be improper.
Once results are available, the researcher faces a number of decisions about how to report them.
Panelists emphasized the need to report both individual and aggregate results in an
understandable and timely fashion. They highlighted several factors for the researcher to
consider:
• Order of communication. As one panelist observed, participants may be unhappy if they
first learn the results of the study at a meeting or through the media. Participants should
be the first to learn the results, not the last.
• Providing timely results. Panelists agreed that timely reporting is an important goal, but
they also recognized the importance of balancing timeliness with the need for quality
control. In some cases, results cannot be processed and returned to the participant until
several years after the data are collected. In other cases, preliminary results can be
provided quickly, but further analysis could lead to a different interpretation. To address
these concerns, one panelist suggested working with communities to find out exactly
what information they feel is most important to have in a timely fashion. For example,
the community may be happy to receive some preliminary results in the form of general
trends. Another panelist suggested that the researcher focus on timely reporting of any
results of "alarm." As this panelist noted, some people believe the researcher also has an
obligation to ensure that intervention occurs in any situation with alarming results, but
this belief is controversial.
• Reporting in a useful fashion. Panelists discussed ways to provide participants with
results that are useful and understandable. One panelist pointed out the distinction
between data and findings, noting that the latter involves explaining what the numbers
actually mean. Another explained the utility of presenting individual results in the context
of aggregate results, so participants know how their results compare to the overall
distribution. In general, the researcher should consult with the community in advance to
find out what type of reporting would be most useful to them.
• Whether to report all results or just the "meaningful" results. One dilemma faced by
researchers is whether to give participants all results or only the "scientifically relevant"
results—i.e., results associated with known risks to one's health or well-being. Panelists
recognized advantages and disadvantages to each approach:
o The "meaningful" results are probably the most useful to the participant, while
reporting every result may seem like a "data dump" with little meaning. Further,
some IRBs involved with child research have restricted reporting out of concern
that too much information can overwhelm participants and lead to undue stress.
19 In post-meeting comments, one panelist (Michael Lebowitz) recommended that community/participant education
also cover the utility of observational exposure studies.
30
-------
For example, if participants learn that several hundred compounds were detected
in their environment, and they do not know what all the levels mean, they may
worry unnecessarily about their health and feel powerless to do anything about it.
o Providing only "meaningful" results means that someone (e.g., the investigator)
must make a judgment about what may or may not be dangerous. Even if
objective criteria are available (e.g., government guidelines), there are often
several different guidelines from which the investigator must choose. Regarding
concerns about added stress resulting from a "data dump," one panelist suggested
that the public may be more capable of dealing with uncertainty than is sometimes
assumed.
To resolve this dilemma, one panelist suggested giving participants a choice. For
example, the informed consent could include check boxes asking how each participant
wants to receive results (or if he/she wants to receive results at all). The researcher also
can ask for each participant's preference once the results are ready, and can provide
participants with the option to contact the researcher if they initially opt out but later
decide they do want to learn the results. Another panelist suggested reporting all results
by default, but making sure to explain clearly whether each result constitutes a risk.
Panelists emphasized the importance of outlining all reporting procedures up front in the
informed consent, and recommended that the study design clearly identify which
guidelines will be used to determine risk.
• Role of stakeholders. The researcher should provide some aggregate data to stakeholders.
Certain stakeholders may have reservations about reporting data to communities, due to
concerns about the potential economic or political impacts of releasing this information.
One panelist recommended that the researcher listen to stakeholder concerns, but not let
these concerns override the researcher's obligation to report back to participants. Another
pointed out that with public funding, the aggregate results are public information anyway,
so they will inevitably find their way to the Internet or another public forum. Results are
likely to end up in peer-reviewed literature as well.
Along with reporting results to participants and the community, the researcher will probably be
expected to report the findings in peer-reviewed literature—which, as two panelists noted, is
frequently required by the sponsor.20 The researcher can continue to engage the community at
this stage. For example, one panelist reported that he often includes a community member as a
coauthor in his publications.
Develop lay language. Reflecting on earlier discussions, panelists reiterated the importance of
defining key terms and communicating in a way that is linguistically appropriate.21
20 In post-meeting comments, one panelist (Michael Lebowitz) added that researchers are often expected to place the
data on web sites and to provide the data to local agencies and the media.
21 In post-meeting comments, one panelist (Sophie Balk) suggested changing the title of this section to "Develop
understandable language," noting that medical and environmental health professionals may not be familiar with all
31
-------
What constitutes "acceptable" conditions? During the course of a study, the research team
may identify exposures of concern, both related to the study and unrelated. Panelists discussed
numerous complexities regarding when and how to report these concerns to participants.
Several panelists emphasized the importance of defining limits up front, including thresholds for
imminent danger. For many substances, federal or state laws define "acceptable" exposure
levels. In other cases, however, the researcher might have to consider whether a surrogate
standard is appropriate—for example, should indoor PM2.5 concentrations be compared to the
outdoor NAAQS? The researcher should also consider guidance from agencies and organizations
like the American Academy of Pediatrics, which has policies on pesticide use, mold,
environmental tobacco smoke, and other hazards in homes and schools.
Panelists agreed that the researcher has a duty to inform participants about risks related to the
study. However, they disagreed on the extent to which it is necessary to inform participants
about risks unrelated to the study.22 Some panelists favored providing all participants with a
general fact sheet on common household hazards (e.g., injury risks, chemical hazards), rather
than assessing and reporting these hazards on an individual basis. A general fact sheet would
prevent the researcher from getting bogged down in assessing every possible hazard (of which
there are many), and would prevent participants from being confronted with a "laundry list" of
criticism from the researcher, which might leave them feeling overwhelmed or insulted.
However, another panelist argued that relying on general education rather than individual
reporting of hazards has led to criticism in the past—including some of the criticism that led
EPA to convene this panel. To improve observational exposure studies in the future, this panelist
recommended that EPA be scrupulous about informing participants about risks in their homes.
To address the issue of risk communication, panelists suggested some precedents for EPA to
consider, including NAS and World Health Organization (WHO) documents that describe
thresholds for reporting—for example, informing participants about indoor exposures that
exceed occupational standards (see Section 10, "Information Resources"). One panelist
suggested giving participants a choice by asking in the informed consent whether they would like
to be told about hazards unrelated to the study. Panelists also discussed the importance of how
the researcher reports these hazards to participants. In one study, the research team offered to
remedy certain physical hazards rather than simply telling the participant to do it. The research
team also can provide recommendations and resources that empower participants to remedy
hazards. For this approach to work, it helps to have a friendly, trusting relationship with the
participant and to know what resources are at his/her disposal. For example, as one panelist
noted, a renter may not have control over mold in his/her apartment. In this case, rather than just
tell the tenants that they have a mold problem they can do nothing about, the researcher can
provide recommendations and resources to empower the community to tackle larger problems
(e.g., landlord issues) through legal or political action.
the relevant terminology either, even though they probably do not consider themselves "lay people." This panelist
also emphasized the importance of defining all acronyms.
22 In post-meeting comments, one panelist (Sophie Balk) discussed the need to reconcile this idea of "duty" with the
way the panel defined the researcher's duties in the "Study Conceptualization" and "Addressing Privacy" chapters.
32
-------
Ownership of data. Panelists identified several issues related to ownership, including access to
data, retention and future use of samples, and specific concerns related to DNA and genetic-
environmental interaction. They also noted that issues of ownership generally fall into two
domains: the legal and the ethical.
Legal aspects of data ownership are prescribed by law or by court precedent. For example, one
panelist noted that new laws require certain datasets to revert to the public domain after a
specified amount of time. Agencies also have their own regulations. Another panelist cited the
court case of Catalona v. Washington University, which deals with ownership of tissue samples
provided to a researcher. The researcher cannot change the rules, but can educate participants
and the community so they are aware of legal rights and restrictions.
For ethical issues regarding data ownership, the researcher has more room to negotiate with the
community in order to meet their needs and address their concerns. As several panelists noted,
communities often feel that they should own the data—particularly if their tax dollars are
funding the study. By engaging the community throughout the study, the researcher can help the
community feel a sense of ownership. One panelist suggested that the researcher also has an
ethical responsibility to give the community access to any data that will help them reduce
exposure in the future. Another panelist reiterated the distinction between data and findings,
noting that the community typically wants the researcher to provide the latter, but may still want
access to the former as well.
Panelists agreed that legal and ethical issues are important to discuss with communities in
advance and should be included in the study design. If some aspects of ownership are negotiated
through dialogue with the community, this agreement can be formalized through a memorandum
of understanding.
Members of the panel discussed some key ownership issues, including confidentiality. One
panelist noted that communities often request identifiable data, but confidentiality rules prevent
these data from being shared. Thus, it is important to educate the community about
confidentiality. Another panelist pointed out that communities, schools, and other organizations
and institutions have a right not to be identified in public datasets and literature. The researcher
should be sure to inform the community about its right not to be identified, which can prevent
stigmatization.
Emerging concerns include the issues of retention, use, and ownership of genetic information,
particularly given recent efforts to patent genes (e.g., a recent court case regarding ownership of
the gene that causes Canavan disease) and a growing number of studies on genetic-
environmental interaction. One panelist noted that IRBs may have special rules regarding sample
retention, although others suggested that certain aspects of retention and use are open to
negotiation with participants. The scientific community is currently engaged in debate about
whether genetic data can ever be de-identified. One panelist noted a growing consensus that
genetic data cannot be de-identified, suggesting that in the future, public use datasets will have to
be modified accordingly.
Building a relationship with the IRB. One panelist emphasized that the researcher should build
a relationship with the IRB, rather than simply view it as an obstacle to progress. However,
33
-------
another panelist cautioned that the members of the IRB may prefer to keep some distance
between themselves and researchers.
Anticipating and responding to criticism. As one panelist pointed out, observational exposure
studies can attract criticism from activists, politicians, and others who may have a different
worldview from the research community. In some cases, this criticism may be based on
misconceptions. To dispel misconceptions and reduce criticism, one panelist recommended that
researchers anticipate criticism in advance, decide how to address it, and then create a strategy to
get the word out and explain the study before others have a chance to mischaracterize it. Another
emphasized the value of a timely response to criticism, suggesting that researchers partner with
agency administrators and public relations staff to respond quickly with a clear explanation and
justification for the study. Researchers may find it helpful to consider approaches used by EPA
and CDC (see Section 10, "Information Resources").
General thoughts on the communication strategy. One panelist urged EPA to treat the
communication strategy as more than an exercise in public relations. As the panelist noted, it is
important to anticipate and respond to criticism, but the overall strategy should be much broader.
Other facets of the communication strategy include two-way communication with participants
and the community—for example, soliciting feedback throughout the study.
Chapter Title
The panel recommended that EPA cover this topic area in a chapter entitled "Designing and
Implementing Strategies for Effective Communication." One panelist noted (and others agreed)
that the title should include the word "strategies," reflecting the idea that communication should
be planned in advance and should be based on shared expectations with the community.
9. Document Title and Other Elements
Toward the end of the workshop, panelists discussed ideas for the title of EPA's document, the
preamble, and other recommended sections.
Title
The panel recommended that EPA's document be titled "Scientific and Ethical Approaches for
Observational Exposure Studies." As several panelists pointed out, ethical issues were a major
part of the panel's discussion, and should be referenced in the document title. Panelists agreed to
keep "observational exposure studies" in the title per EPA's charge, which Larry Reiter (Director
of NERL) confirmed. One panelist expressed concern that limiting the document to
"observational exposure studies" could "tie EPA's hands" and make it hard for the Agency to
conduct other types of studies in the future, noting that the two major studies that prompted this
review—CHEERS and Kennedy Krieger—were technically intervention studies. However,
another panelist pointed out that the major controversies in these intervention studies involved
the observational aspects, which would be covered by the guidance in EPA's proposed
34
-------
document. Panelists agreed that their findings are applicable to many types of studies (including
intervention studies), and suggested highlighting this connection in EPA's document, perhaps in
a preamble.
The panel recommended using the word "approaches" rather than "practices" because the latter
might imply that the report covers specific methods or case studies. They also decided to remove
the word "measurement," which may be redundant or may be perceived as limiting the scope.
Preamble
The panel recommended that EPA include a preamble, which will "set the stage" and provide
context for the rest of the document. Panelists suggested that this preamble cover the following
items:
• The scope of the document which covers scientific and ethical concerns. This document
focuses on observational exposure studies, but the preamble should explain that many of
the concepts in the document can apply to other types of studies as well, including
intervention studies. One panelist suggested encouraging researchers to consider
alternative and innovative designs as well.
• The intended audience. The document should clearly identify that it is intended for NERL
scientists.
• The complexity of the issues. One panelist suggested listing the key scientific and ethical
issues that must be considered; another suggested that this list could take the form of
some type of schematic. Several panelists pointed out that scientific and ethical
approaches can be study-specific, depending on study intensity and duration, risks, and
societal norms. For example, a study with greater risks to participants may require more
scrutiny, and a longer or more intense study might require more engagement with the
community. It may be useful to provide illustrative examples of a few key concerns—
e.g., observing behaviors that may or may not be socially acceptable.
• Key terms. Most terms can be defined in a glossary, but words related to the title and
scope—e.g., "exposure"—need to be defined up front for the reader.
• Context for why this effort came about. Panelists disagreed on the degree of detail to
provide, with one noting the importance of the CHEERS controversy as a precipitating
event but another advising a more general discussion of potential controversies related to
this type of study. The panel ultimately agreed that while case studies can be useful, this
document should not emphasize CHEERS as a case study. They noted that CHEERS was
a complex political problem that would require a great deal of careful analysis to reach
conclusions. The panel did not talk extensively about CHEERS, and they did not have all
the information they would need for a constructive analysis. Panelists also noted EPA's
emphasis on "moving forward." Instead of discussing CHEERS, one panelist
recommended that EPA choose other illustrative case studies—for example, some of the
studies mentioned in Linda Sheldon's presentation (see Appendix F).
35
-------
• The importance of observational exposure studies. One panelist noted that the first two
paragraphs of EPA's charge to the panel provide some good language.
Other Items to Include in the Document
In addition to the preamble and the six main topic chapters, members of the panel recommended
that EPA include the following elements in its document:
• Information about how the document was developed, including the names of the authors,
panelists, and peer reviewers who contributed to the process.23
• An introductory chapter on the science of observational exposure studies. This will allow
EPA to keep the preamble succinct.
• An introductory chapter on some of the major ethical issues associated with observational
exposure studies. Some of the ethical principles that are currently listed under "Elements
to be Considered in Study Conceptualization" could be addressed more generally in this
introductory chapter, leaving the more specialized chapter to focus specifically on issues
related to observational exposure studies.
• At the end, the document could include a section with general conclusions and
recommendations—for example, recommending that similar documents be created for
other types of studies.
• A glossary. The preamble should define terms that are vital to understanding the title and
scope of the document; however, it would be cumbersome to define too many terms in
the preamble. Other key terms (e.g., "community") should be defined in a glossary.
Panelists also suggested that every chapter include a section on information sources.
10. Information Resources
In the charge to panelists, EPA asked the group to identify sources of additional information to
assist with the development of the document. During the meeting, members of the panel
provided a number of references, to which they added several post-meeting suggestions. These
potential sources are listed below by general topic area:
23 In post-meeting comments, one panelist (Sophie Balk) suggested that this information also include the academic
affiliations and titles of the panelists.
36
-------
Elements to Be Considered in Study Conceptualization
• Researchers can review the literature to obtain more information about how to conduct a
study and when to stop the study.
Ensuring Protection of Vulnerable Groups
• The World Health Organization (WHO) and the European Union (EU) have conducted
extensive research on the elderly and populations with preexisting conditions. WHO
documents are available from http://www.who.int/publications/en/; information on EU
research can be found at http://ec.europa.eu/research/index.cfm.
• Researchers may want to consult geriatric and cultural health journals and National
Academy of Sciences (NAS) and Institute of Medicine (IOM) reports. NAS documents
are available from http://www.nap.edu/; IOM reports are available from
http://www.iom.edu/CMS/2955.aspx.
• One panelist emphasized that when studies involve children, the researcher should
consult pediatricians with expertise in dealing with particular environmental hazards.
Addressing Privacy and Other Concerns Related to Personal Exposure Observational Studies
• Researchers can review the literature to obtain more information about how to conduct a
study and when to stop the study. Ethics journals are an important and often overlooked
source of information, particularly for scientists who are used to consulting the
chemical/toxicological literature but less familiar with ethical research. Panelists
suggested that researchers be given rapid access to ethical reviews.
Creating an Appropriate Relationship Between Participant and Investigator
• There is a large body of literature on effective strategies for informed consent and what
constitutes appropriate compensation.
• The Grimes v. Kennedy Krieger Institute court ruling provides information about the
court's views on compensation, influence, and the researcher's duty of care.
• The 2005 NAS report "Ethical Considerations for Research on Housing-Related Health
Hazards Involving Children" discusses issues related to the Kennedy Krieger ruling. See
http: //www. nap. edu/catal og /11450. html #toc.
• Hoffmann, Diane E., and Karen H. Rothenberg. 2002. "Whose Duty Is It Anyway? The
Kennedy-Krieger Opinion and Its Implications for Public Health Research." J. Health
Care law & Policy 109.
37
-------
• Coleman, Carl H., et al. 2005. The Ethics and Regulation of Research with Human
Subjects. Possible categories of duty to the participant are discussed on pp. 580-583. This
discussion responds to Hoffmann and Rothenberg (2002).
• EPA and researchers can consult relational ethics literature regarding issues of power,
hierarchy, and social context.
Building and Maintaining Appropriate Community and Stakeholder Relationships
• The 2005 NAS report, "Ethical Considerations for Research on Housing-Related Health
Hazards Involving Children," has a chapter on engaging communities. See
http: //www. nap. edu/catal og/11450. html #toc.
• An initiative called "Turning Point: Collaborating for a New Century in Public Health," a
national program supported by the W.K. Kellogg and Robert Wood Johnson Foundations,
has published "Fourteen Policy Principles for Advancing Collaborative Activity Among
and Between Tribal Communities and Surrounding Jurisdictions." Several tribes have
signed onto these protocols. See
http://www.naccho.org/topics/infrastructure/TurningPoint.cfm and
http://archive.naccho.org/documents/TP-policy-principles.pdf.
• Panelists recommended that EPA provide researchers with good examples of a
memorandum of understanding with the community. One panelist specifically suggested
working with experts like Barbara Israel at the University of Michigan.
• Several articles on trust, community, etc., appear in Preventing Chronic Disease, a CDC
web-based journal available at http://www.cdc.gov/pcd/. Several other journals discuss
these issues too, including Ethnic Health.
• Blumenthal, Daniel S. 2006. "A Community Coalition Board Creates a Set of Values for
Community-based Research." Preventing Chronic Disease 3(1): 1-7.
http://www.cdc.gov/pcd/issues/2006/jan/pdf/05_0068.pdf.
• Veazie, M.A., Teufel-Shone, N.I., Silverman, G., Connolly, A., Warne, S., King, B.,
Lebowitz, M.D., and Meister, J.S. 2001. "Building community capacity in public health:
The role of action-oriented partnerships." JPHMgt & Prac 7:21-32.
• The University of Kansas has an online tool box with many resources on community-
level public health, including community-campus partnerships
(http ://ctb. ku.edu/tool s/en/tool stoc. htm).
• Community-campus partnerships could be supported by the Community-Campus
Partners for Health web site at the University of Washington
(http://depts.washington.edu/ccph/index.html).
38
-------
• Emergency research rules may provide some insight on ways to communicate with
communities. The FDA version of these rules appears at 21 CFR Section 50.24.
• Coleman, Carl H., et al. 2005. The Ethics and Regulation of Research with Human
Subjects. A general discussion on emergency research rules appears on pages 321-327,
along with citations for other writings on this topic.
• Corburn, Jason. 2006. "Community knowledge in environmental health science: co-
producing policy expertise." Environ. Sci. Policy (in press).
• One panelist suggested the Superfund Program's "Community Involvement Handbook"
(particularly chapters 1 through 3), although some of the material is limited in scope
and/or not fully updated. Therefore, the reader should use this resource in conjunction
with more current sources. It is not specific to observational exposure studies. See
http://www.epa.gov/superfund/tools/cag/ci_handbook.pdf.
• Several resources are available through EPA's Office of Environmental Justice. These
resources are not specific to observational exposure studies, but may still provide helpful
information on community engagement:
o EPA's "Environmental Justice Strategy" provides a framework and underlying
principles. It provides a few examples of pilot projects with community
involvement. See
http://ww.epa.gov/compliance/resources/policies/ej/ej_strategy_1995.pdf.
o "Environmental Justice: Guidance under NEPA." Pages 7 through 17 are focused
on public participation issues throughout NEPA processes. This resource has lists
of innovative involvement methods. One important point is that "the public" is
different at different phases of a project, and staff are cautioned to actively attend
to these changes during the process. See
http://www.epa.gov/compliance/resources/policies/ej/ej_guidance_nepa_ceql297.
pdf.
o The "Enforcement Assessment Tool" has sections on identifying populations with
respect to minority, vulnerability, and /or economic factors. See
http://www.epa.gov/complicance/resources/policies/ej/ej-seat.html.
o Section 3.3 of the "Toolkit for Assessing Potential Allegations of Environmental
Injustice" expands on the "Enforcement Assessment Tool'"s concepts of
indicators of subpopulations. See
http://www.epa.gov/complicance/resources/policies/ej/ej-toolkit.pdf.
o The "Environmental Justice Collaborative Problem-Solving Cooperative
Agreement Factsheet" includes a brief list of the program's key elements for
community and stakeholder involvement. See
http://www.ep.gov/compliance/resources/publications/ej/factsheets/fact-shet-ej-
cps-grants-6-13 -03 .pdf.
39
-------
o The National Environmental Justice Advisory Council's (NEJAC's) "Model for
Public Participation" lists desired participants, as well as core values and guiding
principles. See
http://www.epa.gov/compliance/resources/publications/ej/nejac/model-public-
part-plan.pdf.
o Chapter 4 of the "Guide on Consultation and Collaboration with Indian Tribal
Governments" discusses public participation. This document cross-references
NEJAC's Model for Public Participation. See
http://www.epa.gov/compliance/resources/publications/ej/nehac/ips-consultation-
guide.pdf.
o NEJAC's May 2000 Meeting Report, "Environmental Justice and Community-
Based Health Model Discussion," includes a summary of data gathering efforts
(Section l.A), along with source citations. It also mentions the need for exposure
data. See
http://www.epagov/compliance/resources/publications/ej/nejac/community-based-
health-recom-report.pdf.
o "Environmental Justice and Federal Facilities" includes recommendations to
improve agency-community relationships and agencies' cultural sensitivity. See
http://www.epa.gov/compliance/resources/publications/ej/nejac/ffwg-final-rpt-
102504.pdf.
o "Meaningful Involvement and Fair Treatment by Tribal Environmental
Regulatory Programs" defines "meaningful involvement" and "fair treatment,"
and describes tribal views of appropriate public participation and due process. See
http://www.epa.gov/compliance/resources/publications/ej/nejac/ips-final-
report.pdf.
o "Future Mechanisms to Improve Stakeholder Involvement" provides
recommendations to improve stakeholder involvement in environmental justice-
related processes. See
http://www.epa.gov/compliance/resources/publications/ej/nejac/stakeholder-
involv-9-27-06.pdf.
• Although ATSDR has written risk communication primers, one panelist pointed out that
these documents were written between the late 1980s and mid-1990s, and are not well-
suited to this workshop. ATSDR health assessments or specific sites might offer some
valuable lessons, however. Interviews with staff may be the most expeditious way to
capture relevant lessons learned.
Designing and Implementing Strategies for Effective Communication
• For ideas on anticipating and responding to criticism, researchers may find it helpful to
consider approaches formerly used by EPA's Office of Risk Communication. One
40
-------
panelist suggested that although this office may have been disbanded, it did produce
some useful documents in the past.
• At CDC, the Office of Communication has been developing emergency risk
communication strategies since the anthrax events that occurred a few years ago. These
approaches may be helpful to researchers. See http://www.bt.cdc.gov/erc/.
• To address the issue of risk communication, panelists mentioned that certain NAS and
WHO documents describe thresholds for reporting. NAS documents are available from
http://www.nap.edu/; WHO documents are available from
http://www.who.int/publications/en/.
• Guidelines and standards for indoor air have been published by the following agencies:
o ATSDR publishes guidelines in its "Toxicological Profiles."
o WHO has published air quality guidelines for Europe (indoor and outdoor),
o NIOSH publishes occupational guidelines,
o EPA has its own air and water guidelines and standards.
General Pediatric Resources
• EPA can consult the NAS report entitled "The Ethical Conduct of Clinical Research
Involving Children." See http://www.nap.edu/catalog/10958.html.
• The American Academy of Pediatrics (AAP) Pediatric Environmental Health Handbook,
2nd Edition. Elk Grove Village, IL: American Academy of Pediatrics, November 2003.
This resource is available through the AAP web site (www.aap.org/publications).
• The AAP Committee on Environmental Health Policy has published and continues to
update and publish new statements and technical reports on many subjects
(http://www.aap.org/visit/cmtel6.htm). A new statement on mold appears in the
December issue of Pediatrics
(http://pediatrics.aappublications.Org/cgi/reprint/118/6/el909).
• Pediatricians who staff the Pediatric Environmental Health Specialty Units can be
helpful. See http://www.atsdr.cdc.gov/HEC/natorg/pehsu.html.
• EPA's Office of Children's Health Protection maintains the searchable Toxicity and
Exposure Assessment for Children's Health (TEACH) database, which contains
overviews of scientific literature on children's environmental health. See
http://www.epa.gov/teach.
• Public Health Partners Children's Environmental Health World Wide Web Sampler. This
resource is a directory of pediatric environmental health web sites, including government,
41
-------
academic, and non-governmental sites such as the AAP's Committee on Environmental
Health and Physicians for Social Responsibility. See
http://www.phpartners.org/cehir/sampler.html.
• Many other pediatric resources are available, including several EPA web sites.
11. Observer Comments
Six observers provided oral comments during the two public comment sessions on the first and
second days of the workshop. This section summarizes their remarks.
Tuesday, November 28, 2006
Heather O 'Maonaigh, Oak Ridge Center for Advanced Studies
Noting that the "state of the science" is constantly evolving, Heather O'Maonaigh encouraged
the panel to consider ways to structure the document so it can be adapted in response to changes
in the science and comments from researchers or the public. She suggested that some type of
structured review process—e.g., an annual review—might be a good way to make sure EPA's
document remains a "living" resource in the future.
Wednesday, November 29, 2006
David Marker, Westat
David Marker suggested several additional items for the panel and EPA to keep in mind. He
highlighted the following five considerations:
• Although community engagement is important, some EPA studies are so large in scope
that it can be difficult to reach out to every participant's community. For example, many
EPA studies are national or even international in scope; others are longitudinal studies
that follow a group of participants over a long period of time, during which they may
move to different communities. In studies like these, it may be unrealistic to require a
high level of engagement in every community from which data are collected. Marker
suggested that the panel may want to caveat their comments and perhaps consider other
approaches for large-scale studies, like talking to a certain subset of communities to gain
the necessary insight.
• Language and socioeconomic status (SES) can affect many aspects of a study. For
example, when working with a community with limited English proficiency or basic
literacy, the basic design of the informed consent and subsequent protocols will be crucial
in determining whether information is shared effectively—particularly if a member of the
42
-------
research team is not present to guide the participant through the document. Researchers
should consider these factors throughout the study plan.
• Marker urged the panel to consider the overlap between issues of statistical power and
representativeness of the sample, which can be affected by recruitment and retention
methods.
• Relationships with third parties can be quite complex—particularly relationships with
landlords. Landlords may serve as gatekeepers, requiring their permission to access their
property. Landlords also may prefer not to be given certain results because this disclosure
can lead to new legal requirements. For example, if a landlord is made aware of lead-
based paint on the premises, he/she becomes responsible for disclosing this information
to future buyers, possibly resulting in a lower property value.
• Marker wondered whether some of the panel's recommendations might differ depending
on the funding source. For example, should extra requirements be placed upon privately
funded studies in order to ensure objectivity?
Robert Clickner, Westat
Exploring some of the tradeoffs related to data quality, Robert Clickner noted that increasing the
incentive for participation can boost the sample size yet at the same time reduce the quality of
the data by attracting disinterested participants who are "in it for the money." He explained that
data from observational exposure studies fall into three categories, with the potential for data
quality problems depending on the category. The categories are as follows:
• Data that the field staff collects on its own (e.g., samples). These data are the least likely
to be influenced by participant motivations because the research team is in full control of
data collection.
• Data that the field staff collects by interacting with participants—for example, through a
questionnaire. There is some potential for error here because disinterested participants
can answer questions untruthfully. However, a well-trained interviewer can elicit more
honest responses, and a well-designed questionnaire can minimize the opportunity to be
dishonest.
• Data that the participant collects on his/her own—e.g., by keeping duplicate diet records
or filling out a diary. This category holds the greatest potential for what statisticians call
non-sampling error or item non-response measurement error, in which the disinterested
participant can affect data quality. If the researcher is aware of these issues up front,
however, he/she can take measures to reduce the potential for error.
Noting the concerns raised by the panel about potentially motivating participants to increase
exposure, Clickner observed that in many cases, a more realistic concern is that the study might
motivate participants not to decrease exposure. Both possibilities can lead to criticism of a study.
43
-------
Marcia Nishioka, Battelle Memorial Institute
Speaking on the topic of study design, Marcia Nishioka noted that the extent to which
participants can control exposure varies, and this is an important factor when selecting an
appropriate study design. Control over exposure falls along a broad spectrum, ranging from
PM2.5—an exposure over which people have very little control—to household pesticides and
chemicals like PFOS and PFOA, which people control directly through their use of consumer
products. Certain study designs may be more appropriate for certain types of exposures, both
scientifically and ethically; for example, a longitudinal study may be less ethically defensible if
the exposure can be controlled easily (i.e., it may be harder to justify "sitting back and watching"
for a long period of time while a participant is exposed to a substance over which he/she does
have control). Nishioka suggested that the panel (and EPA) consider whether certain study
designs are more appropriate for particular exposure situations.
Cynthia Yu-Robinson, NERL
Cynthia Yu-Robinson suggested some additional information sources that might reduce the need
for EPA to "reinvent the wheel" in preparing this state-of-the-science document. Drawing upon
her experience working on community involvement with EPA's Superfund program, she noted
that the following agencies and programs have already developed materials that could be useful
in the context of observational exposure studies:24
• EPA's Superfund program publishes a Community Involvement Handbook, which the
program's Community Relations Officers update every year to reflect new lessons
learned. This handbook is publicly available through EPA's web site.
• EPA's Environmental Justice Program may have useful guidance on working with
minority and low-SES communities.
• EPA's Office of Enforcement and Compliance Assurance (OECA) has developed
protocols for audits and other visits to manufacturing facilities. These protocols cover
disclosure of sampling results, community right-to-know rules, imminent hazards, and
other issues that may have parallels in the context of human observational exposure
studies. OECA may have useful guidance manuals on these topics.
• The Agency for Toxic Substances and Disease Registry (ATSDR) has extensive
experience with risk communication. Among other things, ATSDR works with
communities to understand what is known about exposures, what gaps remain, and what
additional measures could help fill those gaps.
24 Following the meeting, one panelist (Rebecca Parkin) researched several of these resources and added specific
citations to Section 10, "Information Resources."
44
-------
Lisa Melnyk, NERL
As one of the researchers who will ultimately use the document EPA plans to produce, Lisa
Melnyk noted that even when the researcher follows all the appropriate steps to develop a study
that is scientifically and ethically sound, his/her efforts can still be stymied by misperceptions
and by the personal or political agendas of others. She urged the panel to give researchers some
recommendations on how they may be able to dispel misconceptions and get their point across
effectively to the politicians and activists who may be preventing a study from going forward.
45
-------
-------
Appendix A
Panelists
A-l
-------
-------
United States
Environmental Protection Agency
National Exposure Research Laboratory
Expert Panel Workshop on the State-of-the-Science
Approaches for Observational Exposure
Measurement Studies
Hilton Durham near Duke University
Durham, NC
November 28-29, 2006
Invited Experts
Sophie Balk
Attending Pediatrician
Children's Hospital at Montefiore
Professor of Clinical Pediatrics
Albert Einstein College of Medicine
3415 Bainbridge Avenue
Bronx, NY 10467-2940
718-405-8090
Fax: 718-405-8091
Email: sbalk@montefiore.org
Timothy Buckley (Panel Chair)
Division of Environmental Health Sciences
School of Public Health
The Ohio State University
1214 Kinnear Road - Room 250D
Columbus, OH 43212
614-292-6180
Fax: 614-292-6485
Email: tbuckley@sph.osu.edu
David Carpenter
Director, Institute of Health and Environment
University of Albany, SUNY
5 University Place - Room A217
Rensselaer, NY 12144
518-525-2660
Fax: 518-525-2665
Email: Carpent@uamail.albany.edu
Giselle Corbie-Smith
Department of Social Medicine
CB# 7240, Medical School Wing D - Room 378-B
University of North Carolina-Chapel Hill
Chapel Hill, NC 27599-7240
919-843-6877
Fax: 919-966-7499
E-mail: gcorbie@med.unc.edu
Alan Fleischman
Senior Advisor
The New York Academy of Medicine
1216 Fifth Avenue at 103rd Street
New York, NY 10029
212-822-7368
Email: afleischman@nyam.org
Natalie Freeman
Center for Environmental and Human Toxicology
Department of Physiological Sciences
University of Florida - College of Veterinary
Medicine
Building 471 Mowry Road
Gainesville, FL 32611
352-392-4700 x 5545
Fax: 352-392-4707
Email: FreemanN@mail.vetmed.fl.edu
Environmental Health
Department of Behavioral Science & Community
Health
College of Public Health and Health Professions
Email: nfreeman@phhp.ufl.edu
A-3
-------
Loretta Jones
Healthy African American Families
3845 Martin Luther King Jr. Boulevard - #209
Los Angeles, CA 90008
323-292-2002
Email: ljonesHAAF@aol.com
Bruce Lanphear
Professor of Pediatrics and of Environmental
Health
Cincinnati Children's Hospital Medical Center
Division of General & Community Pediatrics
3333 Burnet Avenue
Cincinnati, OH 45229-3039
513-636-3778
Fax: 513-636-4402
E-mail: bruce.lanphear@chmcc.org
Michael Lebowitz
Retired, Professor of Medicine & Epidemiology
Res. Professor of Medicine, Pending
University of Arizona
Colleges of Public Health & Medicine
2528 West Criswell Court
Tucson, AZ 85704
520-620-6631
E-Mail: mlebowit@u.arizona.edu
Jerry Menikoff
Department of History & Philosophy of Medicine
University of Kansas Medical Center
3901 Rainbow Boulevard (M.S. 1025)
Kansas City, KS 66160
913-588-7662
Email: jmenikof@kumc.edu
Rebecca Parkin
Associate Dean for
Research and Public Health Practice
Professor of Environmental and
Occupational Health
School of Public Health and Health Service
The George Washington University
Medical Center
202-994-4569
Fax: 202-994-3773
Email: parkinr@gwu.edu
A-4
-------
Appendix B
Charge to Panelists
B-l
-------
-------
United States
Environmental Protection Agency
National Exposure Research Laboratory
Expert Panel Workshop on the State-of-the-Science
Approaches for Observational Exposure
Measurement Studies
November 28-29, 2006
Durham, NC
Charge to the Expert Panel
Purpose of the Workshop
Observational human exposure measurement studies are performed by researchers both within and
outside of EPA to measure human exposure to chemicals in the environment. These studies involve
measurements of chemicals in environmental media (e.g., air, water, food, soil, and dust); biomonitoring
(e.g., collection of urine, saliva, or blood samples), and collection of information about the voluntary
study participants, their homes, their work environments, and their activities. EPA and other federal
agencies have been performing observational human exposure measurement studies for over two
decades. These are critical to improving our understanding of human exposure to chemicals and meeting
our goal of improving public health.
Observational exposure measurement studies are complex in their design and implementation. Many
scientific and ethical issues need to be addressed in the planning, scoping and study design; the
community interactions; study implementation; and communications with the study participants and their
communities. EPA wants to ensure that the observational studies conducted by the Agency for measuring
human exposure to environmental chemicals continue to be based on the most up-to-date sound science
and the highest ethical standards. To meet that objective, researchers in EPA's National Exposure
Research Laboratory (NERL) are examining the latest approaches, methods, techniques, and ethical
standards for performing such studies and intend to compile a set of state-of-the-science approaches for
performing observational human exposure measurement studies. Examples of study elements include
survey design, identification of community groups and interactions with communities during the scoping
and planning of studies, participant recruitment methods, informed consent procedures, identification and
reporting unanticipated results, communication of study results, etc. EPA wants to ensure that their
observational studies not only represent excellence in science, but also the highest standards of ethical
conduct.
Accordingly, the purpose of this workshop is to obtain expert advice and public comment on the
development and content for a "state of the science" document. The workshop panel consists of experts
from different disciplines with relevant expertise on scientific and ethical issues to be addressed in the
design and implementation of observational human exposure measurement studies.
B-3
-------
Charge to the Pane!
The panel is asked to consider these issues prior to the workshop in preparation for discussion during this
workshop meeting:
1. Provide recommendations on the content and organization of the document.
a. Identify the major scientific and ethical areas/issues in the design and implementation of
observational human exposure measurement studies that should be considered for inclusion
in the document.
b. Identify specific elements in each of these major areas that should be considered for
inclusion in the document.
c. Provide recommendations on the type and level of information that should be considered for
inclusion in the document when describing state-of-the-science approaches, methods,
techniques, or standards.
d. Provide recommendations on the criteria that should be considered when evaluating and
identifying the state-of-the-science for the approaches, methods, techniques, or standards.
2. Provide recommendations and listings of sources of information for developing the document
including case studies where available.
3. Identify at least ten specific elements of the design and implementation of these studies that the
panel considers to have the most uncertainty with regard to the "state-of-the-science," discuss these
elements, and provide recommendations on state-of-the-science approaches for them.
B-4
-------
Appendix C
Panelists' Pre-Meeting Input
c-i
-------
-------
Panelists' Pre-Meeting Input: Suggested Topics for
Discussion
Sophie Balk
Ideas for components or sections for a framework for conducting observational human exposure
studies:
• Studies must have an explicit focus on children. Studies should consider children as a
separate category. Children have different and generally higher exposures compared to
adults. The health and developmental consequences of their exposures may be more
serious and lifelong. Pediatricians and other child health experts should be a part of the
study design and approval process.
• Studies must be designed to conform to best pediatric practices. Study planners should
design observational studies to protect children and be consistent with current pediatric
advice. For example, pediatricians usually discourage pesticide spraying in homes where
children reside, preferring other methods of pest control whenever possible. If routine
spraying is allowed as part of an observational study, this contradicts best pediatric
practices.
• The process for approving studies must be transparent and accessible to the public. Study
design and information about Human Subjects Review must be posted for public review.
Information about study design must incorporate detailed information about the study.
Information about Human Subjects Review includes the membership and affiliations of
the Human Subjects Review Panel, questions they asked and answered while reviewing
the study, and any modifications made. Scientific information must be presented in ways
that are understandable to the general public.
• Community participation must be ensured. Community representatives, (including
parents if children are involved) must be included at all points of study design and
approval.
David Carpenter
My suggestions for issues to be discussed at the workshop are as follows:
• When should results of human testing be reported back to the individual, and when is it
ethically justified not to report results?
• How must an investigator deal with explaining results when reporting back to a study
subject if the result is somewhat abnormal, but the health consequence of the abnormal
C-3
-------
result is not clear? Which is worse - scaring the subject or giving the subject a false sense
of security?
• What sort of compensation is appropriate under the circumstances where a biomonitoring
program is being done in order to determine background levels of a contaminant in the
general population and where there little or no immediate benefit to the individual who
participates?
• How far can an investigator ethically go in utilizing invasive procedures when doing
human testing that is being done for the purpose of obtaining background data in a
population? Is something like a fat biopsy appropriate? What about X-rays or other
procedures that carry a small risk?
Gisele Corbie-Smith
The following are what I see as key issues:
• Ethical principles in community based research with underserved populations
• Community engagement in observational exposure research
• Community input in review and IRB approval of community based research
Natalie Freeman
The specific elements of a study need to be based on the objectives of the study and the
hypotheses being considered. Observational studies, while not "experiments," can if properly
designed be used to test hypotheses. Observational studies may be simply descriptive or by
comparing well defined groups, can be used to evaluate and infer differences based on
characteristics of the groups.
Study design issues include: location in terms of region(s) of the country, urban and/or rural,
residential and/or institutional, monitoring period in terms of days, weeks, months, years, study
population(s) and its/their representativeness of the population(s) of concern, monitoring
protocol including sampling times and durations, media of interest, analytes of interest, survey
and questionnaire information.
Ethical issues focus on outreach and consenting activities, providing potential participants with
an adequate understanding of the study, its goals, and both participant and researchers'
expectations.
There has been a great deal of discussion as to the reasonableness of incentives and remuneration
to participants and whether these are in fact bribes. There is an extensive literature on the impact
C-4
-------
of incentives on participation and research outcomes. It is this commenter's opinion that it is
unethical not to remunerate participants for their time and activities. I also believe that incentives
are permissible as long as they are not so great as to modify people's behavior.
A second part of linked study design/ethical issues is related to communicating results to
participants. Sharing exposure results with study participants is a multifaceted process. The
communication begins before the study starts during the outreach and informed consent process
and addresses the objectives of the study, and anticipated results for the participant.
Solicitation of participants is driven by the study design. Presenting to study objectives to
potential participants is only one part of solicitation and outreach. To be successful, the
researcher needs to understand something of the range of motivations and concerns that prompt
individuals to participate in exposure science projects.
Loretta Jones
EPA NERL speaking points:
• Community Engagement
• What's the win-win for community?
• Observational Studies—when to inform Community of a danger that exists?
• Follow-up to Children's Studies
• How to connect to other Research and Advocacy
Bruce Lanphear
I had a few ideas:
• Reporting individual test results
• Definition of community input, involvement and representation
• Funding observational studies; should industry have any involvement?
• Definition of appropriate remuneration
C-5
-------
Michael Lebowitz
I think the five key issues relate to:
• The type of sampling used in the study design (random vs. stratified cluster) with
emphasis on over sampling presumed high exposure & low SES clusters (to maximize
the info on the upper 25th percentiles of exposure and reduce cost)
• Use of lessons learned from NHEXAS & other major exposure studies, especially
regarding recruitment of participants (see below)
• The engagement of the communities involved (including communications strategy - a
NHEXAS draft will be sent if/when I can find it)
• The development of staged sampling with improved screening methods
• Concentration on the cells in the chemical-media that are likely to yield knowledge,
ignoring those known to be close to null, especially those deemed important re:
regulations/laws/etc.
There's more, also very important in my mind: design and modeling to provide input for risk
assessment and for epidemiological studies.
We shouldn't forget the words to the wise either:
Lioy P, Leaderer B, Graham J, Lebret E, Sheldon L, Needham L, Pellizzari E, Lebowitz MD.
"The application of exposure assessment to environmental health science and public policy." J
Expos Analysis & Environ Epidemiol. 2005.
EPA - NHEXAS - LESSONS LEARNED - Supplemental Notes by Mike Lebowitz
(pre- & post- Workshop of 8/14/01) (in partial fulfillment of P.O.)
I. Project Leadership
What was good:
• NHEXAS leadership showed that it was possible to mount and conduct
large-scale, multi-media studies in the field
• Leadership, and the cooperative agreements, worked well in general;
collaborations were very evident
• Communications with all worked well in general
C-6
-------
• Working with communities and other partners (AZ & Region 5) went
well
What were the problems:
• Delays, delays, delays
• Over-committed and under-funded
• Insufficient time and money to perform analyses and disseminate results
Primary lessons for the future include:
• Leadership should prepare and agree upon a study design and the
logistics plan early
• Prepare the rest of the framework so that each new endeavor doesn't
start from scratch, and be flexible
• There should be a limited set of objectives and focus
• Leadership should utilize what's already known, use standard
instruments & pre-test new instruments quickly, and get all-around
approval early
• Get the QSIPs & SOPs done early, partly by modifying existing ones
• Get the IRB and other reviews started right away
• Set milestones and try to adhere to them (but be flexible)
• Budget & forward fund appropriately, and be cognizant of relative costs
II. Design
What went well:
• There was a study design and a research plan (logistics) for each
consortium
• The primary objectives (population distributions of exposure and 90th
percentiles) were met
• There were some hypotheses
What were the problems:
• Too many objectives
C-7
-------
• Strenuous sample size calculations
• Too much and too many SOPs, and too difficult to follow changes
• Logistical problems getting participants (and not knowing compliance)
• Too many non-detects - especially for some compounds in some media
Primary Lessons for the Future:
Study Design:
• Very much need to focus more and determine what the goal is, then
design & calculate sample size
• If what is wanted is Total-Cumulative-Aggregate exposure, especially
longitudinally, then design the studies for that purpose (which is
different than designing it for pop distributions of exposure)
• Balance SOWs, burdens , costs, compatibilities, response rates
• Determine what can be effectively used for screening, in advance
Logistics:
• Secure a dedicated and trained staff to best work in the region for what
is planned
• Pre-plan, pre-test, and set
• Improve response rates and identify the characteristics of the non-
respondents
• Measure compliance, and identify the characteristics of the non-
compliant
• Make, track and document modifications easier
• Script important SOPs
• Maximize the quantification and completeness of the data base
• Streamline everything
• Minimize burden of investigators and participants
• Use reference "labs" throughout
C-8
-------
VI. Data Base(s) (post-workshop summary)
Issues & Recommendations:
• Timeliness
• Input tied to forms, data shells, built-in QA/QC
• Consider CAPI and/or CATI systems
• Use standard programs, formats, codes (including for non-detects), &
SOPs where possible, or create ahead of time
• QA/QC
• Data Tracking and pre-determination of division of labor (and
responsibilities)
• Flag and document problems
• Completeness
• Presence of resources and funding
• Tie data base formation to: reporting, analysis, products, marketing
• Archive originals (SOPs, modifications, data forms, codes, etc.)
VII. OA (post-workshop summary)
(refer also to above)
• Create framework & streamlined procedures
• Needs to be collaborative
• Prepare early, use past experience and SOPs
• Need pre-planning and testing; decrease delays
• Need reference materials and labs
• Need consistency/uniformity and comparability studies
• Need good documentation & audits
C-9
-------
Jerry Menikoff
Here are some ideas for discussion points, as was requested:
• Payment Issues: Payments to subjects should be designed in such a way that they do not
encourage parents to change their behaviors in a way that increases the exposure of
children to chemicals. This goal can require more than merely verifying that payments
are similar to the hourly amounts that the parents would receive in similar jobs; it may
also require verifying that the parents could as readily obtain such similar payments
without major alterations in their daily lives (e.g., if in a study the parents are paid for
time they spend at home, could they receive similar payments for time spent at home if
there was no such study).
A related issue is that payments should be designed so as to not only discourage changes
in behavior leading to increased exposure to chemicals for the purpose of being admitted
to the study, but also to prevent participation in the study from altering decisions made
after enrollment in the study.
• Issues relating to initial identification of prospective subjects: If there is a concern
regarding the possibility that prospective subjects might have an incentive to alter their
behaviors in order to participate in the study, how can we identify prospective subjects in
a way that provides objective evidence of pre-existing behaviors by the subjects, so that
we know they will not have to alter their behavior to qualify for the study? Are there
objective indicia of pre-study behaviors that can be verified as part of recruiting subjects?
A related issue might be to what extent the specific criteria for enrollment in the study
might be kept somewhat confidential. To the extent subjects are not aware of those
criteria, it would make it difficult for them to be attempting to demonstrate that their prior
behaviors are consistent with those eligibility criteria.
• The Ethics of Watching Harmful Behaviors: To what extent is it appropriate for
researchers to sit by and watch subjects participate in potentially harmful behaviors (such
as exposing their children to pesticides)? It would be desirable to address which such
behaviors are considered "acceptable" (and thus it is appropriate for researchers to allow
such behaviors to continue, while watching and collecting information), and which such
behaviors would not be acceptable and under what circumstances the researchers would
therefore intervene and stop the behavior. (E.g., allowing pesticides to be used according
to their labeling instructions, but not permitting subjects to apply pesticides in a way that
violates those labeling instructions).
C-10
-------
Rebecca Parkin
Here are some ideas for key elements for the document. I have not ordered them in any particular
way:
• What is the problem being addressed?
• How is the problem defined and by whom?
• How are affected persons and the community engaged? (e.g., design, feedback, and
decisions)
• What level of exposure assessment precision is needed for addressing the problem?
• What are the sources of uncertainty and types of variability that need to be considered
when interpreting the measurements?
• What is exposure? Is it a single factor, cumulative, aggregate, etc? What is the right
exposure to be measured?
• What is the right set of exposure metrics? Are the metrics reliable?
• What are the key cofounders? Can they be measured?
• What role to do behaviors play? Can they be adequately characterized for addressing the
problem?
• How will "state of the science" disputes be resolved?
• Is there a complete exposure pathway—conceptually and in reality? Can the entire
pathway be measured?
• Are the modes of action, mechanisms understood?
• Is information (e.g. toxicokenetics) available to reliably transfer biological metrics from
animals to humans?
C-ll
-------
-------
Appendix D
Meeting Agenda
D-l
-------
-------
United States
Environmental Protection Agency
National Exposure Research Laboratory
Expert Panel Workshop on State-of-the-Science
Approaches for Observational Exposure
Measurement Studies
Hilton Durham
Durham, NC
November 28-29, 2006
Agenda
DAY 1 - Tuesday, November 28, 2006
7:45 AM Registration/Check-in
8:30 AM Introductory Remarks Jan Connery, ERG
8:40 AM Welcome Remarks Kevin Teichman, EPA/ORD
8:45 AM Opening Remarks Warren Lux, EPA/OSA
9:00 AM Panel Introduction Jan Connery, ERG
9:15 AM Overview of Observational Exposure
Measurement Studies Linda Sheldon, EPA/NERL
9:30 AM Background on State-of-the-Science Approaches for
Observational Exposure Measurement Studies Roy Fortmann, EPA/NERL
9:45 AM BREAK
10:00 AM Observer Comment Period Jan Connery, ERG
11:00 AM Overview of the Charge
and Workshop Format Tim Buckley, Panel Chair, Ohio State University
11:30 AM Panel Discussions: General Comments
12:30 PM LUNCH (on own)
1:45 PM Breakout Groups: Discuss Topics I & II (to be determined)
3:15 PM BREAK
D-3
-------
DAY 1 - Tuesday, November 28, 2006 - continued
3:30 PM Plenary: Group Reports & Discussion on Topics I & II
4:15 PM Breakout Groups: Discuss Topics III & IV (TBD)
5:00 PM ADJOURN
DAY 2 - Wednesday, November 29, 2006
8:00 AM Breakout Groups: Discuss Topics III & IV (continued)
8:45 AM Plenary: Group Reports & Discussion on Topics III & IV
9:30 AM BREAK
9:45 AM Plenary: Discuss Topics V & VI (TBD)
Noon LUNCH (60 minutes)
1:00 PM Observer Comment Period
1:30 PM Plenary Discussions: Development of Recommended Framework
• Discussion and formulation of responses to charge questions
• Framework for document
2:30 PM BREAK
2:45 PM Plenary Discussions (continued)
4:20 PM Concluding Remarks
4:30PM ADJOURN
D-4
-------
Appendix E
Meeting Attendees
E-l
-------
-------
(¦ » }
United States
Environmental Protection Agency
National Exposure Research Laboratory
Expert Panel Workshop on the State-of-the-Science
Approaches for Observational Exposure
Measurement Studies
Hilton Durham near Duke University
Durham, North Carolina
November 28-29, 2006
Observers
Daniel Barker
Scott Clifton
Michael Firestone
Environmental Scientist
Physical Scientist
Science Director
HRSD
Human Exposure &
Child & Aging Health Protection Division
1432 Air Rail Avenue
Atmospheric Sciences Division
U.S. Environmental Protection Agency
Virginia Beach, VA 23455-3002
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
757-460-4247
Attn: Matthew Clifton (MD E205-04)
Washington, DC 20460
Email: dbarker@hrsd.com
Research Triangle Park, NC 27711
202-564-2199
919-541-4612
Email: firestone.michael@epa.gov
Timothy Barzyk
Fax: 919-541-0905
Human Exposure &
Email: clifton.matthew@epa.gov
Roy Fortmann
Atmospheric Sciences Division
Project Leader
U.S. Environmental Protection Agency
Elizabeth Dederick
Human Exposure &
(MD E205-4)
The LifeLine Group
Atmospheric Sciences Division
Research Triangle Park, NC 27711
1453 East Broad Street
U.S. Environmental Protection Agency
919-541-1520
Westfield, NJ 07090
109 TW Alexander Drive (MD E205-04)
Email: barzyk.timothy@epa.gov
443-845-6097
Research Triangle Park, NC 27711
Email: edederic@jhsph.edu
919-541-1021
Annie Chaisson
Email: fortmann.roy@epa.gov
The LifeLine Group
Peter Egeghy
2504 Yorkdale Court
Research Environmental
Sydney Gordon
Durham, NC 27707
Health Specialist
Battelle Memorial Institute
919-360-0716
U.S. Environmental Protection Agency
505 King Avenue
Email: amblol@msn.com
109 TW Alexander Drive (E205-04)
Columbus, OH 43201
Durham, NC 27711
614-424-5278
Robert Clickner
919-541-4103
Email: gordon@battelle.org
Associate Director,
Fax: 919-541-0905
Environmental Studies
Email: egeghy.peter@epa.gov
Richard Hermann
Westat
Director, Human Research Protocol Office
1650 Research Boulevard
Zaida Figueroa
U.S. Environmental Protection Agency
Rockville, MD 20850
Industrial Hygienist
(MD 58A)
301-294-2815
Human Exposure &
Research Triangle Park, NC 27711
Email: Robertclickner@westat.com
Atmospheric Sciences Division
919-966-6217
U.S. Environmental Protection Agency
Email: Hermann.Richard@epa.gov
4930 Page Road (MD E205-04)
Durham, NC 27703
Ross Highsmith
919-541-0875
U.S. Environmental Protection Agency
Fax: 919-541-0905
(D305-01)
Email: figueroa.zaida@epa.gov
Research Triangle Park, NC 27711
919-541-7828
Email: highsmith.ross@epa.gov
E-3
-------
Richard Honeycutt
President
H.E.R.A.C., Inc
220-1 Swing Road
Greensboro, NC 27409
336-294-5559
Fax: 336-294-3533
Email: herac@aol.com
Paul Jones
Math Statistician
U.S. Environmental Protection Agency
109 TW Alexander Drive
Durham, NC 27711
919-541-1516
Email: jones.paul@epa.gov
Eileen Kennedy
Syngenta Crop Protection, Inc
P.O. Box 18300
Greensboro, NC 27419-8300
336-632-7387
Fax: 336-632-7322
Email: eileen.kennedy@syngenta.com
Andrew Lee
Economist
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
703-308-7226
Email: lee.andrew@epa.gov
Warren Lux
Human Subjects Research
Review Official
Office of the Science Advisor
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW (8105R)
Washington, DC 20460
202-564-3746
Email: lux.warren@epa.gov
Christopher Lyu
Associate Director/IRB Member
Battelle
100 Capitola Drive - Suite 301
Durham, NC 27713
919-544-3717
Fax: 919-544-0830
Email: Lyuc@battelle.org
David Marker
Associate Director and
Senior Statistician
Westat
1650 Research Boulevard
Rockville, MD 20850
301-251-4398
Fax: 301-294-2829
Email: davidmarker@westat.com
Suzanne McMaster
Human Studies
U.S. Environmental Protection Agency
104 Mason Farm Road
Chapel Hill, NC 27599
919 966 6385
Fax: 919-966-7584
Email: mcmaster.suzanne@epa.gov
Lisa Melnyk
Microbiological and Chemical Exposure
Assessment Research Division
U.S. Environmental Protection Agency
26 West Martin Luther King Drive
(MD 564)
Cincinnati, OH 45268
513-569-7494
Fax: 513-569-7757
Email: melnyk.lisa@epa.gov
Pauline Mendola
Branch Chief
Human Studies Division
U.S. Environmental Protection Agency
(MD 58C)
Research Triangle Park, NC 27711
919-966-6953
Fax: 919-966-0655
Email: mendola.pauline@epa.gov
Marsha Morgan
Research Environmental Health Scientist
U.S. Environmental Protection Agency
109 TW Alexander Drive
Durham, NC 27709
919-541-2598
Email: morgan.marsha@epa.gov
Marcia Nishioka
Battelle Memorial Institute
505 King Avenue
Columbus, OH 43201
614-424-4964
Fax: 614-424-3638
Email: nishiomg@battelle.org
Heather O'Maonaigh
Research Associate
Oak Ridge Center for Advanced Studies
PO Box 2008 - Building 5100 (MS-6173)
Oak Ridge, TN 37831
865-574-2794
Email: Heather.Omaonaigh@orau.org
James Quackenboss
Acting Assoc. Director for Health
Human Exposure &
Atmospheric Sciences Division
U.S. Environmental Protection Agency
P.O. Box 93478
Las Vegas, NV 89193-3478
702-798-2642
Fax: 702-798-2261
Email: Quackenboss.James@epa.gov
Kristen Rappazzo
ASPH Fellow
Human Studies Division
U.S. Environmental Protection Agency
(MD 58A)
Research Triangle Park, NC 27711
919-843-4435
Email: rappazzo.kristen@epa.gov
Lawrence Reiter
Director
National Exposure Research Laboratory
U.S. Environmental Protection Agency
109 TW Alexander Drive- Room D310
(D305-01)
Research Triangle Park, NC 27709
919-541-2106
Fax: 919-541-0445
Email: reiter.larry@epa.gov
Leah Rosenheck
Senior Human Safety Risk Assessor
Syngenta Crop Protection, Inc.
P.O. Box 18300
Greensboro, NC 27419
336 632-2329
Email: leah.rosenheck@syngenta.com
Linda Sheldon
Acting Director
Human Exposure &
Atmospheric Sciences Division
U.S. Environmental Protection Agency
109 TW Alexander Drive (E205-01)
Research Triangle Park, NC 27709
919-541-2454
Email: Sheldon.Linda@epa.gov
Carvin Stevens
Research Chemist
HEASD/EAMB
U.S. Environmental Protection Agency
109 TW Alexander Drive (E205-04)
Research Triangle Park, NC 27711
919-541-1515
Email: stevens.carvin@epa.gov
Daniel Stout II
Biological Scientist
Human Exposure &
Atmospheric Sciences Division
U.S. Environmental Protection Agency
109 TW Alexander Drive (E205-04)
Research Triangle Park, NC 27711
919-541-5767
Fax: 919-541-0905
Email: stout.dan@epa.gov
Takahiko Suwa
Director
Shiseido Cosmetics ltd.
100 Tokeneke Road - Suite A
Darien, CT 06820
203-656-7999
Fax: 203-656-7995
Email: takahikosuwa@shiseidoatec.com
E-4
-------
Kevin Teichman
Acting Deputy Assistant
Administrator for Science
National Exposure Research Laboratory
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW (8104R)
Washington, DC 20460
202-564-6705
Email: teichman.kevin@epa.gov
Kent Thomas
U.S. Environmental Protection Agency
MD E205-04
Research Triangle Park, NC 27711
919-541-7939
Email: thomas.kent@epa.gov
Chris Trent
Assistant to the Director
U.S. Environmental Protection Agency
109 TW Alexander Drive (D305-01)
Research Triangle Park, NC 27711
919-541-5337
Email: trent.chris@epa.gov
Nicolle Tulve
U.S. Environmental Protection Agency
109 TW Alexander Drive (MD E205-04)
Research Triangle Park, NC 27709
919-541-1077
Email: tulve.nicolle@epa.gov
Elin Ulrich
Human Exposure &
Atmospheric Sciences Division
U.S. Environmental Protection Agency
109 TW Alexander Drive (D205-05)
Research Triangle Park, NC 27711
919-541-3717
Don Whitaker
Human Exposure &
Atmospheric Sciences Division
U.S. Environmental Protection Agency
109 TW Alexander Drive (MD E205-04)
Research Triangle Park, NC 27711
919-541-1571
Fax: 919-541-0905
Email: whitaker.donald@epa.gov
Ron Williams
Research Chemist
U.S. Environmental Protection Agency
(MD E205-04)
Research Triangle Park, NC 27711
919-541-2957
Email: williams.ronald@epa.gov
Nancy Wilson
Battelle
100 Capitola Drive - Suite 301
Durham, NC 27713
919-544-3717
Fax: 919-544-0830
Email: wilsonk@battelle.org
Cynthia Yu-Robinson
Communication Team Leader
National Exposure Research Laboratory
U.S. Environmental Protection Agency
109 TW Alexander Drive (D305-01)
Research Triangle Park, NC 27711
919-541-0682
Fax: 919-541-3615
Email: yu.cynthia@epa.gov
ERG Technical Support Staff:
Jan Connery - Workshop Manager
Eastern Research Group
110 Hartwell Avenue
Lexington, MA 02421
781-674-7200
Fax: 781-674-2851
Email: jan.connery@erg.com
Chris Lamie - Technical Writer
Eastern Research Group
110 Hartwell Avenue
Lexington, MA 02421
781-674-7200
Fax: 781-674-2851
Email: chris.lamie@erg.com
Email: ulrich.elin@epa.gov
E-5
-------
-------
Appendix F
Presentation by Linda Sheldon, EPA
F-l
-------
-------
Linda Sheldon
U.S EPA
Office of Research and Development
National Exposure Research Laboratory
November 28, 2006
Outline
Describe the framework for EPA's research
Describe observational measurement studies
¦ what they are,
¦ how they are conducted,
¦ why they are important.
Describe important considerations for future
studies
RESEARCH & DEVELOPMENT
Building a scientific foundation for sound environmental decisions
-------
-------
-------
-------
Surface Residue Wipes
r.. JP
x a «ni"
*
!
RESEARCH & DEVELOPMENT
Building a scientific foundation for sound environmental decisions
5
-------
Hand wipes, Residue Transfer
RESEARCH & DEVELOPMENT
Building a scientific foundation for sound environmental decisions
Dust Sam
-------
Dietary Samples
Solid food
Beverages
Breast milk
Water
RESEARCH & DEVELOPMENT
Building a scientific foundation for sound environmental decisions
Activity
Diaries
Videotaping
Metabolic activity
Location
RESEARCH & DEVELOPMENT
Building a scientific foundation for sound environmental decisions
-------
-------
-------
10
-------
Appendix G
Presentation by Roy Fortmann, EPA
G-l
-------
-------
State-of-the-Science
Approaches for
Exposure Measurement Studies
ational Exposure Research Laborat
Office of Research and Development
Research Triangle Park, NC
November 28, 2006
-------
-------
Goals of this Project
• Compile information on the current
methods, standards, and approaches for
conducting observational studies
• Prepare a peer-reviewed document that
will serve as an information resource for
researchers both within and outside of EPA
5
RESEARCH & DEVELOPMENT
Building a scientific foundation for sound environmental decisions
The Process and Timeline
3
-------
-------
-------
-------
-------
Charge to the Panel
• Framework and content of the document - what
areas and elements should be addressed?
How should the information be presented?
• Sources of information - what sources are
available to ensure that the state of the science
is captured?
• State of the science - for specific elements
identified by the panel, what is the state of the
science? How do researchers determine what
is the state of the science?
15
RESEARCH & DEVELOPMENT
Building a scientific foundation for sound environmental decisions
Questions
• Questions from the panel?
16
RESEARCH & DEVELOPMENT
Building a scientific foundation for sound environmental decisions
-------
Appendix H
Presentation by Tim Buckley, Panel Chair
H-l
-------
-------
Overview of the Charge and
Meeting Format
Charge to the Panel
. Provide recommendations on the content and
organization of the document.
- Identify the major scientific and ethical areas/issues in the design
and implementation of observational human exposure
measurement studies that should be considered for inclusion in
the document.
- Identify specific elements in each of these major areas that
should be considered for inclusion
- Provide recommendations on the type and level of information
that should be considered for inclusion in the document when
describing state-of-the-science approaches, methods,
techniques, or standards.
- Provide recommendations on the criteria that should be
considered when evaluating and identifying the state-of-the-
science for the approaches, methods, techniques, or standards.
-------
Charge (cont)
2. Provide recommendation and listing of sources
of information for developing the document
including case studies where available.
3. Identify at least ten specific elements of the
design and implementation of these studies
that the panel considers to have the most
uncertainty with regard to the "state-of-the-
science," discuss these elements, and provide
recommendations on state-of-the-science
approaches for them.
Workshop Format
• Breakout session topics: identify and balance
the scientific benefits of human observational
studies with risks to the community and
participating subjects in:
- Study Design (Group A)
- Studies Involving Vulnerable Groups (Group B)
- Recruitment (Group A)
- Studies of Home Hazards (Group B)
- Community Participation (Group A)
- Communication (Group B)
2
-------
Breakout Groups
Group A • Group B
- Carpenter - Balk
- Corbie-Smith - Parkin
- Freeman - Fleischman
- Lanphear - Jones
- Menikoff - Lebowitz
Groups self-select a discussion facilitator and rappateur.
3
-------
-------
Appendix I
Topics and Subtopics Discussed by the Panel
1-1
-------
-------
Workshop to Discuss State-of-the-Science
Approaches for Observational Exposure
Measurement Studies
November 28-29, 2006
Durham, NC
Notes from the Plenary
Discussion
1
Title
• Do we recommend a new/different title?
- Original title: "State of the Science
Approaches for Observational Exposure
Measurement Studies"
- Recommendation: "Scientific and Ethical
Approaches for Observational Exposure
Studies"
2
1
-------
Preamble
Do we recommend a "Preamble"? . Content?
- Define scope of document (scientific + ethical)
- Define the intended audience
- Complexity of issues
• Range of study risks
• Societal norms related to ethical concerns (e.g., observing behavior
that is socially accepted or allowed)
• Variability, depending on type/intensity/duration of study
• Examples
- Define key terms (e.g., "exposure")
• Define other terms in glossary
- Context for understanding and using the rest of the document
- Importance of observational exposure studies
3
Include CHEERS as a case
study? NO.
Pros:
- Value of case studies in general
Cons:
- Political issues: too complex to analyze in this
document
- This document is about moving forward
- This panel would need more information about
CHEERS
-------
Other items to include in the
document
• How the document came to be
- Names of panelists
-Authorship
- Peer review process
5
Elements to be Considered in
Study Conceptualization
6
3
-------
Ethical Principles
Concern that the design of the study will result in an increase in exposure.
- But is this a real likelihood?
- Is it OK to watch kids in a lead-tainted environment?
- Is the study a minimal risk study?
Are incentives appropriate?
- Do you need a third party?
- Should pay minimal wage for time.
- Should consider whether the study will benefit the individual or the population as a whole.
- Should a poor woman who is spending time at home be paid less than another person?
- There have been few studies re: compensation and retention.
- Compensation should be equitable regardless of income of subject.
Consider doing a study involving multiple SES, ethnic populations.
What level of deception, if any, is appropriate?
We should not impose on the investigator a higher duty than our society imposes to protect
children.
- There are two questions - does our society allow kids to play in lead-exposed dust in an old house, and is
this wrong?
- When do we have sufficient information to take some action? Our policies are not intended to protect
human health - rather to protect industry and the economy. But there are imminent hazards that need to be
immediately addressed (child abuse, a hole in the floor, etc.). Must always consider risk vs. benefit.
Investigators should always consider how the community views risk vs. benefit, and where the line is drawn.
7
How is study problem defined, and
by whom?
• Can be defined by an agency, individual,
or community.
• All should be involved.
• Should address a public health question.
8
-------
Clearly articulated study
justification
Should have a targeted and specific goal.
Encourage studies more broadly on
subjects like children's health.
Should consider things like maternal
depression, etc.
Broad focus of environmental influences
on children's health is important.
Address broad community needs.
9
Funding:
Should industry funding be allowed?
- Acceptance makes an investigator vulnerable to community and peer concern.
- Maybe all research should be government funded.
- Should there be a fee on all chemicals that goes into a general fund?
- Should not allow someone from ACC to sit on a group that determines funding
for research studies.
Distinguish between a true conflict of interest and an apparent conflict.
- May be more important to discuss how to accept funds and find ways to create a
process for assuring that there will not be undue influence tied to the funding.
The community should have a voice in determining whether or not to accept
industry funding.
Knowledge of the concerns that industry-funded research tends to give
results favorable to the industry.
Problems when community demands things that cost a lot of money - like
letters, regular reports, travel etc. Problems in that the community may
consider a long-term contract when funding may expire before study
completed.
-------
Innovative and alternative design
strategies:
• Should test something with natural
histories - i.e., compare homes that have
regular (i.e., routine, not family) application
of pesticides vs. those who do not. Or lag
exposed (or intervention) vs. control
population - one for one year then switch.
Involve the community in design of study.
11
Other issues
• Feasibility
- Need to have study that will be accepted by community and will lead to
results.
• Statistical power - designed to meet study objectives?
• Include communication planning: Needs to be done
• Balance: When do you stop the study? Criteria?
• When should a study involve a data safety and monitoring board?
• Need good communication with the communities re: importance of
maintenance of funding.
• Therapeutic misconception
• What constitutes "acceptable" conditions?
• Transparency of IRB:
- Need to make sure that there is real community participation. Need to
communicate how the IRBs work to the communities being studied.
Need to advocate appropriate representation on IRBs.
12
6
-------
Harm Reduction
• What if the chemical of interest has not yet
been tested for safety?
• Need to communicate what harm
reduction is - not to lead to the idea that
all dangerous exposure will be ended.
13
Ensuring Protection of
Vulnerable Groups
14
7
-------
Who are They?
Clarify and define
- Disparities
- Individual and societal scales
- Characteristics
• Ability to voluntarily participate
• Susceptible to coercion
• Economically or educationally disadvantaged
Protection
Two scales for protecting rights
- Individual
- Community
Underlying concepts
- Beneficence (key, first consideration)
• Risk-benefit balance
-Autonomy
• IRB role in considering special protections,
concerns
-------
Minimize Undue Burdens
Where the study occurs
What data are collected
Underlying principle = justice
- Do not stigmatize the community
- Do not blame the victims
- Provide benefits to affected first
How to Study Them?
Make the study
details transparent
Be visibly involved, in
the community
Build knowledge of
the community/group
needs, concerns
Build cultural
awareness
• Create authentic
relationships
• Ensure respect, trust,
empathy
• Include appropriate
experts, participant
advocates
- IRB
- Study design team
18
-------
Information Sources
Ethics journals
- Review ethics literature, just as would do for a
specific chemical
• Ensure rapid access to reviews
- Look for:
• How to do studies
• When to stop a study
WHO and the European Union
- Elderly, pre-existing conditions
Addressing privacy and other
concerns related to personal
exposure observation studies
-------
What IS Personal Exposure?
Define it clearly
- Involve the community
- In terms the community understands
Teach the community and researchers
- Scientific issues and importance
- Ethics and means of monitoring
• Why it must be done in your space? (not lab, clinic)
• Privacy issues
Goal = Increase understanding
21
Ethical Issues (,..5)
Invasion of privacy
- Be upfront about it
- Respect how powerful invasion of privacy can be
- Team needs to be culturally sensitive
- Seek diversity in entire research structure
Confidentiality
- Not the same as personal-space privacy protection
- May need certificates of confidentiality
- Special issues: e.g., adolescents
-------
Ethical Issues (2..S)
Risk-Benefit balance
- Ethics and science
- What is the win-win?
- May not be benefits to the individual participant
• State upfront
• Don't exaggerate benefits or minimize risks, burdens
- Be aware of differences in risk-benefit perception of
investigator, individual participant, and community
- Risks vary by age, invasiveness of sampling, cultural
norms
• Participants may refuse participation in parts of study,
sampling
23
Ethical Issues (3.,5)
Disclosure
- Timely feedback, results to individuals
-When to inform individuals?
• Individual may refuse to receive results
- What information?
• Known and unknown hazards
• Tailored using input from IRB, sponsor, individuals,
community
- How to inform?
• Work out with IRB and community
24
-------
Ethical Issues h.,5,
"Collateral" observations (unrelated to the
study)
- MUST anticipate
- MUST have an action plan (even for
unexpected)
• Train research staff
• PI must be responsible for interventions,
mandatory reporting, etc
• Identify resources to help with decision-making
Ethical Issues (5.,5)
Duty to care
- Not the same as for clinical care
• Doctor-patient duty
• Researcher-participant duty
- Researchers need to address their biases,
perceptions
When to stop?
- Levels
• Participant
• Entire study
- Role of data safety monitoring boards
-------
Third Parties
Who are they?
- Household members not in
the study
- Neighbors
- Community
- Insurers
- Landlords, rental agents
- School officials, teachers
- Caregivers of:
• Children
• Elderly
• Disabled
- Etc
Minimize potential impacts
on them
- Respect their presence
- Announce home entry in
advance
- Recognize they may affect
participation
- Have a plan to address them
When does a third party
become a participant?
27
Justification for the Study
Agency
Researcher
Community
Individual
Application of results to policy
Part of policy change
Altruism
Potential for personal gains
28
-------
Creating an Appropriate
Relationship Between
Participant and Investigator
Informed consent
• Storage and use of samples (particularly
DNA)
• Other issues
30
15
-------
Information to disclose
Question of whether one should discuss with subjects possible hazards of at least the
substance you are studying.
- Like pregnant women who smoke.
- Should err on the side of adequately warning the subjects. But there are too many possible
hazards to cover everything, and many or the risks are uncertain.
Duty of researcher
- no duty,
- duty regarding risk imposed by research on subject,
- duty regarding risks known to be caused by preexisting behavioral of subject and duty
regarding other risks unrelated to the study.
- Refer back to NAS study.
Researcher must be careful regarding what they can promise for long-term activity.
Refer to literature on what kind of informed consent really works.
Do tests to be sure the subjects understand.
Use language appropriate for the population.
Child assent is important - needs attention.
Question of how many adults one should deal with.
Information re: storage and use of samples.
Consent form should state that the family can change behavior without losing
compensation. But this has implications for retention and sample size.
• Transparency of incentive: clearly describe what it is for, i.e. participant
burden, NOT risk
• Relational ethics (Giselle to provide references): issues of power, hierarchy,
social context
- There also is a large literature on what level of compensation is undue influence.
• Can affect the science in representativeness of sample
• Provides significant scientific / practical benefit in recruitment
• Study design can be used as a means to guard against altering behavior
• Less concerned about undue influence when there is minimal risk. Here
issues of equity and importance of incentives for retention more important in
this case.
• We do relatively little to protect children from usual risks, but there remains
the question of whether the study will increase exposure. Some have used
a raffle to improve retention.
• Question of overemphasizing compensation. Hourly rate vs. lump sum?
Involve the community on how to do it.
• Advertisements
31
Incentives
32
16
-------
Creating a supportive environment
for research and interaction
Importance of developing community relationships prior
to initiation of the study.
Planning grants help.
Communication plan helps.
At same time investigator must identify and take action
on imminent hazards he/she observes even if
independent of the subject of the study. But the
community may not see this in the same way - so there
needs to be communication in advance. Investigator
needs to be careful on what level of danger fits level for
action. Researcher does have a duty that is different
from that of a plumber, but the level/threshold should be
set high. This is a complex issue.
33
Ensure that recruitment/retention methods
won't change behavior in ways that would
increase exposure
Real dilemma here - by having informed
consent we may change behavior. Recognize
this is a risk but recognize also this is the ethical
way to go.
May be a benefit to reduce exposure even
though may cause problems in the research.
Need to have a risk/risk statement - this is what
we know about hazards of pesticides and this is
what we know about dangers of pests.
-------
Timely Reporting of Results to
Participants
This is particularly important when we collect biological
data that may reflect hazards to children/adults
Need a fast turn-around time. At least need to be
upfront about timing.
Question of reporting all results is more complex.
Needs to be clear information on whether or not clinically
less significant results will be reported back.
Community representation is important re: giving
information that represents the will of the participants.
35
Other issues
Strategies about the population to be recruited
Question of justification of one population vs. another.
Who do you reach out to? Churches?
Grievance procedures
Ombudsman; Use community advisory board; Find
persons other than the IRB chairperson or PI.
Communication throughout the study
- Is critical for retention in longer term cohort studies. Question of
reporting all results back to subjects - it can be an incentive for
participation.
Set standards for when you stop the study because of
risk: One should consider this issue upfront. See
above.
-------
Building and Maintaining
Appropriate Community and
Stakeholder Relationships
Why be concerned about
community engagement?
The nature of the research—it takes place within
communities
Ethical issues can take on different meanings
Four critical aspects of engagement
- Respect for the individual
- Respect for the culture
- Equity of resources
- Empowerment
Many ways to involve community
- Advisors—utilize community to make sure you're
doing the right thing
- Help conduct research 38
-------
How do we define "community"?
(1 of 3)
• Varies, depending on the study
- Geographic
- Other shared characteristics (cultural, social)
- Breadth of the definition
• Broad sense:
- Anyone not part of research team
• Press
• Local agencies
• Stakeholders
• Etc.
- Anyone without a supervisory role
39
How do we define "community"?
(2 of 3)
• For this effort: population from which study
participants are selected
- Participants and their social/cultural
community
- Community experts
- Excluding government, industry, etc.
40
20
-------
How do we define "community"?
(3 of 3)
• Distinction between "community" and
stakeholders
- Community has a right to speak for its interests
- Stakeholders have information or expertise, but can't
speak for community
- Stakeholders are still important on advisory boards
- There are overlaps
• The chapter should carefully define
"stakeholder" and "community"
41
Who represents the
community?
• Different people can help you identify the
appropriate voices:
- Elected officials
- Community associations
- Community elder, matriarch, etc.
• Considerations for the researcher:
- Important to ask the community who they see as a
legitimate voice
- Loudest voice is not necessarily the most
representative
- Usually not just one person (need multiple viewpoints)
42
21
-------
Who represents the
community? ,2«,2,
Information sources:
- Sociological/ethnographic research about
identifying community leaders
- Large body of literature on community-based
intervention, partnerships
- Emergency research literature
• Some similar issues
• Lessons learned
Importance of language and
culture
Can facilitate or be a barrier to community
engagement
Spoken language
Different norms
- Taking pictures
- Sensitivities about certain types of samples (e.g., hair,
fingernails)
Different decision-making culture
- Length of deliberation
- Consensus building
Keys for researchers
- Avoid over-generalization 44
• Part of process is to appreciate heterogeneities
-------
Building trust (,..2>
Honesty
Equity
Building contracts
- Each side has expectations
- Setting limits
- Some communities like a formal
memorandum of understanding
- Expectations re: a lasting infrastructure
45
Building trust (2.,2)
Honoring voices from both sides
- Beyond simply respecting those voices
- People have a right to expect that their voices will be
taken into consideration
Trust must be built, not assumed
- Build relationships before project begins
- Must demonstrate trustworthiness
Sources:
- Interagency study [M. Lebowitz suggestion]
-------
Honesty, power relationships,
and partnerships (1 of 2)
Partnerships
- Specific definitions may vary
- Some general characteristics
• Must consider power dynamics
Acknowledge that researchers may feel
like they have little power, but in the
context of community relations, they do
Honesty, power relationships,
and partnerships (2 of 2)
Forms of power
- Monetary
- Peer pressure
- Expectations
- Housing relationships
- Imbalance of knowledge
• Community has knowledge the researcher doesn't have
• Researcher has knowledge the community doesn't have
• Potential for knowledge transfer
- "Power to stigmatize"
• How researchers report their findings
• Simply by doing the research (labeling communities)
• Etc.
- Age considerations
• Children have less power
- Access to resources
-------
Leaving a lasting infrastructure
(1 of 3)
Important part of the relationship (don't just take
from the community and then leave)
Goals:
- Empowerment
- Sustainability
- Capacity-building
Researcher should discuss in advance with the
community
Build capacity throughout the project
49
Leaving a lasting infrastructure
(2 of 3)
Include training in the project
- For researchers
- For the community (capacity-building)
Responsibilities of the sponsor
- Many sponsors do make lasting commitments
- Federal agencies should think of similar mechanisms
• Tail-off of funding
• Other approaches
Relationship shouldn't end when funding ends
- Help people learn to write grant applications
- Help ID additional funding sources
- Be available to provide technical support £
-------
Leaving a lasting infrastructure
(3 of 3)
• Researchers need to solicit support within their
own institutions
- Ensure that relationship continues even if individual
researcher leaves
- Formal agreements to sustain relationships between
organizations
- Recognize realities of how some universities prioritize
funding
- Important to emphasize benefits of these
relationships
51
Designing and Implementing
Strategies for Effective
Communication
52
26
-------
Roles and responsibilities (1 of 4)
When to communicate:
- Start with the community
- Then obtain funding
- Involve community in next steps
• Refining the study design
• Finding participants
- During the study
• Communicate on a regular basis
- Community meetings
- Written communications
- Etc.
- Communicating results
• In a timely fashion
• But also recognize need for quality assurance
Roles and responsibilities^)
What to communicate
- Education
• Risks; general info about indoor hazards
• Scientific literacy
• Confidentiality and data ownership
- Methodology
• Study methodology
• How you plan to provide results (explain this in the
informed consent)
- Results [see next slide]
54
-------
Roles and responsibilities^)
• Communicating results
- Aggregate and individual
- Report in an understandable fashion
- At minimum, follow consent form and give participant
the opportunity to receive results
- Pros of full reporting of individual results:
• Researcher won't have to make judgment about what's
"dangerous" enough to merit reporting
- Cons of full reporting of individual results:
• IRB restrictions
• "Data dump" can overwhelm, cause added stress to
participants
- Consider other useful, timely ways to report back
(e.g., trend indicators)
- Recognize difference between data and findings
Roles and responsibilities^)
Other considerations for researchers
- Recognize need for transparency
- Importance of media training for study partners
- Involve community in drafting of manuscripts
- Importance of the order of communications
• People don't want to hear results from the media first
- Ultimately, results are public information
• Tensions
- Stakeholders' feelings about releasing aggregate findings
• Likely to end up in peer-reviewed literature
- Building relationships with IRBs
-------
Develop lay language
Researcher should define key terms up
front
57
What constitutes "acceptable"
conditions?
-------
What constitutes "acceptable" conditions? (2 of 2)
• To what extent should the investigator warn
people about risks?
- Researchers have duty to inform participants about
risks related to the study
- Risks unrelated to the study:
• Difficult to cover all possible risks
• Could provide general (not personalized) fact sheet about
risks ahead of time
• Have a threshold for imminent danger
- Precedents:
• NAS and WHO documents: inform people about indoor
exposures exceeding occupational standards
- Importance of how you say it
• Provide resources
• Empower individuals to remedy situations if possible
• Empower communities to advocate for broader remedies 59
Ownership of data
-------
Ownership of data <2of3)
Must determine ahead of time
- Include in study design
- Dialogue with community
Distinction between data and findings
(interpretation)
- Community wants findings
Importance of education
- Inform people about legal aspects
- People, schools, etc. have a right not to be identified
in publications
- Explain confidentiality issues
61
Ownership of data
Genetic-environmental interaction
- Growing interest in this area
- Debate: can you ever de-identify genetic
data?
• General opinion: NO (at least not in future)
• Must modify public-use datasets accordingly
-------
Communication strategy
Beyond PR: communication throughout the
process
- Get feedback from participants, etc.
Anticipating and dealing with criticism
- Researchers must work with agencies' administration
and public relations staff
- Should be able to respond to criticism in a timely
fashion
• Justify the research
• Explain how the community has been involved throughout
Sources:
- EPA's risk communication strategies
- CDC approaches 6
------- |