*>EPA
S
Board of
Scientific
Counselors
May 14, 2004
Dr. Paul Oilman
Assistant Administrator
Office of Research and Development
U. S. Environmental Protection Agency
Ronald Reagan Office Building
Washington, DC 20460
Re: Communicating Research Results, Report of the Ad Hoc
Communications Subcommittee
Dear Dr. Oilman:
The Board of Scientific Counselors Ad Hoc Subcommittee on
Communications was formed to examine how effectively the results of'
research funded by the EPA. Office of Research and Development (ORD)
currently are communicated, both within and beyond the Agency, and how
they might be more effectively communicated; and to help ORD more
effectively disseminate its research products, explain their significance, and
assist others inside and outside the Agency in applying them.
The Subcommittee reached tentative findings based on the Laboratory and
Center responses to a question about communicating research results that was
asked as part of the BOSC's second review of the ORD Laboratories and .
Centers. The responses were reported to the BOSC at its September 23-24,
2002 meeting. One of the Subcommittee's findings was that the Laboratories
and Centers could benefit from sharing some of their best practices with one
another. To this end, the Subcommittee proposed that the BOSC hold a best
practices workshop on communicating research results, and developed a set of
self-study questions regarding innovative communications practices, for the
Laboratories and Centers to address at the workshop.
Written self-study responses were submitted to the BOSC prior to the
workshop, which took place May 15, 2003, in Washington DC. The
specific aims of the workshop were to share and assess best practices with
regard to communication of scientific results, and to identify specific
opportunities for improvement of EPA/ORD communication of scientific
results. The workshop was designed to increase awareness of the importance
of communications, increase knowledge of what communication tools there
are that could help the Laboratories and Centers, and help them prioritize
their efforts to improve. This Subcommittee report was finalized shortly
after that workshop and submitted to the BOSC Executive Committee for
review and approval. The report also was distributed to ORD management
and the Laboratories and Centers to correct any factual errors.
-------
The meeting and self-study responses showed that most of the Laboratories and
Centers are doing a good job of establishing effective communication practices, but
could still benefit from applying or adapting some of the specific practices described
by other agencies and organizations at the meeting. The Laboratories and Centers
demonstrated a laudable increase in their focus on and efforts to communicate
research results, and described organizational changes to improve their research
results communication efforts, including increases in staff, resources, and
communications products.
Please do not hesitate to call upon us if you have any questions regarding this
report or its recommendations.
Sincerely yours,
Jerald L. Schnoor
Chair, Board of Scientific Counselors
Ann Bostrom, Ph.D.
Chair, BOSC Communications Ad Hoc Subcommittee
-------
v>EPA
Board of Scientific Counselors
Office of Research and Development
United States Environmental Protection Agency
Communicating Research
Results
s
Final Report of the Ad Hoc Subcommittee
on Communications
September 11, 2003
c
-------
NOTICE
,,i '.~~
This report has been written as part of the activities of the Board of Scientific Counselors (BOSC), a
public advisory group that provides objective and independent counsel to the Assistant
Administrator for the Office of Research and Development (ORD) of the U.S. Environmental
Protection Agency (EPA). The Board is structured to provide a balanced expert assessment of the
management and operation of ORD's research programs and its utilization of peer review. This
report has not been reviewed for approval by the Agency, and hence, the contents of this report do
not necessarily represent the views and policies of the EPA or other agencies in the federal
government. Mention of trade names or commercial products does not constitute a
recommendation for use.
-------
TABLE OF CONTENTS
Preface 3
Roster of the Board of Scientific Counselors Executive Committee 5
Roster of the Board of Scientific Counselors Ad Hoc Subcommittee on Communications 6
List of Acronyms 7
1.0 Management of Research Results Communications Efforts 9
2.0 Audience Identification and Communications Goals 11
3.0 Formative Evaluation: Designing Communications Programs and Products 13
4.0 Evaluation: Not Just the Numbers 15
5.0 Risk, Trust, and Strategic Planning 17
6.0 Summary of Recommendations 19
Appendices
A. Summary of the September 23-24, 2002 BOSC Meeting
B. Proceedings of the Communicating Research Results: Best Practices Workshop
September 11, 2003 Communicating Research Results
-------
Communicating Research Results September 11, 2003
-------
Preface
PREFACE
The Board of Scientific Counselors (BOSC)
provides objective and independent counsel to
the Assistant Administrator of the Office of
Research and Development (AA/QRD) on the
management and operation of ORD's research
programs. The primary functions of BOSC are
to: (1) evaluate science and engineering research
programs, laboratories, and research-manage-
ment practices of ORD and recommend ac-
tions to improve their quality and/or strength-
en their relevance to the mission of the EPA;
and (2) evaluate and provide advice concerning
the use of peer review within ORD to sustain
and enhance the quality of science in EPA
The BOSC Ad Hoc Subcommittee on Commu-
nications was formed to examine how effec-
tively the results of research funded by EPA's
ORD currently are communicated, both within
and beyond the Agency, and how they might be
more effectively communicated; and to help
ORD more effectively disseminate its research
products, explain their significance, and assist
others inside and outside the Agency in apply-
ing them. The Subcommittee members were
Ann Bostrom (BOSC Executive Committee
member and Chair of the Subcommittee, Geor-
gia Institute of Technology), Elaine Dorward-
King (BOSC Executive Committee member
and Co-chair of the Subcommittee, Rio Tinto),
Caron Chess (Consultant, Rutgers University),
Anna Harding (BOSC Executive Committee
member, Oregon State University), and Steven
Lewis (Consultant. Exxon-Mobil).
In the BOSC 2001-2002 ORD Laboratory and
Center site reviews (conducted as part of the
BOSC's second review of the ORD Laborato-
ries and Centers), the following question was
posed regarding the communication of re-
search resulu>:
How does (this Laboratory or Center) com-
municate its results within the organization,
within ORD, within EPA, to outside agen-
cies, and to the outside world?
The Ad Hoc Subcommittee on Communica-
tions reached tentative findings based on the
Laboratory and Center responses to this ques-
tion, as reported to the BOSC at its September
23-24, 2002 meeting (see Appendix A), and in
the proceedings of the May 15, 2003 workshop
(available in Appendix B).
One of the Subcommittee's findings was that
the Laboratories and Centers could benefit
from sharing some of their best practices with
one another. To this end, the Subcommittee
proposed that the BOSC hold a best practices
workshop on communicating research results,
and developed a set of self-study questions
regarding innovative communications practices,
for the Laboratories and Centers to address at
the workshop. Written self-study responses
were due to the BOSC prior to the workshop,
which took place May 15, 2003, in Washington
DC The specific aims of the workshop were
to share and assess best practices with regard to
communication of scientific results, and to
identify specific opportunities for improvement
of EPA/ORD communication of scientific
results. The workshop was designed to increase
awareness of the importance of communica-
tions, increase knowledge of what communica-
tion tools there are that could help the Labora-
tories and Centers, and help them prioritize
their efforts to improve. This Subcommittee
recoil was finalized shortly after that workshop
and submitted to the BOSC Executive Com-
mittee for review and approval. The report also
was distributed to ORD management and the
Laboratories and Centers to correct any factual
errors.
The meeting and self-study responses showed
that most of the Laboratories and Centers are
doing a good job of establishing effective corn-
September 11, 2003
Communicating Research Results
-------
Preface
munication practices, but could still benefit
from applying or adapting some of the practices
described by other agencies and organizations
at the workshop. The ORD Laboratories and
Centers demonstrated a laudable increase in
their focus on and efforts to communicate re-
search results, and described organizational
changes to improve their research results com-
munication efforts, including increases in staff,
resources, and communications products.
The report is organized into five sections:
(1) Management of Research Results Commu-
nications Efforts; (2) Audience Identification
and Communications Goals; (3) Formative
Evaluation: Designing Communications Pro-
grams and Products; (4) Evaluation: Not Just
the Numbers; and (5) Risk, Trust, and Strategic
Planning. A summary of the recommenda-
tions concludes the report.
Ac knowledgements
The BOSC gratefully acknowledges the helpful
contributions by all presenters at the May 15,
2003, workshop, without which the meeting
would have accomplished little. The BOSC
extends its thanks to the following EPA and
external participants for their presentations and
participation: Jack Puzak, Acting Director of
the National Center for Environmental Re-
search (NCER); Dr. Hugh McKinnon, Director
of the National Risk Management Research
- Laboratory (NRMRL); Dr. Peter Preuss, Direc-
tor of the National Center for Environmental
Assessment (NCEA); Dr. Gary Foley, Director
of the National Exposure Research Laboratory
(NERL); Steven Hedtke, Deputy Associate
Director for Ecology at the National Health
and Environmental Effects Research Labora-
tory (NHEERL); Dr. Kevin Teichman, Direc-
tor of the Office of Science Policy (OSP); Dr.
Dan Costa, Chief of the Pulmonary Toxicology
Branch, NHEERL; Prudence Goforth, Com-
munications Director for the Office of Air and
Radiation; Dr. Paul Gilman, Assistant Adminis-
trator for Research and Development; Michael
Brown, Associate Assistant Administrator for
Research and Development; Mike Moore, Di-
rector of Communications for ORD; Dr. Mar-
sha Vanderford, Deputy Director of Communi-
cation at the National Center for Environmen-
tal Health, Centers for Disease Control and
Prevention (CDC); Dr. Robert O'Keefe, Vice
President of the Health Effects Institute (HEI);
Dr. Fred Miller, Vice President for Research at
the GIT Centers for Health Research; Dr.
Allen Dearry, Associate Director of Research
Coordination, Planning, and Translation at the
National Institute of Environmental Health
Sciences (NIEHS); and Diane Maple, American
Lung Association.
Communicating Research Results
September 11, 2003
-------
Preface
BOARD OF SCIENTIFIC COUNSELORS
EXECUTIVE COMMITTEE
Chair
Jerald L. Schnoor, Ph.D., Professor, Depart-
ment of Civil & Environmental Engineering,
University of Iowa, Iowa Gty, IA
Members:
Daniel Acosta, Jr., Ph.D.
Dean, College of Pharmacy
University or Cincinnati
Cincinnati, OH
Ann Bostrom, Ph.D.
Associate Professor, School of Public Policy
Georgia Institute of Technology
Atlanta, GA
William L. Chameides, Ph.D.
Smithgall Chair and Regents Professor,
School of Earth and Atmospheric Sciences
Georgia Institute of Technology
Atlanta, GA
James R. Clark, Ph.D.
Exxon Mobil Research & Engineering
Company
Fairfax, VA
Michael T. Clegg, Ph.D.
Distinguished Professor of Genetics, Depart-
ment of Botany and Plant Sciences
University of California
Riverside, CA
George P. Daston, Ph.D.,
Miami Valley Laboratories
The Proctor & Gamble Company
Cincinnati, OH
Elaine J. Dorward-King, Ph.D.
Global Executive
Environment, Health and Safety
Rio Tinto
Valencia, CA
Anna K. Harding, Ph.D., R.S.
Associate Professor, Department of Public
Health
Oregon State University
Cprvallis, OR
James H Johnson, Jr., Ph.D.
Dean, College of Engineering, Architecture,
and Computer Sciences
Howard University
Washington, DC
Juarine Stewart, Ph.D.
Professor, Department of Biological Sciences
Clark Atlanta University
Atlanta, GA
Herbert L. Windom, Ph.D.
Skidaway Institute of Oceanography
Savannah, GA
Committee Staff:
William Farland, Ph.D.
Acting Deputy Assistant Administrator for
Science
ORD BOSC Liaison
Office of Research and Development
U.S. Environmental Protection Agency
Washington, DC
Shirley R Hamilton
Designated Federal Officer,
National Center for Environmental Research
U.S. Environmental Protection Agency
Washington, DC
September 11, 2003
Communicating Research Results
-------
Preface
BOARD OF SCIENTIFIC COUNSELORS
AD HOC SUBCOMMITTEE ON COMMUNICATIONS
Chain
Vice Chain
Members:
BOSCDFO:
Ann Bostrom, Ph.D.
Associate Professor
School of Public Policy
Georgia Institute of Technology
Atlanta, GA
Elaine Dorward-King, Ph.D.
Global Executive, Environment, Health and Safety
RioTinto
Valencia, CA
Steven C Lewis, Ph.D., DABT
Senior Toxicology Associate
Exxon Mobil Biomedical Sciences, Inc.
Annandale, NJ
Caron Chess, Ph.D.
Center for Environmental Communications
Rutgers University
New Brunswick, NJ
Anna Harding, Ph.D.
Associate Professor, Department of Public Health
Oregon State University
Corvallis, OR
Shirley Hamilton
Designated Federal Officer
National Center for Environmental Research
U.S. Environmental Protection Agency
National Center for Environmental Research
Washington, DC
Communicating Research Results
September 11, 2003
-------
Preface
LIST OF ACRONYMS
AA/ORD
BOSC
GARB
CDC
EPA
HEI
NCEA
NCER
NERL
NHEERL
NIEHS
NRC
NRMRL
OAR
ORD
OSP
PM
RTP
STAR
WHO
Assistant Administrator for the Office of Research and Development
Board of Scientific Counselors
California Air Resources Board
Centers for Disease Control and Prevention
Environmental Protection Agency
Health Effects Institute
National Center for Exposure Assessment
National Center for Environmental Assessment
National Exposure Research Laboratory
National Health and Environmental Effects Research Laboratory
National Institute of Environmental Health Sciences
National Research Council
National Risk Management Research Laboratory
Office of Air and Radiation
Office of Research and Development
Office of Science Policy
Particulate Matter
Research Triangle Park
Science To Achieve Results
World Health Organization
September 11, 2003
Communicating Research Results
-------
Preface
Communicating Research Results September 11, 2003
-------
1.0 Management of Research Results Communications Efforts
1.0 MANAGEMENT OF RESEARCH RESULTS
COMMUNICATIONS EFFORTS
Recent approval obtained for ORD to issue its
own press releases illustrates the high level of
organizational commitment in EPA to improv-
ing communication of research results. ORD's
ability to develop and send out its own press
releases, rather than requiring clearance through
a central communication office, is a critical step
forward.
The approval for ORD to issue its own press
releases marks some decentralization for EPA,
which should improve the extent to which re-
search results can be communicated with great-
er timeliness and effectiveness. The Laborato-
ries and Centers must necessarily engage in
public relations efforts as well as in communi-
cation of research results. Public relations
overlap with communication of research re-
sults, and they are important in their own right,
as illustrated by the Office of Air and Radiation
(OAR) presentation at the May workshop.
Central communications activities in ORD in-
clude Office of Science Policy-initiated work-
shops and products, which are issued centrally
from ORD. There also have been new efforts
to coordinate communications efforts and pro-
vide some central training within ORD; for
example, by the Associate Assistant Adminis-
trator for Research and Development.
ORD's communications goal, as stated at the
workshop, is "tc bring about attitudinal and
structural changes that support communica-
tions, because the Office believes that early
communication of research results is essential
for the conversion of scientific knowledge to
policy decisions and the acceptance of policy
decisions." This goal appears to target ORD
staff specifically. However, it also includes an
; i:_:. ^,,«_^_^t,;»,^- ^.-,1 t " »<-'
.i.C- £_V_
ulation of ORD- wide communication goals and
audiences was not presented at the workshop.
If this were done, it would facilitate strategic
management and coordination of such commu-
nications efforts.
In general, each Laboratory and Center carries
out its research results communications efforts
largely independently of the others. Decentral-
ization has its advantages, as discussed above,
and the committee supports the continued in-
dependence of the Laboratories and Centers in
this regard. However, these efforts vary on
several dimensions, including staffing, organiza-
tional structure, and goals. So, for example, the
NCER communications staff is 3 to 4 people
out of a total staff of around 70. NCE A's com-
munication staff is one individual part time,
though NCE A intends to build on this. In
most of the Laboratories and Centers, commu-
nications staff report directly to top manage-
ment. The (wide) variety of goals identified by
the Laboratories and Centers at the workshop
or in the self-study responses are summarized
in Table 1.
The workshop illustrated several ways in which
increased coordination of and interaction be-
tween communications staff across ORD might
be beneficial. Strategic development of com-
munication goals is one such area, along with
others discussed below. In addition, the discus-
sion of the development of the partici'late mat-
ter (PM) synthesis report highlighted the poten-
tial contributions of vested, rather than con-
tracted, efforts to communicate research re-
sults.
Rec omme ndation: ORD should continue
its efforts to improve communications piac-
tlC£S ill 2!! of tHc L=l!?O!r?!fr*r'<"(: ?"iH O*nters.
scientific knowledge to policy decisions" (tar-
geting EPA) "and accepting those decisions"
(possibly targeting the public). Additional artic-
This could include developing a central-
ized, up-to-date annotated directory of
communications expertise within EPA.
September 11, 2003
Communicating Research Results
-------
1.0 Management of Research Results Communications Efforts
Giving communications staff in each Lab-
oratory and Center sufficient time and re-
sources to allow them to assist one another
and to exchange ideas in an ORD-wide
forum would further good communications
practices. Such a forum could even be
Internet- or teleconference-based.
Recommendation: The Laboratories and
Centers are encouraged to have communi-
cations staff report to the Laboratory/
Center Director. This ensures that commu-
nication receives attention at a manage-
ment level, and is considered a best prac-
tice in industry and academia.
10
Communicating Research Results
September 11, 2003
-------
2.0 Audience Identification and Communications Goals
2.0 AUDIENCE IDENTIFICATION AND
COMMUNICATIONS GOALS
Audiences identified by the Laboratories and
Centers in the self-study or at the workshop are
provided in Table 1. Although some of the
Laboratories and Centers identified specific
audiences for the innovations they highlighted
at the workshop, it was evident that the Labora-
tories and Centers have approached the task of
identifying audiences and goals in a variety of
ways, with varying results.
As will be discussed under the section on for-
mative evaluation, a more systematic strategic
effort to characterize specific audiences and
their needs, and to prioritize among them,
would likely improve the success of ORD's
research results communications efforts. For
example, one of the primary audiences for re-
search results from ORD Laboratories and
Centers is EPA itself the Program Offices in
particular, but also Regional Offices. Although
Regions were a focal point of some of the pre-
sentations at the workshop, not much was said
about how the Laboratories and Centers insure
that they understand the needs of and are
reaching Program Office staff.
Identifying audiences for ORD research results
and reaching them more effectively also was a
recommendation of the recent National
Research Council (NRQ report on the EPA
Science To Achieve Results (STAR) program,
which is housed in NCER
Recommendation: The Laboratories and
Centers should formally identify the various
audiences for their research results and
continue their efforts to prioritize among
them.
September 11, 2003
Communicating Research Results
11
-------
2.0 Audience Identification and Communications Goals
12 Communicating Research Results September 11, 2003
-------
3.0 Formative Evaluation: Designing Communications Programs and Products
3.0 FORMATIVE EVALUATION: DESIGNING COMMUNICATIONS
PROGRAMS AND PRODUCTS
Communicating research results is an integral
part of research management. Integrated re-
search management requires planning for re-
sults from the outset, including how to commu-
nicate the results of that research effectively to
those who need them. Early input by end users
into research design (e.g., as in NERL's Strate-
gic Customer Value Analyses) is an important
component of effective research management,
but is not equivalent to involving audiences
early in order to improve the design of commu-
nications products and programs.
So-called formative evaluation is aimed at de-
veloping and improving a program's progress
and is analogous to medical testing that takes
place before treatment and periodically after
initial treatment and diagnosis. Just as test re-
sults allow a physician to assess treatment effec-
tiveness or the likelihood thereof, formative
evaluation provides managers with feedback
during program development and implementa-
tion (Posavac, 1991). This kind of evaluation is
essential to developing good communication
efforts. Formative evaluation can save re-
sources by ensuring that communication reach-
es those who most need or want it in ways
these audiences find useful.
Increasing the effectiveness of communications
depends on comprehensive communication
planning that involves identifying: (1) key audi-
ences; (2) the interests, needs, and concerns of
those audiences; and (3) methods of communi-
cating (workshops, Web sites, newsletters, etc.)
that reach those audiences in ways they find
useful. Such planning ensures that communica-
tion is client-centered. Communication that is
merely one-way (e.g., from inside ORD to
those outside) is far less likely to be effective
ences for input about communication prior to
the communication effort.
Formative evaluation can assess these key com-
ponents of communication before communica-
tion begins, and on a routine basis. Some exam-
ples of formative evaluation include formally or
informally getting input from audiences prior to
developing a communication effort, pre-testing
materials on an intended audience, and solicit-
ing feedback on communication efforts during
early phases of implementation. Based on the
results of formative evaluation, changes can be
made to increase the usefulness and effective-
ness of communication.
Because evaluation is too often an afterthought
or the result of a crisis, formative evaluation,
which must be planned in advance, can be
overlooked. Formative evaluation should range
in comprehensiveness and rigor based on the
importance, scope, and resources expended on
the communication program itself. Develop-
ment of an expensive Web site would merit
more rigorous and comprehensive formative
evaluation than a one-time publication with
limited distribution, unless the publication were
critical for a specific reason (e.g., reaching an
underserved or politically sensitive audience).
Arguably, soliciting limited feedback is better
than soliciting none at all. However, anecdotal
feedback or feedback based on a limited, non-
representative sample will be less generalizable,
and, depending on the method, less reliable. On
the other hand, measuring results by numbers
of people served may nor be as important a
measure as satisfaction of those served.
The workshop provided examples of communi-
cations that involved audience advanced plan-
ning and mechanisms for feedback, as well as
communication that lacked focus on client
needs and was likely to be lesi> effective:. Exarn-
r->W r>f K>p«t r>rarrirp<; in this rppard included the
r i o
HEI pre-briefings; NIEHS participative re-
search, translational research, and partnerships;
September 11, 2003
Communicating Research Results
13
-------
3.0 Formative Evaluation: Designing Communications Programs and Products
and OAR's use of both formal and informal
input to hone its communication efforts. One
of the best examples of formative research pro-
vided at the workshop was CDCs use of quali-
tative audience research to provide insights into
the values, beliefs, and behaviors of the audi-
ences for the National Report on Human Ex-
posure to Environmental Chemicals. The iden-
tification of priority audiences, communication
strategy, messages, and materials were based on
this formative research. Extensive follow-up
research indicated ways to further improve
communication for subsequent reports.
Among EPA presentations, a positive example
was NCER's efforts to listen to internal and
external stakeholders as part of the develop-
ment of their communication plan. NCER
staff worked closely with Regional staff to plan
the workshop for Region I in November 2002.
Undoubtedly, this interaction led to attracting
the number and diversity of participants, posi-
tive feedback on written evaluations, and re-
quests from other Regions for similar work-
shops. NCER also cited intended improve-
ments, such as personal invitations to states,
which reflects a useful post-event focus on
outreach efforts.
NERL's development of a flow chart to de-
scribe its communications efforts reflects a
focus on communication planning. However,
the flow chart did not suggest ways in which
Audience input is used. Collecting input without
"development of mechanisms to use the input is
problematic and suggests limited formative
evaluation. NERL highlighted a number of
communication innovations, including science-
to-science workshops and research abstracts.
We suspect that these innovations might be
even more effective in the future if input is
solicited prior to implementation and at various
stages of development.
NRMRL's communication efforts focused on
the technology to be used to communicate with
its audiences, rather than on getting input from
audiences about their informational needs and
formatting preferences. NRMRL's current
efforts to elicit feedback on these technologi-
cally sophisticated products are an important
means to save resources and increase effective-
ness. NRMRL's evaluation of its pollution
prevention workshop also is a step in the right
direction. However, getting input from clients
prior to such workshops, as NCE A did, is at
least as important.
Recommendation: The Laboratories and
Centers should increase their use of formal
and informal methods of soliciting early
input and involvement from their priority
audiences to improve their communication
of research results.
Reference: Posavac Emil J. and Carey Raymond G. 1991. Program Evaluation: Methods and Case Studies. Prentice
Hall.
14
Communicating Research Results
September 11, 2003
-------
4.0 Evaluation: Not Just the Numbers
4.0 EVALUATION: NOT JUST THE NUMBERS
ORD Laboratory and Center communications
products and programs have received much
positive recognition, including awards such as
that from the Society for Technical Communi-
cation to NHEERL for its annual report.
NCER's Web development has been respon-
sive to, for example, the Region I Science
Needs Survey. Several of the Laboratories and
Centers did mention evaluation efforts being
planned or developed, for which they are to be
commended. However, anecdotal feedback
was mentioned frequently in response to the
question of what performance measures the
Laboratory/Center used. Despite the amount
of positive feedback repotted, there was only
sparse indication at the workshop of how this
feedback is fed into communications planning
and design processes to further improve re-
search results communications.
Outputs and numbers such as the number of
publications, the number of visitors to Web
srtes, and the number of attendees at work-
shops tell only part of the story, and should
be used strategically to achieve greatest effect.
Analyzing the numbers can go a long way to-
ward making them useful. For example, know-
ing what proportion of intended attendees that
actually show up at workshops maybe more
useful than having a count of how many show
up. Knowing how a few representative users
use a Web site may be more useful than know-
ing how many people used it any given day.
For the nurooses of analysis and evaluation in
A 1. ^
particular, it could be helpful to the Laborato-
ries and Centers to have more social scientists
involved in their research results communica-
tions efforts, to assist in the analysis of evalua-
tive data.
Outcomes are more difficult to determine, but
critical to track Although the Laboratories and
Centers contribute to criteria documents and
Agency guidelines, they do not appear to have
any formal measure of this contribution. HE I
tracks citations in California Air Resource
Board (CARB) rulemaking and World Health
Organization (WHO) guidelines. ORD could
benefit from improved tracking of when and
how its research results influence policy.
Recommendation: The Laboratories and
Centers should increase their efforts to in-
corporate the feedback they already collect
into their research results communication
efforts, and to improve the quality of that
feedback. ORD should continue to develop
systematic methods of tracking the influ-
ence of its research on policy.
It was noted at the workshop that NCSA's
bibliography is difficult to use. This contrasts
with the automated data passes and flexibility in
NCER's Web database code, and with the flexi-
bility and increased usefulness to a wide variety
of users implied by the dynamic generation of
Web pages from NCEA's Environmental In-
formation Management System (EIMS).
Recommendation: ORD should continue
its efforts to standardise and increase the
flexibility ot access to and use of research
bibliographies and databases where possi-
ble, in order to facilitate their use.
September 11, 2003
Communicating Research Results
15
-------
4.0 Evaluation: Not Just the Numbers
16 Communicating Research Results September 11,2003
-------
5.0 Risk, Trust, and Strategic Planning
5.0 RISK, TRUST, AND STRATEGIC PLANNING
Among communications topics that BOSC
members raised as potentially neglected at the
workshop were risk, trust, and implicitly, strate-
gic planning. When research concerns risk,
communicating the results will entail communi-
cating risk Risk communication is beyond the
scope of this report, and is but one of several
issues concerning the content and interpreta-
tion of ORD communications that deserves
further attention.
Trust, on the other hand, may have more to do
with organizational structure, practices, and
values than with the content of any given com-
munication. For example, HEI is structured to
maintain credibility and transparency in what
can be controversial national regulatory de-
bates, and does not take policy positions. Al-
though building trust is widely acknowledged as
a critical component of a successful communi-
cations program, many of the findings that
might be useful for ORD regarding how to
achieve this mirror those reviewed in the above
discussion on formative evaluation. Knowing
one's audience and respecting and addressing
its needs consistently are key.
At the outset of its efforts, the Subcommittee
intended to follow up on the NCERQA Com
rwrrioztians Strategy and Implementation Plan of
1998, and hoped to review communications
plans for the other Laboratories and Centers.
Although said review has not been possible,
ORD is to be commended for its progress on
communications in the intervening years since
the BOSC review of the NCERQA plan in
conjunction with the Board's first ORD Labo-
ratory and Center review.
Recommendation: Strategic planning for
communication of research results should
be integrated explicitly into research man-
agement efforts at the Laboratories and
Centers.
September 11, 2003
Communicating Research Results
17
-------
5.0 Risk, Trust, and Strategic Planning
18 Communicating Research Results September 11,2003
-------
6.0 Summary of Recommendations
6.0 SUMMARY OF RECOMMENDATIONS
Management and Staffing: ORD should con-
tinue its efforts to improve communications
practices in all of the Laboratories and Centers.
This could include developing a centralized, up-
to-date annotated directory of communications
expertise within EPA Giving communications
staff within each Laboratory and Center suffi-
cient time and resources to allow them to pro-
vide some assistance to one another and to
exchange ideas in an ORD-wide forum would
also further good communications practices.
Such a forum could even be Internet- or
teleconference-based.
The Laboratories and Centers are encouraged
to have communications staff report to the
Laboratory/Center Director. This ensures that
communication receives attention at a manage-
ment level, and is considered a best practice in
industry and academia.
Audience Identification: The Laboratories
and Centers should formally identify the audi-
ences for their research results and continue
their efforts to prioritize among them.
Formative Evaluation; The Laboratories and
Centers should increase use of formal and in-
formal methods of soliciting early input and
involvement from their priority audiences, to
improve communication of research results.
Incorporating Feedback and Tracking Out-
comes: The Laboratories and Centers should
increase their efforts to incorporate the feed-
back they already collect into their research
results communication efforts, and to improve
the quality of that feedback ORD should con-
tinue to develop systematic methods of tracking
the influence of its research on policy.
Standardizing Records: ORD should con-
tinue its efforts to standardize and increase the
flexibility of access to and use of research bibli-
ographies and databases where possible, to
facilitate their use.
Strategic Planning: Strategic planning for
communication of research results should be
integrated explicitly into research management
efforts at the Laboratories and Centers.
September 11, 2003
Communicating Research Results
19
-------
6.0 Summary of Recommendations
20 Communicating Research Results September 11,2003
-------
Table 1. Communication Goals, Audiences, and Innovations of the
ORD Laboratories and Centers
Laboratory or Center
Goals
Audiences
Innovations and Other Communication Strategies
Highlighted at May Workshop
National Cerier for En-
vironme ital .Research
(NCER)
To "target scientific results to
address the unique interests
and needs of its customers and
[relies on feedback] to make
these results useful to all of its
customers."
^ Own organization
v> EPA Regional Offices
0- EPA Program Offices
"v" Outside agencies
"v" Professional Societies
<> "Outside world" of Capital
Hill
"v" The scientific community
"^ The public
"v" Specific audiences for spe-
cific communications
products (e.g., Web site tar-
gets NCER project officers
and principal investigators,
among others)
^ Re^cnd EmironrrErital Research Senimrs
0- NCER WebSite- Web-based inforrmtion describing
funded research, including the items listed below.
"v" PublicationsProgress and Final research reports,
peer-reviewed journal publications, conference
proceedings, research "capsules" related to topical
areas, STAR reports for lay readers
*v" NCER-sponsored research presentations to EPA
staff and to the scientific community
*v" NCER-sponsored progress review workshops
"v* Presentations byNCER-grantees at professional
scientific conferences
"v" Future efforts: State of the Science reports, news
releases on grant funding and results, one^page -
summaries of NCER-funded research
National Risk Manage-
ment Research Labora-
tory (NPMRL)
To "^ct the right information
in the right format to the peo-
ple who need it." .
To nirse awareness, transfer
NRMRL research results, and
prodace high quality peer-re-
viewed publications.
Technical audiences, includ-
ing academia, and regulated
industries
EPA Regional, state and local
government personnel, and
Tribes
General audiences/general
public
Multirredia CDs
Technical publications: journal articles, books, and
project reports
Technology transfer synthesis documents, cap-
stone reports, bulletins, workshops, and seminars
Materials for general audiences: brochures, press
releases, and public events
National Exposure Re-
search Laboratory
(NERL)
1. Racing awareness of
NERL's relevant high-pri-
oity research
2. Engaging all NERL staff in
the communication effort
NERL staff
EPA Program and Regional
Offices
Decision makers, non-scien-
tific personnel, and the gen-
eral public
"v" The Biological A ssesstrent and Criteria (BA C) workshop
Research Abstracts
v" Strategic Customer Value Analyses, workgroups,
meetings and symposia, and other direct communi-
cation between the scientists and the regulators
"v" Task reviews
-------
Table 1. Communication Goals, Audiences, and Innovations of the
ORD Laboratories and Centers (Continued)
Laboratory or Center
Goals
Audiences
Innovations and Other Communication Strategies
Highlighted at May Workshop
NERL (continued)
3. Ensuring that all NEKL
staff can articulate the work
being done at the Labora-
tory to a variety of audi-
ences
4. Sharing and showcasing
NERL's research
5. Informing NERL's staff
about NERL communica-
tion products.
Task Information System database, which is an
electronic delivery and feedback system to track
annual performance goals, delivery of research
products to customers, and scientific publications
National Health and En-
vironmental Effects Re-
search Laboratory
(NHEERL)
1. To be accurate and innova-
tive in communicating re-
search results to a wide va-
riety of audiences
2. To provide the tools to the
scientists so that they are
prepared to better commu-
nicate their research results
3. To inform decision makers.
NHEERL
ORD
"v" Other federal agencies
"v" Congress
Q The scientific community
"v" Other external audiences (in-
cludes the public)
0- NHEERL Science Report
v* NHEERL Annual Report of Accomplishments
v" Communications program ' ;;.
^ Communications course for scientists, including
emphasis on media relations and fact sheet writing
"v" NHEERL communications desk-top reference
"v" Authoring, co-authoring, or reviewing Program
Office guidance documents
"v" Publications in scientific literature and tracking
thereof; use of Intra- and Internet Web sites, brief-
ings and fact sheets; brochures and printed prod-
ucts to highlight upcoming/ongoing research in a
specific region; and rapid release reports on 'hot'
topics for broad audiences.
National Center for En-
vironmental Assessment
(NCEA)
1. To inform the public and
all stakeholders
2. To tell the complete and
fair story.
3. To develop the appropriate
communication strategy for
each of the key risk assess-
ment efforts, and from the
v* Environmental decision mak-
ers, including the states and
the international community
More specifically:
"v" EPA Regions and programs
Comprehensive communication plan for rollout of
draft cancer guidelines
- Comprehensive Communication Strategy in-
cludes: description of action, background, key
messages, audience, expected reactions from
stakeholders, detailed communication strategy,
announcement notification plan (what, when,
who), and contacts
-------
Table 1. Communication Goals, Audiences, and Innovations of the
ORD Laboratories and Centers (Continued)
Laboratory or Center
Goals
Audiences
Innovations and Other Communication Strategies
Highlighted at May Workshop
NCEA (Continued)
strategy develop useful,
readily available, clear, well-
untten, and timely commu-
nication/outreach materi-
als.
v1 State and local pollution con-
trol agencies
SAB members
"v" Academia
v* Scientific societies (Society of
Toxicology, Society for Risk
Analysis, etc.)
- Briefing presentations designed for audience
(congressional, stakeholders, press)
- Public fact sheets (two)
- Federal Register Notices
-Public Frequently Asked Questions
- Headquarters press release
- Phone calls
-Mailings
v" Single point person at NCEA responsible
"v* Program and issue-oriented Web pages
v" Invest in communications and outreach; build
staff; listen to internal and external stakeholders;
identify Agency needs; work with ORD Laborator-
ies and Centers and EPA Offices; and address -
BOSC recommendations
"v" Use recognized toolbox of communications mate-
rials at EPA
v* Internet
v" Coordinate across the Agency (cross-Agency com-
munications group), with other federal agencies,
and other parties as appropriate.
-------
APPENDIX A:
Summary of the September 23-24, 2002 BOSC Meeting
-------
U.S. Environmental Protection Agency
Office of Research and Development
.,< ':.
BOARD OF SCIENTIFIC COUNSELORS
EXECUTIVE COMMITTEE MEETING
Washington, DC
September 23-24, 2002
Monday September 23, 2002
Welcome, Review of Agenda, Introductions
Dr. Jerry Schnoor (University of Iowa), Chair of the Board of Scientific Counselors (BOSC) called the
meeting to order at 1:00 p.m. He welcomed everyone to the September meeting and mentioned that there
are four new BOSC members, two of whom were present. Dr. Rogine Henderson (Deputy Director,
National Environmental Respiratory Center) introduced herself and described her research interests and
areas of expertise. She is an inhalation toxicologist and her research focuses on biomarkers. She
mentioned that she has worked with Dr. Paul Oilman (AA/ORD) on numerous National Academy of
Sciences (NAS) committees. Dr. George Daston (The Proctor & Gamble Company) is a developmental
toxicologist whose research focuses on chemical teratogenesis and toxicogenomics. Dr. Schnoor
mentioned that the other two new members would be introduced at the next BOSC meeting. He also
announced that Dr. Gilman would be unable to attend today's BOSC meeting because of other
commitments.
With regard to disclosures, Dr. Schnoor said that disclosures were discussed at the May 13-14, 2002
BOSC meeting, and would be discussed at the next BOSC meeting in January 2003. He noted that
tomorrow's agenda includes a discussion of the new disclosure form that replaces the old disclosure form.
Dr. Schnoor quickly reviewed the meeting agenda. He mentioned that the Ad Hoc Communications
Subcommittee has not met since the last BOSC meeting, and that Dr. Ann Bostrom (Georgia Institute of
Technology) would provide an update on the Subcommittee's efforts. He asked the BOSC members to
read the minutes from the May meeting and the draft of the mercury multiyear proposal as homework
before tomorrow's session.
Measures of Success Report
Dr. Schnoor prepared a draft of the Measures of Success Letter Report, which was displayed on the
overhead projector for the BOSC members to review and suggest changes. Dr. Dan Acosta (University of
Cincinnati) noted that he had already submitted his comments on the report and they had been
incorporated by Dr. Schnoor in this version. Dr. Jim Clark (Exxon Mobil Research & Engineering Co.)
also had pointed out some misstatements that have been corrected in this version. Dr. Schnoor
commented that Dr. Gilman seems to prefer shorter, hard-hitting reports, of which this is one. He
pointed out that the report emphasizes that the Strategic Plans of the Laboratories and Centers should not
lose focus and emphasis in the multiyear plans. Dr. Schnoor mentioned that several members of the
BOSC liked the "LOGIC" model and there was agreement that it should be mentioned in the letter report.
September 23-24, 2002 BOSC Executive Committee Meeting Summary
-------
Dr. Rae Zimmerman (New York University) suggested that a sentence be added to the section on
qualitative indicators stating that these indicators, as well as the quantitative indicators, should be mapped
onto the goals in the Laboratory, Center, or Division,,Strategic Plans. Further, it was agreed that the last
sentence of the letter should encourage the Laboratories and Centers to put something in place and try it.
Dr. Schnoor asked to BOSC to vote on the letter report so that it can be submitted to the AA/ORD. Dr.
Clark moved to approve the letter report, and Dr. Acosta seconded that motion. The BOSC approved the
letter report unanimously.
Future Meeting Dates
Dr. Schnoor asked the BOSC members about their availability for meetings in January and May 2003.
Shirley Hamilton (EPA/NCER) proposed the following dates for the next two BOSC meetings: January
9-10, 16-17, or 23-24; and May 19-20, 15-16, or 22-23. She asked the members to check their calendars
tonight so that the next two meetings can be scheduled during tomorrow's session.
Remarks of the NCER Director/BOSC Liaison
Dr. Peter Preuss (EPA/NCER) welcomed the new members to the BOSC. He mentioned that Dr. Oilman
would like the BOSC to move in a new direction by providing more scientific consultation that utilizes
the scientific expertise, as well as the science management expertise, on the Board. Dr. Oilman also
wants the Board to provide input earlier in the planning process. Dr. Preuss noted that there will be a
number of items in the future regarding which ORD will seek the BOSC's advice. He reported that EPA
currently is working on many fronts, including the FY2003 budget, FY2004 budget, and various ideas
and plans for the future. Dr. Oilman plans to discuss these efforts with the BOSC at the January meeting.
When Dr. Preuss indicated that he would not be available to attend tomorrow's session, Dr. Bostrom
asked if the BOSC could discuss the next steps for the Communications Subcommittee while Dr. Preuss
is present, even though it is on the agenda for tomorrow's session. Dr. Bostrom reported that the
Communications Subcommittee had been unable to schedule a site visit at NCER; however, she has read
all five Laboratory/Center reports and believes that there is agreement within the Laboratories and
Centers that more work is needed to effectively communicate research results. Dr. Bostrom asked if the
Subcommittee should continue its attempts to schedule a site visit or if the Subcommittee should
eliminate the site visit from its plans. Dr. Preuss explained that the site visit had to be postponed because
of the ongoing intensive review of the STAR Program by NAS as well as his efforts as the technical lead
on the state of the environment report for the EPA Administrator. He suggested that Dr. Bostrom speak
with Dr. Oilman about the Subcommittee's charge and plans; he noted that Dr. Oilman is very interested
in communications and may have a number of ideas on how the Subcommittee should proceed.
National Center for Environmental Research Draft Report
Dr. Schnoor asked Dr. Jim Johnson (Howard University) to describe the changes made to the NGbK
report since the last meeting. Dr. Johnson identified the minor changes that had been made to the report,
noting that several changes were made to correct statements of fact in the report; some of the wording
regarding NCER and NCER activities had been revised in response to comments at the May meeting. Dr.
Bostrom mentioned that Dr. Johnson has been very receptive to suggested changes and has worked hard
to improve the report. Dr. Johnson said that Dr. Preuss provided more current information for the report
and offered many suggestions for improving it. Dr. Johnson noted that completion of the Center's
Strategic Plan should be a high priority. He reported that all comments received to date had been
incorporated into this draft of the report. Dr. Schnoor caiied for a vote on the report, and it was approved
unanimously by the BOSC. Dr. Schnoor then thanked the BOSC members for the 18-months of effort
devoted to the preparation of the five Laboratory/Center reports. Dr. Schnoor said that his goal is to
submit the five Laboratory/Center reports to Dr. Oilman by October 1, 2002.
September 23-24, 2002 BOSC Executive Committee Meeting Summary
-------
Dr. Bostrom asked about the common themes among the reports. She mentioned that Dr. Schnoor had
agreed to draft a page on the common themes. The BOSC agreed that Dr. Schnoor would look though his
notes concerning the common themes and transmit tjtiis-information in a letter report to Dr. Gilman.
Consultation on Biotechnology
Dr. Larry Reiter (EPA/ORD/NHEERL) indicated that he was pleased to see the "top-down, bottom-up"
approach mentioned in the Measures of Success Letter Report because this is how ORD is approaching
the development of the biotechnology and computational toxicology initiatives. The overall goal is to
create a scientific framework that lays out issues and then to engage the scientists to map out specifics of
the research programs. The regulation and monitoring of genetically modified crops has become an
increasingly important issue for the Office of Prevention, Pesticides and Toxic Substances (OPPTS).
However, both the OPPTS and the NAS agree that more science is needed to assess the risk of genetically
modified (GM) crops to more effectively inform decisionmaking.
The FY2003 budget for biotechnology is $5 million. The focus of this work is geared to OPPTS' most
pressing needGM crops. Dr. Reiter informed the BOSC that he established a steering committee to
create a top-down scientific framework for the work and to then oversee the bottom-up development of
the program with research projects that best fit EPA's needs. To date, an R&D framework has been
established, proposals have been solicited from the Laboratories and Centers, and the steering committee
has met with other organizations involved in the biotechnology area to get their reaction to the developed
framework. The organizationsU.S. Department of Agriculture (USDA), National Institute of Allergy
and Infectious Diseases (NLAID), Food and Drug Administration (FDA), U.S. Agency for International
Development (USAID), European Union (EU), and NASthat have been solicited for advice thought
that the proposed framework was on target in that it seemed to meet EPA's needs and it complemented
the work going on elsewhere. Nothing in their comments led Dr. Reiter or the steering committee to
think that the proposed program needed to be changed.
Dr. Clark wanted to know how this program would fit within ORD's Strategic Plan? Dr. Reiter
responded that the program was being designed with the same philosophy as the plan by identifying high
priority Agency needs and then applying the science to meet these needs. Because the program will not
be initiated until FY2003, the specifics are not in the plan; nevertheless, the program is in the planning
stages and work in this area will begin if and when Congress approves the work in the FY2003 budget.
Dr. Reiter identified three main areas of research: (1) assess the risk of allergenicity from genetically
altered foods as a result of new proteins being introduced into food; (2) assess the possibility for gene
transfer (hybridization and cross-fertilization) and the ecological risks associated with genetically
modified organisms; and (3) manage gene transfer and resistance, specifically resistance of pests to
Bacillus thurigiersus (Bt).
Dr. Daston wanted to know what GM pesticides have been introduced into plants for commercial use. Dr.
Reiter answered that so far, only Bt has been inserted into plants and used commercially. The main
concern is that pests eating the plants will develop resistance to Bt. This is of special concern to organic
farmers whose main weapon against insects is Bt.
Dr. Reiter then presented five slides on problem formulation:
(1) There are no valid animal models to predict the dietary allergenicity and the long-term effects from
consumption of GM crops.
September 23-24, 2002 BOSC Executive Committee Meeting Summary
-------
(2) Regarding ecological effects, there is no standardized, verified assay to test for effect on non-target
species. ».
.,; :"-.
(3) There needs to be investigation to understand the likelihood of gene transfer and how long modified
genes persist in the environment. A major question is whether or not GM crops can cross-hybridize
with wild-type organisms.
(4) Little data exist on whether pesticide resistance occurs in genetically modified crops under field
conditions. The EPA, however, does have a strategy for dealing with pesticide resistance (i.e., the
"High Dose Structured Refugia" strategy).
(5) There are no strategies to identify the key risks of concern nor are there effective risk management
technologies to mitigate key risks.
Dr. Reiter mentioned that in the meetings with key officials from other agencies/organizations, Dr. Kim
Waddell of NAS cross-mapped the problems formulated by EPA against the three NAS reports on GMOs
and found that EPA's propose*! program covers the major issues identified by the NAS.
Dr. Reiter then reviewed the allergenicity research goals. He noted that the proposed work in this area is
complementary to the work being done by the NIALD. There are a number of factors that influence
allergenicity, and they are reflected in the following research goals: (1) determine potency relative to
known allergens, (2) identify windows of vulnerability during early development, (3) identify endpoints
for use in an animal model, (4) relate digestibility to allergenicity, and (5) understand the mechanisms of
allergenicity. Dr. Bill Chameides (Georgia Institute of Technology) asked what was meant by
digestibility? Dr. Janet Anderson (EPA) replied that digestibility is the time to digestion, and in this
context it also would refer to the time required to mount an immune/allergic reaction. Dr. Reiter stated
that the products that will potentially result from this research are an animal model to assess the
allergenicity of transgenic pesticide proteins and an understanding of the factors that contribute to
susceptibility to sensitization to dietary allergens. Dr. James Bus (The Dow Chemical Company) wanted
to know if EPA also would look at the dose-response relationship between allergenicity and the amount
of transgenic protein ingested. Dr. Reiter responded that those types of studies would come later, after a
model has been developed. Dr. Schnoor asked if it was correct to assume that studies measuring the risk
of pesticides on humans have been performed. Dr. Anderson responded that for all approved pesticides,
toxicity on humans has been evaluated.
Dr. Reiter then discussed the specific GMOs that will be studied initially. Bt com and cotton crops will
be used to study pesticide resistance. Bentgrass (its relatives include rye and fescue) and canola will be
used to study gene transfer to non-GMO species. Bentgrass and canola make good models to study gene
transfer because they are commercially important and they are grown in close proximity to non-GMO
relatives. Dr. Bus asked if the studies would be extended to examine the ratio of saturated to unsaturated
fats in canola oil. Further, could studies such as this be done in collaboration with the USDA? Dr. Reiter
responded that although interesting, such studies were beyond the scope of the current proposal. There
was a brief discussion regarding which agency (USDA, FDA, EPA) has regulatory authority over GM
crops and if this should be a collaborate effort between these agencies. Dr. Schnoor stated that pest
resistance is an EPA focus. Dr. Daston commented that this science may not be applicable to other
agencies, given that it focuses on a pesticide problem.
Dr. Reiter then explained "High Dose Structured Refugia," the approach developed by OPP as a
requirement for registrants using GM crops with plant incorporated protectants (PEPs). It is used to
reduce the likelihood that gene transfer from the PIP to an insect will result in the development of
resistance in a large population of insect pests. A small plot of non-GM crop is planted within the
perimeter of a field planted with Bt GM crop. The pests will be present in the non-GM crop patch, but
September 23-24, 2002 BOSC Executive Committee Meeting Summary >
-------
not in the GM crop. However, if there are genetic changes resulting in Bt resistance in the pests, then
there will be a concomitant change in the location of the pests.
.';?' '-.--
Dr. Bus was concerned about whether NCEA had the expertise to perform these types of studies. Dr.
Reiter ensured the BOSC that NCEA has the expertise to perform these experiments and he pointed out
that bridges would be formed among NERL, NCEA, and NRMRL to perform the studies. Furthermore,
NCER's STAR Program also might be used to fill potential gaps.
Dr. Reiter stated that the next steps are to finalize the research plan and to hold a workshop to present the
program and solicit feedback. He asked the BOSC members for their reaction to the program. Dr. Clark
praised the proposal, but expressed concern that $5 million would not be sufficient for a program of this
magnitude. He though it may take closer to $500 million to implement the program presented. Dr. Reiter
suggested that $5 million is sufficient for FY2003. Dr. Bostrom asked about the level of federal funding
that currently is being directed to research on GMOs. Dr. Reiter replied that in the discussions with other
federal agencies, the sense was that there is not much money being spent on this area; he noted, however,
that there might be sizeable programs in other federal agencies that did not participate in the discussions.
The USDA and FDA currently support 90 projects totaling $16 million; therefore, $5 million is a
respectable amount of funding. In contrast, the European Union has spent a total of about $17.5 million
on GMO research. Dr. Bostrom wondered if the Department of Energy (DOE) is supporting any research
on GMOs.
Dr. Daston commented that the areas of proposed research were chosen well, but he also did not think that
$5 million was enough funding to cover the proposal. He suggested that ORD approach this research by
looking at the "worst case scenario." If that scenario was found to be without effects, then there would be
no need to pursue research in that area. For example, if there was no substantial gene transfer between
species, then that issue could be set aside as a priority and ORD could move on to other priorities. Dr.
Reiter expressed concern over trying to prove the negative (i.e., the worst-case scenario). Dr. Henderson
wondered how one would even go about determining the worst case scenario. Dr. Chameides asked if
industry was conducting research on GMOs. Dr. John Glaser (EPA/NRMRL) commented that a
consortium of seed companies for Bt corn recently met in St. Louis, MO. Dr. Anderson said that industry
has chosen to plant GM crops only in regulated areas, rather than fund the research. Dr. Chameides
suggested that industry should be involved at future workshops. Dr. Schnoor commented that ORD was
wisely looking at the regulatory issues, but how might those change with the outcome of the research?
Dr. Reiter answered that information will come out of the research that will allow OPPTS to determine if
they are on the right course with regard to pesticide resistance. Dr. Bus commented that this is the bold
approach that is needed to evaluate the ecological impact over time as more crops are created on which
the EPA must make regulatory decisions. Dr. Anderson mentioned that Bt crop resistance has been
monitored since 1995, and there is no sign yet of pests developing resistance. Dr. Bostrom asked Dr.
Reiter if the workshops would contribute to the prioritization of spending. Dr. Reiter replied that the
purpose of the workshops is to sharpen the focus and prioritization of the research, so that a critical path
is maintained as research programs are being developed.
Dr. Schnoor asked Dr. Reiter if he would like the BOSC to prepare something in writing in response to
this presentation. Dr. Reiter said that feedback from the BOSC regarding the program would be greatly
appreciated. Dr. Bostrom asked if EPA has any written products outlining the program. Dr. Reiter said
that there is a framework document that he can provide to the BOSC. He mentioned that this document
has been revised several times; the fourth draft currently is available. Dr. Schnoor said that the current
draft would be fine and the BOSC was provided copies of that draft. He then asked the members of the
BOSC what they envisioned happening next. Dr. Chameides said that the BOSC should read the EPA
framework report and discuss it at the next meeting or during a conference call. Dr. Schnoor thanked Dr.
Reiter for his presentation and promised to provide feedback on the program.
September 23-24, 2002 BOSC Executive Committee Meeting Summary
-------
Conceptual Report from ORD in Computational Toxicology
<*
After the break, Dr. Reiter described to the BOSC a conceptual framework on computational toxicology.
He stated that this program is an important new initiative for EPA. The computational toxicology
program is a visionary one, and it may very well change how ORD does business in the future.
Furthermore, the goals of this program go beyond the EPA, and the program provides a model of how
science being developed along several pathways can all feed into one program.
Dr. Reiter commented that EPA needs to understand the cascade of events that occurs from source of a
stressor to an adverse outcome. EPA's approach is to do quanitative risk assessments and then develop
risk management options for prevention and/or control. The trend has been to evaluate a single chemical
at a time, which informs our understanding of this continuum.
Dr. Reiter noted that there are several programmatic challenges. There are already a number of priority
lists in the queue, but there is not a risk-based criterion for setting testing priorities. Each authority comes
with different testing requirements, which boils down to what you are going to test and how you are
going to test it. There is not a scientific basis for flexible testing approaches. Finally, there is a lack of
data to reduce the many uncertainties faced in quantitative risk assessments (e.g., extrapolations). Dr.
Reiter indicated that genomics and proteomics have been making major breakthroughs, which promise to
lead to the creation of an important set of tools which ORD can begin to apply.
Dr. Schnoor asked if the computation would be associative or predictive. Dr. Reiter responded that the
application of computation would go in both directions. If one looks at a continuum then it seems that
genomics is a lynchpin. He suggested that the BOSC think about this issue in terms of generating an
understanding of genomic pathways at the cellular level, the tissue level, the organ level, and finally at
adverse outcomes in the whole organism. If we can understand genetic changes and how they relate to
outcomes, then we can use genomic profiling in a predictive manner. Dr. Reiter said that the idea behind
the computational toxicology program is to integrate modern computing and information technology with
molecular biology and chemistry to improve EPA's prioritization of data requirements and to improve
risk assessment for toxic chemicals. Further, if genetic changes are predictive of toxic outcomes,
genomic changes can be used to measure target dose and therefore provide a way to link physiologically
based pharmacokinetics (PBPK) models with biologically based dose-response models.
There are many applications for this research , including but not limited to, delineating toxic pathways,
extending cross- and within-species extrapolation, identifying genomic endpoirits for quantitative
structure-activity relationship (QSAR) models, identifying genomics-based exposure biomarkers, and
understanding cross- and within- species variations on pharmacokinetics.
Dr. Reiter indicated that in FY2002, EPA began to do proof of concept for computational toxicology
while building an integrating research framework for the overall program$4 million appropriated by
Congress to develop alternatives to animal testing. Dr. Reiter noted that endocrine disrupter chemicals
(EDCs) will be used as a first approach for computational toxicology for several reasons: (1) EDCs are a
high EPA priority, (2) these are mechanistically driven outcomes. (3) EDCs allow EPA to integrate health
and ecology, and (4) EDCs are an area of high animal use and computational toxicology approaches could
significantly reduce the number of animals required for testing. Dr. Schnoor wanted to know if the EPA
tracked the use of animals in research. Dr. Reiter responded that EPA does track the use of animals, but
he was not firm un wlictaci luc use of animals in EDC rcscsrch is specifically tracked.
Dr. Reiter presented a slide, which demonstrated the intersection of in silico, in vitro, and in vivo research
assays, the combination of which offers the promise to lead to an understanding of the cascade of events
from target dose to adverse outcomes. Dr. Henderson expressed her amazement at some computation
work in proteomics that does not require detailed understanding of what gene/protein is being analyzed.
September 23-24, 2002 BOSC Executive Committee Meeting Summary (
-------
For example, in a recent report in the journal Lancet, a group examined pollutants and used gene changes
to predict ovarian cancer. They knew very little about the detailed biology, the results were based on
software that detected changes in patterns. Dr. Reiter mentioned that Dr. Henderson was speaking more
about informatics, into which the ORD will eventually invest more effort. Dr. Reiter agreed that there
currently is a lot of work on developing pattern recognition programs, and ORD is doing some work on
pattern recognition and analysis of changes in the pattern of gene and protein expression. He noted,
however, that there can be considerable noise in the system that can cloud interpretation of the results.
Dr. Gary Foley (EPA/NERL) mentioned that his Laboratory is leading the research effort in this area; he
is speaking with IBM, Sandia National Laboratories, and other agencies within the federal government to
find potential partners in the area of bio-informatics. Dr. Daston added that if the discriminant analysis is
so good then one could make the determination between one group and another without needing the
mechanistic information. However, the odds of that happening in toxicology are low, so one needs the
mechanistic information to explain variation from the model. Further, information on dose is needed to
understand the cascade leading from exposure to adverse outcomes.
To illustrate the concept, Dr. Reiter's next slide showed how reproduction might be impaired in various
ways by impacting estrogen pathways, androgen pathways, and thyroid pathways, and combinations of in
silica, in vitro, ex vivo, and in vivo assays would be used to understand pathways leading to altered
development.
Dr. Reiter then presented the Xenopus Metamorphosis Model for Thyroid System Disruption. First,
scientists look at the gene changes on a molecular level in the hypothalamus. Then, on the cellular level,
they look at circulating hormone levels, and how they relate to changes in thyroid histology on the tissue
level. Finally, the scientists look at the altered morphology of the Xenopus on the organismal level.
Combining results in such a way provides a basis for understanding toxic pathways and developing
predictive models. Such a framework will be needed before any computational toxicology model is ready
to go "prime time." Several questions remain however. How can we take what we are doing and expand
it to be useful to inform decisionmaking at EPA? What are the areas of highest priority? Dr. Reiter
presented a slide outlining examples of research areas that will support FY2003 base realignment
including toxicogenomics of drinking water contaminants, genomics of aquatic organisms, and genomic
biomarkers from surrogate tissues.
Dr. Reiter noted that the goal of genomics research in aquatic organisms is to integrate exposure/effects of
EDCs in common organisms. The concept is that an understanding of development patterns as related to
endocrine pathways, will provide the means to begin to integrate SAR models. This will allow us to
refine the fathead minnow test and make it more predictive, providing a bridge between gene regulation
across androgen dose and time. Dr. Foley commented that this research also provides possible
opportunities to define susceptibility and exposure-dose relationships. Dr. Zimmerman wanted to know if
any of this research feeds back into the ORD exposure handbook, and Dr. Reiter replied that it does.
Further, as these areas develop, they will have a tremendous influence on how ORD performs these
experiments.
Dr. Reiter's next slide concerned genomics research in aquatic organisms; he indicated that the major
challenge is the need for gene discovery. Dr. Daston replied that it is not simply the lack of genomic data,
but also the creation of transgenic animals and banks of mutagenized animals that limit the research. Dr.
Daston asked why EPA was using the fathead minnow when there appears to be so much more
information on the zebra fish. Why not use the zebra fish instead of developing the information on the
fathead minnow? An audience member commented that zebra fish are not very hardy, and therefore, not
a good experimental model. Dr. Daston wondered whether there are more hardy strains of zebra fish that
could be used for the experimental model. Dr. Schnoor mentioned that work could be done on both
organismswhile genomic data are being gathered on the fathead minnow.
September 23-24, 2002 BOSC Executive Committee Meeting Summary
-------
In concluding his presentation, Dr. Reiter stated that theTnext steps for this program are to: (1) develop a
strategic plan for computational toxicology, (2) hold pees consultation workshops, (3) coordinate and
collaborate with other agencies, (4) develop a research .'implementation plan, and (5) subject the
computational toxicology plan to peer review. Although ORD has a general roadmap for the program,
there is a need to prepare a written plan of action.
Dr. Reiter pointed out a number of challenges to this type of approach. Computational toxicology has the
potential to change how the EPA evaluates the risk of chemicals. A big challenge will be matching the
needed expertise in genomics, proteomics, and high-performance computing with capabilities to do so.
This will involve coordination with others to meet the computational toxicology objectives. EPA will not
solve these problems alone, but the Agency brings to the table an understanding of the context in which
many of these decisions have to be made. Dr. Reiter stated that interpretation of the data will be the key.
It is easy to generate a lot of genomics data, but one has to interpret what changes mean in terms of
outcome or to what organisms are being exposed.
Dr. Schnoor asked if there were any questions or comments on Dr. Reiter's presentation. Dr. Henderson
asked that ORD keep in mind protein changes and not just changes in genes as it builds its program. Dr.
Daston expressed his enthusiasm for this project. He mentioned that he has been working on genomics;
about $250,000-$400,000 has been spent by Proctor & Gamble on generating genomic data. He said that
he could share these data with EPA, and although these data are significant, they are just the tip of the
iceberg. Collaboration and cooperation will be needed to make progress in this area.
Dr. Clark thought this was a great program and that EPA is moving in the right direction. He also liked
the fact that both human health and ecological issues were being addressed. Dr. Bostrom wanted to know
the amount of ORD resources that currently are dedicated to genomics. Dr. Reiter replied that he is not
certain because EPA has not yet completed the inventory; however, he mentioned that these techniques
are really permeating the Laboratories and Centers. Dr. Reiter also commented that the Agency is still
grappling with issues regarding scope; if care is not taken, everything could be considered computational
toxicology. He noted that it will be important to create this type of crosscutting research without
disrupting the critical path of the individual Laboratories on other key areas of research needed to meet
EPA's needs.
Dr. Schnoor commented that this program is really shaking the foundations of risk assessment. Dr. Clark
pointed out that maintaining such a program is costly. Does that mean the program would wind down in
a few years? Dr. Reiter responded that ORD is being careful not to over promise. It is critical for
management and Congress to see the importance of pursuing this program and to recognize that it will
require a long-term commitment. Dr. Herbert Windom (Skidway Institute of Oceanography) asked how
the ORD plans to engage scientists into this area. Dr. Reiter replied that they are using a number of
approaches to engage the scientists. He noted that these approaches are still in the discussion stage;
hnwe.ver; FPA's scientists appear to embrace the concept. Dr. Schnoor wanted to know if this would
become a multiyear plan. Dr. Reiter responded that he hoped that it would, because this program needs to
move forward and grow over time. Dr. Schnoor thanked Dr. Reiter for his presentations and reminded
the BOSC members of their homework assignments.
Tuesday September 24, 2002
Review of Agenda and Approval of May BOSC Meeting Minutes
The meeting was reconvened by Dr. Schnoor at 9 a.m. There was a brief discussion of the May meeting
minutes, and it was decided that the last full sentence of paragraph 4 in the May 13 meeting minutes
(closed session) would be deleted. Dr. Johnson asked that term "work strategies" be changed to
September 23-24, 2002 BOSC Executive Committee Meeting Summary
-------
"objectives" in the section regarding the Measures of Success Letter Report. He also had a question
regarding the last paragraph of the May 13 meeting minutes, where it was noted that the ORD BOSC was
now the EPA BOSC. Dr. Schnoor commented that .this change reflected the BOSC's expanded domain to
research and scientific issues as well as management advice, but clarified that the BOSC would still report
to the AA/ORD. It was agreed that this sentence would be revised. With the suggested changes, the May
meeting minutes were approved unanimously by the BOSC.
Final Discussion/Vetting of BOSC Subcommittee Reports
Dr. Chameides said that he vetted two of the reportsthe NHEERL and NRMRL reports. Dr. Schnoor
asked if there were any reports that needed more attention before they were submitted to Dr. Oilman. Dr.
Zimmerman said that she had marked several small editorial changes in the NCEA report and given them
to Beverly Campbell'(SCO). Dr. Bostrom mentioned that the NERL report reads differently than the
other four reports. In particular, the portion of the report that deals with communication only mentions
communication between the PI and Laboratory Director. This and other differences make the report seem
inconsistent with the others. Dr. Bostrom suggested that a checklist for the body of the reports be created
in the future to maintain consistency among BOSC Subcommittee reports. Dr. Schnoor said that he
would read the NERL report again before it is finalized and submitted to Dr. Oilman. He asked that Drs.
Chameides and Bostrom also read the NERL report once more. Changes of a vetting nature should be
sent to Beverly Campbell for inclusion in the final version of the report. Dr. Schnoor asked that any
changes also be sent to Dr. Juarine Stewart (Clark Atlanta University) so that she can determine which
changes to incorporate and how best to revise the report.
Dr. Johnson asked that the final Laboratory/Center reports be sent via e-mail to the members of the
respective Subcommittees that worked on the reports. Dr. Schnoor agreed with this suggestion and asked
Beverly Campbell to distribute the reports to the Subcommittee members. Dr. Bostrom pointed out that
the reports could simply be posted on the BOSC Web site if one existed. Ms. Hamilton replied that she
has been looking into the possibility of creating a BOSC Web "site.
Dates for Upcoming BOSC Meetings
Based on the availability of the members, it was decided that the next BOSC meeting would be held on
January 9-10, 2003. The meeting will begin at 10:00 a.m. on the 9th and adjourn at 2:00 p.m. on the 10th.
The subsequent meeting will be held on May 22-23, 2003. That meeting probably will be held in
Research Triangle Park, NC, and will begin early on the morning of the 22nd so that it can conclude
around 12:00 noon on the 23rd.
Discussion of SAB Activities
Dr. Schnoor introduced Dr. Robert Flaak, the Acting Deputy Director of the Science Advisory Board
(SAB), who provided an update on the SAB's activities. Dr. Flaak mentioned that Drs. Oilman and
Schnoor will be attending the SAB executive meeting in October in Washington, DC.
Dr. Flaak presented a detailed outline of the SAB history and role. Congress established the SAB in
1978. The SAB includes 107 members, who are considered special government employees; 400
consultants; and a staff office of 18, 8 of which are Designated Federal Officers. The SAB averages 40
reviews a year, and the Board's agenda is set through the Science Policy Council (SPC). Dr. Flaak noted
that a GAO review that was initiated 2 years ago resulted in a change in panel formation,
disclosure/conflict of interest, and public review at the SAB. In response to GAO concerns, EPA
developed a new 3110-48 conflict of interest form, which will be discussed by Dr. Angela Nugent later
today. Dr. Flaak mentioned the requirement for annual ethics training, and reported that EPA is
September 23-24, 2002 BOSC Executive Committee Meeting Summary
-------
developing a CD/online ethics-training course that could be used by the BOSC. He stated that the SAB
advises the EPA through written reports that are generated during its quarterly meetings or through one or
more teleconferences. A..:.:,-._
Another new initiative of the SAB is the stakeholder process where the public has an opportunity to
comment on issues under consideration by the Board. Dr. Flaak reported that one public meeting has
been held and there were press and industry representatives present. Dr. Henderson expressed some
concern that the stakeholders who attend these meetings will be those geographically located in the
Washington, DC, area. Dr. Flaak replied that individuals could attend these meetings via teleconference
or via the Internet. Dr. Zimmerman asked how the meeting was advertised. Dr. Flaak replied that the
meeting was announced in the federal register and through mailings. Dr. Flaak briefly discussed the new
panel formation process and directed the BOSC members to the SAB Web site
(http://www.epa.gov/SAB) for more information. Dr. Schnoor thanked Dr. Flaak for the informative
review and asked if the BOSC members had any questions.
Dr. Schnoor asked if there were any projects on which the SAB and the BOSC could work together. Dr.
Flaak replied that the SAB should work with the BOSC on the ORD projects. To address questions
regarding the difference between the role of the SAB and the role of the BOSC, Dr. Flaak stated that the
main difference is that the SAB looks across the entire Agency, and the BOSC looks specifically at ORD
programs. Dr. Zimmerman inquired if the different SAB subcommittees work together and if there are
inter-committee interactions. Dr. Flaak said that there are interactions between subcommittees on many
projects, but he listed only the lead committee on the spreadsheet that he distributed to the BOSC
members.
Dr. Daston wanted to know if the new disclosure/conflict of interest procedures affected the timeline for
convening a committee. Dr. Flaak responded that is takes about 1 month to generate a short list for a
committee. That list then is posted on the Internet and comments are solicited from the public. This
process now takes about 3 to 4 months. He pointed out that the review process is becoming more
efficient and so the entire time required for the process (from convening the subcommittee to producing
the report) has been reduced. Using a 3-day meeting structure, the subcommittee prepares the report at
the meeting and it usually is approved within 1 month. Dr. Acosta asked how often a full member of the
BOSC also is. a member of the SAB. Ms. Hamilton stated that the BOSC agreed not to appoint members
of the SAB to the BOSC; however, BOSC members can be SAB consultants. Dr. Flaak added that there
are several individuals who serve on more than one SAB committee. He also said that they are trying to
reduce the size of the committees to 8-9 members so that more expert consultants can be added to the
committees. Dr. Chameides asked if the conflict of interest/disclosure forms could be completed online.
Dr. Flaak said that they could be filled out electronically, but that the form then needed to be printed and
signed, as there is no protocol for collecting electronic signatures. Dr. Flaak encouraged the BOSC
members to call or email him if they have additional questions (202-564-4546 or Flaak.Robert@epa.gov).
Discussion of Confidential Financial Disclosure Form for Special Government Employees
Dr. Angela Nugent (SAB/EPA) provided an overview of the new confidential financial disclosure form
developed for special government employees, which she helped to create. Dr. Nugent said that the new
form is a tool to help DFOs do a better job of serving the committee and fulfilling their duties as a federal
officer and to protect the committee members. She mentioned that the form can be downloaded from the
SAB Web Site. The form enables EPA to collect the data it needs to evaluate conflict of interest and the
appearance of lack of impartiality. Dr. Nugent noted that the old form did not collect data on grant
funding sources, detailed consulting work, or relationship/leadership roles in nonprofit organizations.
She pointed out that the SAB was previously criticized for not collecting.that kind of information.
Further, a GAO report (June 2001) indicated that the EPA was not collecting the right kind of
information. The GAO compared the EPA form with those used by other organizations, such as journals
September 23-24, 2002 BOSC Executive Committee Meeting Summary 10
-------
and peer review boards, and used those forms as the benchmark. Previously, EPA did not think it had the
authority to collect that kind of information, but there was a change in thinking in the ethics office, so the
new form was developed and approved. This new form represents a balance between a form that was too
cumbersome to complete and one that did not collect enough information.
The first page of the new form explicitly defines conflict of interest and appearance of lack of
impartiality. The appearance of lack of impartiality is a personal circumstance that would cause a
reasonable person to question the special government employee's involvement in a matter. The special
government employee is required to inform the DFO of these personal circumstances, such as a spousal
activity, consulting activity, etc. Dr. Johnson about the difference between bias and conflict in regard to
the new form. Dr. Nugent replied that the form only collects financial conflict of interest data; it does not
collect information on bias. However, the SAB does try to ask potential committee members about their
position and if there might be a bias on the particular subject which will be discussed by the committee.
Dr. Nugent mentioned that the CD-ROM-based training on ethics includes a segment on conflict of
interest. Ms. Hamilton mentioned that the BOSC members are required to receive annual ethics training,
and she pointed out that it was time for that annual training. Dr. Schnoor said that he would like to have a
discussion about bias and ethics at the next BOSC meeting.
Dr. Nugent then discussed the differences between the new form and the old one. The most significant
differences are in parts 2 and 5 of the form, which focus on consulting work. Part 2 deals with consulting
work for consulting firms by the individual or spouse. Part 5 has to do with consulting activities for
academic and other institutions. Dr. Zimmerman asked where to indicate the consulting firm's clients.
She also asked about the rationale for selecting $2,500 as the figure above which consulting contracts
must be reported. Dr. Nugent replied that all contracts must be reported, but if the contract exceeds
$2,500, then the box must be checked. Dr. Chameides commented that the purpose of providing this
information is to give the Agency a general idea that there may be a conflict.
Dr. Nugent stated that part 4 is a new section in which the special government employee declares his/her
research support and project funding. Part 6 of the form is the declaration of expert testimony. Dr.
Acosta wanted to know what was meant by expert testimony. Dr. Nugent replied that expert testimony
includes any deposition or testimony in a judicial context, or testimony to the Congress. She noted that it
only includes testimony for which the individual was compensated.
Part 3 of the form also is a novel element dealing with non-compensated employment, excluding
religious, arts, social, fraternal, or political entities or those solely of an honorary nature. For example, a
scientist working for "Friends of the Earth" or the Chairperson of an NAS committee would be included
in part 3. There was a debate as to whether membership on an NRC/NAS committee should be included.
Dr. Nugent said that she would check with the lawyers, but that the idea is to report leadership roles, not
membership roles. Dr. Chameides stated that he planned to enter "various NRC activities" on the form.
Dr. Zimmerman asked if professional societies were included in the non-profit category. Dr. Nugent
replied that professional societies are included as non-profits. She added that these forms are to be
completed annually and before each new panel activity (i.e., each time there is a change in subject
matter).
Dr. Nugent stated that part 9 is a new section that allows the individual to provide a personal narrative.
Part 9 is meant for the special government employee to report anything that may be a conflict of interest
with regard to the committee's activities.
Dr. Nugent pointed out that people's lives are very complex so it was difficult to engineer a universal
form. The lawyers and the SAB staff thought that this new form was a good effort to focus on categories
of information the individuals need to divulge.
September 23-24, 2002 BOSC Executive Committee Meeting Summary 11
-------
Dr. Windom mentioned that he recently served on a panel; before the meeting began, the panelists were
asked to discuss potential conflicts of interest. He thought this was a good mechanism to give individuals
an opportunity to air concerns. Dr. Schnoor said that the new forms must be completed by each BOSC
member and submitted to Ms. Hamilton by October 31, 2002. There was some discussion about how to
save the completed form. Dr. Nugent said that it could be saved using enhanced Adobe Acrobat software;
however, it cannot be saved if the individual is using the free downloadable version of Acrobat. Dr.
Nugent also asked that comments about the new form be submitted to Ms. Hamilton. She noted that SAB
is evaluating the form and soliciting feedback that will be used to revise the form.
Dr. Acosta asked if it was necessary to report the specific stocks that are included in a mutual fund. Dr.
Nugent replied that only the stocks of diversified mutual funds need to be reported. She added that the
name-of the fund must be reported, not the family of funds. Dr. Schnoor thanked Dr. Nugent for her
presentation and clarifications concerning the new form.
Communications Ad Hoc Subcommittee Progress Report
i
Dr. Ann Bustrom provided an update on the progress of the Communications Ad Hoc Subcommittee.
The Subcommittee met informally in Seattle, WA, and the members have participated in one conference
call. Most of the Subcommittee communications have been via e-mail. Dr. Bostrom questioned whether
the Subcommittee should try to reschedule the NCER site visit. She agreed, however, to talk to Dr.
Oilman before solidifying the action plan for the Subcommittee.
Dr. Bostrom noted that the self-study questions distributed to each of the five Laboratories and Centers
included the following question: How does the Laboratory/Center communicate its results within the
organization, within ORD, within EPA, to outside agencies, and to the outside world? This question was
posed to elicit a general overview of the state of communication of research results in the Laboratories
and Centers, and to guide the BOSC Communications Ad Hoc Subcommittee in its selection of cases and
questions for further study. Dr. Bostrom stated that the findings and recommendations from the
Laboratory and Center reviews, from responses to this and other questions, suggest that:
v' Communicating research results is an (often self-identified) area of importance and desired
improvement for the Laboratories and Centers.
"v" The Laboratories and Centers have not formally identified, characterized, or prioritized the audiences
for their research results.
^ Ongoing documentation and assessment of the quantity and quality of research results
communications, covering a range of communications processes and products, are lacking.
V Passive information provision (e.g., Web pages anu journal pu'ulk-aiama) is ccuual to ciuiciit efforts to
communicate research results.
"v1 Several, if not all, of the Laboratories and Centers have insufficient communications expertise on their
staff to improve communication of research results.
v" There are specific cases of good communications practices that could be useful for the Laboratories
and Centers to share.
Letters were drafted to Dr. Peter Preuss at NCER, Dr. George Alapas at NCEA, Dr. Dan Costa at the PM
Program, and Dr. Larry Reiter at NHEERL. These letters contain the self-study questions for the review.
Dr. Bostrom indicated that the next steps for the Subcommittee are to: (1) complete the report on the
September 23-24, 2002 BOSC Executive Committee Meeting Summary 12
-------
relevant findings of the Laboratory/Center site visits, (2) discuss the Subcommittee's planned activities
with Dr. Oilman, (3) send the letters and self-study questions to the Laboratories and Centers, (4)
schedule site visits for January-February 2003, and (.§)^draft a Subcommittee report by April and
distribute it to the BOSC members before the May meeting.
Dr. Schnoor said that there were excellent points in the initial draft report and there was a lot of good
information with which to move forward. Dr. Clark asked what needed to be done to the report before it
is sent to Dr. Oilman in October. Dr. Bostrom replied that she did not think the Subcommittee should
send a report to Dr. Oilman; she suggested that the Subcommittee send him background information on
the Subcommittee's plans and efforts. Dr. Windom wondered if the real problem is communication of the
research plan, rather than communication of the research results. How does the Laboratory/Center ensure
that the right research is being done, especially in the multiyear plans? How is communication with the
stakeholders accomplished, so that the research they need is being done? Dr. Bostrom commented that,
although these are interesting questions, they are beyond the scope of this Subcommittee.
Dr. Zimmerman noted that the communication objectives need to be spelled out; she noted that the
question regarding how the Laboratories and Centers communicate is a very difficult one to answer. She
wondered if there was a mechanism for inter-Laboratory communication or exchange, or a way to map e-
mail communications. Dr. Bostrom replied that there are e-mail-mapping studies, but she noted that such
an undertaking is beyond the scope of this Subcommittee. Dr. Daston followed up on Dr. Windom's
question by asking if the ORD has a mechanism to inventory research projects. This may be a useful tool
to understand what projects are being done. Dr. Schnoor replied that there is a catalog of ORD research
projects. ORD does a good job of tracking its projects, but not tracking communication.
Mike Moore (EPA/ORD), Communications Director for ORD, pointed out that Dr. Oilman is very
interested in communications, and his focus extends beyond ORD and encompasses the entire Agency.
Mr. Moore also said that his office has a draft strategic plan, modeled in large part on the NCERQA
Communications Plan. Mr. Moore said that he would share that draft with the Subcommittee. Dr.
Schnoor asked if Mr. Moore also had a list of all ORD publications. Mr. Moore responded that a
scientific inventory is being developed for all of EPA, and everyone has been asked to contribute to that
inventory. Furthermore, his office has established a communication focal point in each
Laboratory/Center that communicates by telephone once each week. Dr. Bostrom asked if she could have
the list of communication contacts, and Mr. Moore agreed to provide that list, as well as additional
information.
Dr. Schnoor asked the BOSC members if they believed that there was enough information for Dr.
Bostrom to complete a letter report for submission to Dr. Oilman by October. Dr. Johnson said that the
Subcommittee should review the letter report to ensure that it is factually correct. Dr. Bostrom asked Dr.
Johnson to clarify his comment. Dr. Johnson said that the Subcommittee should ensure that the letter
report does not recommend that ORD undertake actions that currently are being done or overlook some
program that is meeting an identified need. Dr. Schnoor said that the Subcommittee should meet (face-to-
face or by conference call) before the teleconference with Dr. Oilman. During this meeting, the
Subcommittee members should be asked to identify and approve the preliminary findings to be included
in the report. Dr. Zimmerman added that the report should define the types of communication that are
being reviewed. Dr. Bostrom commented that the report addresses all types of communication; however,
a handful of case studies that use different types of communication are being selected for more detailed
examination.
Dr. Bostrom asked if the self-study questions should be sent out now, or if the distribution should be
postponed until after the conference call with Dr. Oilman. Dr. Acosta thought that the Subcommittee
should solicit Dr. Oilman's input before the self-study questions are distributed. Dr. Schnoor stated the a
September 23-24, 2002 BOSC Executive Committee Meeting Summary 13
-------
report of the Communication Ad Hoc Subcommittee's activities will be on the agenda for the January
BOSC meeting.
.,'. .
Dr. Schnoor reminded Dr. Bostrom that the focus of the communications review is not limited to NCER.
He stated that NCER is being singled out and criticized even though the Center was proactive in
developing a communications plan. The BOSC members agreed that the Subcommittee should
reschedule a site visit with NCER. Dr. Bostrom summarized the next steps for the Subcommittee
identified during the discussion: the Subcommittee will meet (face-to-face or via conference call) to
discuss the preliminary findings and to approve the draft letter report, and a conference call will be
scheduled with Dr. Oilman. It was agreed that the findings of the Subcommittee should be shared with
the BOSC before they are presented to Dr. Oilman. Moreover, the self-study questions should not be sent
out until Dr. Oilman has been consulted about the Subcommittee's plans. Everyone agreed that the
Subcommittee needs a strong vision from Dr. Oilman before proceeding with the review.
Mercury MMP Discussion
Dr. Schnoor stated that Norine Noonan, Henry Longest, and now Paul Oilman have requested that the
BOSC review some or all of the Multi-Year Plans (MYPs. Sixteen MYPs have been drafted, and two
have been formally reviewed by the SAB the Pollution Prevention MYP and the Drinking Water MYP.
Three more MYPs are targeted for review by the SAB Air Toxics, Particulate Matter, and Contaminated
Sites. The Mercury MYP would be the sixth MYP scheduled for review. Dr. Schnoor noted that review
of the MYPs is an important part of the BOSC's FY2003 duties. He thought the members should discuss
how the BOSC is going to handle the MYP reviews. He mentioned that the Board members may want to
meet with Kevin Teichman, Office of Science Policy, as he is interested in the BOSC review of the
MYPs.
Dr. Windom mentioned that he did not understand how the writers of the Mercury MYP arrived at the
scientific questions listed on page 4 of the plan. Dr. Schnoor responded that those issues should be
resolved in the actual review process. He asked the BOSC to focus on the charge regarding the MYPs. Is
it doable? Dr. Daston said that he was not sure if there is enough information in the MYP to address that
question. Dr. Zimmerman wanted to see a more direct charge statement and wondered if the MYP
included stakeholder involvement issues. Who has input into the plan? Dr. Zimmerman also wanted to
know how the authors prioritized the issues. Dr. Chameides asked about the meaning of question 5. Dr.
Schnoor replied that the question concerns whether they are doing research that will accommodate the
future. Are they looking ahead? Dr. Schnoor asked the BOSC members if they thought there was enough
information to review this MYP.
Dr. Bostrom said that much of the data was out of date. For example, on page 5, none of the references
are more recent than 2000. Is there a mechanism to update the documents? Dr. Schnoor suggested that
GSF update the MY? before the COSC reviews it. Dr. Charncides noted that the discussion cf science in
the MYP is weak. If the document is meant for scientists, there should be more background material
provided. He believes the BOSC should review the Mercury MYP, but agrees that OSP needs to address
these deficiencies before that review. Dr. Schnoor added that the Drinking Water MYP was more
detailed and much more impressive than the Mercury MYP.
Dr. Acosta pointed out that the MYP indicated (on page 2) that it was reviewed by the Executive Council.
Is the BOSC duplicating those efforts? Dr. Schnoor replied that the BUSC review would constitute an
questions regarding the Mercury MYP, Dr. Schnoor asked if the discussion of this MYP should be tabled
until a representative from ORD can be present to address the BOSC's concerns and to answer any
questions. It was agreed that the discussion should be postponed. Dr. Schnoor agreed to provide the
BOSC's preliminary comments about the Mercury MYP to the OSP. He will ask OSP also to provide
September 23-24, 2002 BOSC Executive Committee Meeting Summary 14
-------
information on who had input into this MYP and how it was developed, updated information, more
background material, and results from the previous review by the Executive Council.
*'"~'.
Dr. Chameides asked if Dr. Oilman chose this particular MYP for BOSC review. Ms. Hamilton
responded that the BOSC asked to see an example of an MYP; the Mercury MYP was the next plan ready
for review. Ms. Hamilton added that discussion of the MYP was premature given that Dr. Herman Gibb
was unable to attend the meeting and present the MYP to the BOSC. She suggested that the BOSC
postpone discussing the MYP until there is a presentation that will provide the Board a better
understanding of the MYP. Dr. Daston agreed that the BOSC should table the discussion for now,
however; he noted that it would save considerable time if the BOSC's concerns about the plan were
shared with Dr. Gibb prior to the next meeting. Dr. Bostrom mentioned that if the BOSC is to review this
MYP, the Board should identify several mercury experts to add to the review committee. Ms. Hamilton
encouraged Dr. Schnoor to call Dr. Gibb and share with him the BOSC's concerns about the Mercury
MYP.
Wrap-Up
Dr. Schnoor asked the members if there were any additional items for discussion. Dr. Acosta asked about
the term of each Board member. Ms. Hamilton replied that letters are being sent to each Board member
stating the length of his/her term. Dr. Bostrom asked about other projects to be undertaken by the BOSC
in 2003. Dr. Schnoor replied that the BOSC would be reviewing several MYPs. Both Drs. Acosta and
Chameides mentioned that they would be interested in working on bioterrorism issues. Ms. Hamilton
suggested that Dr. Schnoor discuss that possibility with Dr. Gilman. Dr. Schnoor asked for a motion to
adjourn the meeting. Dr. Chameides moved to adjourn, and Dr. Windom seconded the motion. Dr.
Schnoor adjourned the meeting at 1:45 p.m.
Action Items
The following action items were identified during the meeting discussion:
v- Ms. Beverly Campbell agreed to revise and finalize the Measures of Success Letter Report and send it
to Dr. Schnoor.
& Ms. Campbell will revise the May meeting minutes in accordance with the comments of the
September meeting and send them to the BOSC Chair and the DFO.
v* Ms. Campbell will finalize the NCER Report and send it to the NCER Subcommittee Chair for final
approval.
4- Drs. Schnoor, Bostrom, and Chaimedes will send changes to the NCEA Report to Dr. Juarine Stewart
and Ms. Campbell so that the report can be finalized. Dr. Stewart will determine how best to
incorporate the review comments.
$ Ms. Hamilton agreed to investigate the possibility of creating a BOSC Web Site.
^ The BOSC members will provide feedback to Dr. Reiter regarding the biotechnology and
computational toxicology programs presented at the September meeting.
v" Dr. Daston agreed to share his genomics data with EPA.
September 23-24. 2002 BOSC Executive Committee Meeting Summary 15
-------
Dr. Schnoor asked that all BOSC members complete the new conflict of interest and financial
disclosure forms and send them to Ms. Hamilton byOctober.31, 2002.
!&' '-'"""
Dr. Bostrom will schedule a meeting or conference call for the Communications Ad Hoc
Subcommittee. She also will schedule a conference call with Dr. Oilman to discuss future plans for
the Subcommittee. In addition, she will reschedule the NCER site visit.
Mike Moore agreed to provide copies of ORD's draft strategic communications plan with the
Communications Ad Hoc Subcommittee. He also will provide Dr. Bostrom a list of the
communications contacts in the Laboratories/Centers.
Dr. Bostrom will share the findings of the Communications Subcommittee Letter Report with the
BOSC prior to communicating them to Dr. Oilman on the conference call.
Dr. Schnoor agreed to communicate the BOSC's concerns about the Mercury MYP to Dr. Gibb
before the Mercury MYP is presented to the BOSC.
Dr. Schnoor agreed to ask Dr. Oilman if the BOSC's input on bioterrorism issues would be helpful.
September 23-24, 2002 BOSC Executive Committee Meeting Summary 16
-------
Board of Scientific Counselors Executive Committee
Chair:
Jerald L. Schnoor, Ph.D.
Professor, Department of Civil &
Environmental Engineering
The University of Iowa
4112 Seaman's Center (SC)
Iowa City, IA 52242
Phone: 319-335-5649
Fax: 319-335-5660
E-mail: jerald-schnoor@uiowa.edu
Members:
Daniel Acosta, Jr., Ph.D.
Dean, College of Pharmacy
University of Cincinnati
3223 Eden Avenue
Room 136HPB
Cincinnati, OH 45267-0004
Phone: 513-558-3326
Fax: 513-558-4372
E-mail: daniel.acosta@uc.edu
Ann Bostrom, Ph.D.
Associate Professor
Georgia Institute of Technology
School of Public Policy
685 Cherry Street
Atlanta, GA 30332-0345
Phone: 404-894-3196 (receptionist)
Fax: 404-894-0535
E-mail: ann.bostrom@pubpolicy.gatech.edu
James S. Bus, Ph.D.
Director of External Technology
Toxicology & Environmental Research
& Consulting
The Dow Chemical Company
1803 Building
Midland, MI 48674
Phone: 989-636-4557
Fax: 989-638-9863
E-mail: jbus@dow.com
William L. Chameides, Ph.D.
Smithgall Chair and Regents Professor
School of Earth and Atmospheric Sciences
Georgia Institute of Technology
221 Bobby Dodd Way, OCE Building
Atlanta, GA 30332-0340
Temporary Phone: 404-385-1548
Phone: 404-894-1749
Fax: 404-894-5638
E-mail: wcham@eas.gatech.edu
James R. Clark, Ph.D.
Exxon Mobil Research & Engineering Co.
3225 Gallows Road, Room 3A412
Fairfax, VA 22037
Phone: 703-846-3565
Fax: 703-846-6001
E-mail: jim.r.clark@exxonmobil.com
Michael T. Clegg, Ph.D.
Distinguished Professor of Genetics
Department of Botany & Plant Sciences
University of California
900 University Avenue
2132 Bachelor Hall
Riverside, CA 92521
Phone: 909-787-4672
Fax: 909-787-4437
E-mail: michael.clegg@ucr.edu
George P. Daston, Ph.D.
Miami Valley Laboratories
The Proctor & Gamble Company
11810 E. Miami River Road
Cincinnati, OH 45252
Phone: 513-627-2886
Fax: 513-627-0323
E-mail: daston.gp@pg.com
September 23-24, 2002 BOSC Executive Committee Meeting Summary
17
-------
Board of Scientific Counselors Executive Committee (Continued)
Elaine J. Dorward-King, Ph.D.
Global Executive, Environment, Health
and Safety
Rio Tinto pic
6 St. James's Square
London SW1Y4LD
United Kingdom
Phone: 44 (0)20 7753 2183
Fax: 44 (0)7753 2413
E-mail: elaine.dorward-king@riotinto.com
-or-
3006 Creek Road
Park City, UT 84098
Anna K. Harding, Ph.D., R.S.
Chair and Associate Professor
Department of Public Health
25 6 Waldo Hall
Oregon State University
Corvallis, OR 97331-6406
Phone:'541-737-3825
Fax: 541-737-4001
E-mail: Anna.Harding@orst.edu
Rogene F. Henderson, Ph.D., DABT
Deputy Director
National Environmental Respiratory Center
Lovelace Respiratory Research Institute
2425 Ridgecrest Drive, S.E.
Albuquerque, NM 87108
Phone: 505-348-9464
Fax: 505-348-4983
E-mail: rhenders@lrri.org
James H. Johnson, Jr., Ph.D.
Dean, College of Engineering, Architecture,
and Computer Sciences
n«,_j T r_;;,,_;<-,.
WVVCUU l_>Ili V Wibn-jr
2366 Sixth Street, NW, Room 100
Washington, DC 20059
Phone: 202-806-6565
Fax: 202-462-1810
E-mail: jj@scs.howard.edu
Juarine Stewart, Ph.D.
Professor
Department of Biological Sciences
Clark Atlanta University
223 James P. Brawley Drive, SW
Atlanta, GA 30314
Phone: 404-880-6764
Fax: 404-880-6756
E-mail: jstewart@cau.edu
Herbert L. Windom, Ph.D.
Professor Emeritus
Skidaway Institute of Oceanography
10 Ocean Science Circle
Savannah, GA 31411
Phone: 912-598-2490
Fax: 912-598-2310 .
E-mail: herb@skio.peachnet.edu
Rae Zimmerman, Ph.D.
Professor of Planning and Public Admin.
New York University
Robert F. Wagner Graduate School
of Public Service
4 Washington Square North
New York, NY 10003
Phone: 212-998-7432
Fax: 212-995-3890
E-mail: rae.zimmerman@nyu.edu
Committee Staff:
Peter W. Preuss, Ph.D.
ORD BOSC Liaison
U.S. Environmental Protection Agency
Office of Research and Development
National Center for Environmental Research
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: 202-564-6825
Fax: 202-565-2444
E-mail: preuss.peter@epa.gov
September'23-24, 2002 BOSC Executive Committee Meeting Summary
IS
-------
Board of Scientific Counselors Executive Committee (Continued)
Shirley R. Hamilton
Designated Federal Officer
U.S. Environmental Protection Agency
Office of Research and Development
National Center for Environmental Research
Ariel Rios Building, 8701R
'1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: 202-564-6853
Fax: 202-565-2444
E-mail: hamilton.shirley@epamail.epa.gov
Betty J. Overton
Alternate Designated Federal Officer
U.S. Environmental Protection Agency
Office of Research and Development
National Center for Environmental Research
Ariel Rios Building, 8701R
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: 202-564-6848
Fax: 202-565-2444
E-mail: overton.betty@epa.gov
September 23-24, 2002 BOSC Executive Committee Meeting Summary
19
-------
Additional Meeting Participants
Janet Anderson
U.S. Environmental Protection Agency
Ariel Rios Building, 7511C
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: 703-308-8712
E-mail: anderson.janet@epa.gov
Mark Bagley
U.S. Environmental Protection Agency
USEPA Facilities, 642
26 West Martin Luther King Drive
Cincinnatti, OH 45268
Phone: 513-569-7455
E-mail: bagley.mark@epa.gov
Ms. Beverly Campbell
The Scientific Consulting Group, Inc.
656 Quince Orchard Road, Suite 210
Gaithersburg, MD 20878
Phone: 301-670-4990
Fax: 301-670-3815
E-mail: bcampbell@scgcorp.com
Richard David
U.S. Environmental Protection Agency
Ariel Rios Building, 8101R
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: 202-564-3376
E-mail: David.Richard@epa.gov
Robert Flaak, Ph.D.
The Scientific Advisory Board
U.S. Environmental Protection Agency
National Center for Environmental Research
(1400 A)
Ariel Rios Building
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: 202-564-4546
Fax: 202-565-2444
E-mail: flaak.robert@epa.gov
Jack Fowle
U.S. Environmental Protection Agency
Mailroom, B305-02
Research Triangle Park, NC 27711
Phone: 919-541-3884
E-mail: fowle.jack@epa.gov
Gary Foley, Ph.D.
U.S. Environmental Protection Agency
Mailroom, D305-02
Research Triangle Park, NC 27711
Phone: 919-541-2106
E-mail: foley.gary@epa.gov
John Glaser, Ph.D.
Director, National Exposure Research
Laboratory
U.S. Environmental Protection Agency
USEPA Facilities, 642
26 West Martin Luther King Drive
Cincinnatti, OH 45268
Phone: 513-569-7568
E-mail: glaser.john@epa.gov
Debora Hamernik
U.S. Department of Agriculture
Cooperative State Research, Education, and
Extension Service (CSREES)
Phone: 202-401-4202
E-mail: dhamernik@reeusda.gov
Gary Kayajanian
Consultant
Phone: 703-920-0623
Dennis D. Kopp
USDA/CREES
Stephanie Loranger, Ph.D.
The Scientific Consulting Group, Inc.
656 Quince Orchard Road, Suite 210
Gaithersburg, MD 20878
Phone: 301-670-4990
Fax: 301-670-3815
E.-mail: sloranger@scgcorp.com
September 23-24, 2002 BOSC Executive Committee Meeting Summary
20
-------
Additional Meeting Participants (Continued)
Ann Miracle
U.S. Environmental Protection Agency
USEPA Facilities, 642
26 West Martin Luther King Drive
Cincinnatti, OH 45268
Phone: 513-569-7289
E-mail: miracle.ann@EPA.gov
Angela Nugent, Ph.D.
U.S. Environmental Protection Agency
Science Advisory Board, 1400 A
Ariel Rios Building
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: 202-564-4562
Fax: 202-565-2444
E-mail: nugent.angela@epa.gov
Marshall Plaut, M.D.
Chief, Allergic Mechanisms Section
Division of Allergy, Immunology and
Transplantation
National Institutes of Allergy and
Infectious Disease
National Institutes of Health
6700-B Rockledge Drive, Room 5146
Bethesda, MD 20892
Phone: 301-496-8973
Fax: 301-402-0175
E-mail: mplaut@niaid.nih.gov
Larry Reiter, Ph.D.
Director, National Health and
Environmental Effects Research
Laboratory, B305-01
U.S. Environmental Protection Agency
Research Triangle Park, NC 27711
Phone: 919-541-2281
E-mail: reiter.larry@epa.gov
Gregory Toth
U.S. Environmental Protection Agency
USEPA Facilities, 642
26 West Martin Luther King Drive
Cincinnati, OH; 45268
Phone: 513-569-7242
E-mail: toth.Greg@epa.gov
Estella Waldman
U.S. Environmental Protection Agency
National Center for Environmental Research
Ariel Rios Building, 8722R
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Phone: 202-564-6836
E-mail: waldman.estella@epa.gov
Ms. Susie Warner
The Scientific Consulting Group, Inc.
656 Quince Orchard Road, Suite 210
Gaithersburg, MD 20878
Phone: 301-670-4990
Fax: 301-670-3815
E-mail: swarner@scgcorp.com
September 23-24, 2002 BOSC Executive Committee Meeting Summary
21
-------
APPENDIX B:
Proceedings of the
Communicating Research Results: Best Practices Workshop
BOSC Ad Hoc Subcommittee on Communications
-------
Proceedings of the
Communicating Research Results: Best Practices Workshop
BOSC Communications Ad Hoc Subcommittee
Washington, DC
May 15, 2003
CommunicationsAn Agency Priority
At the first U.S. Environmental Protection
Agency (EPA) Science Forum held in May 2002,
EPA Administrator Christie Todd Whitman
stressed the importance of basing Agency deci-
sions on quality science. She stated that "it is
absolutely essential that EPA managers have the
best possible scientific and economic information
to consider when making decisions. Only a strong
commitment to science can define the environ-
mental challenges of the future and determine the
best methods to address these challenges." The
Administrator went on to say that "to make deci-
sions based on sound science, policymakers need
information that reflects the latest findings in
high-quality research and analyses, usually span-
ning a variety of scientific disciplines. This infor-
mation must be presented in a form that
non-scientists, or even the EPA Administrator,
can understand and use correctly. Communicat-
ing the results of our work in a clear manner will
lead to a better understanding of environmental
risks and how best to manage those risks. As
citizens become better acquainted with the scien-
tific basis for EPA's actions, they can make more
informed decisions concerning the environment,
their health, and the health of their families."
One of the most difficult challenges faced by
EPA, as well as many other organizations, is how
to ensure that its research results are communi-
cated effectively to those who need them at the
time they need them. In 2001, EPA's Board of
Scientific Counselors' was asked by the Assistant
Administrator for Research and Development to
examine how the Office of Research and Devel-
opment (ORD) research results are communi-
cated, both within and outside the Agency, and
how they might be more effectively communi-
cated. To conduct this review, the BOSC formed
the Communications Ad Hoc Subcommittee,
which was chaired by Dr. Ann Bostrom, Associ-
ate Professor at the Georgia Institute of Technol-
ogy. This five-member Subcommittee (see
Attachment 1), which included representatives
from both academia and industry, met informally
on December 2, 2001, to discuss how to approach
its task of assisting ORD in improving its commu-
nication of research results.
Based on the results of this meeting and subse-
quent conference calls and e-mail communica-
tions, the Subcommittee decided to employ an
approach used successfully by previous BOSC
subcommitteesto distribute a list of self-study
questions to the ORD Laboratories and Centers.
The Laboratories and Centers were asked to sub-
mit written responses to two general communica-
tions questions as well as 10 questions regarding
one or two communications innovations. The
Subcommittee also elected to review the Labora-
tory and Center responses to a communications
question that was posed as part of the BOSC's
second review of ORD's Laboratories and Cen-
ters. This question focused on how the Labora-
tory/Center communicates its results within its
organization, within ORD, within EPA, to outside
agencies, and to the outside world. In addition,
the Subcommittee members decided to hold a
workshop in conjunction with the May BOSC
meeting to discuss best communications practices
within the ORD Laboratories and Centers as well
as best practices in other organizations.
The BOSC was established by EPA to provide advice,
information, and recommendations about the ORD research
program. For more information about the Board see the BOSC
Web site at http://www.epa.gov/ edrlupvx/bosc/.
Communicating Research Results: Best Practices Workshop
-------
Findings from the Laboratory/Center Review
The Communications Ad Hoc Subcommittee
members reviewed the responses submitted by the
Laboratories and Centers regarding the communi-
cations question included in the 2002 Labora-
tory/Center review conducted by the BOSC.
Although this review did not focus on
communications, it included one question to
solicit input for the communications review.
After analyzing the responses to these questions,
the Communications Ad Hoc Subcommittee de-
veloped the following findings and recommenda-
tions regarding ORD's communication of
research results:
v" Communicating research results is an (often
self-identified) area of importance and desired
improvement for the Laboratories and Cen-
ters.
"v- The Laboratories and Centers have not for-
mally identified, characterized, or prioritized
the audiences for their research results.
"v- Ongoing documentation and assessment of
the quantity and quality of research results
communications, covering a range of commu-
nications as well as processes and products,
are lacking.
v- Passive information provision (e.g., Web
pages and journal publications) is central to
the current efforts to communicate research
results.
v* Several of the Laboratories and Centers have
insufficient communications expertise on
their staffs to improve their communication of
research results.
v" There are specific cases of good communica-
tions practices that could be useful for the
Laboratories and Centers to share.
The Subcommittee used the responses to the com-
munications question to guide the selection of
innovations for further review. Representatives
from each of the tive Laboratories and Centers
were invited to the workshop to present their
responses to this question and describe these com-
munications innovations. Dr. Bostrom thanked
Mike Moore (EPA/ORD) and Michael Brown
(EPA/ORD) for their assistance in planning and
organizing this workshop. She noted that a major
goal of the workshop is to share best communica-
tion practices within ORD and outside the Agency
and to identify opportunities for ORD to improve
the effectiveness of its efforts to communicate
research results.
Communicating Research Results: Best
Practices Workshop
Representatives from EPA's five ORD Laborato-
ries/Centers, Office of Science Policy, Particulate
Matter Research Program, and Office of Air and
Radiation attended the workshop to describe com-
munications innovations that exemplified the best
communications practices within their respective
organizations. In addition, representatives from
the Centers for Disease Control and Prevention,
Health Effects Institute, CUT Centers for Health
Research, National Institute of Environmental
Health Sciences, and American Lung Association
were invited to the workshop to present examples
of their exemplary communications practices.
This 1-day workshop was held on May 15, 2003,
in Washington, DC, and a summary of the presen-
tations follows.
ORD's National Center for Environmental
Research
Jack Puzak, Acting Director of the National Cen-
ter for Environmental Research (NCER), identi-
fied the Center's primary communication tools
and elaborated on two of the more innovative
communicationsthe Center's Science to
Achieve Results (STAR) Regional Environmental
Research Seminars and the NCER Web site.
Other tools that were mentioned include the an-
nual |jiogrcss review workshops, press releases,
several lecture series, internal e-mails to EPA
staff, the NCER Warehouse, conference displays,
and various publications.
"Science in Our Region"The STAR Environ-
mental Science Seminar Series. A pilot STAR
Regional Environmental Science Seminar was
heid in Region i in Nuvcm'uci 2GG2. For tliis
seminar, NCER staff worked closely with the
Regional staff to ensure that the seminar covered
the research topic(s) of interest to the Region.
Communicating Research Results: Best Practices Workshop
-------
- -
6EPA
New England
. .. ., .
UStPAN«wEnglondond
US EMOfBc* o< lUMardiond Dnolopm*^
State and tribal representatives also were invited
to attend the pilot seminar. Six more STAR Re-
gional Environmental Research Seminars are
planned during the remainder of FY2003 (Dallas,
May 28; Chicago, June 17; Kansas City, August
13; Atlanta, September 10; San Francisco, Octo-
ber 8; Philadelphia, September/October). The
audiences for these seminars include Regional
staff and managers, state and tribal environmental
representatives, local media, industry, and univer-
sity professors and students.
The communication goals of the STAR Regional
Environmental Research Seminars are to:
(1) make staff and managers aware of the STAR
research being conducted at local universities,
(2) let Regional staff know that they can contact
grantees for information, (3) inform the Regional
staff about the NCER Web site and other grants
that they could use in their work, (4) provide re-
search information to state/tribal agencies,
(5) determine whether other Regions would like
to plan seminars, and (6) inform the public
through the media. Other communication tools
used to achieve these goals include a Regional
Web site, a brochure that is mailed to state agen-
cies and provided to Regional staff and managers,
e-mails that are sent to staff from the Region's
management, and notification sent to the local
media.
The success of the Region 1 seminar (and the
ones that followed in 2003) was measured by a
number of factors. More than 200 people attend-
ed the opening session, and more than 90 people
attended the scientific sessions. In addition, there
was high attendance by Regional staff and man-
agement as well as attendance from state agen-
cies, nonprofits, industry, universities, and other
federal agencies. The written feedback from par-
ticipants was overwhelmingly positive and verbal
feedback from the STAR grantees was positive.
Another indicator of the success of the Region 1
seminar is that six other Regions have expressed a
desire to hold similar seminars. The Region I
seminar's success was attributed to the follow-
ing:
"v- The Region asked for the information so the
audience was interested in the topics.
& Upper-level management in NCER and the
Region supported the seminar.
v- The Assistant Administrator for Research and
Development (AA/ORD) was supportive of
the seminar.
) The seminar was held in the Regional Office
for each access.
*v" The NCER and Regional staff responsible for
the seminar worked hard to make it happen.
Mr. Puzak identified a few suggestions for im-
proving NCER's Regional seminars. These in-
cluded: extending personal invitations to state
agencies, rather than just sending out mailings;
adding teleconference and simulcast options for
those with travel restrictions; increasing the atten-
dance of nonprofits, tribes, and academics in the
Region; and sending out followup questions to
participants to determine if and how the seminar
information was used.
Communicating Research Results: Best Practices Workshop
-------
NCER's Web Site. The NCER Web site conveys
information on applying for research grants, fel-
lowships, and contracts as well as research pro-
jects and their results. It was redeveloped in
FY2002 and now includes an Oracle database
with several new or improved features, including:
4- Easy access to NCER research abstracts and
progress and final reports.
O- Dynamically generated pages that are auto-
matically updated.
v" Improved and flexible search results reports
and tailored search functions
v* Links to results and investigator publications.
0- The ability to pass NCER data to other data-
bases such as the Environmental Information
Management System (EIMS), Science Inven-
tory, and Web Inventory.
In May 2003, the Oracle database contained 2,600
research project abstracts; 1,500 progress reports;
1,000 final reports; 16,300 investigator publica-
tion bibliographic citations; 4,800 journal article
bibliographic citations; and 500 full-text pdf jour-
nal articles.
More recent Web site advancements include the
projects by Region function, top-awarded institu-
tions function, highly cited researchers page,
EIMS drinking water portal, science pages, New
User Quick Guide, and home page research news
and events. The projects by Region function
allows Regions to compile grants and fellowships
by state, and permits advanced sorting to generate
reports by Region with state and institution sub-
sorts as well as project funding amounts and state
and Regional funding tallies, i he top-awarded
institutions function facilitates the generation of
reports for the institutions with the highest awards
based on either funding or number of grants. The
highly cited researchers page was compiled using
ISI's highly cited researchers list and it identifies
the most highly cited investigators funded by
NCER. The EIMS drinking water portal groups
ORD projects related to drinking water. The
Office of Water (OW) will use the data shared
through this portal for its upcoming DRINK data-
base. The science topics pages on the NCER
Web site show NCER goals, requests for applica-
tions, grant recipients, and results for multiple
years on a particular science topic. These pages
convey the breadth, direction, and success of
NCER research for a specific science topic.
Several indicators of success are used by NCER
to determine the effectiveness of its Web site.
The Web statistics indicate high usage of the site
(i.e., about 40,000 users and more than 500,000
hits per month). The number of list serve sub-
scribers has increased by 34 percent since 1999,
and currently, there are about 11,000 list serve
subscribers. NCER also receives positive feed-
back on the Web site from Project Officers, the
ORD Web group, Regional scientists, and ORD
management.
The success of the NCER Web site was attributed
to its design, which meets the specific needs of
several user groups. Early and frequent commu-
nication was critical for identifying the initial
requiremerus and ii has been essential in defining
evolving needs. NCER's quick response to Re-
gional feedback concerning the Web site, resulted
in the development of tools to meet Regional
needs. Frequent communication with OEI has
helped improve database efficiency and search
times. Mr. Puzak noted, however, that the NCER
Web site could be improved by developing better
tools to assess user satisfaction and the use of
research results. NCER has prepared an online
user survey, which can be accessed at http://www.
epa.gov/ncer/draftsurvey, and plans to develop a
Communicating Research Results: Best Practices Workshop
-------
database to analyze survey feedback over time, as
well as a tool to analyze the e-mail inquiries re-
ceived via the Web site. NCER also could im-
prove the dissemination of results used to support
EPA rulemaking by developing a new list serve
for the Program Offices.
National Risk Management Research Laboratory
Dr. Hugh McKinnon, Director of the National
Risk Management Research Laboratory
(NRMRL), pointed out that communication is an
important part of the Laboratory's function.
NRMRL has three types of communications:
direct, written, and electronic. The direct commu-
nications serve many nontechnical customers.
Each year, NRMRL responds to approximately
3,000 telephone requests, operates a free public
video library of environmental subjects, distrib-
utes hundreds of EPA general audience publica-
tions, maintains a Public Affairs liaison with local
Congressional offices, and manages local media
relations. NRMRL also sponsors a variety of
scientific meetings. In 2002, the Laboratory
reached more than 100,000 people through its
exhibits.
With regard to written communications, NRMRL
offers 134 technology transfer documents for
distribution within and outside EPA. NRMRL's
research results are published in peer reviewed
journals (e.g., Environmental Science & Technol-
ogy, Journal of Environmental Engineering, At-
mospheric Environment, Ground Water Monitor-
ing and Remediation, and Remediation). From
1998 to 2002, NRMRL researchers published 61 1
articles in 271 refereed journals. Dr. McKinnon
mentioned the risk communication tool series and
several publications for a general audience.
As part of its electronic communications,
NRMRL creates in-house interactive multimedia
CDs for training, research, and workshops/confer-
ences. This innovative multimedia technology
presents and enhances information in a radically
new and engaging way. It incorporates digital
video, audio, 3-D animation, high-end graphics,
and peer reviewed content. NRMRL uses interac-
tive CD and DVD formats that can be linked or
converted to Web sites. Dr. McKinnon presented
several video excerpts from conferences and
training sessions.
National Rex. Management Research Laboratory's
Quality Management Plan QMP
Training Course
Effective Risk Management of
Endocrine Disrupting Chemicals
Workshop Newmedia CO
OH
Preparation of multimedia products involves a
five-step cycle: (1) create video, audio, 3-D ani-
mation, graphics, and image media; (2) create the
technical content; (3) design the interactivity
structure; (4) integrate media into interactive
software; (5) subject the product to peer review;
and (6) obtain the necessary clearance and publish
the product. NRMRL's virtual tour is an example
of this technology.
NRMRL is measuring its communications success
through a Customer Satisfaction Program. This
program provides a tool for measuring success
and input for strengthening future products with
regards to reaching the target audience, selecting
topics of interest, and effective delivery to the
intended audience.
Using the Pollution Prevention Workshop as an
example to illustrate how NRMRL seeks cus-
Communicating Research Results: Best Practices Workshop
-------
tomer feedback, Dr. McKinnon indicated that the
customer feedback regarding the workshop rated
the quality of the workshop as a 4.4 on a I to 5
scale. The participants also provided feedback
regarding the delivery methods (i.e., direct com-
munication, Internet postings, and guidance docu-
ments). NRMRL also uses DIALOG Science
Citation Statistics to measure success. The Labo-
ratory searches approximately 4,500 scientific and
technical journals for citations and uses those
statistics to track improvements over time.
NRMRL has identified some ways to improve its
communications efforts. There are plans for a
NRMRL Multimedia Laboratory, which is in-
tended to increase the use of digital technologies
to enhance communications. The Multimedia
Laboratory would be in a centralized location and
would provide cost-effective sharing of software
and hardware. NRMRL also has established the
Science Results Integration Program to improve
its communications. This program is intended to:
(1) integrate science results across ORD and de-
liver information without organizational bound-
aries, and (2) expand the use of ORD's research
and measure outcomes. Dr. McKinnon concluded
his presentation with NRMRL's communication
goal: "We get the right information in the right
format to the people who need it."
National Center for Environmental Assessment
Dr. Peter Preuss, Director of the National Center
for Environmental Assessment (NCEA), stated
that NCEA's focus is national-level assessments,
risk assessment guidance, risk assessment meth-
odology, and consultation and training. NCEA's
mission is to serve "as a national resource center
for the overall process of human health and eco-
logical risk assessments and the integration of
hazard, dose-response, and exposure data and
models to characterize risk." The Center's vision
is to be "a high performing assessment center
providing timely and high-quality risk information
to environmental decisionmakers." NCEA has
three divisions located in Washington, DC; Re-
search Triangle Park, NC; and Cincinnati, OH.
The purpose of NCEA's Human Health Assess-
ment Program is to develop contaminant-specific
risk assessments on chemicals/stressors that are of
high public concern, which then are used by EPA,
the states, and the international community.
Some of the contaminants addressed by NCEA
include diesel, dioxin, mercury, fuel and fuel
additives, paniculate matter and ozone, and
polychlorinated biphenyls (PCBs). The purpose
of NCEA's Ecological Risk Assessment Program
is to: (1) improve the science of ecological risk
assessment, (2) develop ecological risk assess-
ment approaches, and (3) integrate human dimen-
sions into ecological risk assessments. NCEA
also manages nationally recognized programs
such as the Integrated Risk Information System
(IRIS), Global Change Assessment, and Risk
Assessment Forum.
Some specific steps and products are necessary
for the "roll-out" of a major NCEA product: coor-
dination across EPA, with other federal agencies,
and the Executive Office of the President; audi-
ence identification; accurate information; clear
Communicating Research Results: Best Practices Workshop
-------
information; timely information; plain language
information; different types of information; out-
reach to stakeholders; and media outreach. The
first step in rolling out the Draft Final Guidelines
for Carcinogen Risk Assessment and the Supple-
mental Guidance for Assessing Cancer Suscepti-
bility From Early-Life Exposure to Carcinogens
(available online at http://www.epa.gov/ncea/raf/
cancer2003.htm), for example, is to develop a
comprehensive communication plan.
Description of Action 5 Expected Reactions from Stakeholders
Background . Detailed Communication Strategy
Key Messages ~ Announcement Notification Plan
Audience : Contacts
KPA'.l
Jrjfl ikojnvM fofnkj. -SuwlcflvalJ (nuibnfe (IT Aneiiiotf ('inca SunrrtUjhlHy fiinn IUrt
I'-jlctutifen»"
PHOJEtTEBANVH'M'KMEVrhATE: \LttOl '.
I. AUxU\«3C" jnj St-u-ntilk- S
nul (luiJamji; on Rj*t^ fnin l-jrl) -lifc IUp>nurc AL«. KMH-
1 INTACT: ItaMJ I
un. 2(12-.V»t-7K.M
iJ(>li'gii-s, I-I'A fa>
nviri ncnul pi'Ii
ifCMsi' ilto l-JW, ^
goiJv-JiiX-.lui
1'iJjj's Jral'i i
von llu *ohjcct of ptiNif ivi
any of UK » mum-nit and st
This plan describes the action (e.g., public com-
ment on the draft guidelines), background, key
messages, audience, expected reactions from
stakeholders, detailed communication strategy,
announcement notification plan, and contacts.
The next step is the preparation of a Federal Reg-
ister announcement to seek public comment on
the draft guidelines. For this example, NCEA
also developed fact sheets on the draft guidelines
and made them available on the Internet.
Of CM * CWUFT
IHCt UO> tXtOCU'
taM OflU'T lUVPiEUEHTU OUOUMCC FOX PU
6» n. MUM > raeg n^wu
rsa
NCEA also has developed program/issue-oriented
Web pages that are user-friendly, provide one-
stop shopping, focus on high-profile activities/
products, and involve cross-Agency coordination.
The Center has designed program/issue-oriented
Web pages for the Risk Assessment Forum, IRIS,
Global Change Research Program, and Dioxin
and Draft Reassessment; NCEA also was instru-
mental in the development of the MTBE (methyl
tertiary butyl ether) Home Page and assisted OW
in developing the Drinking Water Research
Tracking Portal (for Intranet use).
Wdcome to fe DUS bam page, brought to you by the VS.
Eovironmtotd PraiKtim Agency (EPA) and is Officejf
Research and Dtvtlopmea, SaBl£««tfeL!iasiB!saB!
A||e$smeni. DU3 u a database of birnaa heallb effects (bat may
result bum exposure to v^lui^ BtHlJlceh found to QK
eorirotmea. IRIS was initially developed for EPA staff in
response to a growing demaod for coosisteot information on
chemical substances for use brisk assessments, decision-making
and regulatory activities. Ibe information in IRIS a intended for
(hose wrJfaoat eHensive training in toxicology, but woo some
knowledge of fatal* sciences. For more information about IRIS.
readlnis Introduction.
For definitions of tenns in the IRIS Web rite, refer to da DUS
UstollRSSutslances
Communicating Research Results: Best Practices Workshop
-------
In concluding his presentation, Dr. Preuss listed a
number of NCEA's communication/outreach
innovations, which included: investing in com-
munications and outreach, building the staff, lis-
tening to stakeholders (internal and external),
identifying Agency needs, working with ORD
Laboratories/Centers and EPA Offices, and ad-
dressing BOSC recommendations.
National Exposure Research Laboratory
Dr. Gary Foley, Director of the National Exposure
Research Laboratory, described NERL's commu-
nication goals, the Laboratory's delivery and
feedback system, and two of NERL's specific
communication innovationsresearch abstracts
and scientist to scientist meetings.
NERL's communication goals are: (1) raising
awareness of NERL's relevant high-priority re-
search; (2) engaging all NERL staff in the com-
munication effort; (3) ensuring that all NERL
staff can articulate the work being done at the
Laboratory to a variety of audiences; (4) sharing
and showcasing NERL's research through the
right channels and at the right times; and (5) keep-
ing NERL staff aware of the communication
products being produced. Dr. Foley described
NERL's Delivery and Feedback System, which is
updated every year and includes a Task Informa-
tion System (TIS) that provides a production
tracking system with feedback loops to communi-
cate the research through publication in scientific
journals, posting on the Internet, and inclusion in
the Agency's Science Inventory.
NERL cfeffuecy & feedbacks/stem
One of NERL's communication innovations is its
research abstracts (RAs). For the past 4 years,
NERL has posted its RAs on the Web. These
abstracts are intended to highlight outstanding
research that would draw attention both within
and outside the Agency. They communicate in
"plain English" and link NERL science to regula-
tory, public health, and policy outcomes. These
RAs also are intended to encourage discussion of
other types of communication that may be helpful
to clients, reduce inquiries about how NERL's
research should be interpreted and communicated
to others, and simplify the annual process of re-
porting major accomplishments.
Nftioul Cipoccnt Research Laboratory
h AbttrilinKrn\in*ifncni>ltTuiuKintcnl itnicitui. Sfoiul an!
miof fine [unicuUtf n}incr(PM,--)andii(Nxnc:ur toxic
ooibnloraWj jL-n>i< urtuo arcai and undci di
iJdinuiKR|,-nwi.. tor t
vg in; fcstuws ub>ct\od on
ciothtUru
-------
one on the lessons learned at the World Trade
Center, and one on the use of genomics for mea-
suring endocrine disrupting compounds.
Scientist to Scientist Workshops
For the Biological Assessment and Criteria
(BAC) Workshop, which was co-sponsored by
EPA and the National Council of State Govern-
ments, NERL worked closely with OW to identify
experts and coordinate the workshop. It was at-
tended by 246 biologists from 47 states, 26 tribes,
and 1 territory. The workshop consisted of 5 days
of training (including 4 tracks with 18 courses),
informal meetings, and problem-solving sessions.
Dr. Foley identified a number of benefits to the
workshop participants, which included direct
technology transfer from EPA scientists, seeing
other states demonstrate the use of EPA tools, and
greater use and higher esteem of EPA science.
He also listed a number of benefits for NERL:
facilitation of the use of research methods, tools,
and models; direct communication and interaction
with end users of science; and fortification of
relationships with customers inside and outside
the Agency. NERL management recognizes that
continued improvement is important and should
include broader outreach to stakeholder clients
and more emphasis on integrating products across
ORD.
Dr. Foley stated that NERL's research is success-
ful only if it is completed on time, it is of high
quality, it is completed within cost, it is delivered
to the client, and the client is using it either di-
rectly or indirectly. However, he did not believe
that this is enough to define success. Success
occurs only when the client uses the research
results to make better environmental decisions
that have a positive impact on the environment.
Dr. Foley stressed the need to integrate products
across ORD and to do a better job of working
with clients earlier in the process and ensuring
that clients understand how to use the tools devel-
oped by NERL.
National Health and Environmental Effects
Research Laboratory
Steven Hedtke, Deputy Associate Director for
Ecology at the National Health and Environmen-
tal Effects Research Laboratory (NHEERL),
stated that NHEERL has nine Divisions and two
field stations in eight different geographic loca-
tions, which makes communications a challenge
even within the organization. Mr. Hedtke empha-
sized that communications should occur at all
levels, and that there is more to communicating
research than just developing and disseminating
products. NHEERL's communication goals are
to: (1) be accurate and innovative in communicat-
ing research results to a wide variety of audi-
ences, (2) provide the tools to NHEERL scientists
to assist them in communicating their research,
and (3) inform decisionmakers.
NHEERL's communications program practices
involve clearly defining the audience; using a
high-quality, audience-appropriate writing style;
and using modern, graphics-rich formats. In addi-
tion, NHEERL believes that obtaining feedback
from ORD scientists, management, and staff;
Program Offices and Regions (through scientists
and NHEERL ALDs); and the scientific commu-
nity and public regarding publications and re-
quests for information is very important to im-
prove future communications efforts.
NHEERL produces a variety of communication
products, including Coastal Communications,
journal articles, high profile reports, science re-
ports, and annual reports. Each of these products
has different objectives, target audiences, and
measures of success.
The objectives of NHEERL's Coastal Communi-
cations are to: (I) provide an alert to upcom-
ing/ongoing research in a region of interest, and
(2) improve recognition of ORD's role in coastal
Communicating Research Results: Best Practices Workshop
-------
Coastal Communic
EM Co.M.1 Communication.
.«.._
research. The target audience for the Coastal
Communications includes scientists and managers
within coastal communities. Two of the measures
of success for this product are the receipt of re-
quests to be added to the distribution list and the
positive feedback from users.
Results of NHEERL research often are reported
in journal articles. The objectives of publishing
in peer reviewed journals are to: (1) report on
scientific advances discovered by NHEERL re-
searchers, and (2) build the scientific credibility
of Laboratory staff. The target audience for these
articles is the scientific community. The mea-
sures of success used by NHEERL are the approx-
imately 260 articles published in journals each
year and the receipt of awards such as EPA Honor
Awards, Science and Technological Achievement
Awards, and awards from professional societies.
NHEERL research results are sometimes commu-
nicated in high profile reports. These reports are
intended to rapidly release scientific findings on
particularly "hot" topics, and they target decision-
makers, the scientific community, and the public.
Measures of success for these reports include
feedback from Program Offices and Regions, the
number of copies requested from the EPA Ware-
house, and the number and type of requests re-
ceived by ORD and the NHEERL communica-
tions team.
Science reports are produced by NHEERL to
report on scientific advances and they target
Agency staff and the scientific community. Mea-
sures of success for the science reports include
feedback from NHEERL and other EPA scien-
tists, as well as the Program Offices and Regions.
NHEERL prepares an annual report each year that
makes research highlights available to a wide
audience, including government agencies, Con-
gress, the scientific community, and the interested
public. Measures of success for the most recent
annual report include: requests received by the
EPA Warehouse for 1,900 publications; numerous
requests from NHEERL ALDs, Program and
Regional Offices, and the public; the NHEERL
annual report served as a model for the ORD
10
Communicating Research Results: Best Practices Workshop
-------
Accomplishments Report (Spring 2002); and
receipt of an award of merit for design excellence
from the Society for Technical Communications.
Mr. Hedtke acknowledged that one of NHEERL's
biggest challenges is measuring the success of its
communications efforts.
Office of Science Policy
Dr. Kevin Teichman, Director of ORD's Office of
Science Policy (OSP), explained that OSP per-
forms three major roles within ORD: science
integration, science coordination, and science
communication. To integrate science, OSP devel-
ops unified ORD positions on the use of science,
manages the ORD research planning process, and
coordinates the implementation of Agency sci-
ence policies. To coordinate science, OSP en-
sures that ORD's research addresses EPA's prior-
ities, brings together ORD and the 10 EPA
Regions, and manages ORD's program support
function. To communicate science, OSP repre-
sents ORD to tribes, states, and local communi-
ties, and sponsors colloquia, workshops, and sci-
entific meetings. OSP's primary target audience
is EPA's Program and Regional Offices; its sec-
ondary target audiences include state and local
governments, tribes, the environmental justice
community, other federal agencies, professional
societies and associations, and the general public.
OSP reaches its audiences for science integration
through progress reviews; meetings with Program
and Regional senior management; and the Pro-
gram Support Priorities List. OSP reaches its
audience for science coordination through the
Research Coordination Teams (RCTs); scientist
to scientist meetings; Tribal Science Council;
National Environmental Justice Advisory Coun-
cil; and workshops, symposia, and colloquia.
OSP reaches its audiences for science communi-
cation through the ORD Accomplishments Re-
port, OSP Quarterly Report, and Web sites/por-
tals.
The Regional Science Source Book is an example
of a successful OSP communications tool. The
Regional Science Resource Book was initiated by
OSP to provide the Deputy Regional Administra-
tors (DRAs) with information on research in their
respective Regions. This book was prepared for a
Region-ORD Summit, held in Atlanta on Septem-
ber 25, 2002, that was attended by senior leader-
ship from ORD and each EPA Region. The Re-
gional Science Resource Book was considered a
success by OSP because:
v- OSP received positive feedback from the
DRAs.
v- The AA for the Office of International Affairs
(OIA) requested a similar book for interna-
tional research activities.
& The book was used as a "scorecard" to track
progress on action items.
& The book was used as a reference by both
ORD and Regions in subsequent meetings.
The success of the Regional Science Resource
Book was attributed to the support of the AA/
ORD, the commitment of ORD to expend the
Communicating Research Results: Best Practices Workshop
-------
necessary resources to produce the book, the con-
tributions by ORD Laboratories/ Centers and each
EPA Region, and the usefulness of the informa-
tion to DRAs. Dr. Teichman identified a number
of challenges to developing additional resource
books, including the cost of producing them, pre-
paring books that are useful to a specific audi-
ence, collecting the information to include in the
book, and keeping the book current.
A second example of a successful OSP communi-
cations product is the Region-ORD Critical Eco-
systems Workshop. The topics of the workshop
were initiated by the Regions, and the workshop
was intended to improve the participants' under-
standing of the science completed, underway, and
needed for ecological assessments. The outputs
of the workshop included presentations, papers,
and a proceedings report. The workshop resulted
in a better understanding of science issues and
needs, and development of a network of Regional
and ORD scientists who will collaborate beyond
the workshop.
Dr. Teichman stated that OSP's communication
matches its roles of science integration, coordina-
tion, and communication, and OSP's audiences
are its clients and stakeholders. He noted that the
communication tools developed by OSP can be
resource intensive; however, they can pay large
dividends in furthering ORD's role of providing
science to support EPA's mission. Dr. Teichman
commented that journal articles are usually not
the best source of scientific information for
decisionmakers; better environmental decisions
will be made only if managers and policymakers
have a better understanding of the science, and
that will require communicating by more innova-
tive, targeted means.
Particulate Matter Research Program
Dr. Dan Costa, Chief of the Pulmonary Toxicol-
ogy Branch at NHEERL and Acting ORD Particu-
late Matter (PM) Program Manager from January
2002 to October 2002, provided some background
* nforr^^t'O". nbout OR-D's
needs. Since that report, NRC has published two
additional reports on PM research and recom-
mended a multi-year portfolio of the highest pri-
ority research topics.
In 1998, Congress added $22.4 million per year to
ORD's $27.8 million budget to address PM re-
search. ORD based its PM research strategy on
the 1 1 issues identified in the National Research
Council's (NRC) first report on PM research
The add-on Congressional funding in 1998 was a
substantial investment in the PM issue and as the
5 years drew to a close, the idea emerged within
ORD that a "highlights" report with an assess-
ment of program productivity and advances in
knowledge would be appropriate and timely. It
also could lay out a conceptual plan to address
remaining important knowledge gaps. This report
became known as the PM Synthesis Report.
The "idea" and the "need" for the PM Synthesis
Report emerged in January 2002, and the concep-
tual plan and schedule for the report were devel-
oped from April to June. The sections of the
report were drafted from August to October, and a
full draft with appendices was produced in Octo-
ber. Revisions were made to the report through
February 2003, and a near final draft was pre-
pared by March 15, closely followed by a briefing
for the AA/ORD on March 27. Final editing of
the report is ongoing and it is expected to be com-
pleted in June 2003.
The report is intended to communicate the PM
Program to diverse clients (e.g., Assistant Admin-
istrators, Congress, NRC, management, investiga-
tors) in a succinct and positive manner. It cap-
tures the essence and highlights of the program
over 'ts 5-year fundin" au^rnsntciticn in li^ht of
the NRC priorities. It provides a comprehensive
and balanced report and includes a global narra-
tive, project descriptions, budget information, and
a bibliography. The PM Synthesis Report was
designed to be very user friendly, and as part of
12
Communicating Research Results: Best Practices Workshop
-------
the report preparation efforts, a PowerPoint brief-
ing was developed that could serve as a resource
for AA Office presentations.
The ALDs coordinated the writing efforts to ad-
dress the 11 Research Topic Areas of the NRC
priority needs. Each section was laid out in a
format to address the uncertainties, relevance,
accomplishments, program implications, and
future directions of each Topic Area. The PM
Program Manager then took responsibility for the
major revisions and rewriting of the report, and
once the draft was completed, a number of NCER
staff were asked to review it. Dr. Costa noted that
considerable effort was expended on the Execu-
tive Summary and the 3-page overview.
The report was prepared almost exclusively by
EPA staff from across ORD Laboratories and
Centers. Several face-to-face meetings, e-mails,
and teleconferences were the primary conduits of
communications among the staff working on the
report, and they were very responsive in perform-
ing their tasks. Senior managers were briefed and
given an opportunity to review the report before it
was submitted to the AA/ORD. Professional
editors now are working to finalize the report.
Dr. Costa identified several lessons learned from
preparing the report. It forced program leadership
to critically examine the program's accomplish-
ments, overall plan, and future directions. He
stated that a vested (and not contracted) and
sometimes blinded effort drove the project to
completion. The PM Program Manager clearly
needs more administrative support to complete
such tasks, and there is a definite need for a PM
Program Web site to serve as a resource for intra-
mural and extramural scientists and regulators.
The success of the PM Synthesis Report is mea-
sured by the positive feedback received from
many managers throughout ORD and by the sup-
. port from the AA's Office. One of the greatest
challenges to the development of the report was
gaining trust among the various contributors with-
in the Agency, especially during the time that
funding shifts were being made across the Labo-
ratories/Centers to meet the Agency's needs.
However, this challenge was overcome and the
document development team successfully focused
on completing the report.
EPA Office of Air and Radiation
Prudence Goforth, Communications Director for
the Office of Air and Radiation (OAR), indicated
that her primary role in OAR is to connect to the
public and end users so that they understand the
science behind EPA decisions, policies, and regu-
lations. She is in contact with the press on a daily
basis and the range and complexity of the subjects
that she covers is extensive. Ms. Goforth stressed
the importance of communicating the research
underpinning Agency decisions. To illustrate
some of her responsibilities, Ms. Goforth men-
tioned that she was involved in the recent release
of the Ozone Implementation Plan, and has been
working on the 5-year review of the National
Ambient Air Quality Standards (NAAQS) that are
due to be released very soon.
One of her roles is to capture the interest of the
reporters who are seeking 30-second sound bites.
She posed the question: What do you do when
you are asked to communicate something before
the research and analysis has been completed?
She warned against assuming that the data speak
for themselves. It is essential that the communi-
cator take the time to put the data in context. Ms.
Goforth said she learned this lesson the hard way
with regard to EPA's involvement with the World
Trade Center. Her office received thousands of
inquiries about air quality during the weeks and
months following September 11. The staff were
overwhelmed and, as a result, made the critical
mistake of posting a massive amount of air moni-
toring data on the Web without giving it context.
Everyone with an agenda started mining the data
for their own purposes. They often would focus
on a 1-hour spike or a 1-day spike to exaggerate
the risk. Ms. Goforth said that it has taken EPA
Communicating Research Results: Best Practices Workshop
13
-------
months to undo the damage caused by the misin-
formation reported those data. She noted that
similar lessons were learned when working on the
anthrax incidents.
The most effective way to get the research results
used by decisionmakers is to make a connection
between the researcher (technical) staff, the com-
munications staff, and a Web specialist. This will
ensure that the researcher is focusing on how the
results will be announced or released and made
available on the Web. This group should be
asked to identify the level of interest of various
audience groups and to anticipate their reactions.
What concerns will be raised? Who is likely to be
interested in the results? What will be of interest
to the public? Ms. Goforth emphasized the im-
portance of anticipating the worst information to
be conveyed and thinking of the best approach to
communicate that information. She said that her
approach is to have someone on her staff write the
best story that could be communicated as well as
the worst story. These are not for distribution
outside the Agency, but they are very helpful in
preparing EPA staff for the release of the results.
Another issue to consider is who should be in-
volved in releasing the information. Who will
give the announcement credibility (e.g., EPA
Administrator, American Lung Association)?
Should the information being communicated be
put in a larger context? What has been done in
the past and where is it headed in the future?
What relationship does this research have to is-
sues the government is facing? How will the
research findings help or hurt a regulated organi-
zation? Knowing the answers to these questions
helps to formulate an effective communication
strategy.
More and more universities and other organiza-
tions are using press conferences to announce
their research results. For example, the Presi-
dent's Clear Skies Initiative was announced at a
press conference. Reporters should be invited to
attend such press conferences and given the
opportunity to discuss the results with the re-
searchers. Research results also are commonly
communicated through fact sheets, questions and
answers, and the Web. Although the Web may be
the best way to communicate with some audi-
ences, there is a need to do a better job of inform-
ing users of what to expect from the links. Ms.
Goforth encouraged those in attendance to use
their communications staff and emphasized the
importance of putting the science into context.
Centers for Disease Control and Prevention
Dr. Marsha L. Vanderford, Deputy Director of
Communication at the National Center for Envi-
ronmental Health (NCEH), Centers for Disease
Control and Prevention (CDC), emphasized the
importance of being consistent in communicating
information to the public. Her presentation fo-
cused on communication research associated with
the National Report on Human Exposure to Envi-
ronmental Chemicals. The first National Report
included information on the measurement of 27
toxicants in humans at levels that were previously
unknown. Most of the data compiled were base-
line measurements, and there were trend data for
only 3 of the 27 chemicals. In addition, most of
the health effects of the toxicants were unknown.
The first report received positive feedback from
scientists, and this led to the development of a
second report in January 2003, which focused on
116 chemicals measured between 1999 and 2000
(most of which were first time measures). The
Office of Communications received hundreds of
requests for interviews about the report, and early
media coverage suggested that the public was
Communicating Research Results: Best Practices Workshop
-------
interested in the report as well. In addition, previ-
ous qualitative studies suggested public desire for
more information/attitudes about environmental
health hazards. As a result, formative research
was conducted on the communication of the re-
ports.
The primary audiences for the first report were
intended to be public health agencies, the scien-
tific community, and policymakers. The audi-
ences for the second report included the public,
environmental advocacy organizations, and pro-
fessional audiences (scientists, state and local
pubic health agency staff, health care providers,
and federal partners).
Qualitative audience research provides insights on
the target audience and increases understanding of
the motivations and reactions of the audience.
This type of research answers the question "why"
rather than "how" prevalent. It also provides
access to responses the researcher might not have
considered and offers insights into the core values
and underlying beliefs, behaviors, and perceptions
of the audience.
Telephone interviews were conducted with four
focus groups comprised of people from across the
United States. The materials reviewed during the
research process included news reports, fact
sheets and Web information, and links to EPA
information. Findings from the formative re-
search showed that there was personalization of
the information (i.e., relationship to local events
or to family's or friends' illnesses), expectations
that the report would indicate high levels of expo-
sure and dangerous health effects, and skepticism
that the findings would be used to improve public
health. The findings also indicated that some
individuals would prefer not to know about possi-
ble exposures.
The formative research also investigated what
participants would be motivated to do (e.g., seek
information, limit further exposure, seek personal
testing, share information with others, talk to a
physician, participate in limited activism). The
implications of the formative research findings
were that: (I) communication materials should be
developed specifically for lay audiences and
should include FAQs to respond to questions
from the public, fact sheets for public dissemina-
tion, and information posted on Web pages; and
(2) audiences should be directed to credible
sources where more local and personal informa-
tion is available. The study also indicated that
given the unknown risk, two-way communications
should be available and boundaries should be
placed around the uncertainty (e.g., attention
should be focused on lead and cotinine, for which
there are known health effects and chronological
trends, with the surprise of "good news").
There were four major messages to be communi-
cated: (I) advances in biomonitoring are a major
step forward in tracking exposure; (2) it is too
early to judge new baselines, but the good news is
that there is a decline in exposure to lead and
cotinine; (3) in time, there will be many valuable
uses of the dataall of which will be aimed at
protecting public health; and (4) everyone (i.e.,
science, government, industry, and the public)
Placement of Messages 1,2,3 and 4
Help find leukemia cause, Fallen mother asks
ta UK UtHlature: Thnx days of
heanngs end on child cancer c
Message 1
Message 2
Messages
n Upper Third
Middle Third
n Lower Third
Message 4
Communicating Research Results: Best Practices Workshop
-------
benefits from knowing more about exposures to
these chemicals.
An evaluation of media coverage from March to
April 2001, revealed 84 news reports (TV, radio,
print, Web, and wire) and 66 print reports. A
media analysis of the print coverage was con-
ducted to determine the effectiveness of the com-
munication strategy in emphasizing priority mes-
sages. The analysis indicated that 99 percent of
the reports included one or more priority mes-
sages and these messages filled 41 percent of all
print media space. In most cases, at least one of
the priority messages was placed in the first third
of the report.
CDC decided to assess how effectively it commu-
nicated about the first report to the primary audi-
ences (i.e., scientists, public health staff and offi-
cials, health care providers, and federal partners),
so that it could do a better job communicating the
second report. Dr. Vanderford pointed out that
these primary audiences are the ones who will use
the report to: (I) determine if chemicals are get-
ting into the bodies of Americans, (2) assess the
effectiveness of public health efforts to reduce
exposure, (3) establish reference ranges for com-
parison with at-risk populations or individuals,
(4) track trends in levels of exposure, (5) deter-
mine whether levels are lower in different demo-
graphic groups, and (6) set priorities for research
on human health effects of chemicals. The audi-
ence research was designed to obtain information
from the audience regarding the relevance of the
report to participants' work and professional in-
terests, channels for accessing the report, impres-
sions of the report, impact of the report on
participants' work, evaluation of the report for-
mat, and awareness and expectations for the sec-
ond report.
The audience research process involved semi-
structured in-depth interviews with 54 partici-
pants (15 scientists, 13 health care providers, 13
public health officials, and 13 federal partners).
Dr. Vanderford added that there was a high level
of awareness of the report among ee.rtain grouns
of participants. The channels used for the initial
notification were professional organizations, list
servs, and colleagues and contacts with the CDC.
The primary sources of information about chemi-
cals and human exposure came from published
literature; state and federal databases; federal
agencies such as the Agency for Toxic Substances
and Disease Registry (ATSDR), National Institute
for Occupational Safety and Health (NIOSH),
EPA, National Institutes of Health (NIH), and
CDC; and professional associations.
Most participants expected to use the report in the
future; current uses of the report included com-
parison with other studies, setting/changing re-
search agendas, and serving as a model for sam-
pling and analytic methods. The value of the
report was in providing reference ranges, a geo-
metric mean, percentiles, and sufficient informa-
tion to model the data. The participants were
dissatisfied with the report because of the demo-
graphic variations and the lack of health effects,
safety thresholds, criteria for selection, access to
raw data, and accessibility to lay audiences. The
level of awareness varied among the audience
groups and most health care providers were un-
aware of the reports. With regard to expectations
about future reports, there were some who were
anticipating the next report, and they wanted to
know when the next report would be released and
what it would cover.
Health Effects Institute
Dr. Robert O'Keefe, Vice President of the Health
Effects Institute (HEI), provided HEI's view on
how to communicate scientific information. He
stressed the importance of building trust in the
messenger. The perceived credibility of the orga-
nization affects the receptiveness to the message.
HEI is an independent nonprofit research institute
that is jointly funded by industry and government.
Its mission is to provide independent research on
the health effects of air pollution from sources in
the environment. In addition, HEI's core function
is to provide research that directly informs regula-
tory decisions.
HEI is structured to maintain credibility and trans-
parency in controversial national regulatory de-
bates. The Institute not only has joint and equal
funding, hut also has an independent Board of
Directors and a standing Research and Peer Re-
view Committee. HEI is not affiliated with spon-
sors, has no perceived point of view, and does not
take policy positions. HEI's various science prod-
ucts include research reports, reviews of the sci-
16
Communicating Research Results: Best Practices Workshop
-------
entific literature, reanalysis of studies, and HEI
Perspectives.
SPECIAL REPORT
ii ii ALT ii Reanalysis of the Harvard
! N*STIrfuTi! Six Cities Study and the
American Cancer Society
>*m° Study of Paniculate Air
Pollution and Mortality
A Special Report of the Initltute't Parti
Epidemiology Reanalyiii Protect
Research reports are highly technical in nature
and are HEI's core scientific product. Examples
of these reports include epidemiological and toxi-
cological studies of particulates, benzene, etc.
HEI usually conducts reviews of the scientific
literature on key topics (e.g., MTBE or diesel
exhaust), and these reviews are moderate to high-
ly technical. HEI also reanalyzes studies central
to the regulatory process, such as the Harvard Six
Cities Study. HEI Perspectives is a synthesis/
primer on key issues that are central to under-
standing health effects of pollutants (e.g., PM
mechanism, epidemiology).
The HEI audience includes sponsors (EPA, indus-
try); science community (investigators); stake-
holders (environmental nongovernmental organi-
zations, industry associations, citizens, etc.); Con-
gress; other federal agencies (Office of Manage-
ment and Budget, Department of Energy, Federal
Highway Administration, etc.); international/other
regulatory bodies (World Health Organization
[WHO], European Commission, Japan EPA, In-
ternational Agency for Research on Cancer
[IARC], California Air Resources Board
[CARB]); international lenders (World Bank,
Asian Development Bank); and the press. HEI
publications are targeted to a diverse constitu-
ency, and reanalysis often is needed to communi-
cate to a broad spectrum of policy and scientific
professionals. Congress wants to know if the
study held up under scrutiny, if there was an open
pYocess, and if adequate data were provided.
Regulators may want to know about the reana-
lyzed relative risks and the implications for other
studies. Scientists want to know about the meth-
odological approaches, implications for science,
and future research. Industry, NGOs, and others
also have unique interests.
HEI publications are organized to communicate to
a diverse audience. Therefore, each contains an
HEI statement that provides a synopsis in lay
terms of the project context, results, implications,
and conclusions (2-4 pages). Each also includes a
preface, which provides details on the process; an
investigators report, which is a detailed technical
report by the scientific research team including
data, methods, and scientific approaches; and a
review committee commentary, which is a techni-
cal peer review and integrated distillation of key
findings, strengths and weaknesses, conclusions,
and implications for regulatory decisions.
Dr. O'Keefe described four phases in a study
release. Study Release I begins with a decision to
undertake the major research area or project. HEI
communicates its motivation to stakeholders to
explain why they should trust HEI, the relevance
of the project to its sponsors, the regulatory agen-
da, the scientific rationale for undertaking the
H3 Perspec
AIRBORNE PARTICLES AND HEALTH: HEI EPTOEMIOUXJIC EVIDENCE
ntnrkHU herwrfm panlcaht, mantr IPMf and
ttiwtbOt* no*akirme«f+1rriaatrjtfrtOmlurtfHt f/M ITMlMn pncdrill to ito- US «od M'xlna Eutap*. v»
lcl*
mnl miH lomhKh-oolunr tlr ponmlaa
coaibuMlaa wm luurd. la pubilr buli
««l la inmuiaKi Borulllv (B*lxt I
wmm, dbast«d oo wtwitui itw [
lui PM opHUi> lud ni»«J lh» «tt
fWi. (UU.8 and Son.* l-ViaL Th^ (
liwn iotpUtntbl* (inn tt»- l»k of.t
Communicating Research Results: Best Practices Workshop
17
-------
research, the public health implications, and the
expected benefits. He noted that an early face-to-
face meeting is ideal. In Study Release II, pro-
gram summaries are prepared providing key sci-
entific background, and workshops are held with
stakeholders during study design/implementation.
HEI also provides briefings and updates of study
progress, and holds an annual conference where
posters and abstracts of the latest results are dis-
played. In Study Release III, HEI conducts a pre-
briefing with the sponsors to inform them of key
findings 24-48 hours prior to release.
HEI typically prepares a press release for major
studies and a press backgrounder for complex
studies or exceptional circumstances. On the day
of release, HEI calls key stakeholders to inform
them of the release. Notification of the release is
sent via e-mail and hard copy, and it is posted on
the Web. In the weeks following the release, HEI
visits key stakeholders to provide face-to-face
briefings and to answer detailed questions. Al-
though this process is labor intensive, it has been
effective and appreciated. It sets the stage for
future releases and prepares the sponsors and
stakeholders for unfavorable results. Study Re-
lease IV involves followup synthesis, such as
publication of HEI Perspectives.
Dr. O'Keefe acknowledged that tracking and
evaluating the communication of scientific infor
mation is very difficult. HEI tracks the citing of
its research studies/reviews in rulemaking (EPA
criteria documents, CARB rulemaking, WHO
guidelines processes, IARC monographs, etc.).
HEI also tracks journal publications from its stud-
ies, the demand for HEI Perspectives, and the
number of visitors on the Web site and the docu-
ments downloaded.
The soft measures include invitations to present
HEI's results at key stakeholder forums or to
Congress. HEI also tracks the reactions of spon-
sors and stakeholders, and whether new groups or
the press (trade and popular) are taking an interest
in the studies. Dr. O'Keefe stated that the key
elements to communicating scientific information
are:
^ Pay attention to organizational perceptions,
which matter and require early and consistent
maintenance.
-------
v* Subject the publication to editing (by non-
scientists); this can be a challenge but is
worth the effort.
3* Understand individual reporter interests and
prepare supplemental press materi-
alsespecially for technical documents.
4- Track both hard and soft measures.
Dr. O'Keefe concluded his presentation with
some additional points for consideration by EPA.
He stated that the Agency should track the publi-
cation of important studies it sponsors and pursue
joint press strategies with the investigators. In
addition, EPA-funded results should be summa-
rized and communicated in a synthesis document,
on the Web, and in the science press.
CUT Centers for Health Research
Dr. Fred Miller, Vice President for Research at
the CUT Centers for Health Research, described
the process for developing, implementing, and
communicating research at CIIT. He said CUT is
a small research institute that publishes its results
in the open literature; however, CIIT does not
stop with mere publication of results. CIIT com-
municates its results through a quarterly newslet-
ter, its Web site, an educational outreach program
(for K-12), and a number of other communication
tools. GIFT'S core program consists of about $16
million of research, funded by the American
Chemistry Council (ACC), that focuses on sys-
tems biologythe what and the why.
The fundamental characteristics of environmental
health research at CIIT are: (1) risk assessment
(RA) orientation, which involves institutional
commitment and experience to bring science to
bear on the decisionmaking process; (2) integra-
tion of basic and applied science; and (3) commit-
ment to address uncertainties that often arise in
RAs such as low dose responses, interspecies
extrapolation, and susceptible populations. Sys-
tems biology is the quantitative study of biologi-
cal processes as whole systems instead of isolated
parts. A systems level view is needed to under-
stand the complex dynamics that underlie the
physiology in both the normal and diseased states.
CIIT's approach employs a synergistic integra-
tion of theory, computation, and experiment.
:| Tissue
Dose ..
Cellular
Effects
Functional
Deficits
11 A bunch of
Mother data, too
How do we plan the work so it will be used!!
E
x
p
o
s
u
R
E
Tissue
Dose
Cellular
Effects
Functional
Deficits
Exposure
Tissue
Interactions
Organ Level
The expected outcomes from using a systems
biology paradigm at CIIT are that complex bio-
logical data are more effectively integrated into
risk assessments; uncertainties such as relevance
of animal data to humans are reduced and some-
times eliminated; the mode of action is better
understood; and the determinants of interindivid-
ual variability and, by extension, potential devel-
opmental and gender-specific susceptibilities are
defined. The nature and extent to which proposed
research will address and reduce uncertainties in
assessing human health risks is a major factor for
identifying and prioritizing core research on is-
sues and topics. The long-term viability and ef-
fectiveness of the core research requires a mixed
portfolio comprised of program projects to ad-
dress major issues that require an extensive, inte-
grated research strategy; individual projects that
address important topics; methods development
activities; and an investment in and use of cutting-
edge technologies.
CIIT uses a program orientation and implementa-
tion guidelines document that covers relevance
and scientific questions specific to identifying and
prioritizing program and individual projects as
well as methods development projects. The ques-
tions that should be asked when identifying and
prioritizing program and individual projects are:
What is the issue and what part do we want to
work on? What risk assessment uncertainties
would be addressed? What would represent an
impact and how likely can we achieve it? The
scientific approach questions to ask are: What is
the hypothesis? What is the scope of the program
needed? Is a systems biology approach feasible
Communicating Research Results: Best Practices Workshop
19
-------
and would it add value? The areas of emphasis
for core research include the developing organ-
ism, risk assessment elements, and susceptibility
factors.
Dr. Miller provided an example from the CUT
research program and how it links to the focus
areas of respiratory biology/toxicology. In the
study on mechanisms of adaptive and adverse
responses in the respiratory tract following low-
level exposure to inhaled reactive gases, the is-
sues were that human exposures to reactive gases
are often low level. Extrapolation of animal ex-
posure data to humans is needed. Another issue
was that many reactive gases induce rodent nasal
lesions, and the predictability of rodent nasal
lesions for humans is uncertain. Additionally,
there is a need to understand the dose-response
relationships over time; the pathogenesis of le-
sions; the susceptibility factors, including gender,
age, ethnicity, and genetics; and the risk assess-
ment context. The relevance of this study is that
it: (1) addresses Long-Range Research Initiative
(LRI) research strategy issues, including real-
world human exposures, demonstration of adver-
sity, and biological sensitivity; (2) has a high
probability of impact (i.e., it builds on existing
strengths and knowledge at CIIT, is a current
focus of regulatory attention, and will provide a
template for broader RA efforts); and (3) ad-
dresses uncertainties in risk assessment, including
interspecies and intraspecies extrapolation as well
as acute to subchronic to chronic extrapolation.
Human
Interspecies Extrapolation
ppm
*
£sS Exposure I Highest Lqcat Do
^^ ml I (no/mm -sec)
Highest Local Dose =
mm2-sec
± X
One of the research program goals of this study
was to compare ?nd confnast fociisp.Hj coordinated
studies on two reactive gases to elucidate modes
of action for insight on adaptive and adverse ef-
fects, and then to apply this knowledge to CU and
H2S risk assessments. Additional goals were to
use a systems biology approach and to focus on
the primary responses. This project has examined
the differences in complexity of the nose geom-
etry of the rat and the human. CIIT has developed
anatomically rich, computational fluid dynamic
(CFD) models built from rat, monkey, and human
data. These models describe how the complex
anatomy of the nasal passages affects airflow
patterns in the nose. The goal is to use this model
to extrapolate dose data for human exposures. In
concluding his presentation, Dr. Miller presented
a diagram that illustrated the interactions and
synergies of three CIIT projectsOlfactory Tox-
icity of H,S, C12 Risk and VOI Analysis, and Cl,
Dosimetry and Pathobiology. He noted that all
three of these projects are building on CIIT's
dosimetry modeling core.
National Institute of Environmental Health
Sciences
Dr. Allen Dearry, Associate Director of Research
Coordination, Planning, and Translation at the
National Institute of Environmental Health Sci-
ences (NEEHS), provided a brief overview of the
Institute. The mission of NEEHS is to reduce the
burden of human illness and dysfunction from
environmental exposures by understanding each
element and how they interrelate. The NIEHS
achieves its mission through multidisciplinary
biomedical research, prevention and intervention
efforts, and communication strategies that encom-
pass training, education, technology transfer, and
community outreach. Dr. Dearry explained that
environmental health, in its broadest sense, com-
prises those aspects of human health, disease, and
injury that are determined or influenced by factors
in the environment.
The NIEHS target audiences include the general
oublic. communitv and advocacv prouos. orofes-
i. J * *_ L. t
sional organizations, grantees, health profession-
als, other agencies, and Congress. NEEHS' Office
of Communications and Public Liaison (OCPL) is
responsible for developing press releases, pam-
phlets, videos, fact sheets, conference exhibits,
public service announcements, responses to pub-
lic and media inouiries, and devHcoina informa-
tion for posting on the NEEHS Web site (http://
niehs.nih.gov/home.htm). En addition, NEEHS
publishes the journal Environmental Health Per-
spectives, which has global distribution and in-
cludes 17 issues per year. The journal covers the
20
Communicating Research Results: Best Practices Workshop
-------
Translational Research Interrelationship
topics of toxicogenomics, children's health, and
environmental medicine. NIEHS has made the
journal available to underdeveloped countries and
published a Chinese edition.
NIEHS also coordinates the National Toxicology
Program (NTP), which is an interagency program
that coordinates toxicological testing programs
within the Department of Health and Human Ser-
vices (DHHS), develops and validates improved
testing methods, and provides information about
potentially toxic chemicals to health regulatory
and research agencies, the scientific and medical
communities, and the public. The NTP issues an
annual report on carcinogens that provides data
on substances known to be carcinogenic. The
three centers of the NTP facilitate information
sharing among various federal agencies on alter-
native toxicological methods, risks to human
reproduction, and phototoxicology. The NTP also
maintains a Web site, including a list serv; holds
public meetings; and produces technical reports.
The translational research that is conducted at the
NIEHS strives to improve our understanding of
how physical and social environmental factors
affect human health; develop better ways of pre-
venting environmentally related health problems;
and promote partnerships among scientists, health
care providers, and community members. One of
NIEHS' roles is translational research, which is
the conversion of environmental health research
into information, resources, or tools that can be
used by public health and medical professionals
and by the public to improve overall health and
well being. Translational research at the NIEHS
is intended to improve understanding of how
physical and social factors affect human health,
develop better means of preventing environmen-
tally related health problems, and promote part-
nerships among scientists, health care providers,
and community members. NIEHS' translational
research programs focus on environmental justice,
community-based participatory research; health
disparities, children's environmental health and
disease prevention; and ethical, legal, and social
implications. The NIEHS Translational Research
Web site can be found at http://www.niehs.nih.
gov/translat/home.htm.
Dr. Dearry emphasized the importance of two-
way communication, noting that true communica-
tion is always two way. The benefits of two-way
communication include more collaborative com-
munication, improved research that is relevant to
public health, and more cost-effective approaches
that will link research to health outcomes as well
as translate and disseminate the information to the
Communicating Research Results: Best Practices Workshop
-------
target audiences. The challenges of two-way
communication, however, are in identifying key
participants, investing the required time, being
proactive in the face of controversies and crises,
and meeting the expectations and needs of the
audience. The two-way communication at the
NIEHS includes research (Community Outreach
and Education Programs), education (K-12 Pro-
gram), and priority setting (workshops, roundtable
meetings, retreats, brainstorming sessions; the
NAS and Institute of Medicine (IOM) Commit-
tees; town meetings; Interagency Working
Groups; and Public Interest Liaison Group).
NIEHS also supports numerous Centers across the
United States.
NIEHS' future directions will include translation-
al research that involves creating environmental
justice partnerships for communication, establish-
ing seven to eight Centers for Population Health
and Health Disparities, working with the National
Cancer Institute on breast cancer and the environ-
mental centers, and the built environment (man-
made structures, land-use planning). Additional
future directions include environmental medicine
(nurse training, research, and practice), and the
Division of Research Coordination, Planning, and
Translation.
Communications Subcommittee Synthesis
Following the workshop presentations, Dr.
Bostrom led a discussion to synthesize findings
and recommendations to improve ORD's commu-
nication of research results. She pointed out that
ORD has made a laudable effort to improve its
practices and innovations in communicating re-
search results. There is increased focus and ef-
forts taken by the ORD Laboratories and Centers
to communicate to various target audiences. In
addition, ORD has made organizational changes,
hired more staff, and allocated more resources to
communications.
At the front end of communications is defining
the audiences and goals. Most of the Laboratories
pnH ^V^nt^rc fiirrr^t tfirf*(=* r\r frviir pnHipr»r*g ""f^^'DS
, . _ . . i._ . ^J ,_,.. Q_ « »...,, ^ .. . V It* I-.*-.*-..*-..** ^. V -~|~J .
Should the press be considered an audience? She
noted that most Laboratories/Centers recognize
the importance of obtaining input on communica-
tions design earlier in the process, and there is
increased outreach to end users in the design of
research programs. She commented that a similar
approach could improve ORD communications.
In examining outputs versus outcomes, Dr.
Bostrom asked if ORD can actually determine if
its research is affecting EPA policy. Dr.
Henderson pointed out that risk communication
had not been discussed during the presentations.
Dr. Bostrom replied that the CDC presentation
mentioned risk communication, but she added that
risk communication was not a specific part of the
Subcommittee's charge. The Subcommittee de-
cided to limit its focus to the communication of
research results. Dr. William Farland, Acting
Deputy Assistant Administrator for Science,
agreed that risk communication is an important
issue, adding that it should be dealt with inde-
pendent of this review.
Dr. Farland asked the BOSC if the press should
be viewed as an audience or a tool. Dr. Herb
Windom (Skidaway Institute of Oceanography),
commented that the press is an important audi-
ence, because of the tremendous amount of infor-
mation to be communicated from ORD. He
thought it was important that ORD consider the
press an audience and develop specific messages
for them. He added that ORD can build public
trust through the news media. EPA should be
proactive in identifying the good that it does, and
This should be communicated to the press to build
public trust. Dr. Jerry Schnoor (University of
Iowa), Chair of the BOSC, agreed that the press is
important and media reports of ORD research
should be tracked. He added that academia close-
ly tracks its press coverage by both the popular
and trade press. Dr. Bostrom thought that the
press should be considered an audience, and ORD
should make a concerted effort to involve the
press in its communication efforts. Dr. Steve
Lewis (Exxon Biomedical Sciences Inc ), member
of the Communications Ad Hoc Subcommittee,
noted that the scientific literacy of the press has
increased substantially in the past 10 years. Dr.
James Bus (Dow Chemical Company) said that he
was struck by the fact that each Labora-
tory/Center develops its own methods and outputs
for communication. He asked if it is in OP-D's
best interest to maintain all of these different
methods and outputs. Should ORD consider cen-
tralizing certain aspects of communication, such
as Web sites? Is it cost effective for each Labora-
tory/Center to maintain its own Web site? More
22
Communicating Research Results: Best Practices Workshop
-------
efficient communications could free resources for
the research activities. Dr. Fred Miller comment-
ed that the inclusion of the media as an audience
depends on what is being communicated. In some
cases the media becomes a public driver. Ms.
Cindy Yu (EPA/NERL) pointed out that centraliz-
ing ORD communications might separate the
communicators from the scientists, leaving the
communicators with just a cursory understanding
of the science.
Dr. Jim Clark (Exxon Mobil Research & Engi-
neering Co.) said that ORD needs to prioritize its
messages and its audiences. He noted that the
public and the scientific community are among
ORD's priority audiences. Mr. Mike Moore
(EPA/ORD) commented that most EPA staff
communicate regularly with the trade press, and
there has been an increase in the popular press
picking up ideas from the trade press. He stressed
the importance of building public confidencer
ORD needs to be perceived by the public as an
organization that is conducting valuable research
in a cost-effective manner to improve public
health.
Dr. Bostrom stated that there has been a great deal
of positive feedback on the products developed by
ORD's Laboratories and Centers. Some of the
products also have received awards. The question
arises, "How successful is EPA's communication
of research results?" Also, "Does EPA's research
affect policy? To what extent and how do the
communication efforts influence the degree to
which science influences policy? What changes
in communication strategies can help EPA to
obtain greater results?"
Dr. Caron Chess (Rutgers University), a member
of the Communications Ad Hoc Subcommittee,
was struck by the progress EPA has made in com-
munications in the past 10 years. Although the
communication goals could be better defined, it is
clear that EPA recognizes the value of communi-
cation. Dr. Bostrom mentioned that, although
Diane Maple, Director of Media Relations for the
American Lung Association was unable to attend
the meeting because of illness, her presentation
was included in the handouts for the meeting.
Dr. Bostrom thanked the speakers for their out-
standing presentations and the participants for
their insightful comments. She then provided an
outline for the Communications Ad Hoc Subcom-
mittee Report. It will include a mission state-
ment, a section on managing communications,
and the goals for communication. In addition it
will address audience selection, content design
and format, and staffing. The report also will
present the conclusions and recommendations of
the Subcommittee. Dr. Bostrom expects that the
report will be about 10 pages.
It is clear that ORD recognizes that it can no lon-
ger rely primarily on journal publications to com-
municate its research results to its various audi-
ences. ORD's investment in communications
training is evidence that the managers recognize
that effective communicators need a broad array
of skills and they must understand the technolo-
gies and nuances of multimedia, interactivity, and
the Web. They also need to keep abreast of the
advancements in the communications field and be
familiar with the latest communication research
techniques. In addition, communications staff
must be actively engaged in the day-to-day activi-
ties and decisionmaking of their organizations so
that they understand and can effectively commu-
nicate the significance of the research results.
Dr. Schnoor thanked Dr. Bostrom and the other
Communications Ad Hoc Subcommittee members
for their efforts in organizing such a valuable
session on ORD communications. He indicated
that the BOSC Executive Committee is looking
forward to reviewing their report.
Communicating Research Results: Best Practices Workshop
23
-------
24 Communicating Research Results: Best Practices Workshop
-------
Presentations for the
Communicating Research Results:
Best Practices Workshop
Communicating Research Results: Best Practices Workshop 25
-------
Jack Puzak
Communicating Results
Jack Puzak
National Center for Environmental Research
U.S. Environmental Protection Agency
Office of Research and Development
Presentation to the Board of Scientific Coinsebrs (BQSC)/
May 15, 2003
Building a
tcitntijlc
foundation
far found
environmental
decision;
NCER's Communication Tools
Annual research progress
review workshops
STAR Regional
Environmental Science
Seminars
News releases and
advisories
Publications
- STAR Reports
- Stata-of-Scfence Reports
- SQRSuccass Stories
- Workshop Procradinoi
tcitnlifix
foaadaiian
far found
awvamcnul
NCER's Communication Tools
Web site
- Annual progress, llnal reports
- Requests for Applications
- Science pages
- Powerful search engines
- New and events
- Personalized pages
- Research capsules
Quarterly updates on grant
awards
NCER Presents and
OEI/ORD lecture series
NCER warehouse
Ktentf/it
feuntoti,,,
far touiut
cmHranauiuat
iecifions
STAR Regional Environmental
Research Seminars
STAR researchers present and
discuss their work with regional
staff and managers
Pilot seminar held in Region 1
last November
Region chooses research it
wants to hear
State/tribal representatives also
attend
NCER works closely with region
to ensure they get exactly what
they want
-------
Jack Puzak
Bu&Kaff *
fcuntific
founJatian
for found
ennnnaental
decuioni
1
Upcoming STAR Regional
Kg Dallas
jraggji. Chicago
Kansas City
gijffijj Atlanta
H|p| San Francisco
£JJ Philadelphia
Seminars - 2003
May 28
June 17
August 13
September 10
October 8 (?)
Sept/Oct
5
BuHdutg a
icientific
foundation
for sound
en vtn> nmentat
decisions
Audience
Regional staff and
managers are primary
audience '
State and tribal
environmental
representatives
Local media
Industry
University professors and
students
fnuitjatutn
Jar found
ttvirattmtrual
Communication Goals
Make staff and managers aware of STAR research in
regional universities
Let regional staff know that they can contact grantees for
information
Inform region about NCER website and other grants they
can use in their work
Give research information to state/tribal agencies
Determine whether other regions would want seminars
Inform public through the media
BliMiag a
scientific
faunda&in
far unad
nvvanmrnlat
Communication Tools
Regional website
Brochure
- Mailed to state
agencies
- Handed to regional
staff and managers
Emails sent to sta" from
region's management
Local media notified
Z
-------
Jack Puzak
BuBitinga
stitmifa
f»anda£aa
far touad
tarimunenul
decisions
Presentation Topics
Region 1
Drinking water
Air pollution and human health
Harmful algal blooms
' Water and air monitoring '
Mercury
Funding land conservation
Phytoremediatioh
Climate change
Building a
scientific
foundation
far sound
environmental
decisions
1
Evidence of Success
Region 1 Seminar
- More than 200 people attended the opening session
- More than 90 people attended the scientific sessions
- High attendance from regional staff and management
- Attendance from state agencies, nonprofit, industry, universities.
and other federal agencies
- Written feedback was overwhelmingly positive
- Verbal feedback from grantees was positive
- Six other regions want similar seminars that are being planned now
Created in response to a need expressed by the regions
10
. BuUdutga
scientific
foundation
fersauad
decistaa*
Why Did this Seminar Work?
Region asked for this information so the audience was
interested
Upper-level management in NCER'and the region
supported the effort
Support by Paul Oilman
Topics directly-applied to regional needs
STAR grantees were willing to present
We worked REALLY hard!
KUnSfit
foriounj
ttfiratuBfiuat
dfcuiata
Making Our Regional Seminars Even Better
Personal invitations will be extended to state agencies - not
just mailings
Add teleconference and simulcast options for those with
travel restrictions
Audience increases
- More nonprofits
- Tribes
- University professors and students
Send out follow-up questions to see how seminar
information was used
-------
Jack Puzak
Building t
Ktfnrifte
fauiutotian
. far tnuitd
nviroiimetital
decisions
NCER Web Site Overview
NCER's web site conveys
information on:
- Applying for research
grants, fellowships, and
contracts
- Research projects and their
associated research results
BmUinc a
scientific
/aandaaan
faruuiul
entvttn'mental
decisions
NCER Web Site Overview (cent)
Web site was redeveloped in FY02 to include an Grade
database that provides:
- Easy access to NCER research abstracts, progress and Final reports
- Dynamically generated (automatically updated) pages
- Improved search results and tailored search functions
- Rexible search results reports
- Links to results and PI publications
- Ability to pass NCER data to other databases including EIMS,
Science Inventory and Web Inventory
Building a
scientific
f»r tfiuut
environmental
NCER Web Site Overview (com.)
Grade database implementation was successful and
contains:
- 2,600 research project abstracts
- 1,500 progress reports
- 1.000 final reports
- 16.300 PI publication bibliographic citations
- 4,800 journal article bbfographic citations
- 500 lull-text pdf journal articles
scientific
foutularvin
en viranntfntal
Recent Web Site Advancements
Projects by Region function
Top Awarded Institutions
function
Highly Cited Researchers
page
EIMS drinking water portal
Science pages
New User Quick Guide
Home page Research Mews
and Events
-------
Jack Puzak
HutUing*
K'tenttfit
faaaJatian
farioutut
environmental
dtckianx
1
"Projects by Region" Function
Regions have a need to compile grants and fellowships in
. each state
"Projects by Region" is an advanced sort function that:
- Generates reports by region with state and institution sub-sorts'
- Includes project funding amounts and state and regional funding
tallies .
[httD^/cfDub.eDa.aov/ncer abstracts/index.cfm/Iuseaction/j
[ . reQional.main . . J
17
tcuntifix
foundation
for sound
tnviroRmtntol
"Top Awarded Institutions" Function
Regional need
Generates reports for the top
awarded institutions based
on either funding or
number of grants
http://cfDub.epa.gov/ncer abstracts/
special reports/lop25.cfm
BuiMinaa
faun&toatt
fawiuut
environmental
decision.
1
"Highly Cited Researchers" Page
Compiled using ISI's highly cited researchers list
Shows highly died NCER Pis
. Linked to contact, abstract, progress and publication
information
Future link to highly died publications and citation rates
Important public relations tool conveys relevance of PI pubs
in the scientific literature
http://es.eDa.aov/ncer/results/riiqhlv dted.html 1
J '
19
Building a
faanJattaa
for sound
tnvironateiUal
decisions
1
EIMS Drinking Water Portal
ORD-wide Data Sharing Task
Groups ORD projects related to drinking water
. Topics include Microbial/Disinfection Byproducts. Arsenic, CCL
- Subtopics: exposure, effects, risk management, risk assessment
More than 200 of NCER's extramural grants, fellowships, center and .
SBIR projects included
Important data pass of an NCER topical data subset
- NCER used a database-to-database link to implement
- Automatically updated
OW wi use these data in their upcoming DRINK database
The DMS Drinking Water portal is sffl draft
httD'J/cfintrtDnc.eoa.aov/dwoortal/cfm/dwHome.cfm 1
J
-------
Jack Puzak
Butlding m
icitntifif
Jaurulatlea
for tau tut
enrirpnnitiKal
dfcu'mns
1
"Science Topic" Pages
Show NCER goals, requests for applications, recipients,
and results tor multiple years on a particular science topic
Conveys the breadth, direction, and success of NCER
research for a particular science topic
Economics and Particulate Matter posted in FY02
Nanotechnology under review
f
1 MtD://www.eDa.qov/ncer/science/
I
21
Building a
tcienti/u:
foiftufatun
for sound
environmental
decisions
New User/Quick Guide
Provides guidance to
new users on purpose
of NCER site and how
to use it
Brings solicitation "
information to the
surface
scientific
JOT sound
irattmcntal
tfecisiont
Research "News and Events"
Front page news articles give visibility to NCER research
- Features
- Awards
- Recognized research'
- Upcoming events
Articles indicate the impact of NCER research projects
Buildutg a
. faitfuiaiani
fmrxattnd
en vim nmeittal
1
Indicators of Success
Positive feedback and statistics showing high use of web site
and are primary indicators of success
Web statistics show high numbers of users: about 40,000 users and more
than 500.000 hits per month
"Contact Us' button - 1 50 inquiries per month
Workshop registralbn pages 2600 registrants tor 31 events since 1999
- NCERrJatabaseaccounlslor2S%olpagespassed(rona«EPA
ColdFusion databases
List serve subscribers have increased an average ol 34% since 1999
Currently more than 1 1 .000 subscrfcers
. httn://es.eDa.Qov/stats/ncer/index.html !
I J
- - " H
-------
Jack Puzak
BttiMinga
fcientific
foundation
for launtt
ettvinnmenial
decuiata
Indicators of Success (com.)
Anecdotal feedback Irom POs, ORD web group, regional
scientists, and ORD management is positive
- Topical Science pages supported by ORD AA
Used as example by ORD web group
- ORD management has complimented NCER on:
Linkage ol NCER database with QMS and the Science
Inventory
Positive effect that NCER project abstracts and research results
had on EIMS and the Science Inventory
- Regional offices and regional scientists find the NCER database
sorting tools extremely helpful
for nuuJ
nrirortmtatal
dccubm
Why Does This Communication Tool Work?
Designed to meet specific needs of several users groups
Early and frequent communication identified initial
requirements and evolving needs
- NCER user groups tested draft tools prior to public release
Quick response to regional feedback yielded tools to meet
their needs
Frequent communication with OEI improved database
efficiency and search times
faitndatutH
fortounj
How Can This Communication Tool
be Improved?
Develop better tools to assess user satisfaction and
research results use
- Online user survey fhtartfwww.epa-oov/hcef/dratisutvgvl
- Database to analyze survey feedback over time
- Tool to analyze email inquiries
Improve dissemination of-resutts used to support EPA rule
making '
- Develop a news fist serve lor program offices
-------
NRMRL Presentation
iTCr. .National Risk Management
((Research Laboratory
Efforts
.
^Com/mm/cot/on ta the Bridge Between
Research end Implementation
Communications
Direct
Written
Electronic
Innovative Communication Multimedia CD
Measuring Success
Plans for Enhancement
We Serve Many Non-Technical Customers
Each year, we respond to about 3,000 phone requests.
We operate a free public video library of environmental
subjects and distribute hundreds of EPA general audience
publications.
We maintain a Public Affairs liaison with local Congressional
offices and manage local media relations.
NRMRL sponsors a variety of scientific meetings
In 2002 we reached over 100,000 people through
exhibits !
Direct Communications
-------
NRMRL Presentation
NRMRL offers 134 Technology Transfer documents
Research Results
NRMRL Researchers published most
frequently in these refereed journals
Environmental Science and Technology
Journal ol Environmental Engineering
Atmospheric Environment
Ground Water Monitoring and Remediation
Remediation
Journal ol Environmental Monitoring
Journal ol the Air and Waste Management Assoc
Chemospriere
Analytical Chemistry
NRMRL researchers published 611 articles
in 271 refereed journals (1998-2002)
Written Communications
The Risk
Communication
Tool Series
Publications for
a Genera!
Audience
Written Communications
NRMRL creates in-house interactive multimedia CDs
for'Training
for Research
for Workshops/
Conferences
Electronic Communications
-------
NRMRL Presentation
Innovative Multimedia Technology
Presents and enhances
information in a radically
new and engaging way
Incorporates digital video,
audio, 3-0 animation,
high-end graphics, and
peer-reviewed content
Uses interactive CD and
DVD formats that can be
linked or converted to
web sites
Electronic Communications
Video Excerpts from Multimedia Products
Electronic Communications
Typical Multimedia Development Cycle
1. Create Video. Audio,
3-D Animation, Graphics,
and Image Media
2. Create 3. Design
Technical Interactivity
Content Structure
Electronic Communications
NRMRL initiates
Virtual Tour concept
Electronic Communications
-------
NRMRL Presentation
Customer Satisfaction Program
Tool to measure success
Way to strengthen future products
-Audience
- Topics of interest
Delivery mechanism
Customer Feedback
Data from Pollution
Prevention Workshop Series
- Overall Quality Average
4.4 (on a 1 -5 scale)
Ranking of Delivery
Methods
- Direct Communication
- Internet Postings
- Guidance Documents
Measuring Success
DIALOG Science Citation Statistics
Citations searched by:
Author
- Journal Article Title
Approximately 4500 scientific and technical
journals searched
Use statistics to track imorovement over time
Measuring Success
Future NRMRL Multimedia Lab
Increased use of digital
technologies to enhance
communications
Centralized location
Cost-effective sharing of
software and hardware
-------
NRMRL Presentation
Science Results Integration Program
Implementation of Goal 4
of ORD's Strategic Plan
Internal Vision
- Integrate science results
across ORD and deliver
information without
organizational boundaries
External Vision
- Expand the use of ORD's
research and measure
outcomes
Three Ms of.: -;:'' :::'$} -^izjfe
^jl L Science ResidU Integration]^
Plans lor Enhancement
Science Results Integration Program
Three Pilot Projects
- Regional Seminars on Molecular Biology
- Mercury Risk Communication
- Understanding Drinking Water Disinfection
By-Products and Microbial Contaminants
Plans lorEnhancement
Waste-^y
Citizens
State.P/
Engine ei
Agencfe
C^ng^
Pffi&
i'$cient(:
Environr
Croups
Educatbi
Com'mur
at£?r:J$ana'gers - Program. .Off
Industry'- International Com
We get the, '
' Right Information .--,.
I rv- T h^-- """ " ''"i - ~ '
,.**' - I'l-H 1 W :.','-%. t . . t
, Right Format " ^
5 ;vviitb. the people < *
who need it
ces - Int)
Ttunity T
$$$8
r. Fed en
'/; ' ^',-:V-". ..'
Tribes -.
gde.qnc
ssoc/otii
j-4~Y * ri"fi5S'"
.fltCf* ti/t
:>; -, , -*
-s - Local Governments(:7;Dea,s/on,/)ioJc
i/ty , En vironmental ^Organizations'- 'lad
-------
Peter Preuss, NCEA
> Major Product/Assessment Outreach
> Program/lssue-Oriented Web Pages
Peter W. Preuss, Ph.D, Director
National Center for Environmental Assessment
U.S. Environmental Protection Agency
Office of Research and Development
Presentation to the Board of Scientific Counselors
xMay15,2003
uUaiTu
foumtatuln
forioiuid
enviratuncnlal
factions
National Center for
Environmental Assessment
Main Foci:
> National-level
Assessments
> Risk Assessment
Guidance
> Risk Assessment
Methodology
> Consultation and
Training
BuUdinftt
scientific
foundation
farsauad
en viranmentol
decisions
National Center for
Environmental Assessment
MISSION
NCEA serves as a national resource center for
the overall process of human health and
ecological risk assessments and the integration
of hazard, dose-response, and exposure data
and models to characterize risk.
VISION
NCEA is a high performing assessment
center providing timely and high quality risk
information to environmental decision-makers.
NCEA's Three Divisions are located in:
-------
Peter Preuss, NCEA
Human Health Assessment Program
> Diesel supports recent diesel truck
rulemaking that will lead to reduced risks
>Dioxln - identifies major human exposure
pathways
> Mercury Congressional report that
led to increased attention on risks and
efforts to reduce use. Revised RfD
> Fuel and fuel additives (e.g. MTBE,
ethanol. methanol) - providing health
risk evaluations and supporting
cost/benefit analyses
> Air Quality Criteria Documents
(Particulate Matter, Ozone) -
supporting Clean Air Act standa:
> PCBs - support Hudson River am
other Regional site clean-up
decisions '
NCEA develops
contaminant-specific
risk assessments on
chemicals/stressors
that are of high concern
to the public. The
assessments are used
by EPA, the States, and
International
community.
Ecological Risk Assessment Program
"I- - r... - -. :;':
' > Improve the science ol ecological risk assessment
': Conduct research on risk methods --_/.-'
- Provide Tmodefecolojical risk assessments ,i'?\'"
Demonstrate innovative risk assessment techniques
..:.- Continue developmenl of ecological risk guidelines
> Develop ecological risk assessment approaches,*
£,' , Place-based assessments al multiple scales
,, '^ ' ^ GbbaiChange ', :''. ;
'. ^ , Emerging Issues (biotechnobc^.mvasiyespecies);;;!;;--;.;,^
"4 .V Integrate human dimensions into ecolc^1cal:|isl<
assessments ' ,. . ' ' - ;:''-"''"!
i » ' l ' ' M JJ;:' .. :-:'-l ^"i'vM,1^
'..-," :S-:;;G^$:/'^-i:.?;%pf«
i . '. > ^i/..$.v*&Ate-;3$ml!ffi
NCEA also includes the staffs and the
management of nationally-recognized programs.
SEbi
Integrated Risk Information System was
developed in the 1980s by OHEA, NCEA's
edecessor. IRIS is the Agency's primary
source for toxicity information for human health
risk assessment. It is an important information
resource for programs, regions, and states.
Global Change Assessment is an important
component of the larger ORD Global Change
Research Program. The Global Change Research
Program is an assessment-oriented program with
primary empfiasis on understanding the potential
consequences ot global change.
Risk Assessment Forum is a committee
of EPA senior scientists that develops
Agency-wide risk assessment guidance..
Many NCEA scientists also have
leadership roles in Forum activities.
Building a
tciettli/tf
foundation
for taiutd
Roll-Out of a Major Product
What is Needed:
> Coordination across Agency
> Coordination with other Federal Agencies, EOF
> Audience Identification
> Accurate information
> Clear Information
> Timely Information
> Plain Language Information
> Different Types of Information
> Outreach to Stakeholders
> Media Outreach
-------
Peter Preuss, NCEA
NCEA's Rollout of Draft Cancer Guidelines
i
~ : ? ..-"" - -r-;i.'v:;i.::'v^
Dr*n Fbwl GwhMltM Ur
C^rrtfafn Rlrt Aj.naamrt
li "IMg In- brirlX- JIL| .in w
Boti the draft (ha! Qutialirvs and the draft Supptanonla/ Qutdanca are avaSabte at
www . e pa .oov/neea/cflf/ca neer2003.htm
Develop a Comprehensive Communication Plan
1. Description of Action
2. Background
3. Key Messages
4. Audience
5. Expected Reactions from Stakeholders
G. Detailed Communication Strategy
7. Announcement Notification Plan
8. Contacts
COMMUNICATIONS FLAN
OFFICE OF RESEARCH AND OEV£LOI*M£NT
TJTUji
frwr, £.f (7-
PHOJKCTEn A
M«n*3.M03
1. ACTION:
ETA B rcqncftiiif. via fWera/ JbfMM-rfFR) notkc. pubfic common urC«rnm^wRUAM«MMM((C«icCTGttkk&»eiorG«ideBan). b the IUIK HI notice. EPA at
ifao fckuiat for jwbBe eamnKnl drafl oiipkmcnul (nkhncn br tucsn&f dtlaJmi'i (fat- TTui draft
life cxprauuc. Both tkicuincnK wifl te *bn* with the rtippfemenuiy |uiducc on duklren'* tiik)
win «t fntth rcvUed *d of jwrocipfc* and pmcctlurei (o (aide EPA icieaiixtx anil mt ancMon h
netimg the canctr rUU rerohm; from ctponut to dxmioh or other igcnU
-------
Peter Preuss, NCEA
OUTREACH: Briefings
24 Public Questions and Answers (9
pages); Available on the Internet
IWrtMHCnUMIFll
'
Development of Media Advisory
f* | f > A LHud SIMM . CommunicMorw, Educ^ian,
^CVCT^rA Enwra-nwitil Prw'octton And ««« fVUixxk
ilw oncer inWeCnw
U Out Apency cancer Tukuxcnoncnix he bo Thorough
> High Quality
> Timely
> Successful!
"New EPA 'guidelines would better gauge
danger of chemicals."
'Cancer-risk guidelines face EPA overhaul"
SHjc 2\cXu JJork Simcs
j "Agency says cniiareivs risk is nigner
for some cancers."
AND...
AP/Philadelphia Inquirer: "EPA to get tough on child cancer risks.'
Orlando Sentinel: "EPA proposes concer-risk guidelines for kids."
. Reuters: "EPA proposes cancer guidelines for children."
-------
Peter Preuss, NCEA
Program/Issue-Oriented Web Pages
User Friendly
"One-Stop Shopping"
Focus on High Profile Activity/
Product
Cross-Agency Coordination
NCEA has developed (our Program/Issue-Oriented
Web Pages:
> Risk Assessment Forum .
> Integrated Risk Information System
> Global Change Research Program
> Dioxin
NCEA was instrumental in the development of:
'r MTBE Homepage
Also, NCEA and the Office of Water developed for
Intranet use:
> Drinking Water Research Tracking Portal
NCEA's Homepage
[^ www. epa.gov/ncea I
Risk Assessment Forum Homepage
I www.epa.gov/ncea/raf |
(AS; EffviraMWiM Protection Agtnty
Integrated Risk Information System
(IRIS) Homepage | »vww.epa.oov/irir]
fSmtma t Sutl«t^CH
"- --' * Ot*c*rtftgg' -r^*t/ I
3&3£
3"o"
Wcleom* 10 flw QUS hww p^e. brau^d M ww by fc U5,
EnviraanetuI Protertioo A«ct»c) (EPA) «d ilc OHicc of
Rt «t ychjfij r^vfbfflMT*- t^jBlSSsLCMftidlfcr &!.v(r«'** niaitq m toxicology, bid with laax
knowttdg* of b**ld) (cicnc**. For men qrfwnwtion ibotd QUS,
For dcfmilioM af terms in tbe HUS Web nit. rtfer (o die J
-------
Peter Preuss, NCEA
Global Change Homepage
| wvOTf.epa.QOv/gcrp [
)ioxin and Draft Reassessment Homepage
I www.epa^gov7ncea/dTo^m]
H livm apwi *nd pifUttxiUvr Eacti inUon erf Uv n
m coltBlinmttan Mrfi cm
imwiit. In i-m-ntw, 1994, [PA .^..tl UM public .*.«, drWl of
tfr* CMniln OiuMammanr DncunwntB "tilOi IrdwclMf tttnK wvtjnf f«|vxt*:
o p-dloOn (TCOO) *
Dioxin (continued)
Building a
scientific
far sound
tlecUiam
MTBE Homepage
Protection Aqentf
Wl>/endnhtr«UT^itus«dingasoir« IMjfarprauod Slorja* Tintji WnrtEPAitdama ,
pasoto* Mhn cortainra WTB£
HcMMTBctnyanwtya/ pUtn iff inri Truimwn Hc»rMrB£-
-------
Peter Preuss, NCEA
ORD Drinking Water Portal (Intranet)
Drinking Water Research j
Tritcki-ttg System
WekomeoORC''*Dm«ano Wiser
Rese»rch Trictang System r
partnership
Information Management System
£EWS|.
gHiii^
Building a
tcientVtc
foundation
for found
nriranmtatat
decisions
SUMMARY: NCEA Communication/
Outreach Innovations...The Near Future!
> Invest in Communications and Outreach
> Build the Staff
> Listen to Stakeholders (Internal and
External)
> Identify Agency Needs
>. Work with ORD Labs and Centers and
EPA offices
> Address BOSC Recommendations
-------
Gary Foley
Today's presentation
NERL's communication goals
NERL's delivery and. feedback
system
Communication Innovations
1. Research Abstracts
2. Scientist-to-Scientist Workshops
NERL's
communication goals
Raising awareness of NERL's relevant
high priority research
Engaging all NERL staff in
communication effort
Ensuring all NERL staff can articulate
our work to a variety of audiences
Sharing and showcasing NERL's
research through right channels and
right times
Keeping NERL staff abreast of our
communication products
NERL delivery & feedback system
NERL's business plan Mem
NERL's buwvess plan
-------
Gary Foley
NERL Research Abstracts
Research Abstracts
GOALS
Highlight outstanding research in a way that
draws Agency attention
Communicate in "plain English"but link NERL
science to regulatory, public health. & policy
outcomes
Encourage discussion of other types of
communication that may help our clients
Reduce inquiries about how our research
should.be interpreted and communicated to
others
Simplifies annual process of reporting major
accomplishments
Research Abstracts
CLEARANCE
Data for a RA is entered into TIS with
built-in approvals similar to other tasks.
A RA must be prepared for every
significant research output/outcome,
including but not limited to APM.
An approved Abstract is transmitted to
the client along with g peer-reviewed
major product
Approximately 20 - 40 RA's per year
Research Abstracts
DISTRIBUTION
Lab Director, Deputy Lab Director,
Associate Directors for Health and
Ecology, Assistant Lab Directors, and
scientists disseminate Research
Abstracts
Research Abstracts are posted in TIS
and on the NERL internet
Additional notification of potential
audience and further marketing are
necessary
-------
Gary Foley
Scientist to Scientist Workshops
Biological Assessment
and Criteria Workshop
"^^ritfiSia:i,iii«i>mJint; Co-sponsdred'by
"""^Slig^S^-^ EPA and National
^u^u. ~jnAd ita-m.*,. council of State
Governments
Office of Water
sought out national
experts
NERL'sEERD
provided science
and leadership
BAC Workshop
BENEFITS to Participants
1. Received direct
technology transfer
from EPA scientists
2. Seeing other states
demonstrate the use of
EPA tools (overcoming
the Fear Factor)
3. Greater use and higher
esteem of EPA science
BENEFITS to NERL
1. Facilitates use of
research methods,
tools, and models
2. Direct communication
and interaction with
end users of science
3. Fortifies relationship-
with customers inside
and outside the
Agency
BAC Workshop
numbers
246 biologists from 47 states, 26
tribes, and 1 territory
5 days of training, informal
meetings, and problem-solving
4 tracks with 18 courses
-------
NHEERL presentation
ttEsearch^sut5 to'^vyWe'Variety of audiences '>
viae.tne.toc
-------
NHEERL presentation
-------
NHEERL presentation
lMake^eseairl.jiighlightsavailable to 3 -wide audience?
""''"'"''"'''
-------
Kevin Teichman, OSP
Communicating ORD Science
Office of Science Policy
Kevin Y. Teichman, Ph.D.
Director, Office of Science Policy
Presentation to the Board of Scientific Counselors
May 15,2003
OSP's Roles Within ORD
Science Integration
- Developing unified ORD positions on the use ol science
- Managing the ORD research planning process
- Coordinating the implementation of Agency science policies
Science Coordination
- Ensuring that ORD's research addresses EPA's priorities
- Bringing together ORD and the 10 EPA Regions
- Managing ORD's program support function
Science Communication
- Representing ORD to tribes, states, and local communities
- Sponsoring coiloquia, workshops, and scientific meetings
OSP's Target Audiences
Primary
- EPA's Program Offices
- EPA's Regional Offices
Secondary
- State and Local Governments
- Tribes
- Environmental Justice Community
:' -- Other Federal Agencies
- Professional Societies and Associations
- General Public
How OSP Reaches Its Audiences
Science Integration
- Progress reviews
- Meetings with Program and Regional senior management
- Program Support Priorities List
Science Coordination
- Research Coordination Teams (RCT)
- Scientist-to-scientist meetings
- Tribal Science Council, Natl. Environmental Justice Advisory Council
- Workshops, Symposia, Coiloquia (e.g., Regional, Wilson Center)
Science Communication
- ORD Accomplishments Report
- OSP Quarterly Report, web sites/portals
-------
Kevin Teichman, OSP
Example: Regional Science.
Resource Book
Initiated by OSP
To provide each Deputy Regional Administrator
(DRA) with information on research in their Region
For a Region-ORD "Summit" held in Atlanta on
September 25,2002
Attended by senior leadership from ORD and each
EPA Region
Regional Science Resource Book
Why We Consider It a Success
Positive feedback from the DRAs
AA/OIA requested similar book for international
research activities
Used as "scorecard" to track progress on action
items
Used as a reference by both ORD and Regions
in subsequent meetings
Regional Science Resource Book
Why It Succeeded
Support by the ORD Assistant Administrator
Commitment to expend necessary resources
(expensive and time consuming to produce)
Contributions by ORD Laboratories and each
EPA Region (ALD/ACDs, RSLs, HSTLs)
Useful information for DRAs
Challenges to Doing Additional
Resource Books
Cost. Expensive in terms of both time and money
Utility to Audience. Regional book produced for a
specific event (summit), for a specific audience (DRAs)
Information Collection. Keep requests to ORD's
Laboratories and Centers to a minimum
Keeping it Current. Wiii it be a living' document,
or a one-time effort?
-------
Kevin Teichman, OSP
A Second Example: Region-ORD
Critical Ecosystems Workshop
Topic. Initiated by the Regions
Purpose. Understand science completed,
underway, and needed for ecological assessments
Outputs. Presentations, papers, proceedings
report
Outcomes. Better understanding of science issues
and needs; network of Regional and ORD scientists
who collaborate beyond the workshop
Summary: OSP's Communication
of ORD Science
Our communication matches our roles. Science
integration, coordination, and communication
Our audiences are our clients and stakeholders.
Principally, EPA's Program and Regional Offices
Communication tools can be resource intensive.
However, they can pay big dividends in furthering ORD's role
of providing science to support EPA's mission
-------
Dan Costa
Communicating the ORD PM Program
Dan Costa, Sc.D. - Acting ORD PM
Program Manager (Jan - Oct '02)
The Situation
..
''v:/!ncreased1heEPA".; 7
'JjudgBi.ol$27:B.mililor).
:-'-.-for Ihe-'PM program by
."S22.4;rnii!ion'per year- '
:..'.an increasETthat has.- '.
: teen largely sustained
-.through the ensuing ;
;:i'..?:iiveyears. .''. ''
./The NRC.has'.issue'd 3 reports';;.-.
y;.lhat idehiifyjmporiant research;.-.
< needs and recommend.a multi-T
j.iryear portfolio of theMghest'.< '
{priority research ;tppics7tThe final;.
,'- ''report jsdueiiiMay2003.';','/..
History of Report Development
The 'idea' and the 'need' - Jan '02
Conceptual plan & schedule - April to June
Draft sections submitted - Aug to Oct
Full draft and appendices - Oct
Revisions continued through Feb '03
Near-final draft - Mar 15
Oilman Briefing - Mar 27
Final editing - ongoing...expected final in June
, Vision & Objectives
Communicate the PM Program to diverse 'clients' (AA,
Congress, NRC, management, Pi's) in a succinct and
positive manner
Capture the essence and highlights of the program over
its five year funding augmentation - in light of the NRC
research priorities
Provide a 'comprehensive' and 'balanced' report -
(global narrative, projects descriptors, budget, and
bibliography.. .user friendly)
Develop a ppt briefing that could serve as resource for
AA Office presentations (diverse venues)
-------
Dan Costa
How the Report Evolved
ALD's coordinated writing efforts to address the 11
Research Topic Areas of the NRC Priorities
Each section was laid out in a format to address the
uncertainties, relevance, accomplishments, program
implications, and future directions of each Topic'Area
Main revisions and writing taken over by PM Prog Mgr
A limited circle of reviewers/editors - mainly in NCER
Considerable effort focused on the Executive Summary
and 3-page overview
Resources Involved
Almost exclusively EPA staff from across ORD Labs and
Centers - exception was a single contractor from NCEA
who assisted with the bibliography
No additional funding, nor subordinate support staff
A couple face-to-face meetings with email and telephone
as the primary conduit of communication
Highly responsive contributors and reviewers!
Sr. managers were given an opportunity (not long) to
review the report / briefing before submission to AA
Now contracted to professional editors by the PM Mgr.
Lessons Learned
Vision: What did the U.S. get for a quarter billion dollars?!
The report forced the program leadership to look critically
at its accomplishments, overall plan, and future directions
A vested (and not contracted) and sometimes blinded
effort drove the project to completion
The PM Prog Mgr needs more administrative support
There is a clear need (or a PM Program Web ailfe to serve
as a resource for intramural and extramural scientists and
regulators - bibliography and reference resource
-------
Dan Costa
Contributors
NERL
Tim Watklns
Ken Schere
Ed Edney
Jim Vfckeiy
Ron Williams
Linda Sheldon
Paul Solomon
-------
Marsha Vanderford-final
Reaching Public and Professional Audiences:
Communication Research on the National Report
on Human Exposure to Environmental Chemicals
Marsha L. Vanderford, PhD
Deputy Director of Communication
National Center for Environmental Health
1st National Report
Measurement of 27 toxicants in
humans
- At levels heretofore unknown
- Almost ail were baseline measures
- Most of the health effects unknown
- Primary Audiences:
Public Health Agencies
Scientific Community
Policy Makers
2ud National Report on
Human Exposure to
Environmental Chemicals
- January 2003
- 116 Chemicals
- Most first time measures
- Measured 1999-2000
Communication Research
1fl Report
- Public Audiences
2nd Report:
- Public Audiences
Environmental Advocacy
Organizations
- Priuary Audiences
Professional Audiences
- Scientific
- SIBIC UK! Local Public HcaBlt
AfewcySU/T
- Raleral Pirtucn
-------
Marsha Vanderford-final
ln Exposure Report
How should we treat the public as an audience?
100's of media Inquiries
Early media coventge suggested public reaction
Previous qualitative studies suggested public desire for mort
Information/attitudes about environmental health hazards
I Help find leukemia cause, Fallen mother asks
Formative Research Participants
Sex: Female (20); Male (12)
Ages: 18-24 (3); 25-34 (6); 35-44 (5); 45-54 (6); 55-64 (4);
65" (8)
Education: High School Degree Only (7); Current HS
Senior (1); Some College (5); Four-year College Degree (9);
Some Grad School or Graduate Degree (9); Unknown (1)
Household Income; <20K (3); 20-40K (8); 40-60K (9);
>60K (6); NA (Retired or Student) (6)
Formative Research Participants
Formative Research Process
Materials Reviewed
1. News Reports
2. Fact Sheets - Web Information
3. Links to EPA Information
-------
Marsha Vanderford-final
Purposes of Qualitative Audience
Research
Provides insights on target audiences
Increase understanding of motivations and reactions
Answers the question "why"rather than "how
prevalent
Provides access to responses researcher might not have
considered
Offers insights into core values, and underlying beliefs,
behaviors and perceptions
Findings of the Formative Research
Personalization of
information
- Relationship to local
events . .
- Relationships to
family's or friends'
illnesses
Findings of the Formative Research
All Environmental News is Bad News
-Expectations that the report would indicate
- High levels of exposure and dangerous health effects
. Rather not know results
Surprise afcout "First Report"
- Skepticism that findings won't be used to improve
public health
Findings of the Formative Research
What would participants be motivated to do?.
Information Seeking
Limit Further Exposures
Personal Testing
Share Information with others
Talk to Physician
Limited Activism
-------
Marsha Vanderford-final
Implications
Develop materials specifically for lay audiences
- Translate web pages
- FAQ's to respond to public questions
- Fact sheets for public dissemination
Point to credible sources where more local and
personal information is available.
Implications: Continued
With unknown risks provide
2-way communication:
- ATSDR Phot* Unt
Put boundaries around
uncertainty
- Focus on lead and cotinme
Knoin health effect*
' Chronological trend* known
Focus attention on lead and
cotinihe with the surprise of
"good news'*
Four Major Messages
Advances in biomonitoring = a major step forward in
tracking exposure.
Too early to judge new baselines, but good news in
declining exposure to lead, cotinine
In tune, many valuable usesall aimed at protecting
public health
E»etyt*ne benefits fmrn knowing more....science.
government, industry, the public
Evaluation of Media Coverage
March-April 2001
Media Penetration:
84 news reports (TV, Radio, Print, Web, Wire)
66 Print Reports .
Help find leukemia cause, Fallon mother asks
-------
Marsha Vanderford-final
Media Analysis:
Focus: Print Coverage:
Method: Content Analysis
Question: How effective was communication strategy
emphasizing priority messages?
Results of Media Analysis
99% included one or
more priority messages
41 % of all print media
space
Totil UK* Coutinj Prfartf Ueuicet
(DM of 1711 Toll! Unei]
Placement of Priority Messages
Number of Unaa Containing Meaaagea t ,2,3 and 4
(Out of 2791 Total Unei)
S soo
5 «»
I W°
| 200
Z 100
0
' '.! i,:-.
i" . ~ -: '
' ' 82 ' .. , v- ' M '.'.
i 4-'^3< '.'. ..'l?ff.:H- .
Muaaga 1 Total MssftQS 2 Tola! htoasaga a Total Moaaaga 4
Priority Messages
Placement of Mosaagac 1^^1 end 4
-------
Marsha Vanderford-final
2nd National Exposure Report
- How effectively did CDC communicate about the 1"
National Report on Human Exposure to primary
audiences?
Scientists
Public Health Staff and Officials
Health Care Providers
Federal Partners
- How can we do a better job with the 2nd Report?
IWiTi
Primary Audiences:
Professionals who can fulfill purposes of
the Exposure Report
- Scientists
- Physicians
- Public Health Officials
- Federal partners with
environmental regulatory
or research agendas
Professional Audiences Use Report to
Determine if chemicals are getting into bodies of Americans
« Assess effectiveness of public health efforts to reduce exposure
Establish reference ranges for comparison with at-risk populations
or individuals
Track trends in levels of exposure
Determine whether levels arc lower in different demographic
groups
Set pnonues for research on human health effects of chemicals
Purposes of Audience Research
Relevance of Report to participants' work and
professional interest
Channels for accessing the Report
Impressions of the Report
Impact of the Report on participants' work
Evaluation of Report format
Awareness and expectations for 2nd Report
-------
Marsha Vanderford-final
Audience Research Process
Semi-structured in-
depth interviews
54 participants
- 15 Scientists
- 13 Health care.-
providers
- 13 Public health
officials
- 13 Federal partners
High Level of Awareness
Channel for initial notification
- Professional organizations
- Listservs
- Colleagues and contacts with
CDC
Primary source of information
about chemicals and human
exposure
- Pub-fished literature
- State and federal data bases
- Federal agencies
ATSDR
N1OSH !"
EPA
NIH
CDC
- Professional associations
Uses of the Report
Primarily in the future
Current uses:
- Comparison with other
studies
- Setting/changing research
agendas
- As a model for
Sampling methods
Analytic methods
Value of Report
Satisfaction
Provides reference ranges
Baseline for comparisons
- Geometric mean
- Percentiles
- Enough information to
model the data
- "Good place to startlong
way to go."
Dissatisfaction
- Demographic variations
- Lack of
* Health effects
Safety thresholds
Criteria for selection
Access to raw data
Accessibility to by
audiences
-------
Marsha Vanderford-final
Expectations about Future Reports
Level of awareness varied
- Most health care providers did not know
Expectations
- Anticipate next Report
- Few details on
Dale
Contents IIT
-------
Robert O'Keefe
Communicating Scientific
Information
A View From the Health Effects Institute
Robert O'Keefe
Vice President
HEI
Communicating scientific information:
Overview
Building trust in the messenger: perceptions
of the organization
HEI audience.
HEI science products & their
communication
Tracking and evaluation
Summary thoughts
The Organization
HEI independent non-profit research institute .
Jointly funded by industry & government
Mission: provide independent research on the
health effects of air pollution from sources in
the environment
A core function: provide research that directly
informs regulatory decisions
Perceived credibility of the organization
affects receptiveness to message
HEI structured to maintain credibility &
transparency in often controversial national
regulatory debates
- Joint & equal funding
- Independent Board of Directors
- Standing Research and Peer Review Committee
- Not affiliated with sponsors - no perceived
"point of view"
- Does not take policy positions
-------
Robert O'Keefe
HEI Science Products
Research Reports: Core scientific product (e.g.
epidemiological, lexicological studies of
particulates, benzene etc. (highly technical in nature)
Reviews of the scientific literature: on key topics
(e.g. MTBE, diesel exhaust, moderate 10 highly
technical)
Reanalysis of studies central to regulatory process
(e.g. Harvard Six Cities, GAM impact
assessment, moderate to highly technical)
"Perspectives" Synthesis / primer on key issues
central to understanding health effects of
pollutants (e.g. PM mechanism, epidemiology,
(less technical)
HEI Audience: science to policy
Sponsors: USEPA. Industry
Science Community (investigators)
Stakeholders (environmental NGOs, industry
associations, citizens, etc)
Congress (House Commerce, Senate EPW,
Science)
Other federal agencies (OMB, DOE, FHWA, etc)
InternationalNother regulatory bodies (WHO,
European Commission, Japan EPA, IARC CARB)
International Lenders (World Bank, Asian
Development Bank)
Press (trade, popular, occasionally TV, radio)
HEI Publications: Targeted to a diverse
constituency
tl* of tlwj Harvard
s Sludyutdtho
m C*nrrr Sfxdccy
HEI publications: Organized to communicate
to a diverse constituency .
' Reanalysis needed to communicate to a broad
spectrum of policy and scientific professionals
Congress: Did study hold up under scrutiny? Was
there an open process? Was data provided?
Regulators: What are the reanalyzed relative
risks? What are the implications for other studies?
Scientists: What are the methodological
approaches, implications for scicacc, fulurs
research
Industry; NGOs, others also have unique interests
-------
Robert O'Keefe
HEI publications: organized to communicate
to a diverse constituency
HEI Statement: Synopsis in lay terms of project
context, results, implications and conclusions in 2-
4 pages
Preface providing detailed process background
Investigators Report: Detailed technical report by
scientific research team, including extensive
reporting of data, methods, scientific approaches'
used (225 pages in reanalysis)
Review Committee Commentary: Technical peer
review and integrated distillation of key findings,
strengths & weaknesses, conclusions &
implications for regulatory decisions (21 pages)
Study Release 1:
Early involvement
Study release begins with decision to undertake
major research area or project
Communicate HEI motivation to stakeholders:
- Who the heck is HEI, why should you trust us?
- relevance to sponsors, regulatory agenda
scientific rationale for undertaking research
public health implications
- expected benefits
Ideally face to face meetings, at early stage
Study Release II:
Intermediate communication
Program Summaries: Key scientific background
and new projects funded
Workshops with stakeholders during study
design/implementation
Briefings and updates of study progress
Annual Conference: Posters and abstracts of latest
results
Study Release III
No surprises: pre-briefing for sponsors of key findings 24-
48 hours prior to release
Typically press release for major studies (1-2 pages)
Often press "backgrounder" for complex studies or
exceptional circumstances (4-10 pages)
Day of release calls to key stakeholders (other industries,
NGOs etc)
Email notification of release, web post, email & hardcopy
distribution.
Weeks following release: pilgrimages to key stakeholders
to provide face to face briefings, answer detailed questions
Labor intensive, but appreciated, sets stage for future
releases, unfavorable results
-------
Robert O'Keefe
Study Release IV:
Follow-up synthesis.
Communicating Scientific
Information: Tracking & Evaluation
Hard measures:
Are HEI research studies, reviews cited in
rulemakings?
EPA Criteria documents
- CARB rulemaking
WHO guidelines processes
- IARC monographs etc
Are there additional journal publications from a study?
- HEI average <6
Is'Terspectives" in demand?
Is the HEI website being visited? What documents are
being downloaded?
Visitor Sesioos
oifl W09 era es/is oso
MM MIS OMB OS/11 05/19
Most Downloaded Files
(WB 04(1.1 Otffl fl?» KJP 05/15 I
IDI OMB 04/17 IKK CSA3 OS/11 CKflS
. TueMm/2003-Won06/30C003 (8 Week Scale)
lRF«2003|xll
^ GetlGnapd
-------
Robert O'Keefe
Communicating Scientific Information:
Tracking and Evaluation
Soft Measures:
Is HEI being invited to present results?
- Key stakeholder forums
- Congress
Sponsor, stakeholder reaction?
Questions, understanding
- Calls for further HE] work
Who is asking? Is there a diverse interest? Are
new groups taking an interest?
Press interest? Trade popular
Closing
Organizational perceptions matter, and require
early and consistent maintenance
Interact with audience, minimize surprises
Tailor publications, in type and internal structure
to communicate effectively
Editing (by non-scientists) can be a challenge, but
its worth the effort
Understand individual reporter interests, prepare
supplemental press materials especially for
technical documents
Track hard & soft measures
Additional thoughts for EPA
Key challenge: .
EPA funds significant level of research
Since EPA does not publish results itself, does EPA know when
important study results will be published?
If yes, then joint press strategy with investigator may be
possible
Summarize and Communicate EPA-Funded results
Synthesis document, web posting, science press
» Thank You!
-------
Fred Miller
/ Developing, Implementing, and
V Communicating Research at CUT
Communicating Research
Results
EPA BOSC Meeting
May 15, 2003
Fred J. Miller, Ph.D.
Vice President for Research
CUT Centers for Health Research
ropites
Core Program Characteristics
Systems Biology - the What and the Why
Guiding Principles for the Core
Example of CUT research program project in
respiratory biology/toxicology
Fundamental Characteristics of
Environmerrtat-HeatttrResearch at CUT
Risk assessment orientation: institutional
commitment and experience to bring
science to bear on the decision making
process
Integration of basic and applied science
Commitment to address uncertainties (low
dose responses, interspecies extrapolation,
susceptible populations) that continually
arise in RAs
4
Levels of Biological Organization
~\ Systems Biology
Vr - Atoms - \ *^
Molecules
Molecular complexes
Organelles
Cells
Tissues
Physiological systems
Individuals
Populations
(he quantitative study of
biological processes as
whole systems instead of
isolated parts
systems level view is
> needed to understand the
complex dynamics that
underlie the physiology in
both the normal and
, diseased states '
a synergistic integration
of theory, computation &
experiment
-------
Fred Miller
I
Systems Integration
Determine Health Outcc
to
>mes
^y ,-x HEALTH OUTCOMES
n *V^ /"l^^sr* ( [Prevention/Protection
si | } \^^_ "2^««i \ ' Detection/Treatment
Tissue, blood, C:'jrii|iiitai:nnia
c5^. / M"
-------
Fred Mi Her
Guiding Principles for Qore Research:
' ' Composition
The long-term viability and effectiveness of
the Core research requires a mixed portfolio
comprised of
Program projects to address major issues that
require an extensive, Integrated research strategy
Individual projects that address Important topips
(narrow scope, series of sequential projects)
Methods development activities
Investment in & use of cutting edge technologies
CGuiding Principles for Core^Research: Use of
*rm<
^Program Orientation & Implementation
Guidelines Document
Among other things, the document covers relevance
and scientific questions specific to
Identifying & prioritizing Program & Individual
projects
Methods Development projects
Prioritizing Program & Individual
Projects: Relevance Questions
What is the issue and what part do we want
to work on?
What RA uncertainties would be
addressed?
What would represent an impact & how
likely can we achieve it?
12
f
^Identifying 8-
Prinriti-rinn Prnriram X,. Individual
Projects: Scientific Approach Questions
What is the
What is the
hypothesis?
scope of the program needed?
- E-D-R; E-D; D-R, D, R, mechanism
action
, mode of
- Parallel and/or serial activities
Is a Systems Biology approach
feasible and
would it add value?
-------
Fred Miller
Guiding Principles for Pore Research:
Areas of Emphasis
The Developing Organism
Risk Assessment Elements
- Interspecies extrapolations
- Shape of the D - R curve
- Uncertainty factors
- Default assumptions
Susceptibility Factors
Major RA Ur certainty: Shape of D-R at Low D
Example of HUT Rpggarr;h Program and How It
ih_Ero
Links to Focus Areas
Respiratory Biology/Toxicology
Mechanisms of Adaptive & Adverse Responses in the
Respiratory Tract Following Low-Level Exposure to
Inhaled Reactive Gases J. Kimbell, P.I.
Dosimetry Modeling Core (J. Kimbell)
Olfactory Toxlcity of Hydrogen Sulfide (D. Dorman)
Chlorine Risk and VOI Analysis (P. Schlosser & A. Jarabek)
Chio'ine Do!=!rn?try ?nd Pathoblnlogy IP. Schlosser & A. Jarabek)
-------
Fred Miller
-Issues^
Human exposures to reactive gases are often low-
level
- Need to extrapolate animal exposure data to people
Many reactive gases Induce rodent nasal lesions
-' Predictability of rodent nasal lesions for humans Is uncertain
- Adaptive vs. adverse
We need to understand:
- Dose-response relationships over time
- Pathogenesls of lesions
- Susceptibility factors
Gender, age, ethnicity, genetics
- Risk assessment context
-Relevance
LRI Research Strategy issues
- Real-world human exposures
- Demonstration of adversity
- Biological sensitivity
High probability of impact
- Builds on existing strengths and knowledge at CUT
- Current focus of regulatory attention
- Template for broader RA efforts
Uncertainties In risk assessment
- Interspecies arid intraspecies extrapolation '
- Acute to subchronic to chronic extrapolation
^Research Program Goals)and Objectives
Compare and contrast focused,
coordinated studies on two reactive gases
- Elucidate modes of action -> Insight on
adaptive and adverse effects
- Apply this knowledge to CI2 and H2S risk
assessments
Use a systems biology approach
Focus on primary responses
Dosimetry Modeling Core
Serial Step Sections through the Nose
-------
Fred Miller
Nasal CFD Meshes to
Rat
Mouse is next
Turblnates - Human Nose
Hypothesis Testing
Interspecies Extrapolation
M
ppm
= X
-------
Fred Miller
Chlorine Pathobiology
Systems Biojogy Considerations
Project Interactions & Synergies
Prelect 3
CI2 Dosimetry and
Pathobiology
Program Background:
an-CHR Expwtenc*
Nasal Utton Workshop
; EPA. PSU. «nd UCSF j
J Human <
Biochemical j
Gone array !
CUT Centers for Health Research
-------
Allen Dearry
Communicating Kesearcn
Rariiltr- Tlnrf Pi-o/-fi'-oc
.CaqiLIS. JLJCSl 1 idOLlOCa
National Institute of Environmental
Health Sciences
EPA - BOSC
May 15-16, 2003
. Allen Dearry, Ph.D. -^fWlEHS
Overview
NIE
Auc
Cor
Tra
Tw<
Put
HS
iences
imunications and Public Liaison
islational Research
'-Way Communication
ire Directions
.^^ __
~&AV KDEKS
ie National Institute of
arrvitrortmental Health Sciences
Individual
Susceptibility
Age/Time
NIEHS - Implementation
The
thrc
rest
inte
con
enc
tecf
out
NIEHS achieves its mission
ugh multidisciplinary biomedical
;arch programs, prevention and
rvention efforts, and
imunication strategies that
ampass training, education,
inology transfer, and community
each.
c^^ NIEHS
-------
Allen Dearry
Environmental Health
"In it^ broadest sense, environmental health
>rises those aspects of human health,
se, and Injury that are determined or
snced by factors in the environment. This
com
disei
Influ
Includes not only the study of the direct
path
logical effects of various chemical,
deve
indu
physical, and biological agents, but also the
effects on health of the broad physical and social
environment, which includes housing, urban
opment, land-use arid transportation,
:try, and agriculture."
- Healthy People 2010
Target Audiences
_L
Gen
jral Public
Community and Advocacy |
s
Professional Organizations
Hea
Othi
Grantees
th Professionals
r Agencies
Congress
Office of Communications and
ic Liaisoir
Press Releases
Pamphlets, Video, Fact sheets, etc.
Cor ference Exhibits
Public Service Announcements
Put lie and Media Inquiries
NIEHS Website
h1tp://niehs.nih.gov/home.htm
Environmental Health
crspecti ves
17 Is::ues/Yr, Global Distribution
Toxic ogenomics
Child
en's Health
Environmental Medicine
Chinese Edition
Underdeveloped Countries
FOR MORE INFO.. |
See the
EHP Home Page - http://ehp.niehs.nih.gov/
'Jg HIEKS
-------
Allen Dearry
National Toxicology Program
! ,
Repor
- Publ
. NTPC
Websi
Public
NIEHS
Techn
op Carcinogens
c, Industry, Government
inters
e - includes listserve
Meetings
WlH Central Data Management Office -
cal Reports, Annual Report
FOR MORE INFO...|
See the
NTP Home Page - http://ntp.niehs.nih.gov/
~w,
-------
Allen Dearry
Translational Research
Two-Way Communication
Tru 5 Communication is Two-Way
"If you're trying to listen and talk at the same time,
you'll stink at both of them."
6th 3r«d«f
Benefits Challenges
Commu
- Collab
Better P
- Relev;
health
- Hypot
Result
- Sustal
i Cost Efi
- Resea
health
- Trans
disser
nication Identifying Key
orative Participants
esearch Establishing criteria
nee to public Time
Initial investment
leses, Design,
s Controversies and
lability Crises
ectiveness - Being proactive
ch linked to Expectations and
outcomes Nee(js
itlon,
ilnation rf%
-^TKIEKS
1 t v
(Two-Way Communication at
fflBHS
B
Research
immunity Outreach and Education Programs
Edu nation
- K-12 Program
Prio-ity Setting
- W jrkshops, Roundtables, Retreats,
ainstorming Sessions
- NAS, IOM Committees
- NIEHS Town Meetings
- Interagency Working Groups
- Public Interest Liaison Group
-------
Allen Dearry
Current and Future Directions
Transla
-EJ-P
- Cent
Disp
- Brea
- Built
Enviror
- Nurs
- EHP
t Divisioi
Transla
- OPP
:ional Research
artnerships for Communication
ers for Population Health and Health
arities
st Cancer and the Environment Centers
Environment
mental Medicine
e Training, Research, and Practice
Grand Rounds
of Research Coordination, Planning, and
ion
:, OCPL, EHP, Library, Tech Transfer *£&IUEH8
-------
t
AMERICAN
LUNQ
ASSOCIATION*
"-- '^mm&K&KKi- . /:
:-,;: pia,
.;, TbVencbufage policy^cn
.:' «SK??!S»ffi?₯te,?, j V_>;ii:ifeSg
-------
...... _....-.,.,. ... ,., >Mg|
Rank? 25 metro areas with^pjstozpne:;i;;
i
?... . .vOFiAS.iiS?* v,'f'^!iVh;fcv5rtV^..iiTis?A^*ir^r-rsfe//-S<}!qiss
^$g? ^VT'ifAMERICANlvfc:;;
:-*-^"i j>-; i -.'.-iTf^UJNG'.- ' "" '"
- - AMERICAN LUNG ASSOCIATION.
-,».i;.- ,
"-..' ± AHERICAN LUNG ASS
;'>! I State of the Air:
2
-------
State of the Air 2003
LUNGrr
ASSOCIATION
} , _ (137jfnitlion) live In areas with -^
unhealthy jevelsofozohe*"**^*-*-1 **^v
^ 55 5% of all monitored courlties received an' F
^ * Over 7 4 million adults and 2 mllliory:hiTdren with
asthma live fn F counties ""*"
Over 1 5 million Americans with emphysema live
who live-in -monitored::: --
.,. --.
.... Prepare national mediaspoRes
-iW gsfeia*^^- .- v. Ss^^s
>,">- Maintain earl and continuous.
-------
". ::Sfatefo^He'AiK^d03&:- ' -.?$|«&s|£:
' ^^g$&%^^;-..^^&^f*5-j:~ - ij.vJr&fe*^-;
-, ' -"r^VS-gJ^;!,-^.. ;Vv|7r.
;-';: Telemuhd6rIDa1iy.shbwli(G^,:^^_p:''i
- ; ' /^'-i ';v/-,^»*»*feiggsaa»f3K^^*;;Si^^^ '
;;;.:, Central), Discovery^hannel^'f^^iS^^
Ady^pcJ^bulreach to jpcaj
rnecllialvf;; ^yj;- . . .
-------
.
v Why was repo
':.' «TProvided t
-;*'- ";*. - - v1ii*-'"«**?-%-J
V-;. r. materials in a
AMERlCAN.--4«::
.,,-. .. --«JNG*!»:«-- -.'^.:...
S*'-i-ol : ASSOCIATION, «r
-:x:?i^t^i,,,,v^l^,
' !'-: ^l.^?|^a*iH).?'"i?ar9??§i?'
r.;.' . ..!Helpecl?rep6rteris fihi
''.'.':'. i'--fi;»;&-'.ii.T.t54.ri-:£,-,-iarw?5i
. OiUNQ'-asW^ifK-'.;
ASSOCIAT1O
rnal cbrnmuhicatiohs
v i^omijsuppora TO neip «
-/.-. -.J"v2ses*-^^fe&^-- ., --sSsSi
-------
.^ -ry«'-:3l;;':i'^-*3Sa*S5i'v;'--.iS*
Tel econference media
-------
.;;'"» S^F-;- w''**;«.".' - - c~ci-.',;jti^»
orderwou can:rnu
-«i-ii''-'' -->>S
7
------- |